Social security workers rallied in front of the agency building in Baltimore in fear they will be next in the round of federal worker layoffs.