Most of those workers are seeking religious or medical exemptions