New York City Issues Proposed Rules On Law Aimed At Curbing Artificial Intelligence Bias In Employment Decisions – Employee Rights/ Labour Relations

Seyfarth Synopsis: On September
23, 2022, New York City’s Department of Consumer and Workplace
Protection (“DCWP”) released the highly anticipated
proposed rules implementing Local Law Int. No. 1894-A, which
regulates the use of automated employment decision tools
(“AEDT”) in hiring and promotion decisions and takes
effect on January 1, 2023. The proposed rules attempt to address
some of the ambiguities under the law by providing additional
details on employer obligations. DCWP will hold a public hearing on
the proposed rules on October 24, 2022 and is also accepting public
comments on the proposed rules until that date.

As background, the New York City Council passed Local Law Int.
No. 1894-A on November 10, 2021 to amend the City’s
Administrative Code. The law was designed to prohibit employers and
employment agencies from using an AEDT unless such tool has been
subject to a bias audit within one year of the use of the tool,
information about such audit is publicly available, and certain
notices have been provided to employees or job candidates. You can
read more about the law in our past coverage here.

The administrative rules being proposed aim to address some of
the ambiguities under the current law and provide additional
clarification on employer obligations. Several provisions are
highlighted here: (1) new and expanded definitions for certain
terms found in the law, (2) clarification on the requirements for
bias audits and the results of such audits that must be made
publicly available, and (3) notices regarding the use of an AEDT
that employers and employment agencies must provide to employees
and candidates for employment.

What Is An Automated Employment Decision Tool

As currently defined, an AEDT is “any computational
process, derived from machine learning, statistical modeling, data
analytics, or artificial intelligence, that issues simplified
output, including a score, classification, or recommendation, that
is used to substantially assist or replace discretionary decision
making for making employment decisions that impact natural
persons.” While the proposed rules maintain this broad
definition, the rules provide additional context by defining
certain terminology and phrases found in the AEDT definition
including: “machine learning, statistical modeling, data
analytics, or artificial intelligence,” “simplified
output,” and “to substantially assist or replace
discretionary decision making.”

• Machine learning, statistical modelling, data
  analytics, or artificial intelligence: means a group of
  mathematical, computer-based techniques: (1) that generate a
  prediction (an expected outcome for an observation, such as an
  assessment of a candidate’s fit) or a classification (an
  assignment of an observation to a group, such as categorizations
  based on skill sets or aptitude), (2) where a computer at least in
  part identifies the inputs, the relative importance placed on those
  inputs, and other parameters for the models in order to improve the
  accuracy of the prediction or classification, and (3) for which the
  inputs and parameters are refined through cross-validation or by
  using training and testing data.




• Simplified output: means a prediction or
  classification as specified in the definition for “machine
  learning, statistical modelling, data analytics, or artificial
  intelligence.” A simplified output may take the form of a
  score (e.g., rating a candidate’s estimated technical skills),
  tag or categorization (e.g., categorizing a candidate’s resume
  based on key words, assigning a skill or trait to a candidate),
  recommendation (e.g., whether a candidate should be given an
  interview), or ranking (e.g., arranging a list of candidates based
  on how well their cover letters match the job description). It does
  not refer to the output from analytical tools that translate or
  transcribe existing text, e.g., convert a resume from a PDF or
  transcribe a video or audio interview.




• To substantially assist or replace discretionary
  decision making: means to rely solely on a simplified
  output (score, tag, classification, ranking, etc.), with no other
  factors considered, or to use a simplified output as one of a set
  of criteria where the output is weighted more than any other
  criterion in the set, or to use a simplified output to overrule or
  modify conclusions derived from other factors including human
  decision making.

These broad definitions mean that a wide variety of automated
tools will likely be covered by these rules. Some examples include
resume scanners that prioritize applications using certain
keywords, employee monitoring software that rates employees on the
basis of their keystrokes or other factors, virtual assistants or
chatbots that ask job candidates about their qualifications and
reject those who do not meet pre-defined requirements, and testing
software that provides job fit scores for applicants or employees
regarding their personalities, aptitudes, or cognitive skills.

Independent Bias Audits and Publication of Results

The proposed rules also address the lack of substantive
requirements for bias audits as provided under the law. As
currently defined, a bias audit is “an impartial evaluation by
an independent auditor” that includes testing the AEDT to
assess its “disparate impact” on employees and applicants
based on race, ethnicity, and sex.

Although the text of the law omitted a definition of what
constitutes an independent auditor, the proposed rules define an
independent auditor as “a person or group that is not involved
in using or developing an AEDT that is responsible for conducting a
bias audit of such AEDT.” While this seems to indicate that a
third-party will be required to conduct the AEDT bias audits, the
definition fails to specify the level of independence required
(i.e., whether individuals within a company that do not use the
AEDT can conduct the independent audit).

The rules also address the disparate impact mechanism employers
and employment agencies need to use when conducting the bias audit.
The proposed rules establish that an auditor must calculate the
selection rate for each EEO-1 race, ethnicity, and sex reporting
category (“EEO-1 demographic category”), and compare
those selection rates to the most favored group to determine an
impact ratio. In other words, the analysis must first identify the
specific group with the highest selection ratio and then analyze
the impact of the tool against every other relevant demographic
category. The examples set forth in the proposed rules suggest the
required analyses are “cross-sectional” analyses based on
the prescribed EEO-1 race/ethnicity and sex categories.

The proposed rules go on to explain that there are two distinct
audits that must be run depending on the type of AEDT being
utilized. The first involves situations where an AEDT selects
individuals to move forward in the hiring process or classifies
individuals into groups. Under this type of AEDT the bias audit
must, at a minimum: (1) calculate the selection rate for each EEO-1
demographic category; (2) calculate the impact ratio for each EEO-1
demographic category, and (3) where an AEDT classifies individuals
into groups, the calculations in paragraphs (1) and (2) above must
be performed for each such classification. On the other hand, when
an AEDT scores applicants or candidates, the bias audit must, at a
minimum: (1) calculate the average score for individuals in each
category and (2) calculate the impact ratio for each category.

The proposed rules provide additional guidance on the
requirements for posting AEDT audit results. Before using a
specific AEDT tool, employers and employment agencies in New York
City must make the following information publicly available: (1)
the date of the most recent bias audit for the AEDT being used, (2)
a summary of the results, including the selection rates and impact
ratios for all EEO-1 demographic categories, and (3) the date the
employer or employment agency began using a specific AEDT. These
requirements can be met by including a clearly identified hyperlink
on the website of an employer or employment agency. The proposed
rules require that this information remain posted for at least six
months after last using such AEDT for an employment decision.

Even with this additional guidance, important questions with
regard to the audit and publication provisions still remain and
will need to be addressed during the notice and comment period.
Among others, such questions include whether company or vendor
personnel not involved with the use or development of an AEDT can
qualify as an “independent” auditor, how the audits
findings are to be utilized, and whether the AEDT must
“pass” an audit to be utilized, and if so, what is
considered a passing score under the NYC law.

While certain provisions of the law clearly apply only to NYC
residents, it is not clear whether the audits may appropriately
include the results of decisions made outside of NYC or based on
candidates or employees that do not reside in NYC. Indeed, it is
rarely the case that an AEDT is focused on such a narrow
demographic region so the feasibility of limiting the results to
“residents” of NYC may present unforeseen challenges to
employers and vendors.

The proposed rules also appear to lack clarity on the timing of
the audits. The text of the law specifies that AEDT tools cannot be
used unless the tool has been the subject of a bias audit “no
more than one year prior to the use of the tool.” Given the
law’s upcoming January 2023 effective date and the lack of
finalized rules, it is not clear whether NYC employers will have
the opportunity to use AEDT tools in January 2023 unless the rules
are quickly finalized and employer audits that comply with the
proposed rules are quickly conducted.

There are concerns that certain of these timing issues and
requirements will slow down innovative technological advances for
NYC workers and employers. NYC employers who want to use AEDT may
have to deal with a “chicken and the egg” problem. Put
differently, if a tool cannot be used unless an audit is conducted,
what is the appropriate population on which to conduct such an
audit. And to the extent employers are permitted to rely on audits
that use the same AEDT based on other geographic regions or indeed,
as applied to other employers, the proposed rules should provide
further clarification.

Notice Requirements

Employers who use an AEDT must notify all candidates and
employees residing within New York City that such tools will be
used to assess their candidacy and must provide the job
qualifications and characteristics the tool will be assessing. This
notice must be given at least ten business days before the AEDT is
used to ensure employees and candidates have an adequate
opportunity to “request an alternative selection process or
accommodation.”

The proposed rules add to this already stringent notice
requirement by defining a “candidate for employment” as
“a person who has applied for a specific employment position
by submitting the necessary information and/or items in the format
required by the employer or employment agency.” If adopted,
this overly broad definition could require employers and employment
agencies to notify any candidate that submits a job application for
an open position regardless of whether or not they are considered
or qualify for the job. This could be especially problematic for
those employers that utilize high volume requisitions to fill high
turnover or entry level positions.

The proposed rules attempt to limit this burden by providing
several ways to comply with the notification requirement. Employers
and employer agencies may provide notice by either: (1) including
the notice on their careers website in a clear and conspicuous
manner (for candidates) or in a written policy or procedure (for
employees), (2) listing the notice on a job posting, or (3)
providing the notice via U.S. mail or e-mail. In a seeming
contradiction, the proposed rules require that the notice include
instructions for how to request an alternative selection process or
accommodation, but also clarify that nothing requires that an
employer or employment agency provide an alternative selection
process.

Further, employers and employment agencies are required to
retain information about the data collected for the AEDT being used
and its data retention policy. This information can be disclosed by
either: (1) including notice on the careers or jobs section of its
website in a clear and conspicuous manner, or (2) providing written
notice in person, via U.S. mail or e-mail within 30 days of receipt
of a written request for such information. If the notice is not
included on the career website, employers and employment agencies
must post instructions for how to make a written request for such
information.

Public Hearing and Opportunity to Comment

DCWP will hold a public hearing on the proposed rules on Monday,
October 24, 2022 at 11:00 a.m. ET which will be accessible by phone
and videoconference. For more information on how to attend the
public hearing please click here. Persons that wish to comment on the
proposed rules at the public hearing will be given up to three
minutes to speak and must sign up to speak prior to the hearing. As
for those interested in submitting public comments on the proposed
rules to DCWP, the deadline is also October 24, 2022. Comments can
be submitted through the NYC Rules website or by email to [email protected].

The content of this article is intended to provide a general
guide to the subject matter. Specialist advice should be sought
about your specific circumstances.