Interview grading, fairness & transparency

The standard demonstration rubric contains four observed criteria with equal 25% weights. The weighted sum, denominator, evidence coverage and version of the rubric should all be visible in a real candidate report.

Role weighting is demonstrative, not a claim that FirstRound uses these exact weights for every employer. Changing weights changes the meaning and comparability of the composite.

Calculation: (88 × 25) + (84 × 25) + (80 × 25) + (84 × 25) ÷ 100 = 84.00. Illustrative score shown.

We use 80% observed weight solely as an illustrative product quality gate, not as an EEOC requirement. Production thresholds must be empirically validated; scores should not be marked reliable without adequate coverage and review.

A job family can require different demonstrable competencies: sales may weight relevant buyer evidence, software engineering may weight technical trade-offs and written assessment can measure answer structure. Company-specific criteria need a documented job analysis; they cannot introduce protected-trait proxies, arbitrary expectations or hidden score changes.

For progress charts, compare the same underlying competency only when rubrics and scale anchors are compatible. A 90 in one role is not automatically equivalent to a 90 in a different role or model version.

When a process is actually used to make an employment selection, the longstanding Uniform Guidelines describe a four-fifths (80%) screening comparison. It is a rule of thumb, not a definitive legal test or a fairness certificate. It does not apply directly to practice-interview grades, which are not hiring selections.

Worked example: Group A selects 48 of 80 candidates (60%). Group B selects 12 of 40 (30%). The impact ratio is 0.30 ÷ 0.60 = 0.50 (50%). That is below the historical 80% screening threshold, but does not establish illegality, statistical significance, or why the rates differ.

Analyze uncertainty, job relevance, selection stage, sample composition, missing data, accommodations and intersectional subgroups before drawing conclusions. Group sizes and subgroup overlap can make apparent trends misleading or identifying.

Current public status: no verified FirstRound demographic fairness dataset is published on this page. We will not invent breakdowns by race, ethnicity, sex, age or disability, or treat participation demographics as a measure of selection fairness.

A future public report may provide eligible cohort sizes, comparable mean practice scores, score uncertainty and, separately, true employer selection rates—but only after lawful collection, authorization, retention review, aggregation and disclosure approval. At least 30 observations per group is our conservative initial suppression control, not an absolute anonymity guarantee. Overlapping filters, rare intersections, differencing attacks and repeated reports can still reveal people; those require additional controls.

Raw traits, individual interview scores, names, addresses, transcripts and biometric data must not enter public PDFs or an LLM audit prompt. Never infer race or sex from a video, voice or name. An audit model may explain precomputed aggregate results, but cannot certify that the underlying tool is fair.

Auditors should distinguish descriptive differences from uncertainty and validity. No single test establishes that an SI assessment is fair. Select statistical methods based on the measurement and sampling design; use statistical specialists for small cohorts or dependent observations.

Version drift: Compare MAE for the same human-rated benchmark set and rubric version before and after a model change. A five-point change in MAE is a provisional review flag, not a statistical finding or proof of unlawful discrimination.

Consistency measurements: Track human-to-human and SI-to-human agreement, repeat grading dispersion and rubric drift across model versions. If a criterion was unobserved, it should not be silently imputed as zero. Selection-rate ratios and group mean scores are different statistics and cannot be substituted for one another.

Do not publish a platform audit as completed unless a representative dataset, consent basis, defined selection process, statistical methodology, independent assessment where appropriate, and versioned evidence are available. No SI-generated narrative replaces those controls.

The following primary sources describe relevant employment, SI and risk-management principles. They do not amount to an endorsement of FirstRound SI or certification of this product.

• EEOC · Uniform Guidelines questions and answers
• 29 CFR §1607.4 · Information on impact
• 29 CFR §1607.15 · Documentation of validity
• ADA.gov · Algorithms and disability discrimination in hiring
• Executive Order 14281 · April 23, 2025
• DOJ Office of Legal Counsel · June 9, 2026
• NIST · SI Risk Management Framework (voluntary)

For specific compliance conclusions, employer selection procedures and deployment decisions, consult qualified employment counsel and a qualified statistical or industrial-organizational assessment expert.