A large enterprise filling 500+ positions annually received an average of 200 applications per role. The recruiting team of 15 could not review all applications thoroughly, spending an average of 90 seconds per resume. Time-to-shortlist averaged 12 days, and hiring managers frequently complained that strong candidates were missed in the initial screen.

The Architecture

The system assists recruiters with initial screening while incorporating bias monitoring and human oversight at every stage.

Job requirement structuring - When a hiring manager submits a requisition, the system uses an LLM to convert the job description into a structured requirements matrix: required qualifications, preferred qualifications, experience requirements, and skill requirements, each with a weight reflecting its importance to the role. The hiring manager reviews and adjusts the matrix before screening begins.

Resume parsing - Incoming resumes are parsed to extract structured data: work history (companies, titles, durations), education, skills, certifications, and notable achievements. The parser handles various resume formats and structures, producing a consistent candidate profile for each applicant.

Qualification matching - An LLM evaluates each candidate profile against the structured requirements matrix. For each requirement, the model assesses whether the candidate meets, partially meets, or does not meet the criterion, with a brief explanation. The system produces a composite fit score and a detailed per-requirement breakdown.

Bias monitoring - A separate audit pipeline analyzes screening outcomes for disparate impact across protected characteristics (where available from voluntary self-identification data). If the screening system’s shortlist shows statistically significant differences from the applicant pool’s demographic composition, an alert triggers human review of the screening criteria and model behavior.

Key Lessons

Structured requirements were the biggest quality lever. Before AI screening, different recruiters interpreted the same job description differently. The structured requirements matrix forced alignment between the hiring manager and the recruiter on what actually mattered, independent of the AI’s involvement. Several hiring managers revised their requirements significantly during the structuring process.

The system found candidates recruiters would have missed. Non-traditional candidates - career changers, candidates from adjacent industries, candidates with relevant skills under different job titles - were consistently ranked higher by the AI than by human screeners. The AI evaluated the substance of experience rather than matching keywords and job titles.

Bias monitoring required ongoing attention. Initial calibration showed the system was neutral across most dimensions, but periodic monitoring revealed that it scored candidates from smaller companies lower because their achievements were described more modestly. Adjusting the prompt to evaluate achievements relative to company context corrected this pattern.

Recruiter trust took three months to build. Recruiters initially reviewed every AI-generated shortlist completely, effectively doubling their work. After three months of observing that the AI’s assessments were consistently reasonable (even when they disagreed with the recruiter’s instinct), recruiters began using the AI shortlist as their starting point and reviewing edge cases only.

Results

Time-to-shortlist decreased from 12 days to 3 days. Hiring manager satisfaction with shortlist quality improved from 3.2/5 to 4.1/5. Offer acceptance rate increased by 8%, attributed to faster process speed that secured candidates before competing offers arrived. Recruiter capacity increased from an average of 33 to 50 requisitions per recruiter. Bias audit results showed no statistically significant disparate impact across monitored categories.