Skip to main content

Skill · AI Hiring Index · as of 2026-11-19

Evals in AI job postings

21% of technical postings at the AI companies we track mention Evals (1,037 postings). Research Scientist postings ask for it most: 66%.

Technical postings that mention it
21%
1,037 postings
Change since the previous Index
+5 pt

Which roles ask for it

Share of each technical role family's open postings that mention Evals.

Role familyShare, as a barShare
Research Scientist66%
AI / ML Engineer61%
Research Engineer60%
Evals / Data38%
FDE / Applied28%
Safety / Policy20%
Inference / Perf16%
Software Eng12%
Infra / Hardware5%
Security5%

Companies asking for it most

CompanyPostingsOf its technical postings
OpenAI14228%
Anthropic11132%
Sierra6479%
Scale AI4952%
Databricks448%
Mistral AI3827%
Harvey3236%
Mercor2959%
Wayve2623%
Cohere2336%

Open postings that mention Evals

How to learn it

  • AI Evals: Everything You Need to Know — Hamel Husain and Shreya Shankar free

    Start from error analysis on real traces, not from a metric; then judges you check against people — the practice behind 'evaluation frameworks' in these postings.

  • promptfoo free

    Runs an eval set as CI — cases in YAML, assertions by rule or by judge, red when the score drops — turning one-off eval studies into a regression suite. Open source; part of OpenAI since March 2026.

  • Inspect free

    An evaluation institute's open-source framework: a clean task / dataset / solver / scorer split, with agent tasks and tool use first-class — the shape to grow a reusable harness in.

Project: An eval harness that gates a change — Label 100 real outputs by hand, build an LLM judge and measure its agreement with you, and make CI fail when a prompt or model change drops the score.
A posting counts when its description mentions the skill in what the role does or asks for — the company's description of itself is left out. Shares are over technical postings only; the tags are checked against the postings every month and a skill below90% precision is not shown. How we count.

Spot a mistake on this page?