Skill · AI Hiring Index · as of 2026-11-19
Evals in AI job postings
21% of technical postings at the AI companies we track mention Evals (1,037 postings). Research Scientist postings ask for it most: 66%.
Which roles ask for it
Share of each technical role family's open postings that mention Evals.
| Role family | Share, as a bar | Share |
|---|---|---|
| Research Scientist | 66% | |
| AI / ML Engineer | 61% | |
| Research Engineer | 60% | |
| Evals / Data | 38% | |
| FDE / Applied | 28% | |
| Safety / Policy | 20% | |
| Inference / Perf | 16% | |
| Software Eng | 12% | |
| Infra / Hardware | 5% | |
| Security | 5% |
Companies asking for it most
| Company | Postings | Of its technical postings |
|---|---|---|
| OpenAI | 142 | 28% |
| Anthropic | 111 | 32% |
| Sierra | 64 | 79% |
| Scale AI | 49 | 52% |
| Databricks | 44 | 8% |
| Mistral AI | 38 | 27% |
| Harvey | 32 | 36% |
| Mercor | 29 | 59% |
| Wayve | 26 | 23% |
| Cohere | 23 | 36% |
How to learn it
- AI Evals: Everything You Need to Know — Hamel Husain and Shreya Shankar free
Start from error analysis on real traces, not from a metric; then judges you check against people — the practice behind 'evaluation frameworks' in these postings.
- promptfoo free
Runs an eval set as CI — cases in YAML, assertions by rule or by judge, red when the score drops — turning one-off eval studies into a regression suite. Open source; part of OpenAI since March 2026.
- Inspect free
An evaluation institute's open-source framework: a clean task / dataset / solver / scorer split, with agent tasks and tool use first-class — the shape to grow a reusable harness in.