Physix FrontierPhysix Frontier · News Briefing Card (Hacker News · Nov 6, 2026)
AI firms recruit professionals to set model evaluation standards
KEY FACTS
- AI training providers are recruiting professionals such as biostatisticians and air traffic controllers to design evaluation tasks.
- Biostatisticians must build clinical trial evaluation tasks and define more than 35 scoring criteria.
- Air traffic controllers are asked to transcribe real aviation radio communications, with word error rate required to stay below 2%.
- Engineers test whether AI agents can complete professional tasks in CAD and CFD software.
- The core of these roles is defining correct answers and evaluation standards, not developing AI.
KEY DATA
35+Number of scoring criteria
≤2%Word error rate threshold
PHYSIX OBSERVATION
AI evaluation is shifting from the language level to professional workflows, and the industry has begun using practitioners' tacit knowledge to define what counts as "qualified." This means high-quality evaluation data is becoming a scarce asset, and it also reminds us: the ceiling of model capability depends on whether genuinely knowledgeable people can be recruited to set the questions and grade the answers.
Source: Hacker News report