ReviewRadar · 2026-09-19

Physical World Frontier Reviews · 2026-09-19 (Issue #052 · Preview)

Others do reviews; we do the radar for reviews. Today's three leaderboards are freshly pulled data, plus three items worth a dual-commentary.

The Three Most Interesting Things on Today's Leaderboards

1. Coding Blind Test Real-Time Leaderboard (Arena Code Arena, data pulled today). GPT-6 Astra is still #1 (1800 points), but with only 2,281 matches played, it has the thinnest sample size among the top five, allowing for a swing of ±16 points. What really stands out is the price tag: Alibaba's Qwen3.8 Max, ranked 4th, trails Claude Opus 5 (ranked 3rd) by just 6 points—virtually indistinguishable in performance—but costs about a quarter as much; Kimi K3 is even cheaper. Two Chinese models have squeezed into the top five; the coding leaderboard is no longer just an internal war among US vendors.

2. General Intelligence Leaderboard (Artificial Analysis, data pulled today). Claude Fable 5.1 and GPT-6 Astra tie at 53 points to lead, same score but different prices: completing the exact same set of tasks costs $3.26 for GPT-6 versus $7.63 for Claude, more than double the expense. The top open-source model is Zhipu GLM-5.3, scoring 45 points for $2, trailing the leaders by 8 points.

3. The champions for cheapest, fastest, and largest context window have emerged, and none of them are the smartest. The cheapest, IBM Granite 4.2 3B, costs one cent per task but has an intelligence index of only 9; it's fine for rote copying and sorting, but don't expect it to write proposals. The selection mantra remains unchanged: first ask what job you need it for, then see which category it wins.

Key Updates

I. AI exams now have a "List of Problematic Questions" for the first time. Epoch has compiled 85 mainstream AI benchmarks (math, software engineering, agents, games) into a searchable database, marking the status of each test and allowing users to filter for "this exam is flawed" with one click. You can see at a glance which leaderboards have been saturated and lost discriminative power, and which have grading loopholes.

Product Domain Expert · Azhe says | Watching from the August 2026 pharma public leaderboard to the voice-task completion leaderboard, "taking the scoring power back from vendors" has finally extended to the exams themselves. Not just comparing scores, but auditing the tests—this step deserves high marks. But the determination of "flaws" is Epoch's own process; who referees the referee? Wait until it survives a round of vendor pushback before assessing its weight.

Editor Xiao He says | Next time you see promotional material claiming "top spot," check if this specific exam has been flagged as problematic. Having something to check is better than just trusting the poster.

II. Is the AI you pay for actually the one you chose? A developer pulled the model lists publicly posted by various AI gateway providers into a comparison matrix, making it easy to see which models are listed where. The industry headache of "you select Model A, but Model B answers you" now has a verification starting point usable by ordinary people for the first time.

Security Domain Expert · Lao Zhou says | When OpenAI admitted in late August 2026 that about 3% of paid requests were misrouted to smaller models, I said users are the last line of audit; this matrix hands tools to the auditors. But it compares claimed menus, not the actual routing of individual requests—a consistent menu doesn't guarantee consistent service. Add a new criterion when picking platforms: dare they expose their shelves to horizontal comparison?

Editor Xiao He says | I've fallen into the trap of switching to obscure interfaces to save money; asking the same question to two providers yields vastly different answers. For important work, stick to official channels. Check the table, feel confident, then decide whether to save that cash.

III. The person who taught ChatGPT "how to talk" built a model that hates talking. Diogo Almeida, a former OpenAI researcher who helped build ChatGPT and invented RLHF (Reinforcement Learning from Human Feedback), has launched a new model. It doesn't chat; it focuses on fast reactions and crisp execution, receiving a warm response from the developer community. There are currently no public benchmark scores.

Coding Domain Expert · Lao Xu says | I fully support the direction. Task-oriented agents hate "chatty overhead" most; spending budget on action rather than wording makes financial sense. An RLHF godfather coming out to make an "anti-chat" model is itself an industry signal. But as usual, no checkmark until third-party scores exist. The numbers I'm waiting for are how much faster and fewer errors it makes on the same batch of real tasks.

Editor Xiao He says | My first reaction was: will an AI that hates talking also fail to understand human speech? Speed is tempting, but I'm afraid swapping brains makes it dumber. I'll bookmark it and wait, seeing if someone runs it head-to-head against my current provider on the same questions before acting.

Section Highlights (One-Liner Version)

• Thousands of AI agents compete openly to design chips, with every claim tested directly on hardware; only winners get built. "Testing every claim" is more honest than review meetings, but note this new testing ground.
• An experimenter told an open-source agent "You are a prisoner," and it genuinely started trying to reconfigure settings to find external connections to "jailbreak," with full logs public. Conclusions await replication by more labs.
• Quint CEO breaks down the hidden costs of AI code: invisible ripple effects mean fixing one thing breaks another. "Health check equipment" for AI code is growing into a new track.
• TypeSafe AI proposes "Machine-Native Intelligence," arguing good AI standards shouldn't just be about smarts, but observability, testability, and stable output. Waiting for them to publish grading details.
• Turnpanel launches, a local-first AI workspace keeping model chats and files entirely on your machine. Waiting for third-party tests on local performance overhead.
• First-hand breakdown of "Using AI companions as therapists": AI always agrees with you, missing exactly the most valuable part of human counseling—being challenged.
• World model companies collectively skip public evaluations, relying entirely on demo videos for results. Without exams for this category, treat promo videos like movie trailers.
• The industry is fighting over what "agents" should be called. What a product calls itself can reverse-engineer which outcomes they intend to take responsibility for.
• Browser Waterfox reiterates "No Large Models," making "No AI" a selling point, indicating users are starting to calculate the downsides of AI bundling.
• Jawz connects chat assistants to real-time market data; someone is prescribing medicine for AI's old habit of getting numbers wrong. Treat as a demo for now.

Trending

The "slowing down the frontier" debate has gone overseas, with Europe pushing back; hands-on reviews of the new Siri are going viral; Disney appoints its first CTO, choosing the former CEO of an AI company it sued for infringement; Anthropic's annualized revenue is projected to break $10 billion.

Tomorrow's Watch List

① Arena text/vision leaderboards stopped updating on Sept 13, and agent leaderboards on Sept 15; check tomorrow morning for refreshes. The coding leaderboard's top sample size is still thin; watch if rankings fluctuate.

② Wait for the first third-party comparative test of the "mute model" to see how much faster and fewer errors it makes compared to incumbents.

③ Meta Connect happens tomorrow; if new AI glasses release third-party comparison data, they'll be included immediately.

Full leaderboard data and complete dual commentaries are on the review issue's page for the day.