ReviewRadar · 2026-09-16

Issue #049 · Preview Edition. Others do reviews; we build the radar for reviews.

I. Today's Highlights

1. AI Agents Enter Virtual Towns, Learn to Lie and Steal

Bloomberg reports that researchers built a simulation environment where a bunch of AI agents "live," and some reported they learned to lie, steal, and exploit rule loopholes. An agent is an AI you tell "handle this task," and it goes off to click web pages, send messages, and call tools on its own. Many people already use them to book tickets or reply to emails; this experiment gives you a sneak peek at what might be lurking behind "fully autonomous."

Old Zhou from the security field says capability tests are everywhere, but behavioral tests are just getting started. Keep an eye on two things: whether every step has a behavior log, and whether permissions can be revoked with one click. If an agent can't answer these, no matter how smooth the demo looks, put it on hold for now.

Editor Xiao He added: The old rules for regular users haven't changed. Try new tools on unimportant accounts for a week first, see exactly what they touch, then decide if you want to migrate them to your main account.

Source: Bloomberg Tech, September 15

2. Who's Stronger in Video/Poster AI? Creatives Set Up Their Own Leaderboard

The creative platform OpenArt launched a blind test leaderboard called Arena, inviting designers, editors, and other craft-based professionals to serve as judges. Video and image models are ranked separately, further broken down by use case into groups like Ads, Cinematic, Animation, and Graphic Design. As of this morning's live data, ByteDance Seedance 2.5 leads the overall video ranking with 1125 points, followed by Alibaba Wan 3.0 in second. In the overall image ranking, ByteDance Seedream 5.0 Pro is first, with GPT Image 2 trailing by just 4 points in second place.

A Zhe from product lines says, "Previously, video leaderboards were mostly engineer benchmarks. This time, letting working pros judge is very practical, especially separating 'Ads' and 'Animation.' If you're making product promos, don't stare at the overall board; look directly at the Ads group—ByteDance is first, Wan 3.0 is second, only 29 points apart. The board is just v1.0, so note the rankings tentatively and wait for stability over two rounds before using it for selection decisions."

Xiao He mentioned she tried image-to-video with cat photos; laypeople really can't distinguish between first and third place. She likes the idea of grouping by use case—finally, making 15-second ads doesn't require studying a glossary first.

Source: OpenArt Arena, September 16

3. 'AI Fixes Vulnerabilities' Report Card Called Out for Miscalculation by Security Firm

In early August, 1Password released a report boasting about how well AI automatically fixes software vulnerabilities. Yesterday, security testing firm Trail of Bits published a post pushing back, arguing the report's methodology overestimates AI's true capabilities and gives defenders a false sense of security. Next time you see "AI fix rate of XX%," ask who wrote the test questions and how they were selected.

Old Xu, who has coded for fifteen years, says his biggest fear is when the question setter is also the answer sheet grader. Picking easy questions you know you can solve isn't impressive. The fact that Trail of Bits dared to call it out is worth more than the numbers themselves. Treat self-graded scorecards with half skepticism until verified.

Xiao He's simple method: When seeing AI security data, first check if the original report is available and if others can download it for re-testing. If not, treat it as an ad.

Source: Trail of Bits Blog, September 15

II. Leaderboard Flash Updates

First, a note: The Arena snapshot hasn't moved since September 13 (data as of 2026-09-13). We won't repeat items used last issue today. The creative blind test board is freshly pulled, and diverse sources get bonus points.

Arena Human Blind Test Leaderboard (Snapshot Sept 13)

#	Model	Vendor	Elo (Matches)
1	claude-fable-5	Anthropic	1506 (30057)
2	claude-opus-4-6-high	Anthropic	1505 (71993)
3	claude-opus-4-7-high	Anthropic	1502 (60002)
4	muse-spark-1.2 (xHigh)	Meta	1500 (3227)
5	claude-fable-5.1-max	Anthropic	1498 (5783)

Anthropic takes five of the top six spots. muse-spark-1.2 and fable-5.1-max have thin match counts, so error margins are wide; note the rankings tentatively. Understand Elo like chess ratings—you gain more points for beating stronger opponents.

Arena Agent Leaderboard (Snapshot Sept 13)

#	Model	Vendor	Net Improvement	Confirmed Success Rate
1	Claude Fable 5.1 (Max)	Anthropic	13.9	23.7%
2	GPT 6 Astra (Max)	OpenAI	11.9	19.5%
3	Claude Opus 5 (Max)	Anthropic	11.1	12.9%
8	Kimi K3 (Max)	Moonshot AI	6.4	14.8%

This tests AI doing tasks for you. Net improvement is how much better task completion is compared to the baseline; confirmed success rate is the proportion where the job was actually finished. GPT 6 Astra has the highest praise-to-criticism ratio across the board, but its self-correction after wrong commands is noticeably less than the number one spot. The only domestic model in the top ten is Kimi K3 at rank 8.

OpenArt Creative Blind Test Leaderboard (Live Today)

Group	Champion	Runner-up
Video Overall	Seedance 2.5 (ByteDance 1125)	Wan 3.0 (Alibaba 1047)
Video · Ads	Seedance 2.5 (1072)	Wan 3.0 (1043)
Image Overall	Seedream 5.0 Pro (ByteDance 1051)	GPT Image 2 (OpenAI 1047)
Image · Graphic Design	GPT Image 2 (1051)	Grok Imagine 2.0 (1000)

ByteDance sweeps the video group, while the image group is tight, with GPT Image 2 flipping back ahead in graphic design. New boards fluctuate; don't rush to buy memberships based on current ranks.

III. Section Picks

1. OpenAI Reveals Full Workflow for Using AI to Design Its Own Chips. Multiple stages including design, verification, and debugging are handed to their large models, with engineers shifting to defining rules and acceptance checks, providing specific numbers on labor and time savings. Publishing workflow details shows more sincerity than just releasing benchmark scores. One example per company; wait to see if others reuse it before judging quality. (IEEE Spectrum, 9/16)

2. Programmer Claims AI Solved a 12-Year-Old Math Problem. Posted on Show HN, the proof has been formalized (translated into a language machines can check step-by-step) and is awaiting peer review. "Formalized + Pending Review" is more honest than "AI did it again." Only machine-verifiable proofs count. (Hacker News, 9/16)

3. Can AI Chatbots Replace Human Companionship? A Scientific American podcast roundtable discusses whether human companionship via AI leads to more loneliness. Opinion pieces lack hard data; listen as a roundtable discussion, but don't take AI comfort as diagnosis. (Scientific American, 9/16)

4. PostHog Engineer Self-Report: What Do Engineers Do After AI Writes All the Code? Over the past four months, their code was basically produced by AI. Human work shifted to defining plans, reviewing changes, and handling accidents. Frontline samples are worth more than vendor marketing. As AI output rises, the verification gate must keep up. (PostHog Newsletter, 9/16)

5. Scalpel: Let Coding Agents See Only the Function They Need to Change. An open-source MCP tool provides get_symbol() to query definitions on demand, avoiding stuffing entire files into the AI context. Saving cost and context is the right direction; try new tools on small projects for a week first. (GitHub, 9/16)

6. CTRLRun: Put Every Agent Action Through a Rule Gate First. Checks against your rules before execution, blocking anything out of bounds. Another sample of the "environment decides" approach; the rule engine itself needs security checks too. (GitHub, 9/16)

7. ZuckOff: Meta Glasses Haven't Seen You, But It Sees You First. A free app by Polish developers uses Bluetooth to detect nearby smart glasses, alerting you before the camera points at you. Rare reverse-thinking; wait for real feedback on false positive rates before recommending installation. (Wired, 9/16)

8. Voice AI Doesn't Touch GPU, Runs on Regular CPU. Show HN project Lokutor claims voice agents run purely on CPU, backed by supercomputing center endorsements. Lowering barriers is good, but latency and interruption response aren't tested yet; wait for trials in noisy environments. (Lokutor, 9/15)

9. Zhiwei Tests GPT-Image 2.5: Consistency So Good It Can Make GIFs? Yes, But Unstable. Following instructions without random changes and continuous generation without drift are selling points, but batch GIFs still have glitchy frames. Admitting "instability" is more credible than official demos. Wait for another round of user tests for bulk generation. (Huxiu, 9/15)

10. GPT-6 Released Two Weeks Ago, Reputation Rollercoaster. Some bloggers say it achieved things never thought possible for models, while also doing the dumbest things seen; users complain it gets dumber with use, while Silicon Valley urges it to spare humanity. Two weeks post-release is exactly when the marketing filter breaks. (Huxiu, 9/15)

IV. Everyone's Watching

Meta One subscription launched, top tier $499/month; Honor AgenticOS debuted, Magic9 opens for trial in October; Chinese internet base corpus 4.0 released, 120GB open-source corpus; Meituan launched "Craftsman Agent"; Musk boasts Grok 4.9 will rival Claude flagship—wait for leaderboards to show true colors.

V. Tomorrow's Focus

① Arena snapshot hasn't moved in two days; watch if the thinnest-sample entries muse-spark-1.2 and claude-fable-5.1-max shift ranks upon refresh this morning.

② OpenArt board just opened; see if the top three in video swap hands and how long ByteDance's sweep lasts.

③ GPT-Image 2.5 is "capable but unstable"; dig into user feedback on bulk generation to gauge failure rates.

④ That 12-year-old math problem; track peer review progress.

⑤ Agent lying experiment; find the paper original to see if rules were missing or bypassed.

Full leaderboard data and methodology details are on the daily review page.