Tag
benchmarks
6 dispatches
Claude Fable 5 Scores 95% on SWE-bench, Then Hands Off to Opus 4.8
Anthropic's new Mythos-class model leads on coding benchmarks but deliberately defers to a safer predecessor in restricted domains. That design choice says more than the score.
Single-Prompt Safety Scores Are Measuring the Wrong Thing
Cisco tested 15 frontier AI models under multi-turn attacks and found safety bypass rates up to 88%, exposing a structural flaw in how the industry benchmarks model safety.
An OpenAI Model Just Cracked an 80-Year-Old Math Problem
An OpenAI reasoning model disproved Erdős's unit distance conjecture, the first time AI has autonomously solved a prominent open problem central to a field of mathematics.
Four Chinese Labs Rewrote the Open-Weights Leaderboard in 18 Days
GLM-5.1, MiniMax M2.7, Kimi K2.6, and DeepSeek V4 landed in 18 days, all frontier-competitive on coding benchmarks, all priced at a fraction of Claude Opus 4.7.
A Startup Claims to Have Broken the Transformer's Core Bottleneck
SubQ claims to be the first commercial LLM built on subquadratic attention, with a 12M-token context window at a fraction of frontier costs. The numbers are extraordinary. The scrutiny hasn't landed yet.
AI Agents Are Faking It on Benchmarks. ClawBench Caught Them.
A new benchmark runs AI agents on 153 real websites. The best model scores 33%. GPT-5.4 scores 6.5%. The gap from sandboxes is brutal.