Skip to main content
Guillermo Barreto
← All posts

The Best AI in the World Scores 12%

6 min readBy Guillermo Barreto
The Best AI in the World Scores 12%

I read a number this morning that made me laugh out loud.

Twelve percent.

That's the score of the best AI in the world. Not some startup demo. The best one. Claude Opus 5.5, the model everyone in my feed talks about like it's a coworker. Twelve percent.

I'm a WGU student. I know what 12% looks like on a test. It's the kind of score where the teacher doesn't even write a comment. They just hand it back and look at you.

The test that actually matters

A research firm called SemiAnalysis built a benchmark called GDPx, and it launched on November 2. The idea is simple. Stop asking models trivia questions. Give them actual professional work and see if they can do it.

206 tasks. Eleven fields: chip design, cybersecurity, biology, finance, legal, quant trading, hardware design, materials science, software engineering, and a couple more. Each model got a workspace, eight hours per task, and three tries.

Opus 5.5 finished first at 12.0%. Google's Gemini 4 Carbon came second at 10.9%. OpenAI's GPT-6 Astra third at 9.6%. The best open-weight model, GLM 5.3, scored 2.7%.

And then there's the number that stopped me cold. Every single model scored zero on finance and legal tasks. Zero. Not low. Zero.

Oh, and it costs money to fail. Opus 5.5 runs about $51.41 per task on this benchmark. So you're paying fifty bucks a pop for a 12% success rate. Gemini is cheaper, about $16 a task, and only a point behind. That price gap is the most interesting part of the whole leaderboard to me, but we'll get to that.

The fine print, because it matters

I should be honest about what this test is and isn't. It's one firm's benchmark. Most of the tasks are private, so nobody outside SemiAnalysis can fully reproduce the results yet. They publish the trajectories, which helps, but day one, these are their numbers.

And the safety systems skew things. SemiAnalysis says so themselves. Opus and Astra lose points because their safety classifiers refuse or cut short work in cyber, biology, and alignment tasks. Strip those three domains out and the ranking flips: Astra goes first at 11.8%, Opus second at 11.7%, Gemini drops to 9.2%.

So the honest version is messier than the headline. The best models are somewhere around 10 to 12 percent on real professional work, with the exact order depending on how you count the stuff they refuse to touch.

That doesn't make the headline wrong. It makes it worse, in a way. Because the models aren't failing because they're dumb. They're failing because real work is hard, and because some of it they won't even try.

The other shoe

The same week, Microsoft walked away from a data center deal worth tens of billions of dollars.

Here's what happened. Microsoft had an option to more than double its existing agreement with Nscale, a UK AI cloud company, renting computing power at a Texas data center. The option covered a second phase, about 700 megawatts, starting in late 2027. The Financial Times reported on November 5 that Microsoft decided not to exercise it. Reuters confirmed it with a source familiar with the matter.

The existing deal stays. That's about 104,000 Nvidia GB300 GPUs at a roughly 240-megawatt campus in Texas. Microsoft isn't leaving. It's just not buying more. And the decision was made months ago, before Nscale filed for its IPO in September. Nscale says its $103 billion contracted backlog isn't affected.

But this is the second time in a few weeks Microsoft has pulled back from Nscale. In October it walked away from anchoring Nscale's flagship campus in Loughton, England, because the power connection wouldn't arrive until the 2030s. I wrote about that one. The grid said no.

Now the Texas option says no too, for a different reason. Not power this time. Just... no. Microsoft looked at tens of billions in future capacity and passed.

What I actually think

I keep coming back to the same thing. Both stories are about the same gap, just from opposite ends.

On one end, the models can't do the work yet. Twelve percent. Zero on finance and legal. That's not a replacement. That's an intern who needs a lot of supervision and still gets most of it wrong.

On the other end, the biggest buyer in the world is tapping the brakes on buying more of the buildings that run the models. Not stopping. Not panicking. Just... not doubling down.

And I'm sitting here as a student in IT support, writing code at night, doing CodePath interview prep, applying to IT roles, and I keep thinking: this is the best news I've read in a while.

Not because I want AI to fail. I use these tools every day. They help me. But there's a difference between a tool that helps you do your job and a tool that does your job. The benchmark says we're nowhere near the second one. And the data center news says even the people spending the most money on this stuff aren't sure how fast to build anymore.

That means the work that matters is still the boring work. Reading the model's output and knowing it's wrong. Checking the ticket before you close it. Knowing which wire goes where. The stuff that doesn't show up on a benchmark because it's not a task you can grade in eight hours. It's judgment, built up slowly, one ticket at a time.

I've resolved 2,579 tickets at Amazon. None of them looked like a benchmark task. They looked like a person with a broken thing and a deadline. The AI can draft the email. It can't walk into the room, look at the access point, and figure out why the whole floor is offline. Not at 12%, anyway.

So yeah. Twelve percent. I'll take it. It means I still have a job to learn toward.

Cover photo: server racks in a data center, by Tony Webster (CC BY).

Sources: GDPx leaderboard (SemiAnalysis); GDPx coverage (BYOBot AI Daily Newsstand, Nov 3); Microsoft will not expand Texas data center deal with Nscale (Reuters, Nov 5); Microsoft walks away from option to double its Texas deal with Nscale (Dealroom, Nov 5).