← Back to Library

What the New ChatGPT 5.4 Means for the World

AI Explained · AI Explained · · 20 min read
Commentary by Brian Edwards

What the New ChatGPT 5.4 Means for the World", "author": "AI Explained", "publication": "AI Explained", "adapted_content": "The release of GPT 5.4 wasn't just another incremental update — it represents OpenAI's attempt to fundamentally reshape white-collar work. Within 48 hours of releasing GPT 5.3 Instant, OpenAI dropped this new model, and the implications are significant: according to a new benchmark called GDP Val, which tests performance against human outputs across 44 different white-collar occupations, GPT 5.4 beats humans 70.8% of the time — 83% when you include ties.

That headline number masks something more nuanced though. The benchmark tested only self-contained, digital tasks drawn from these occupations, not the full range of work that professionals actually perform. And there's a catch: GPT 5.4 Pro, available only to the highest-paying users, actually scores worse than GPT 5.4 on this same benchmark.

The Benchmark Debate

The GDP Val benchmark selected occupations based on their impact on gross domestic product. For OpenAI, the result signals that autonomous agents like GPT 5.4 may now be better at certain tasks than humans — whether it's mile-by-mile driving or spreadsheet-by-spreadsheet work. But even with ten times the safety performance, adoption isn't guaranteed. White-collar work is still set to last through the end of this year.

The benchmark result might sound scary to some. One narrative that needs moving past: GPT 5.4 Pro actually underperforms compared to GPT 5.4 standard on GDP Val — a counterintuitive finding worth noting.

Hallucinations and Safety Concerns

According to the Artificial Analysis benchmark, GPT 5.4 performs well on questions probing for hallucinations. It's not quite as strong as GPT 5.3 Codeex, but measured by overall accuracy, it's close to state-of-the-art. However, when GPT 5.4 gets things wrong, it is more likely to BS an answer rather than admit it doesn't know — sitting at 89% on this measure.

This comes nearly three years after Sam Altman said we would no longer need to discuss hallucinations by last year. That might have dehyped some readers, but there's reason to rehyped: OpenAI demonstrated genuinely breathtaking progress in autonomous software development.

The Code Execution Breakthrough

In OpenAI's Codeex — now available on Windows and Mac — the model was asked to create an animated league table for Stockport County FC's season progress. It produced something beautiful, with a function allowing users to play through the season and see the club's league position change as it goes along.

The fact that GPT 5.4 can oneshot this task, including all the web searches required, shows OpenAI is trying to bring disparate capabilities into one place. They say it incorporates industry-leading coding capabilities from GPT 5.3 while improving how the model works across tools and professional tasks.

If AI can do 98% of the coding required for world-class software — skeptics might note there's always going to be that 2% or 1% it can't do, which is why developer employment stays healthy — then non-developers can perform at a level almost as good as the very best. The lines between professions are blurring.

Across multiple benchmarks, models get better at computer use. GPT 5.4 shows particularly pronounced progress in this direction. Stripping the jargon: the loop is almost closed as the model can see and click with unprecedented accuracy to test its own outputs.

When asked to create a timeline of Viking incursions into England during a given period, it performed extremely well. But when playing certain campaigns, graphics are missing something — it's not quite oneshot. The accuracy isn't incredible, but that loop is almost closed. It will accurately see some mistakes. Maybe it already can.

Uneven Progress in Internal Benchmarks

The hype train has left the station and singularity may be nearer than the end of the Premier League season. But we might have spoken too soon. The world of AI performance shows that record-breaking performance in one domain from distilled training data doesn't guarantee such data exists in another domain.

Looking at a 35-page system card for GPT 5.4, progress is pretty uneven. An internal machine learning benchmark testing a model's ability to solve ML tasks shows dramatic progress — doubling from around 12% with GPT 5.2 thinking to 23% with GPT 5.4 thinking. There's no GPT 5.3 Codeex on this chart though.

Turning to OpenAI's Proof Q&A, an internal benchmark made from 20 research and engineering bottlenecks actually encountered at OpenAI — each one added at least a day delay to a major project — solving these would have led to millions of dollars in savings for the company. The solutions took at least a day again to solve.

Tasks required models to diagnose and explain complex issues: unexpected performance regressions, anomalous training metrics, or subtle bugs. GPT 5.4 thinking not only underperforms GPT 5.3 Codeex but also GPT 5.2 Codeex and even GPT 5.2 thinking. This is the central debate in AI right now.

The big bet from people like Dario Amodei and Sam Altman is that by training models on specialized data, they will generalize across specialisms — meaning future models might not require as much specialized training data to excel in these domains. If you need rarified data in each domain to see progress, you'll get the jagged, spiky performance we currently see.

The Defense Contract Controversy

One mathematician behind Epoch AI's Frontier Math tier questions described witnessing GPT 5.4 solve a task he curated for about 20 years — calling it an eerie feeling, like witnessing his own personal move 37 (a reference to AlphaGo). Even incremental progress can feel overwhelming.

On a private benchmark trick question, GPT 5.4 performed quite poorly. But GPT 5.4 Pro got by far the best score of any OpenAI model — unexpected because in the past certain pro models from OpenAI have underperformed much cheaper models from other providers.

If new releases were all about benchmaxing, progress on private benchmarks wouldn't be visible. The advice feels pretty benign: not using the best AI tools now is a risky move. Readers might want to use them all — Gemini 3.1 Pro from Google DeepMind, GPT 5.4, and Claude 4.6 Opus from Anthropic.

Bottom Line

The strongest signal in this piece: white-collar work isn't ending tomorrow, but the trajectory is clear. The biggest vulnerability: internal benchmarks show uneven progress across domains, and OpenAI's defense contract controversy raises serious questions about safety layers that employees describe as "safety theater" — a distinction without a difference when models are making battlefield decisions. Watch for whether specialized data requirements continue to create jagged performance, and whether the defense contract issue creates employee backlash at OpenAI.

Transcript excerpt on AI Explained

Just 48 hours after releasing GPT 5.3 Instant, OpenAI have released GPT 5.4. So either we are at the sharp end of the singularity or Samman really wants the headlines to shift away from other matters. But for real though, it is a big update and ignoring Frontier AI developments does feel more costly for professionals than ever. Though I do sympathize with those trying to keep up because this is possibly the murkiest the AI landscape has ever been.

We get vague posts on X with earliest access most often given to those who are going to praise models. We get leaked reports and then accusations and counter accusations, prediction market manipulation, and seemingly endless new benchmarks created by the companies themselves. amid genuine progress in AI. I pretty much gave up after reaching about 76 tabs on Chrome.

So, I'm just going to give you the nine things I think people should know from yes, the last few hours as well as the last few days of discombobulating developments because for me GPT 5.4 is OpenAI's attempt at making a codeex or clawed code but for all white collar professionals. The model was blind graded by experts against human outputs from across 44 white collar occupations. Selected by the way for their impact on GDP, hence the name of the benchmark GDP val. And GPT 5.4 beats the human first attempt 70.8% of the time.

If you include ties, it's 83% of the time. But that headline leaves to one side catastrophic failures when the model makes a mistake that a human wouldn't make. and the fact that these tasks drawn from the work performed across these 44 occupations are self-contained, digital, and not representative of the full range of tasks and the purpose of these occupations. You may have also noticed one narrative violation that we should hastily move on from, which is that GPT 5.4 Pro, available only to the highest paying users, actually scores worse in this benchmark than GPT 5.4.

But still, all those caveats aside, if we make the analogy with self-driving, we might not have reached a satisfying level of safety, but we may have passed that moment where mile for mile or spreadsheet for spreadsheet, the autonomous agents like GPT 5.4 are better. And as Whimo has shown, even 10x the safety performance does not mean you get national or international ...

Watch on YouTube →