OpenAI o3 Explained — The Reasoning Model

Photo: Sanket Mishra / Pexels
On December 20, 2024, OpenAI streamed a preview of a model called o3, and the number it led with was 87.5% on ARC-AGI, a puzzle test built to be easy for people and hard for language models. This article explains what o3 is, how a "reasoning model" differs from a chat model, what each benchmark actually measures, what the December results cost to produce, and where the claims need caveats.
What OpenAI announced in December 2024
The December 20 event was a preview. OpenAI did not release o3 that day; safety researchers could apply for access to o3-mini, and Sam Altman said o3-mini would launch near the end of January with the full model to follow, according to TechCrunch's coverage of the announcement. The name skips "o2." TechCrunch reports, citing The Information, that OpenAI avoided a possible trademark conflict with the British telecom provider O2, and that Altman "somewhat confirmed" this during the stream.
The numbers shown were OpenAI's own internal evaluations. They are worth reading with that label attached, because several of them were later qualified by the groups that built the benchmarks.
| Benchmark | What it tests | Result reported in December 2024 |
|---|---|---|
| ARC-AGI (semi-private set) | Inferring a rule from a few visual examples | 75.7% at low compute, 87.5% at high compute |
| AIME 2024 | High-school competition mathematics | 96.7%, missing one question |
| GPQA Diamond | Graduate-level biology, physics, and chemistry | 87.7% |
| Codeforces | Competitive programming rating | 2727 |
| SWE-bench Verified | Fixing real software issues | 22.8 percentage points above o1 |
| FrontierMath | Unpublished research-style math problems | 25.2%; no earlier model above 2% |
Sources: TechCrunch for AIME, GPQA, Codeforces, FrontierMath, and the SWE-bench margin; ARC Prize for the ARC-AGI figures. TechCrunch gives the SWE-bench result only as a margin over o1.
A note on the table: the model tested in December was a preview, and ARC Prize said it looked forward to testing the production version once API access was available. The December scores describe a system, a set of settings, and a compute budget, not the product people can buy.
The cost behind the headline: the 87.5% ARC-AGI result cost about $456,000 to run across 100 tasks, which is $4,560 per task. A human paid to solve ARC tasks costs roughly $5 per task, according to ARC Prize.
What "reasoning model" actually means
A standard chat model reads a prompt and starts writing the answer. Every token it produces is part of the response you see. A reasoning model is trained to write a long stretch of intermediate working first, and only then commit to a final answer.
Think of two students sitting an exam. One must write the final answer directly on the answer sheet, with no scratch paper. The other is allowed to fill pages of scratch work, cross things out, try a second method, and check the result before copying down the answer. Both students may know the same material, but the second one solves harder problems because working through steps is itself a tool.
OpenAI's system card for o3 and o4-mini describes the approach in plain terms. The o-series models "are trained with large-scale reinforcement learning on chains of thought," and "through training, these models learn to refine their thinking process, try different strategies, and recognize their mistakes." The card adds that they can produce "a long internal chain of thought before responding to the user."
Reinforcement learning (RL) is a training method where a model is rewarded for good outcomes rather than shown the right answer word by word. For a math problem, the reward can be as simple as "did the final answer match?" The model gradually learns which patterns of working lead to rewards. For background on how this differs from supervised training, see our explainers on how AI models learn from feedback and on RLHF, RLAIF, and DPO.
Where o3 sits in the o-series
The line began with o1-preview on September 12, 2024. OpenAI described a large-scale RL algorithm that teaches the model to use its chain of thought productively, and it decided "not to show the raw chains of thought to users," citing user experience, competitive advantage, and the option to monitor the reasoning, as Simon Willison's notes on the launch record. The reasoning appears in the API as "reasoning tokens" that are invisible but still billed as output tokens.
That billing detail matters for cost. A reasoning model can spend thousands of tokens thinking before it writes a ten-line answer, and the customer pays for all of them.
| Model | Date | Notable point |
|---|---|---|
| o1-preview | September 12, 2024 | First public reasoning model; raw reasoning hidden |
| o3 (preview results) | December 20, 2024 | ARC-AGI 87.5% at high compute; not released |
| o3-mini | January 31, 2025 | Low, medium, and high reasoning effort settings |
| o3 and o4-mini | April 16, 2025 | Reasoning combined with tools such as web browsing and Python |
| o3 price cut | June 10, 2025 | o3 pricing cut 80%; Altman's announcement also referred to o3-pro |
Sources: Willison (o1), TechCrunch (December preview and o3-mini), OpenAI system card (April 16), VentureBeat (June 10).
Why extra thinking time can beat a bigger model
The idea underneath o3 is called test-time compute, meaning computing power spent when the model is answering rather than when it was trained. For years, the main route to better models was making them bigger and training them on more data (the pattern covered in our piece on neural scaling laws). Test-time compute opens a second route.
A 2024 paper by Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar, Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, put numbers on the trade. In a comparison that held total computation fixed, a smaller model given extra thinking time on a problem could "outperform a 14x larger model." The authors also found that allocating the thinking budget adaptively, spending more on hard prompts and less on easy ones, was more than four times as efficient as a plain best-of-N baseline.
Our article on test-time compute covers the mechanics. For this article, the key point is that thinking time behaves like a dial. Turn it up and accuracy rises on problems that have checkable answers, at the price of delay and money.
A worked example of why more attempts help
Suppose a model solves a certain puzzle correctly 10% of the time on any single attempt, and suppose you could reliably recognize a correct answer when you saw one. The chance that at least one of N independent attempts succeeds is 1 minus 0.9 raised to the power N.
- With 1 attempt: 10%.
- With 6 attempts: about 47%.
- With 1,024 attempts: effectively 100%.
Real attempts are not fully independent, and the hard part is knowing which attempt to trust. But the arithmetic shows why the ARC Prize table lists a very different number of samples for the two o3 settings: 6 samples for the efficient configuration and 1,024 for the high-compute one. The point of the arithmetic is the direction, not the exact figures. More attempts, and more thinking per attempt, convert spare computing power into accuracy.
What ARC-AGI measures, and what 87.5% means
ARC stands for the Abstraction and Reasoning Corpus. François Chollet introduced it in his 2019 paper On the Measure of Intelligence, where he proposed it as a way to measure "a human-like form of general fluid intelligence." Each ARC task shows a few small examples of a transformation, typically on a grid of colored cells, and asks the solver to apply the same transformation to a new grid. The puzzles rely only on basic ideas such as objects, counting, and symmetry, so there is little to memorize.
That design is why the test mattered. Chollet's argument was that skill on a familiar task does not show intelligence, because skill can come from enormous practice. What counts is how efficiently a system acquires a new skill. ARC was built to punish memorization and reward quick adaptation.
The December results, with costs
ARC Prize published the full run. The semi-private set (100 tasks) is held out from public view, and the public set has 400 tasks.
| Set | Setting | Score | Samples per task | Cost per task | Tokens (total) |
|---|---|---|---|---|---|
| Semi-private (100 tasks) | High efficiency (low compute) | 75.7% | 6 | $26 | 33.5 million |
| Semi-private (100 tasks) | Low efficiency (high compute) | 87.5% | 1,024 | $4,560 | 5.7 billion |
| Public (400 tasks) | High efficiency | 82.8% | 6 | $167 | 111 million |
| Public (400 tasks) | Low efficiency | 91.5% | 1,024 | $1,900 | 9.5 billion |
The labels can confuse. ARC Prize calls the cheaper run "high efficiency" and the expensive run "low efficiency," and the high-compute setting drew 1,024 samples per task against 6 for the low-compute one. Time per task on the semi-private set was 1.3 minutes at low compute and 13.8 minutes at high compute.
The table also implies a unit price: $456,000 divided by 5.7 billion tokens works out to about $80 per million tokens, and the cheaper run's $2,680 over 33.5 million tokens gives the same rate. The high-compute run therefore used about 57 million tokens per task, which is why the bill reached $4,560 per task. Against the roughly $5 ARC Prize cites for paying a human, that is about 900 times the cost.
Chollet's caveats
The creator of the benchmark was careful about what the result did and did not show. ARC Prize reports several qualifications from Chollet:
- "Passing ARC-AGI does not equate to achieving AGI, and, as a matter of fact, I don't think o3 is AGI yet."
- OpenAI trained the tested o3 on 75% of the ARC public training set, and ARC Prize had not yet tested a version that was not trained on ARC-style data, so how much of the score comes from that training was unknown.
For the wider debate about what "AGI" means and how it gets claimed, see our explainer on artificial general intelligence.
ARC-AGI-2 reset the scoreboard
ARC Prize announced ARC-AGI-2 on March 24, 2025, and it was designed to be harder for AI while staying easy for people. On that test, ARC Prize estimated that o3-preview at low compute would score around 4%, and said o3 at high compute might reach 15 to 20% at a cost of thousands of dollars per task. The same page lists o1-pro at about 1%, and o3-mini-high and GPT-4.5 at 0.0%.
Human performance on ARC-AGI-2 was measured with panels of testers. ARC Prize reports that the average human panel scored 60% and that at least two humans in the panel solved every task, at about $17 per task. The gap between the 87.5% headline and a single-digit score on the follow-up test shows how much of the first result depended on the first test's design and on the volume of attempts.
The other benchmarks, and what each one means
A single score tells you little without knowing the test. Here is what the remaining December figures were measuring. Our guide to AI benchmarks and the companion piece on why benchmarks can mislead cover the general problems.
GPQA Diamond
GPQA is a set of 448 multiple-choice questions in biology, physics, and chemistry, written by domain experts, and described in the original paper as "Google-proof." In that paper, experts with or pursuing PhDs in the relevant field scored 65%, skilled non-experts with more than 30 minutes and unrestricted web access scored 34%, and the strongest GPT-4-based baseline scored 39%. OpenAI's reported 87.7% for o3 is therefore well above the expert average in the paper, though the paper's expert figure rose to 74% once the experts' own identified mistakes in the questions were excluded.
Reading that comparison correctly matters. A multiple-choice test answered by a model in seconds is not the same as an expert doing research. The score says the model handles this question format at a high level. It does not say the model can replace a chemist.
FrontierMath
FrontierMath is a benchmark of original, very difficult math problems written and checked by expert mathematicians, using unpublished problems and automated verification to limit contamination. The paper, first submitted November 7, 2024, reported that the leading models of the time solved under 2% of the problems. OpenAI's reported 25.2% for o3 was therefore a very large jump, which is why it drew attention.
It also drew scrutiny. As TechCrunch reported, Epoch AI, the nonprofit behind the benchmark, disclosed on December 20 that OpenAI had funded its creation and had access to many of the problems and solutions. Epoch co-founder Tamay Besiroglu said OpenAI had a verbal agreement not to train on the problem set and that Epoch "made a mistake" by not being more transparent. Epoch's lead mathematician, Elliot Glazer, said Epoch had not independently verified the o3 result and that "we can't vouch for them until our independent evaluation is complete."
None of this shows the score was wrong. It does show why a number reported by a model's maker, on a benchmark that maker helped fund, deserves a caveat until an independent evaluation exists.
SWE-bench and Codeforces
SWE-bench, introduced in a 2023 paper, contains 2,294 software engineering problems drawn from 12 popular Python repositories. The best model in the original paper, Claude 2, resolved 1.96% of the issues. The Verified version used in December is a curated subset, and OpenAI's reported gain was 22.8 percentage points over o1.
Codeforces is a competitive programming site that assigns ratings to participants, and OpenAI reported a rating of 2727 for o3. A rating compares performance on timed contest problems that have clear right answers. It says little about the messier work of maintaining a large codebase, which is what SWE-bench tries to approximate.
The released model: April 16, 2025
OpenAI released o3 and o4-mini together on April 16, 2025. The system card for the pair says they "combine state-of-the-art reasoning with full tool capabilities," listing web browsing, Python, image and file analysis, image generation, canvas, automations, file search, and memory. The models "use tools in their chains of thought to augment their capabilities," for example by cropping or transforming images, searching the web, or running Python while they think.
This is a real change in what a reasoning model does. The reasoning loop can call a search engine, a Python interpreter, or an image tool partway through and use the result before answering. That makes it closer to the AI agents described elsewhere on this site, where a model decides what step to take next.
OpenAI's Safety Advisory Group reviewed the models under version 2 of its Preparedness Framework and, according to the card, determined that neither reached the "High" threshold in the three tracked categories: biological and chemical capability, cybersecurity, and AI self-improvement.
The card also notes that o3 and o4-mini perform "approximately on par" with o1 on its disallowed-content evaluations. In the standard refusal test, o3 scored 0.84 on the "not overrefuse" aggregate, against 0.86 for o1.
What o3 costs
Reasoning models change the unit economics of AI. A chat model's cost scales with the length of the answer. A reasoning model's cost scales with the length of the answer plus all the hidden thinking.
| Date | Model or event | Price per million tokens (input / output) |
|---|---|---|
| January 31, 2025 | o3-mini launch | $0.55 cached input; $4.40 output |
| April 16, 2025 | o3 at launch | $10 / $40 |
| June 10, 2025 | o3 after 80% cut | $2 / $8 |
| June 10, 2025 | o3 flex processing | $5 / $20 |
Sources: TechCrunch on o3-mini; VentureBeat on the o3 price cut. The launch price of $10 and $40 is the "old" price in VentureBeat's report of the cut.
VentureBeat, reporting on June 10, 2025, quoted Altman: "we dropped the price of o3 by 80%!!"
Per-token price is only part of the bill. Two models can have the same price per token and very different bills, because one thinks twice as long. When comparing reasoning models, compare the cost of finishing a task, not the price list.
OpenAI also offers o3-pro as a higher tier than o3. This article does not quote its per-token price, because the sources above do not state it.
The effort dial
o3-mini introduced a setting that made the test-time compute idea concrete. Developers could pick low, medium, or high reasoning effort, and in ChatGPT the default was medium. TechCrunch's launch coverage found the differences were modest on some tests: at high effort, o3-mini beat o1 on AIME 2024 by 0.3 percentage points, and on GPQA Diamond it did not surpass o1 even at high effort. Low effort performed comparably to o1-mini, and medium effort comparably to o1 on math, coding, and science.
This is the practical meaning of thinking time as a dial. Higher settings help, with diminishing returns, and the best setting depends on whether a wrong answer or a slow, expensive answer is the bigger problem.
The hallucination problem
More reasoning did not mean fewer invented facts. OpenAI's technical report on o3 and o4-mini included a benchmark called PersonQA, which asks questions about people. As TechCrunch reported, o3 hallucinated on 33% of PersonQA questions, about double the rate of o1 (16%) and o3-mini (14.8%), while o4-mini reached 48%. The report said "more research is needed" to explain the rise, and OpenAI did not have a clear cause.
This is a reminder that "reasoning" in this context means a trained habit of working through steps, not a guarantee of truth. A model can reason fluently toward a false claim about a person because the failure sits in what it remembers, not in how it argues. Our article on why AI hallucinates facts explains the underlying mechanism.
Where o3 fits among reasoning models
o3 was not the only reasoning model, and the field moved quickly after it. Rather than a table of benchmark scores that go stale within months, here is how to think about the alternatives, with links to our own explainers.
| Family | Maker | What to know | Further reading |
|---|---|---|---|
| o-series (o3, o3-pro, o4-mini) | OpenAI | Raw reasoning not shown to users | This article |
| DeepSeek R1 | DeepSeek | Reasoning model | DeepSeek R1 explained |
| Gemini 2.5 Pro | Reasoning-capable Gemini model | Gemini 2.5 Pro explained | |
| Claude 4 family | Anthropic | Claude models with extended thinking | Claude 4 explained |
For anyone who wants to run open models on their own hardware instead of paying per token, our guide to running LLMs locally with Ollama covers the practical side. For how the newest OpenAI line relates to these, see What Is GPT-5?.
Benchmark tables comparing these families go out of date within weeks, and different labs measure under different settings: effort level, number of attempts, tool access. When a chart shows one model beating another by two points, ask what settings each used. A score is a property of the model plus the test plus the compute budget.
When a reasoning model is the right tool
A reasoning model earns its cost where three things are true: the problem has multiple steps, a wrong intermediate step ruins the answer, and there is some way to check the result. Mathematics, formal logic, and debugging fit well, because a proof or a failing test gives a clear signal.
It earns little on tasks where the answer is a matter of tone or taste. Rewording an email or summarizing a meeting does not improve with ten minutes of hidden deliberation, and the delay and cost add up. Our explainer on model routing describes how products solve this by sending easy requests to fast models and hard ones to reasoning models.
None of this is advice for any particular purchase or workflow. It describes how the technology behaves as documented in the sources above.
What o3 does not mean
It does not mean AGI. Chollet said so directly. A system that tops one benchmark by spending $4,560 per task is a strong result in a particular category, not general intelligence.
It does not mean the benchmark scores transfer cleanly to the released product. The December numbers came from a preview, with high-compute settings and a training set that included 75% of ARC's public training data.
It does not mean reasoning models are more truthful. The PersonQA results point the other way on at least one measure.
And it does not mean the trend will continue at the same rate. The jump from under 2% to 25.2% on FrontierMath was reported by the lab that funded the benchmark, and an independent check was still pending at the time.
Frequently asked questions
Q: What is the difference between o3 and GPT-4o? A: GPT-4o is a chat model that writes its answer directly. o3 is trained with reinforcement learning on chains of thought, so it works through a long internal sequence of steps before answering. That helps on math, coding, and multi-step science questions and costs more time and money per answer.
Q: Did o3 achieve AGI? A: No. François Chollet, who created ARC-AGI, said "I don't think o3 is AGI yet." He added that passing ARC-AGI does not equate to achieving artificial general intelligence.
Q: How much did the 87.5% ARC-AGI result cost? A: On the 100-task semi-private set, the high-compute run cost about $456,000, or $4,560 per task, and used about 5.7 billion tokens. The cheaper run scored 75.7% at about $26 per task. ARC Prize puts the cost of paying a human to solve a task at roughly $5.
Q: What does o3 cost now? A: At launch in April 2025, o3 cost $10 per million input tokens and $40 per million output tokens. On June 10, 2025, OpenAI cut both prices by 80%, to $2 and $8. Prices change often, so check OpenAI's current pricing page before budgeting.
Q: Can I see o3's chain of thought? A: OpenAI has said it decided not to show raw chains of thought to users, citing user experience, competitive advantage, and the option to monitor the reasoning for safety. The system card refers to a chain-of-thought summarizer, which is consistent with users seeing a summary rather than the raw text.
Q: Does o3 hallucinate less than older models? A: Not on the benchmark OpenAI reported. On PersonQA, o3 hallucinated 33% of the time versus 16% for o1, and OpenAI said more research was needed to explain why.
Q: Why did OpenAI skip the name o2? A: Per The Information, as reported by TechCrunch, to avoid a possible trademark conflict with the British telecom provider O2. Altman somewhat confirmed this during the livestream.
The takeaway
o3 matters less for any single score than for what it showed about where improvement can come from. A model trained to reason, and given room to think, solved problems that a larger model answering in one pass could not. The price of that room is real: thousands of dollars per task at the extreme, and a bill that grows with every hidden token.
The most useful habit is to read each number with its conditions attached: who ran the test, on what settings, with how many attempts, and with whose money behind the benchmark. By that standard, o3 is a genuine advance on checkable, multi-step problems and a less clear one everywhere else.
Read next:
Sources
- ARC Prize, "OpenAI o3 Breakthrough High Score on ARC-AGI-Pub", December 2024
- ARC Prize, "Announcing ARC-AGI-2 and ARC Prize 2025", March 24, 2025
- OpenAI, o3 and o4-mini System Card, April 16, 2025
- TechCrunch, "OpenAI announces new o3 models", December 20, 2024
- TechCrunch, on FrontierMath funding disclosure, January 19, 2025
- TechCrunch, "OpenAI launches o3-mini", January 31, 2025
- TechCrunch, on o3 and o4-mini hallucination rates, April 18, 2025
- VentureBeat, "OpenAI announces 80% price drop for o3", June 10, 2025
- Simon Willison, "Notes on OpenAI's new o1 chain-of-thought models", September 12, 2024
- Snell et al., Scaling LLM Test-Time Compute Optimally, 2024
- Rein et al., GPQA: A Graduate-Level Google-Proof Q&A Benchmark, 2023
- Glazer et al., FrontierMath, 2024
- Jimenez et al., SWE-bench, 2023
- Chollet, On the Measure of Intelligence, 2019
Follow new articles
Add SciHub101 to your feed reader to see new science and technology stories.
About the Author
SciHub101 is an independent science and technology publication founded and edited by Buğra Sözeri, a software engineer and app developer based in Türkiye. AI tools assist with drafting and research. Source documentation in the existing archive is uneven, and articles are being reviewed and corrected. The Editorial Team byline identifies editorial responsibility; it does not certify specialist or independent human review. Where we compare products, anything built by our own team is labelled as such — see the editorial guidelines and About page for how corrections and disclosures work.
Related Articles

What Is Apple Intelligence? Apple's AI Stack
Apple Intelligence is Apple's on-device and Private Cloud Compute AI system. How it works, which devices run it, and what Siri AI changed in 2026.

What Is the Vercel AI SDK? LLM Toolkit
The Vercel AI SDK is a TypeScript toolkit for building LLM apps with React, Next.js, Svelte, and Vue. Unified API, streaming, tools, structured outputs.

How Large Language Models Actually Reason — Inside ChatGPT, Claude, and Gemini (2026)
Do LLMs actually reason or just predict text? A deep technical guide to transformer attention, chain-of-thought, test-time compute, and frontier model benchmarks in 2026.

Test-Time Compute Explained — LLM Reasoning
Test-time compute lets LLMs think longer at inference. Learn how o1, o3, DeepSeek-R1, and Claude reasoning models work and when to use them.

What Is AGI? Artificial General Intelligence Explained
AGI — artificial general intelligence — is the holy grail of AI: a machine that can do anything a human can. Here's what it really means, how close we are, and why definitions matter so much.

What If Artificial General Intelligence Arrived Tomorrow?
Not in five years, not after careful preparation — tomorrow. An AI that can do anything a human can, better. The world is not ready.