GPT-4: Large Language Models Explained

Photo: Matheus Bertelli / Pexels
In March 2023, OpenAI released GPT-4 and published a technical report claiming a simulated bar exam score near the top 10 percent of test takers. The same report declined to say how many parameters the model has, what hardware trained it, or how its data was assembled. That pairing of a strong result and a closed box is the right place to start, because most of what circulates about GPT-4 is either measured in that report or guessed. This article separates the two: what GPT-4 is, what it scored, why it works, where it breaks, and what replaced it.
Key Takeaways
- What it is: GPT-4 is a Transformer-based model pre-trained to predict the next token, then refined with human feedback. It accepts image and text input and produces text output.
- What is known: OpenAI published benchmark results, but states that its report contains no further details about architecture (including model size), hardware, training compute, dataset construction, or training method.
- What the scores mean: The headline exam results are real measurements, but they depend on which test-takers serve as the comparison group. A 2024 reanalysis of the bar exam result lowered the percentile substantially.
- What it cannot do reliably: GPT-4 still states false things with confidence, loses track of facts in long inputs, and fails some tasks that are trivial for people.
- Where it stands now: GPT-4 was retired from ChatGPT on April 30, 2025, and its successors handle most of the same jobs faster and cheaper.
What OpenAI actually disclosed about GPT-4
Start with the primary source. OpenAI's GPT-4 Technical Report describes a large multimodal model that accepts image and text inputs and produces text outputs. It is Transformer-based and pre-trained to predict the next token in a document, and a post-training alignment step then improves its factuality and adherence to desired behavior.
What the report leaves out is just as important. It states that, given the competitive landscape and the safety implications of large-scale models, it contains no further details about the architecture (including model size), hardware, training compute, dataset construction, or training method. Every specific figure you may have seen for GPT-4's parameter count, expert count, or training cost is therefore an outside estimate or a rumor, not something OpenAI published.
That matters because GPT-3 was handled differently. The GPT-3 paper reported 175 billion parameters, and its authors trained eight model sizes from 125 million to 175 billion parameters so readers could see how performance changed with size. With GPT-4, the scientific record shrank to benchmark outcomes and a few methodological notes.
Everything public about GPT-4 comes from benchmark results and OpenAI's own statements. The parameter count, the layout of any expert networks, and the training data have never been published by OpenAI.
Public claims about a mixture-of-experts design (a model that routes each token to a few specialist sub-networks, explained in our article on mixture-of-experts routing) have circulated for years. OpenAI has not confirmed them in its technical report, so this article treats the architecture as unknown.
From GPT-3 to GPT-4: what changed
To see what GPT-4 added, look at the steps before it. GPT-3, described in a 2020 paper by Brown and colleagues, was trained on roughly 300 billion tokens (a token is a word or word fragment) and showed that a single model could perform tasks from a few examples in its prompt. In the paper's arithmetic tests, the 175-billion-parameter model reached 100.0 percent accuracy on two-digit addition and about 80 percent on three-digit addition in the few-shot setting.
The next step was not a bigger model but a better-behaved one. In the 2022 InstructGPT paper, Ouyang and colleagues fine-tuned GPT-3 using human feedback and found that human raters preferred outputs from a 1.3-billion-parameter InstructGPT model to outputs from the 175-billion-parameter GPT-3, a model more than 100 times larger. The method, reinforcement learning from human feedback (RLHF), is the same family of technique that shaped ChatGPT, and RLHF, RLAIF, and DPO compared covers how it works in detail.
GPT-4 combined three ingredients: more pre-training, the alignment stage, and image input. The report's own comparison with GPT-3.5 shows where the gain was largest.
| Test | GPT-4 | GPT-3.5 |
|---|---|---|
| Uniform Bar Exam (simulated) | 298 of 400, about 90th percentile | 213 of 400, about 10th percentile |
| LSAT | 163, about 88th percentile | 149, about 40th percentile |
| SAT Evidence-Based Reading and Writing | 710 of 800, about 93rd percentile | 670 of 800, about 87th percentile |
| SAT Math | 700 of 800, about 89th percentile | 590 of 800, about 70th percentile |
| MMLU (57 subjects, 5-shot) | 86.4 percent | 70.0 percent |
| HumanEval (code, 0-shot) | 67.0 percent | 48.1 percent |
| LeetCode hard problems solved | 3 of 45 | 0 of 45 |
Source: OpenAI, GPT-4 Technical Report (2023).
Two details in that table stand out. First, the bar exam jump from about the 10th to about the 90th percentile is the largest single change, and it is the number most often repeated. Second, the LeetCode row shows how uneven the progress was: on easy problems GPT-4 solved 31 of 41, but on the hard ones it solved three of 45. A model can look expert on one slice of a task and be a beginner on the next.
How a next-token predictor ends up passing exams
The odd thing about GPT-4 is the gap between how it is built and what it does. At bottom it performs one operation: given a sequence of tokens, produce a probability for each possible next token, pick one, append it, and repeat. A good analogy is autocomplete on your phone, scaled up by many orders of magnitude and trained on enough text that completing a sentence often requires modeling the situation the sentence describes.
Consider the prompt "The defendant argued that the contract was void because." To continue it sensibly, a model needs something like a working picture of contract law, even if that picture is statistical rather than explicit. The more text a model has absorbed, and the more capacity it has to store regularities, the more of that picture it can hold. Passing a bar exam then becomes a case of completing text in the style of a competent legal answer.
Attention: how the model decides what matters
The mechanism that makes this workable is attention, introduced in the 2017 paper Attention Is All You Need. Earlier language models read text in order and had to squeeze everything they had seen into a fixed-size memory. The Transformer dropped that bottleneck: for every token, it computes how relevant every other token in the context is, and mixes their information accordingly. The authors reported that their model was more parallelizable and trained faster than earlier designs, with the English-to-French translation model training in 3.5 days on eight GPUs.
A useful analogy is a student working through an open-book exam. Rather than memorizing the book in order, the student keeps the whole book open and, for each question, flips to whichever pages matter. Attention is the flipping. Stacked in dozens of layers, it lets early layers resolve small things, such as which noun a pronoun refers to, and later layers combine those resolutions into larger structure. Our explainer on how transformers pay attention to words works through a full example, and how LLM tokenization works explains why the model sees word fragments rather than letters.
Two stages: pre-training, then alignment
Pre-training is the expensive stage. The model reads an enormous amount of text and code and adjusts its parameters to predict each next token better. At the end it is a powerful text completer but not a helpful assistant: ask it a question and it may continue with more questions, as a forum thread would.
Alignment turns the completer into an assistant. Human labelers rank candidate responses, a reward model learns those preferences, and the language model is further trained to produce responses the reward model scores highly. The InstructGPT result above is the clearest public evidence of how much this matters. The GPT-4 report says its post-training alignment improves factuality and adherence to desired behavior, but it does not describe the procedure.
Why OpenAI could predict GPT-4's performance in advance
One of the less-discussed claims in the technical report is that some aspects of GPT-4's performance were predicted from models trained with far less compute. That is only possible because of an empirical regularity first laid out in 2020 by Kaplan and colleagues in Scaling Laws for Neural Language Models: a language model's loss (its error at predicting the next token) falls as a smooth power law in model size, dataset size, and training compute, over more than seven orders of magnitude.
Think of it as tasting a sauce in a small pot to predict how a huge batch will taste. If the recipe scales cleanly, a cheap experiment tells you something about an expensive one. For a training run that costs many millions of dollars, being able to forecast the outcome from small runs is a major engineering advantage, regardless of whether you believe scaling is the whole story.
The scaling picture was refined in 2022. Hoffmann and colleagues trained a 70-billion-parameter model called Chinchilla on about four times more data than the 280-billion-parameter Gopher, using the same compute budget, and found it outperformed Gopher, GPT-3, and other larger models. Their conclusion was that model size and training tokens should grow together: for every doubling of model size, the number of training tokens should also double. Our explainer on scaling laws from Kaplan to Chinchilla covers the details.
A caution applies here. Smooth loss curves do not guarantee smooth abilities. Loss is an average over every token, while a skill such as multi-step arithmetic can look absent until a model is good enough to get every step right. That distinction sits at the center of the next debate.
Emergent abilities: real, or a measuring artifact?
In 2022, Wei and colleagues defined an ability as emergent if it is not present in smaller models but is present in larger ones, and they argued such abilities cannot be predicted by extrapolating from smaller models. The idea was attractive because it explained why GPT-4 could do things GPT-3 could not.
A 2023 paper by Schaeffer, Miranda, and Koyejo challenged it. They argued that apparent emergence often comes from the metric rather than the model: all-or-nothing scoring, such as exact-match accuracy, produces sudden-looking jumps, whereas continuous metrics applied to the same outputs show gradual, predictable improvement. They concluded that some reported emergent abilities may disappear under different metrics or better statistics.
Both findings can hold at once. A model may improve smoothly in its underlying competence while crossing a usable threshold on a particular task at a particular size. For a user, the threshold is what matters; for a scientist, the smooth curve is. When you read that GPT-4 "suddenly" understood jokes or code, ask which metric produced the sudden jump.
What GPT-4 can do with images
The "multimodal" in the technical report refers to image input. A user can supply a photo, chart, or screenshot alongside text and receive a text answer. Mechanically, images are converted into sequences of vectors that the Transformer processes alongside text tokens, which is the approach most vision-language systems share, though OpenAI has not published GPT-4's specifics.
Independent testing gives a more cautious picture than demonstrations do. On MMMU, a benchmark of 11,500 college-level multimodal questions across six disciplines, its authors reported that GPT-4V and Gemini Ultra reached only 56 percent and 59 percent. On BLINK, a benchmark that reformats 14 classic computer-vision tasks into 3,807 multiple-choice questions, humans averaged 95.70 percent while GPT-4V reached 51.26 percent, only 13.17 percentage points above random guessing. The BLINK authors note that specialist computer vision models solve these problems much better.
The pattern is consistent: general scene description and reading text in images work well, while tasks that need exact depth, correspondence, or geometry remain weak. A model that can explain a meme may still be unable to say which of two lines is longer.
Where the benchmarks mislead
Benchmark scores are measurements of a model on a particular test, under particular conditions, compared with a particular group. Each of those three qualifiers can move the headline.
The bar exam is the best-documented case. OpenAI's report placed GPT-4 around the 90th percentile of test takers. In a 2024 paper in Artificial Intelligence and Law, Eric Martínez re-evaluated that result. His analysis argued that the 90th-percentile figure relied on data from a February administration of the Illinois bar exam, whose test-takers skew toward people retaking after failing in July and who score lower than the general population. Comparing against more representative groups, he estimated GPT-4's overall percentile at below the 69th, and about the 48th on the essay portion.
The lesson is not that GPT-4 was weak. It is that a percentile is a statement about a comparison group, and the comparison group was chosen after the fact. The same caution applies to every exam in the table above.
Other problems affect benchmarks generally:
- Contamination: If test questions or close paraphrases appear in training data, a model may have effectively seen the answers. Without published training data, nobody outside the lab can check for it.
- Format sensitivity: Accuracy can shift with how a question is worded or how many examples are shown, which is why the report specifies settings such as 5-shot for MMLU and 0-shot for HumanEval.
- Saturation: When most strong models cluster near a ceiling, a benchmark stops separating them.
For a fuller treatment, see why AI benchmarks can be misleading and how AI benchmarks measure machine intelligence.
Where GPT-4 still fails
The technical report itself says GPT-4 is not fully reliable. It reports that GPT-4 scored 19 percentage points higher than the latest GPT-3.5 on OpenAI's internal adversarial factuality evaluations, an improvement, but one that leaves a substantial error rate and says nothing about every domain. Several independent studies document specific, repeatable failure types.
Hallucination
A hallucination is a fluent statement that is not true. In September 2025, Kalai and colleagues argued in Why Language Models Hallucinate that hallucinations arise partly because training and evaluation reward guessing over admitting uncertainty, so models become good test-takers that guess when unsure. If that account is right, hallucination is less a malfunction than a predictable result of how models are optimized and graded. Why AI hallucinates facts walks through the mechanisms and mitigations.
The reversal curse
Berglund and colleagues described the reversal curse: models trained on a statement like "A is B" do not automatically learn "B is A." For GPT-4, they reported that it correctly answered questions like "Who is Tom Cruise's mother?" 79 percent of the time, but only 33 percent of the time for the reverse, "Who is Mary Lee Pfeiffer's son?" A person who knows one fact knows the other. A language model stores the direction in which it saw the text.
Long inputs
Liu and colleagues found that performance is often highest when relevant information is at the beginning or end of the input, and drops significantly when it sits in the middle, even for models built for long contexts. More room in the prompt does not guarantee that every part of it gets used equally. Our article on context windows explains why.
Planning and search
A single left-to-right pass is a poor way to search. In the Tree of Thoughts paper, GPT-4 with chain-of-thought prompting solved about 4 percent of Game of 24 puzzles (combine four numbers with arithmetic to reach 24), while a method that let it explore and backtrack among partial solutions reached about 74 percent. The model had the arithmetic skill; it lacked a way to try alternatives and discard dead ends. This is one reason later systems add explicit reasoning steps, a topic covered in test-time compute explained.
Did GPT-4 show general intelligence?
In March 2023, a Microsoft Research team published Sparks of Artificial General Intelligence, testing an early version of GPT-4. They argued it exhibited more general intelligence than earlier systems, performing novel and difficult tasks across mathematics, coding, vision, medicine, law, and psychology, with performance close to human level on many. They suggested it could be viewed as an early but incomplete version of an artificial general intelligence system, while also noting that reaching deeper AGI may require a paradigm beyond next-word prediction.
Readers have taken opposite lessons from this paper, and that disagreement is informative. The result was a qualitative exploration of an early model, not a standardized test, and the authors themselves focused on finding weaknesses. Whether "general intelligence" is the right frame depends on what you count as intelligence, a definitional dispute that no benchmark settles. For the debate over what AGI would even mean, see what AGI is.
A more modest framing is easier to defend: GPT-4 was broad. A single model handled legal questions, code, translation, and image description at a level that previously required separate specialized systems. Breadth is measurable. Whether it amounts to understanding is not.
What people used it for, and what the evidence says
Claims about productivity gains are common and often unsourced, so it helps to look at controlled studies. In a controlled experiment on GitHub Copilot, an AI pair programmer, Peng and colleagues asked developers to implement an HTTP server in JavaScript as quickly as possible. The group with Copilot completed the task 55.8 percent faster than the control group, with heterogeneous effects across participants. Note the limits: one task, one language, and a model that was not GPT-4 itself.
Evidence for other domains is thinner. Reports of GPT-4 helping lawyers draft, researchers summarize papers, or teachers generate practice problems are mostly case studies and vendor statements. A reasonable reading is that the model reduces the cost of a first draft, and that the cost of checking the draft does not fall by the same amount. Where an error is cheap to catch, such as code that either runs or does not, the trade is favorable. Where an error is hard to catch, such as a plausible but invented legal citation, the trade can reverse.
GPT-4's afterlife: Turbo, GPT-4o, and retirement
GPT-4 was not a single fixed product. It was the first of a family, and the details of that family explain much of the confusion in older articles.
| Model | Date | Notable change |
|---|---|---|
| GPT-4 (8K context) | March 2023 | Launch version. API price of $0.03 per 1,000 input tokens and $0.06 per 1,000 output tokens |
| GPT-4 Turbo | November 2023 | 128,000-token context; priced at $0.01 input and $0.03 output per 1,000 tokens |
| GPT-4o | May 2024 | One network trained end-to-end across text, vision, and audio |
| GPT-4 in ChatGPT | Retired April 30, 2025 | Replaced by GPT-4o; still available through the API at that time |
Sources: OpenAI GPT-4 Technical Report; OpenAI DevDay announcement as summarized by Gigazine; GPT-4o System Card; Engadget.
The GPT-4o System Card describes an autoregressive "omni" model that accepts any mix of text, audio, image, and video inputs and generates text, audio, and image outputs, trained end-to-end across text, vision, and audio. It responds to audio in as little as 232 milliseconds, averaging 320 milliseconds, matches GPT-4 Turbo on English text and code, and is faster and 50 percent cheaper in the API.
According to Engadget's report on OpenAI's changelog, GPT-4 was retired from ChatGPT on April 30, 2025, with GPT-4o described as its natural successor, while API users could still access it after that date. The question of what comes after is covered in what GPT-5 is. Check OpenAI's current model documentation before relying on any model name or price in this article, because both change.
A worked cost example
At the launch prices of the 8K model, a request using 2,000 input tokens and producing 1,000 output tokens costs 2 × $0.03 + 1 × $0.06 = $0.12. At GPT-4 Turbo's launch prices, the same request costs 2 × $0.01 + 1 × $0.03 = $0.05. Prices have fallen since and continue to change, which is why older cost estimates, including ones quoted in earlier versions of this article, should not be treated as current.
The research that led up to GPT-4, in order
GPT-4 did not arrive from nowhere. Each of the results cited above answered a specific question, and the order of those answers explains why OpenAI bet on scale and alignment together.
| Year | Result | What it established |
|---|---|---|
| 2017 | Attention Is All You Need | An architecture built on attention alone, easier to parallelize than recurrent models |
| January 2020 | Kaplan et al., scaling laws | Loss falls as a power law in model size, data, and compute |
| May 2020 | Brown et al., GPT-3 | A 175-billion-parameter model can learn tasks from examples in the prompt |
| January 2022 | Wei et al., chain-of-thought prompting | Showing intermediate steps lifts a 540-billion-parameter model to state-of-the-art accuracy on GSM8K math word problems |
| March 2022 | Ouyang et al., InstructGPT | Human feedback makes a 1.3-billion-parameter model preferable to a 175-billion-parameter one |
| March 2022 | Hoffmann et al., Chinchilla | Model size and training tokens should scale together |
| March 2023 | GPT-4 Technical Report | Image and text input; benchmark results; no architecture details |
Read in sequence, the table shows a pattern. First came an architecture that scaled, then evidence that scale paid off predictably, then evidence that training data and training method mattered as much as raw size, and finally a system that combined them. The chain-of-thought paper found that including a few worked examples with intermediate reasoning in the prompt lifted a 540-billion-parameter model to state-of-the-art accuracy on GSM8K, surpassing even a fine-tuned GPT-3 with a verifier. Our explainer on AI reasoning and chain of thought picks up that thread.
Five questions to ask of any GPT-4 claim
Because so much about GPT-4 comes through press coverage, a short checklist helps separate measurement from folklore.
- Who measured it? A result in OpenAI's report is a vendor result. An independent replication, such as the bar exam reanalysis or the BLINK benchmark, carries different weight.
- Which version of the model? GPT-4, GPT-4 Turbo, GPT-4V, and GPT-4o are different systems. The Sparks paper tested an early version still in development, and its authors say so.
- Which comparison group? A percentile is only meaningful relative to the people it compares against.
- Which setting? Zero-shot, few-shot, and tool-assisted results are not interchangeable. The report's own table specifies 5-shot for MMLU and 0-shot for HumanEval for that reason.
- Is the number current? Prices, context limits, and availability changed within 12 months of launch and have kept changing.
None of these questions requires technical expertise. They require only the habit of asking what was actually tested, which is the habit that makes the difference between reading a benchmark and being sold one.
Common confusions about GPT-4
"GPT-4 has a trillion parameters." OpenAI has never published the figure. Treat any specific number as an unconfirmed estimate.
"It looks things up." The base model does not browse or query a database. It generates text from patterns stored in its weights, which were fixed at training time. Products built around it may add search or retrieval, which is a separate system, described in what retrieval-augmented generation is.
"A high exam percentile means expert competence." Exams sample a narrow slice of what experts do. As the bar exam reanalysis shows, the percentile also depends on who the comparison group is.
"Bigger always means smarter." InstructGPT showed a 1.3-billion-parameter model preferred over a 175-billion-parameter one after alignment, and Chinchilla showed a 70-billion-parameter model beating larger ones with more data. Size, data, and training method all matter.
"Prompting tricks can fix anything." Good prompts help, and prompt engineering is a real skill, but they cannot supply knowledge the model never learned or remove the failure types described above.
The takeaway
GPT-4 matters less as a product than as a demonstration. A model trained only to predict the next token, then shaped by human feedback, passed professional-style exams, read charts, and wrote working code. That result made the broader case that scale plus alignment can produce general-purpose behavior, and it shaped the next generation of systems from every major lab.
It also set a pattern that has not gone away: the most important facts about frontier models are disclosed selectively, the headline numbers depend on choices the reader cannot see, and the failures are as systematic as the successes. Reading GPT-4 well means holding both: the measured breadth of what it did, and the documented, repeatable ways it breaks.
FAQ
Q: Is GPT-4 actually thinking, or just predicting words? A: Mechanically, it predicts the next token. Whether that amounts to thinking is a definitional question. Evidence points both ways: GPT-4 handles novel multi-step problems, yet it fails on the reversal curse and on search-heavy puzzles that a person finds simple. Describing it as a powerful predictor with uneven reasoning is accurate without settling the philosophy.
Q: How many parameters does GPT-4 have? A: OpenAI has not said. The GPT-4 Technical Report states that it contains no further details about architecture, including model size. Figures reported elsewhere are estimates or rumors. By contrast, GPT-3's 175 billion parameters were published in its paper.
Q: What score did GPT-4 get on the bar exam? A: OpenAI reported a simulated Uniform Bar Exam score of 298 out of 400, around the 90th percentile of test takers. A 2024 reanalysis by Eric Martínez argued that the comparison group inflated this and estimated GPT-4's overall percentile at below the 69th, and about the 48th on essays.
Q: Can GPT-4 do math? A: It is much stronger than GPT-3.5. The report lists 700 of 800 on SAT Math, about the 89th percentile. But multi-step arithmetic and proofs still produce errors, and it solved only three of 45 hard LeetCode problems. Tools such as code execution, where the model writes a program and a computer runs it, improve reliability.
Q: What is the difference between GPT-4, GPT-4 Turbo, and GPT-4o? A: GPT-4 launched in March 2023 with an 8,000-token context. GPT-4 Turbo, announced in November 2023, extended the context to 128,000 tokens and cut prices. GPT-4o, released in 2024, is a single network trained end-to-end across text, vision, and audio, and is faster and cheaper than Turbo.
Q: Is GPT-4 still available? A: OpenAI retired GPT-4 from ChatGPT on April 30, 2025, replacing it with GPT-4o. According to coverage of that announcement, API users could still access it afterward. Availability changes, so check OpenAI's current documentation.
Q: Why does GPT-4 make things up? A: Because it is trained to produce likely text, not verified facts, and because training and grading often reward a confident guess over an admission of uncertainty. GPT-4 scored 19 percentage points higher than GPT-3.5 on OpenAI's internal adversarial factuality tests, an improvement, but one the report describes as short of full reliability.
Sources:
- OpenAI. (2023). "GPT-4 Technical Report." arXiv:2303.08774. https://arxiv.org/abs/2303.08774
- Brown, T., et al. (2020). "Language Models are Few-Shot Learners." arXiv:2005.14165. https://arxiv.org/abs/2005.14165
- Ouyang, L., et al. (2022). "Training language models to follow instructions with human feedback." arXiv:2203.02155. https://arxiv.org/abs/2203.02155
- Vaswani, A., et al. (2017). "Attention Is All You Need." arXiv:1706.03762. https://arxiv.org/abs/1706.03762
- Kaplan, J., et al. (2020). "Scaling Laws for Neural Language Models." arXiv:2001.08361. https://arxiv.org/abs/2001.08361
- Hoffmann, J., et al. (2022). "Training Compute-Optimal Large Language Models." arXiv:2203.15556. https://arxiv.org/abs/2203.15556
- Wei, J., et al. (2022). "Emergent Abilities of Large Language Models." arXiv:2206.07682. https://arxiv.org/abs/2206.07682
- Schaeffer, R., et al. (2023). "Are Emergent Abilities of Large Language Models a Mirage?" arXiv:2304.15004. https://arxiv.org/abs/2304.15004
- Martínez, E. (2024). "Re-evaluating GPT-4's bar exam performance." Artificial Intelligence and Law. https://dspace.mit.edu/handle/1721.1/153986
- Berglund, L., et al. (2023). "The Reversal Curse." arXiv:2309.12288. https://arxiv.org/abs/2309.12288
- Liu, N., et al. (2023). "Lost in the Middle." arXiv:2307.03172. https://arxiv.org/abs/2307.03172
- Kalai, A., et al. (2025). "Why Language Models Hallucinate." arXiv:2509.04664. https://arxiv.org/abs/2509.04664
- Yue, X., et al. (2023). "MMMU." arXiv:2311.16502. https://arxiv.org/abs/2311.16502
- Fu, X., et al. (2024). "BLINK: Multimodal Large Language Models Can See but Not Perceive." arXiv:2404.12390. https://arxiv.org/abs/2404.12390
- Bubeck, S., et al. (2023). "Sparks of Artificial General Intelligence: Early experiments with GPT-4." arXiv:2303.12712. https://arxiv.org/abs/2303.12712
- Peng, S., et al. (2023). "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot." arXiv:2302.06590. https://arxiv.org/abs/2302.06590
- OpenAI. (2024). "GPT-4o System Card." arXiv:2410.21276. https://arxiv.org/abs/2410.21276
Read next:
Follow new articles
Add SciHub101 to your feed reader to see new science and technology stories.
About the Author
SciHub101 is an independent science and technology publication founded and edited by Buğra Sözeri, a software engineer and app developer based in Türkiye. AI tools assist with drafting and research. Source documentation in the existing archive is uneven, and articles are being reviewed and corrected. The Editorial Team byline identifies editorial responsibility; it does not certify specialist or independent human review. Where we compare products, anything built by our own team is labelled as such — see the editorial guidelines and About page for how corrections and disclosures work.
Related Articles

What Is Machine Learning? Explained Simply, with Real Examples
Machine learning is how computers learn patterns from data instead of being told the rules. It powers everything from email spam filters to self-driving cars. Here's how it actually works.

AlphaZero: Self-Teaching Chess AI
How DeepMind's AlphaZero learned chess, shogi, and Go from the rules alone, what its self-play loop actually does, and what the Stockfish matches did and did not prove.

How AI Models Are Trained: The Full Pipeline Explained
Training a large language model is not a single act — it is a five-stage industrial process that begins with petabytes of scraped text, passes through weeks of GPU computation consuming millions of dollars of electricity, and ends with a model that can hold a coherent conversation. Each stage has a specific job, a distinct failure mode, and a measurable cost. Here is how the whole pipeline works, from raw data to deployed product.

How Image Recognition Works: The Science of Convolutional Neural Networks
In 2010 the best image classifier missed the right label in its top five guesses about 28% of the time. By 2015 that was 3.57%. Here is what changed, and how a CNN actually sees.

How Large Language Models Work: Transformers, Attention, and Why They Hallucinate
Meta trained Llama 3 on more than 15 trillion tokens using 7.7 million H100 GPU-hours. Here is how next-token prediction at that scale turns into a conversation partner.

Edge AI: How AI Runs on Your Phone
Server-side chatbots run on racks of GPUs in data centers. Face ID, voice dictation, and a growing number of language features run on a chip the size of a fingernail in your pocket. The science of shrinking large models to fit is called edge AI.