What Is an LLM (Large Language Model)? A Plain-English Explainer

A large language model, or LLM, is a type of neural network trained on enormous volumes of text and code so it can predict, and then generate, the next piece of language in a sequence — a mechanism that turns out to be powerful enough to write essays, answer questions, translate, summarize, and hold a conversation. Chatbots like ChatGPT, Claude, and Gemini are built on top of LLMs, but the term refers specifically to the underlying model, not the app, the chat window, or any workflow wrapped around it.

What Is an LLM, Exactly?

At its core, an LLM is, in the words of its own general reference entry, a language model notable mainly for its scale. It is a statistical model of language. It has been shown so much text — books, articles, code, forum posts, reference sites — that it has learned the patterns of how words, phrases, and ideas tend to follow one another. Ask it to continue a sentence, and it calculates, for every word in its vocabulary, how likely that word is to come next, then samples from that probability distribution. Do that one word (technically one token) at a time, thousands of times in a row, and the result can look like reasoning, explanation, or code.

That description sounds mechanical, almost too simple to explain what these systems can do — write working software, pass professional exams, or hold a multi-turn conversation about a legal contract. The honest answer from the people who build them is that scale changes behavior in ways that are not fully predictable in advance. Anthropic's own interpretability researchers have described the inside of a trained model as largely opaque even to its creators: the model's internal state is a long list of numbers (neuron activations), and a single idea is typically spread across many of those numbers at once rather than sitting in one tidy location, which is part of why "why did the model say that?" is still a hard question to answer precisely.

Neural Networks, in Plain English

An LLM is a particular kind of neural network, which is itself a very old idea in computer science: a system of simple, interconnected mathematical units ("neurons") loosely inspired by the brain, organized into layers, that learns by adjusting the strength of the connections between those units. What made today's LLMs possible is a specific neural network design called the transformer, introduced by Google researchers in a 2017 paper titled "Attention Is All You Need." The transformer's key trick is self-attention: for every word in a passage, the model computes how relevant every other word is to it, so it can weigh context from anywhere in the text rather than only the words immediately before it. Because that computation can run on many words in parallel instead of one at a time, transformers train far faster on modern chip hardware than the recurrent neural networks that came before them — which is a large part of why model sizes and training datasets were able to grow so quickly after 2017.

Almost every well-known LLM today — the GPT family, Claude, Gemini, and the large open-weight models researchers publish on platforms like GitHub — is a transformer, or a close variant of one.

How Training Works, Part One: Pretraining

Building an LLM happens in stages, and the first and most expensive stage is called pretraining. The model is given a simple, repetitive task: look at a chunk of real text with the ending hidden, and predict what comes next. It does this billions of times across a training set that can include a meaningful fraction of the publicly available internet, digitized books, and code repositories. Each time it guesses wrong, an algorithm nudges its internal settings — its parameters — a tiny bit in the direction that would have made the correct answer more likely.

There is nothing in that loop that explicitly teaches facts, grammar, or logic. Those capabilities emerge as side effects of getting extremely good at next-word prediction across a huge and varied dataset. Pretraining is also where most of the cost lives: training runs can take enormous amounts of specialized computing hardware over weeks, which is why only a small number of organizations train frontier-scale models from scratch, even though many more fine-tune or build on top of them afterward.

How Training Works, Part Two: Fine-Tuning and RLHF

A freshly pretrained model is good at continuing text in a statistically plausible way, but it has no particular reason to be helpful, follow instructions, or refuse harmful requests — it is simply modeling "what comes next" based on everything it read, good and bad. Turning that raw model into something like a usable assistant takes a second stage, usually with two parts.

The first part is instruction fine-tuning: the model is trained further on examples of instructions paired with good responses, so it learns the general shape of "answer the question that was asked, in a helpful format." The second part is commonly reinforcement learning from human feedback, or RLHF. Human raters compare different candidate responses to the same prompt and say which one they prefer; those preferences are used to train a separate "reward" model, and the LLM is then adjusted with reinforcement learning to produce more responses that the reward model scores highly. Anthropic has published its own variant of this idea, which it calls Constitutional AI, where a set of written principles helps guide the model's own self-critique during training rather than relying solely on human comparisons for every step.

Fine-tuning and RLHF are why a raw pretrained model and a shipped product like a chat assistant can behave so differently from one another, even though the underlying network and most of its learned knowledge are the same.

Tokens, Parameters, and Context Windows, Decoded

Three words come up constantly in LLM coverage, and they are each simpler than they sound.

Tokens are the chunks of text a model actually processes. Computers work with numbers, not letters, so before any text reaches the model it gets broken into pieces — often word fragments rather than whole words — and each piece is mapped to a number. "Unbelievable" might become a few tokens rather than one. Roughly speaking, a token is somewhere around three to four characters of English text on average, so a 1,000-word document is typically a bit more than 1,000 tokens once split this way.

Parameters are the model's internal settings — the adjustable numeric "dials" that pretraining tunes. A model with more parameters has more capacity to represent patterns, which is one reason "parameter count" became an informal (and imperfect) shorthand for a model's scale. GPT-3, OpenAI's 2020 model, had 175 billion parameters; its predecessor GPT-2, released the year before, had 1.5 billion. Parameter counts for the newest frontier models are frequently not published at all, so treat any specific number you see for a current flagship model with caution unless it comes from the company itself.

Context window is the amount of text — measured in tokens — that a model can "see" and weigh against itself at one time, in a single conversation or document. Ask it something that depends on information outside that window and it simply cannot refer back to it, the same way you can't quote a page of a book you've already set down. Context windows have grown enormously since the transformer era began — Anthropic's Claude 3 family, for instance, launched with a standard 200,000-token window in 2024 — which is part of why today's chat assistants can hold long conversations or read entire documents that would have been impossible for earlier models to process in one pass.

Figures drawn from Wikipedia's "Large language model" and "GPT-3" articles and Anthropic's Claude 3 announcement; see Sources below. Parameter counts for current frontier models are often undisclosed by their makers.

How Inference Works When You Hit "Send"

Training happens once (or periodically, for updated versions). Every time you actually use a model — typing a prompt into a chat app, or a piece of software calling it through an API — that is called inference, and no further learning happens in that moment; the model's parameters are frozen. What happens instead: your prompt is broken into tokens, those tokens are converted into the model's internal numeric representation, and the network runs its calculations layer by layer to produce a probability distribution over what token should come next. One token is chosen, appended to the running text, and the whole process repeats for the next token, and the next, until the response is complete or the model produces a designated stop signal. This token-by-token generation is why longer responses visibly take longer to produce, and why a model can occasionally "paint itself into a corner" partway through an answer, since each new token is chosen based on everything generated so far rather than planned out in advance.

How LLMs Differ From Older Chatbots and Narrow AI

Before LLMs, most conversational software was what's often called a rule-based or narrow AI system: a developer wrote explicit decision trees or pattern-matching rules ("if the user's message contains 'refund,' respond with X"), or a statistical model was trained for one specific job, like spam filtering or sentiment scoring, and could do only that job. Earlier statistical language models existed too — simple "n-gram" models that predicted text purely from counts of how often short word sequences appeared together, without any deeper representation of meaning.

LLMs are different in two important ways. First, they are general-purpose: the same pretrained network that can draft an email can also summarize a document, explain a concept, or write code, without being separately programmed for each task — a property that didn't really exist in earlier narrow systems. Second, they don't rely on hand-written rules for how to respond; their behavior emerges from patterns learned across a training set, which is also exactly why they can be unpredictable, confidently wrong, or sensitive to how a question is phrased in ways a hand-coded rule engine simply wouldn't be.

Where LLMs Show Up in Everyday Software

LLMs are no longer confined to a standalone chat window. A few places they now sit inside ordinary software:

• Chat assistants — ChatGPT, Claude, and Google's Gemini family of models are the most visible examples, used for writing, research, tutoring, and general Q&A.
• Coding assistants — tools built into code editors suggest completions, explain unfamiliar code, and generate whole functions from a plain-English description, drawing on an LLM trained heavily on public code repositories.
• Search — major search engines now generate summarized answers above traditional results by having an LLM read retrieved web pages and synthesize a response, rather than only returning a ranked list of links.
• Autonomous and semi-autonomous agents — software that uses an LLM's reasoning as a planning engine to carry out multi-step tasks, calling other tools or services along the way.

That last category is a genuinely different layer on top of the model itself, and it's worth being precise about the distinction. An LLM is the underlying language engine. An "AI agent" is a system built around an LLM that can take actions, call tools, and pursue a goal across multiple steps — a related but separate concept covered in pandromeda's plain-English explainer on AI agents. Likewise, the question of how an LLM-powered application actually connects to outside tools and data sources — files, databases, other software — is its own topic, explained in the guide to MCP, the Model Context Protocol. And the increasingly common practice of leaning on an LLM-based coding assistant to build software quickly, sometimes without reading every line it produces, is covered separately in pandromeda's explainer on vibe coding. This article focuses specifically on the model underneath all of them.

The Limits: Hallucination, Bias, and Opacity

Because an LLM generates text by predicting plausible continuations rather than by looking up verified facts, it can produce fluent, confident statements that are simply wrong — a widely used shorthand for this failure mode is "hallucination." The model has no built-in mechanism that distinguishes a well-supported claim from an invented one; both are just sequences of tokens that scored as plausible continuations during generation. Training data can also carry forward the biases present in the text it was drawn from, and fine-tuning and RLHF, while they help, don't eliminate the underlying statistical nature of how the model produces output.

There's also a deeper, more structural limit: even the organizations that build these models can't fully explain why a specific response came out the way it did. Anthropic's interpretability research has used a technique called dictionary learning to identify millions of recurring, interpretable "features" inside one of its Claude models — internal patterns that correspond to recognizable concepts, from specific places and people to abstract ideas — and found that amplifying a given feature can predictably change the model's output. That work is a genuine step toward opening up what Anthropic describes as largely a "black box," but the researchers themselves note that the features identified so far are only a subset of what the model has learned, and knowing a representation exists doesn't yet reveal the full circuitry that puts it to use.

What's Next for Large Language Models

Several trends are shaping where LLMs go from here. Context windows keep expanding, which is steadily reducing the practical need to chop long documents into pieces before a model can work with them. "Reasoning" or "thinking" model variants, which generate intermediate reasoning steps before producing a final answer, have become a distinct category rather than a single-pass predict-the-next-token approach for every query. Interpretability research, like Anthropic's feature-extraction work described above, is an active area aimed at making these systems less opaque rather than simply making them bigger. And the agent and tool-use layer built on top of LLMs — where a model doesn't just answer a question but plans and executes multi-step tasks using other software — is where much of the visible product innovation is currently happening, even though the language model itself remains the component actually doing the reading, reasoning, and writing underneath it all.

Written byFelix Moreau

Felix Moreau writes Pandromeda's software coverage and how-to guides. He covers Windows, macOS and Linux updates, the apps people rely on, emulators and developer tools, and turns official documentation into clear, numbered steps that work on the current version.

More from Felix MoreauAll Software & Guides coverage