How Large Language Models Work: The Technology Behind Modern AI

Large language models like GPT-5.6, Claude Opus 5, and Gemini 3.1 Pro can hold a conversation, write code, and answer questions across nearly any topic. Underneath, they're doing one specific thing: predicting the most likely next piece of text, over and over, given everything that came before it. That simple objective, applied at enormous scale, is what produces behavior that looks like understanding.

Step one: tokenization

A model doesn't read text as letters or whole words. It first breaks text into tokens - chunks that are often a word, part of a word, or a punctuation mark. "Tokenization" is why LLM context windows and pricing are measured in tokens rather than words: a rough rule of thumb is that 1,000 tokens is roughly 750 words in English, though this varies by language and by content.

Step two: the transformer architecture

Modern LLMs are built on the transformer architecture, introduced in the 2017 paper "Attention Is All You Need" (linked in our Resources page). Its key mechanism is self-attention: for every token, the model computes how relevant every other token in the input is to it, and weighs its prediction accordingly. This lets the model connect a pronoun to the noun it refers to several sentences earlier, or track a variable's meaning across a long piece of code, regardless of the distance between them in the text - something earlier architectures struggled with.

Step three: training

Training happens in stages:

1. Pre-training. The model is fed enormous amounts of text and learns, one prediction at a time, to guess the next token. This stage is where the model absorbs grammar, facts, reasoning patterns, and writing style - purely as a side effect of getting better at next-token prediction. It requires no human-labeled data at this stage.
2. Fine-tuning. The pre-trained model is then trained further on curated examples of the kind of behavior developers actually want - following instructions, refusing harmful requests, formatting answers helpfully.
3. Preference alignment. Human raters (or, increasingly, other models) compare pairs of model outputs and indicate which is better; the model is adjusted to produce more outputs like the preferred ones. This is the stage that shapes a lot of what people experience as a model's "personality."

Where "reasoning" and "thinking" modes fit in

Some current models - Claude Opus 5's adaptive thinking is one documented example - can allocate extra computation to work through a problem step by step before producing a final answer, rather than generating a response in one pass. This measurably improves performance on tasks that benefit from multi-step reasoning, like math or debugging, at the cost of extra latency. It is not evidence that the model has human-like reasoning in a general sense - it's a specific mechanism with a specific, testable effect on specific task types.

Why this explains both the strengths and the limits

Because an LLM is fundamentally predicting plausible continuations of text based on patterns in its training data, it's naturally good at tasks with a clear textual pattern (writing in a given style, transforming data formats, generating code that resembles code it has seen). It's also naturally prone to hallucination - producing fluent, confident, and sometimes entirely wrong statements - because "plausible-sounding continuation" and "true" are not the same target, and the model has no built-in mechanism to distinguish them. That's why verifying anything factually important an LLM tells you remains essential, no matter how confident the answer sounds.