What a Large Language Model Actually Does
A plain explanation of how today's text-generating systems are built and trained, and why the way they work explains both their strengths and their mistakes.
Large language models are the technology behind most of today’s conversational AI assistants. Their behaviour can look mysterious, but the basic recipe is simple to describe, and knowing it makes their strengths and failures easier to understand.
Text as a sequence of tokens
A language model does not see words the way people do. Text is first split into tokens — whole words, pieces of words or punctuation marks — and each token is turned into a list of numbers. The model then works entirely with these numbers.
Learning by predicting what comes next
The core training task is easy to state: given a stretch of text, predict the next token. The model makes a guess, compares it with the token that actually follows, and its internal weights are adjusted slightly to make a better guess next time. Repeated over a very large collection of text, this simple exercise forces the model to pick up grammar, facts that appear frequently, common patterns of reasoning and the styles of many kinds of writing.
Most current models use the transformer architecture, introduced in 2017. Its attention mechanism lets the model consider every earlier token when predicting the next one, which helps it keep track of context over long passages.
Learning from examples in the prompt
A 2020 study of a very large model of this kind found something useful: the model could often carry out a new task when shown only a handful of worked examples in its input, without any further training. The authors called this “few-shot” learning. It is the reason why carefully written instructions and examples — the prompt — make such a difference to the results.
From text predictor to assistant
A model trained only to continue text is not yet a helpful assistant. It may continue a question with more questions, or produce text that is fluent but unhelpful. A further stage of training adjusts the model using examples of good answers and human judgements about which of two responses is better. Research published in 2022 showed that this kind of training with human feedback made models considerably better at following instructions as people intended.
What follows from the recipe
- Fluency is not the same as accuracy. The training objective rewards plausible text. Checking facts requires a separate step, by a person or by connecting the model to reliable sources.
- Knowledge has a horizon. A model learns from data collected up to a certain point. Anything that happened later is unknown to it unless supplied in the prompt or through a connected tool.
- Context matters. The same model can give very different answers depending on how much relevant information and how clear an instruction it receives.
Practical advice on getting reliable results is collected in A Practical Guide to Working with AI Assistants.
Sources
- Vaswani, A. et al. Attention Is All You Need. NeurIPS 2017.
- Brown, T. B. et al. Language Models are Few-Shot Learners. NeurIPS 2020.
- Ouyang, L. et al. Training language models to follow instructions with human feedback. NeurIPS 2022.
- National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), 2024.