A Field Guide to the LLM Models Behind Modern AI
The phrase "large language model" gets used as if all LLMs are the same kind of thing. They are not. The model that finishes your email, the model that solves a math proof, the model that reads an image and describes what is in it, and the model that turns a sentence into a numeric vector for search are all called LLMs, but they have been built and trained for different jobs. Understanding the major families, what each is for, and how the training process produces each one is useful for anyone trying to make an informed decision about which model to use or build on.
The foundation of nearly every LLM you have ever interacted with is a pretrained base model. Base models are produced by taking a large neural network with billions of parameters and training it on enormous corpora of text, often trillions of tokens drawn from the public internet, books, academic papers, and code repositories. The training objective is deceptively simple: given a sequence of tokens, predict the next one. Repeated across vast and diverse text, that single objective forces the network to internalize grammar, factual knowledge, reasoning patterns, and stylistic conventions. The result is a model that can continue almost any text, but does not yet know how to follow instructions, answer questions politely, or refuse harmful requests. A raw base model is powerful and unrefined. It is the substrate everything else is built on, and it is rarely what end users see directly. Publicly released base models include Meta's Llama family, Mistral's open-weight base models, and Alibaba's Qwen base models. The base versions of OpenAI's GPT series and Anthropic's underlying models exist too, but those are kept internal; only the tuned versions reach users.
What most people actually use are instruction-tuned and chat models, which take a base model and refine its behavior through a series of additional training stages. The first stage is supervised fine-tuning on carefully curated examples of instructions paired with high-quality responses, which teaches the model to answer questions, follow directions, and produce output in a useful format. The second stage is typically reinforcement learning from human feedback, where human raters compare pairs of model responses and the model is updated to prefer the outputs humans rate higher. Some labs replace that human signal with a model trained against a written set of principles, which is faster and cheaper at the cost of being one step further from real human judgment. The combination of these stages is what turns a raw next-token predictor into something that feels like a helpful assistant. The base capability comes from pretraining; the personality, helpfulness, and safety behavior come from this fine-tuning layer. Familiar products in this category include OpenAI's ChatGPT, Anthropic's Claude, Google's Gemini, and the Instruct variants of Meta's Llama. ChatGPT is the instruction-tuned and RLHF-refined version of OpenAI's underlying GPT model; Claude is the corresponding product built on Anthropic's base models; Gemini is Google's. Each one is the same kind of object: a base model wrapped in this fine-tuning stack.
Beyond general-purpose chat models, there is a growing set of specialized models, each built for a narrower job. Reasoning models are trained to spend extra compute "thinking" before producing an answer, often by generating long internal chains of thought the user never sees. OpenAI's o-series (o1, o3) and DeepSeek's R1 are well-known reasoning-focused models, and recent versions of Claude and Gemini include an "extended thinking" mode that does the same kind of work. They are typically built on top of a strong base model and then further trained on math, science, and logic tasks where step-by-step working improves accuracy. Code models are fine-tuned with a much heavier emphasis on code repositories than general models, and they tend to outperform general models at autocompletion, refactoring, and bug analysis. Anthropic's Claude and OpenAI's GPT family are both heavily used for coding inside tools like GitHub Copilot and Cursor; DeepSeek-Coder and Meta's Code Llama are well-known open-weight examples. Multimodal models extend the input space beyond text to include images, audio, or video, and they require additional training stages that align those modalities with the model's language representation. OpenAI's GPT-4o, Anthropic's Claude with vision, and Google's Gemini are all multimodal. Embedding models are a different shape of LLM entirely: instead of producing text, they output a fixed-length numeric vector that captures the meaning of an input, which is what powers semantic search and retrieval-augmented generation systems. OpenAI's text-embedding models, Cohere's embeddings, and Voyage AI are common production choices.
Running alongside the trend toward larger frontier models is a parallel push toward smaller and more efficient ones. Small language models with a few billion parameters or fewer are produced through a combination of careful training data curation, architectural choices that improve parameter efficiency, and distillation, where a small student model is trained to mimic the outputs of a much larger teacher. Examples in active use include Meta's Llama 3.2 1B and 3B variants, Microsoft's Phi-3 family, Google's Gemma, Mistral 7B, and Anthropic's Claude Haiku tier. These models accept a real reduction in raw capability in exchange for speed, lower cost per inference, and the ability to run on a laptop or phone rather than a data center. For tasks where the largest models would be overkill, or where privacy and latency are dominant constraints, a small model can be the correct choice. The gap between what a frontier model and a well-tuned small model can do on specific tasks has been closing steadily, and for many practical applications the small model is more than enough.
The choice of which kind of LLM to use is one of the more consequential decisions in designing any AI product. A general-purpose chat model is not the right tool for semantic search; an embedding model is not the right tool for open-ended writing; a reasoning model is overkill for autocompletion. Knowing which family of model does what, and what the training process implies about its strengths and blind spots, is part of being deliberate about building on AI rather than just plugging into whichever default API a provider hands you. The label "LLM" hides a lot of structure, and most of the interesting engineering decisions live in that structure.