Parameters and Weights: What the Numbers in a Model Actually Are

Q: Does more parameters mean a smarter model?

No. Parameter count measures capacity — how much the model *could* store — not how well it was trained. Data quality, training compute, architecture, and post-training often matter more. A well-trained smaller model routinely beats a poorly-trained larger one, and modern small models outperform the giants of a few years ago. Use benchmarks and real testing to judge quality, not the headline number.

Q: What's the difference between parameters and training tokens?

Parameters are the model's *capacity* — how many learned numbers it has to store patterns in. Training tokens are how much text it was *shown* while learning. They're independent: a big model fed too little data leaves much of its capacity unused, while a small model fed enormous data eventually saturates. Quality depends on balancing the two, not maxing either. This is why two models with the same parameter count can differ enormously — one may have been trained on far more data per parameter than the other.

When someone says a model has "70 billion parameters," they are telling you it contains 70 billion individual numbers — plain decimal values like 0.0231 or -1.887 — that were adjusted during training and then frozen. That's it. A parameter is not a fact, a rule, or a concept. It's a single tunable number, and a modern model is a giant grid of them. "70 billion parameters" means the file that is the model holds 70 billion of these values, and running the model means multiplying your input through all of them.

So the number everyone quotes is really a size measurement, closer to "this engine has a 6.2-litre displacement" than "this car does 0-60 in 3 seconds." It tells you how much machinery is in there, which loosely tracks how much the model can store and how expensive it is to run — but it does not tell you how good the model is. Two models with identical parameter counts can differ wildly in quality, and a smaller model routinely beats a larger one. This post explains what those numbers physically are, why the count matters for memory and hardware, and why it's a proxy, not a guarantee.

The stakes are practical. If you ever want to run a model yourself, fine-tune one, budget for an API, or just cut through vendor marketing, the parameter count is the first number you'll meet and the most consistently misunderstood. People treat it as an IQ score. It is closer to a shipping weight. By the end of this piece you'll be able to look at "a 7B model quantized to 4-bit with 32 active experts" and know exactly what each part of that sentence claims, what it costs you, and what it does not promise about output quality.

Key takeaways
What a parameter physically is
Where the parameters actually live: inside a transformer block
Weights vs. parameters vs. "the model"
What is not a parameter (and why the confusion costs you)
How parameters encode knowledge: superposition and distributed representation
Why the count maps to memory and cost
Precision: same count, fewer bytes
Parameters vs. tokens: size is not the same as how much it learned
Why bigger isn't automatically better
Dense vs. sparse: total vs. active parameters
Total vs. active parameters (the MoE twist)
Reading a model card like a pro
FAQ
The bottom line
Further reading

Key takeaways

A parameter is one learned number. A "weight" is the most common kind of parameter — a multiplier applied to a value as it flows through the network. Most people use "parameters" and "weights" interchangeably, and for practical purposes you can too.
Parameter count measures capacity, not intelligence. It's roughly how many adjustable knobs the model has for storing patterns. More knobs can hold more, but only if training and data actually fill them well.
The count directly sets memory and cost. Each parameter occupies a fixed number of bytes. Multiply parameters by bytes-per-parameter and you get the memory footprint — the single most important number for figuring out what hardware you need.
Bigger is not automatically better. Data quality, training method, architecture, and post-training often matter more than raw size. A well-trained small model beats a poorly-trained large one all the time.
Precision changes the byte cost, not the count. Quantization shrinks how many bytes each parameter uses (e.g. 16 bits down to 4) without changing how many parameters there are.
"Active" parameters differ from "total" in some models. Mixture-of-Experts models quote a large total but only use a fraction per token, which decouples size from run-cost.

What a parameter physically is

Imagine an enormous spreadsheet — millions of cells, each holding one number. That's the mental model. A neural network is organised into layers, and each layer is essentially a big table of numbers (a matrix) that your input gets multiplied against. Every number in those tables is a weight: a multiplier that says "when this signal comes through, scale it by 0.42." Alongside the weights sit biases — offset numbers added after the multiplication. Weights and biases together are the parameters. Weights dominate the count so heavily that people just say "weights" for all of it.

To make the mechanism concrete: your input to a layer is a list of numbers (a vector). The layer holds a matrix of weights. The core operation is a matrix multiplication — every output number is a weighted sum of every input number, where the weights are exactly those learned parameters. If a layer takes in 4,096 numbers and puts out 4,096 numbers, its main weight matrix alone is 4,096 × 4,096 ≈ 16.8 million parameters. Stack roughly a hundred layers, each with several such matrices, and the millions become billions. That is the entire arithmetic of "how do you get to 70 billion": it's just the sum of the sizes of every weight matrix in the network. Nothing exotic — a very large number of very simple multiply-and-add operations, all governed by these stored constants.

The reason a parameter can be any real number, not just a 0 or 1, is what gives the network its expressive power. A weight near zero effectively ignores its input; a large positive weight amplifies it; a negative weight inverts it. Between those extremes lie infinitely many gradations, and it's the precise pattern of billions of such gradations — tuned against each other — that lets a fixed pile of numbers behave like it "understands" text. The behaviour is emergent: no line of code says "detect sarcasm." Sarcasm detection, where it exists, is a side effect of how the numbers landed.

Here's the important part: these numbers are not designed by anyone. They start as random noise. During training, the model makes a prediction, sees how wrong it was, and every parameter gets nudged a hair in the direction that would have made the answer better. Repeat that across trillions of words, and the once-random numbers settle into values that happen to encode grammar, facts, coding patterns, and reasoning shortcuts. If you want the mechanism behind that nudging, see how neural networks learn. The takeaway here is simpler: the "knowledge" in a model is not stored as text or rules. It is smeared across billions of decimal numbers, and no single parameter means anything on its own.

This is why you can't open a model and "find where it knows that Paris is the capital of France." That fact is distributed across countless weights, entangled with everything else. The parameters are the substrate; meaning is what emerges when you run data through all of them at once.

Where the parameters actually live: inside a transformer block

"A giant pile of numbers" is true but unsatisfying. It helps to know where those numbers sit, because it explains why the count grows the way it does and why some architectures are more parameter-efficient than others. Modern language models are transformers, and a transformer is a stack of near-identical blocks. Almost every parameter lives in one of two places inside each block, and knowing which is which demystifies the whole count. For the full picture of what these components do, see how transformers work; here we care only about where the weights hide.

1. The attention weights. Each block has an attention mechanism that lets every token look at other tokens and decide what's relevant. Mechanically, this is done by projecting each token's vector into three new vectors — query, key, and value — using three weight matrices (plus an output projection). Those four matrices are pure parameters. They don't store facts so much as store how to route information: which words should attend to which. On a model with a hidden size of, say, 4,096, each of these projection matrices is on the order of 4,096 × 4,096 parameters, and every block has its own set.

2. The feed-forward (MLP) weights. After attention, each block passes every token through a small two-layer network — the feed-forward block, or MLP. This is where the bulk of a transformer's parameters actually live, typically two-thirds or more of the total. The feed-forward block expands the hidden dimension by a factor of roughly four, does a nonlinearity, then projects back down. Those expand-and-contract matrices are enormous, and interpretability research increasingly points to the feed-forward layers as where much of a model's factual knowledge is stored — the attention layers move information around, the MLP layers hold a lot of the "what."

A smaller share of parameters sits in the embedding matrix (which turns token IDs into vectors, and vectors back into token probabilities) and in tiny normalization layers. But the headline number is dominated by attention and MLP weights, repeated across every layer. This is why "depth times width" governs size: more layers or a wider hidden dimension multiplies out into more parameters fast. It's also why two models can share a parameter count yet allocate it very differently — one deep and narrow, another shallow and wide — with real consequences for what they're good at.

The practical upshot: when a provider says a model is "70B," they are summing the attention projections, the feed-forward matrices, the embeddings, and the biases across every one of its blocks. There is no mystery reservoir of intelligence hiding elsewhere. The number is a straightforward inventory of the multiplication tables inside a repeated architectural motif.

Weights vs. parameters vs. "the model"

The words get used loosely, so here's the clean version:

Parameter — any learned number in the model. Umbrella term.
Weight — a parameter that multiplies a value. The vast majority of parameters are weights.
Bias — a parameter that's added, not multiplied. A small minority of the count.
Weights (plural, informal) — often used to mean "the whole set of learned numbers," i.e. the model's actual content. When people talk about "open weights," they mean the provider published this giant pile of numbers so you can download and run it yourself.

Two things are not parameters, and confusing them is common:

Activations are the temporary numbers produced while the model runs on your specific input. They live for one forward pass and vanish. Parameters are permanent; activations are scratch paper.
Hyperparameters are settings about training (learning rate, number of layers, batch size) chosen by humans before training. They shape how parameters get learned but aren't part of the count.

So "the model" you download is, concretely, a file (or set of files) containing the frozen parameters plus a small config describing how they're wired together. Everything else — the tokenizer, the serving code — is scaffolding around that pile of numbers.

What is not a parameter (and why the confusion costs you)

The single most common source of confused reasoning about model size is lumping together things that are counted separately. Three categories get mistaken for parameters, and each one distorts a different practical decision.

Activations are not parameters. When you send a prompt through the model, every layer produces intermediate numbers — the activations — as your specific input flows forward. These are transient: they exist for one forward pass and are discarded. Crucially, activations consume memory while the model runs, on top of the memory the parameters already occupy. That's why "the model is 14 GB, so my 16 GB GPU is fine" is a trap — the parameters fit, but the activations and other runtime buffers may not. Parameters are the fixed cost of owning the model; activations are part of the variable cost of using it.

The KV cache is not a parameter either. As the model generates text, it stores the keys and values it computed for every previous token so it doesn't recompute them — the KV cache. This grows with the length of your conversation and can, on long contexts, rival or exceed the size of the model's weights. It is often the reason a model that "fits" runs out of memory halfway through a long document. If you're sizing hardware and only budget for parameters, you will be wrong precisely when it matters most — on long inputs. The KV cache and its inference memory math is a whole subject of its own.

Hyperparameters are not parameters. This one is purely a naming collision, but it trips people up constantly. Hyperparameters are the human-chosen settings that govern training and inference: learning rate, batch size, number of layers, and — at inference time — things like temperature and top-p. They are not learned, not counted, and not part of the downloaded weights. When a model card says "trained with a learning rate of 3e-4," that number is a hyperparameter; it shaped how the parameters were learned but is not itself one of them.

Why does the distinction earn a whole section? Because each confusion produces a specific, expensive mistake. Confusing activations or KV cache with parameters makes you under-provision memory and hit out-of-memory crashes. Confusing hyperparameters with parameters makes you think "tuning the model" and "changing a setting like temperature" are the same act — they're worlds apart, one requiring retraining and the other a single line in your API call. Keep the mental buckets separate: parameters are permanent and learned; activations and KV cache are temporary and computed; hyperparameters are chosen and external.

How parameters encode knowledge: superposition and distributed representation

If knowledge isn't stored as text or rules, and no single weight means anything, then how do billions of numbers come to hold facts and skills? The honest answer is that we don't fully understand it — this is the frontier field of interpretability — but the working picture is worth carrying, because it explains several otherwise-baffling behaviours.

The core idea is distributed representation. A concept like "the capital of France" is not stored in one weight or even one neuron. It's represented as a pattern of activity across many neurons, and the weights that produce that pattern are spread across many matrices. Conversely, any single neuron participates in representing many unrelated concepts. This many-to-many mapping is the opposite of a database, where one row holds one fact. It's why you can't delete a fact from a model by zeroing out a weight, and why models "know" things fuzzily — with confidence that shades smoothly from certain to hallucinated rather than a clean hit-or-miss lookup.

A sharper version of this is superposition: the observation that models appear to pack more distinct concepts into a layer than it has neurons, by representing concepts as overlapping directions in a high-dimensional space rather than assigning each its own dedicated neuron. Because the space has thousands of dimensions, you can fit far more "almost-orthogonal" directions than dimensions, as long as any given input only lights up a sparse handful at once. Superposition is a leading explanation for why interpretability is hard: the features are entangled by design, not by accident. It also gives intuition for why scaling helps — more parameters means more room to hold distinct features with less interference — and why compression (quantization, pruning) eventually breaks things: you're crowding an already-crowded representation.

The practical takeaways from all this are concrete, not philosophical. First, you cannot surgically edit what a model knows by poking individual weights — knowledge editing is an active research problem precisely because facts are smeared and entangled. Second, a model's confidence is not calibrated knowledge; because retrieval is a soft pattern-match across distributed weights rather than a lookup, a model can produce a fluent, wrong answer with the same machinery it uses for a right one. Third, more parameters buys representational headroom, which is part (only part) of why bigger models can hold more — but that headroom is wasted if training never fills it, which brings us to the economics of the count.

Why the count maps to memory and cost

This is where the parameter number earns its keep, because it translates directly into hardware requirements. The formula is almost embarrassingly simple:

Memory to hold the model ≈ (number of parameters) × (bytes per parameter).

Each parameter is stored at some numerical precision. A 16-bit parameter takes 2 bytes; an 8-bit one takes 1 byte; a 4-bit one takes half a byte. So:

Model size	16-bit (2 bytes)	8-bit (1 byte)	4-bit (0.5 byte)
7 billion	~14 GB	~7 GB	~3.5 GB
13 billion	~26 GB	~13 GB	~6.5 GB
70 billion	~140 GB	~70 GB	~35 GB
400 billion	~800 GB	~400 GB	~200 GB

These are just-to-load numbers; actually running the model needs extra memory for activations and the context window's KV cache, so real requirements run higher. But the pattern is the point: a 70B model at 16-bit precision won't fit on a single consumer GPU (typically 16-24 GB), which is exactly why people either quantize it, split it across multiple GPUs, or rent cloud hardware. The parameter count is the first thing you check when asking "can I run this?" — and it's the backbone of running LLMs locally.

Cost follows the same logic. More parameters means more multiplications per token generated, which means more compute, more energy, and more money per response. The economics of that are their own rabbit hole — see AI inference cost economics — but the intuition is clean: parameter count is a decent proxy for how expensive a model is to serve, because it sets both the memory you must rent and the arithmetic you must pay for.

There's a second memory story that inference hides: training a model costs several times the parameter memory, not one times. To train or fully fine-tune, you don't just hold the weights — you also hold a gradient for every parameter, plus optimizer state (common optimizers keep two extra numbers per parameter), plus the activations needed to compute the backward pass. A rough rule of thumb is that full training in 16-bit needs on the order of sixteen-plus bytes per parameter once you count weights, gradients, and optimizer momentum terms, which is why fine-tuning a 70B model the naive way is out of reach for almost everyone, and why parameter-efficient methods like LoRA — which train a tiny add-on set of weights while freezing the original billions — exist at all. If you're headed down that road, how to fine-tune a model covers the tradeoffs. The point for now: the same parameter count implies very different memory depending on whether you're running the model (roughly one to two bytes per parameter) or training it (many times that).

One more subtlety worth internalizing: the multiplication above gives you the steady-state weight memory, but the reason a model "fits" or "doesn't fit" on a given card is almost never a close call on the weights alone — it's the weights plus activations plus KV cache plus framework overhead. Practitioners typically budget a comfortable margin above the raw weight figure rather than assuming the table above is the whole bill.

Precision: same count, fewer bytes

A frequent confusion: people assume a "smaller" version of a model has fewer parameters. Usually it doesn't — it has the same parameters stored at lower precision. This is quantization, and it changes the bytes-per-parameter term in the formula above, not the parameter count.

Think of it as rounding. A weight of 0.4213897 stored in high precision might become 0.42 in a compressed format. You lose a little fidelity, but you halve or quarter the storage. A 70B model that needs 140 GB at 16-bit can be squeezed to ~35 GB at 4-bit — same 70 billion numbers, just recorded more coarsely. Quality degrades gradually as you compress harder, and modern 4-bit methods are good enough that the loss is often barely noticeable for everyday use. The precise where-it-breaks details, and why some layers tolerate compression better than others, are covered in the tradeoffs of quantization.

Why does this work at all? Because of the distributed, redundant way knowledge is encoded (the superposition story above): the model's behaviour depends on the overall pattern of weights, not the exact seventh decimal place of any one of them, so coarsening the numbers slightly perturbs the pattern rather than destroying it. Push too far — 3-bit, 2-bit — and the perturbations compound, the crowded feature directions start colliding, and quality falls off a cliff. There's a real floor; quantization is a discount, not a free lunch. This is also why quantization is the workhorse of running LLMs locally: it's usually the only way to fit a capable model into consumer memory without changing which model you're running.

The lesson: when comparing models, "70B" tells you the count, but the precision tells you the footprint. Two copies of the same 70B model can demand 140 GB or 35 GB depending only on how the numbers are stored. Don't conflate the two — and when someone quotes a memory requirement without stating the precision, they've told you half a fact.

Parameters vs. tokens: size is not the same as how much it learned

There are two "big numbers" in any model, and conflating them is the deepest misconception about size. One is the parameter count — how much machinery the model has. The other is the training token count — how much text it was shown while learning. A parameter is capacity to store; a training token is an opportunity to learn. These are independent dials, and the quality of a model depends on getting the ratio right, not on maxing either one.

The intuition: capacity you never fill is dead weight. A huge model shown too little data ends up with billions of parameters sitting near their random starting values, because there was never enough signal to nudge them into useful configurations. Conversely, a small model shown enormous amounts of data eventually saturates — it can't store any more of what it's seeing — and further data yields diminishing returns. Somewhere between those failure modes is a compute-optimal balance: for a fixed training budget, there's a sweet spot for how big the model should be versus how many tokens it should see.

This is the conceptual heart of what's often called the Chinchilla finding: for a long time the field built ever-larger models but under-fed them, and it turned out that, for a given amount of training compute, you often do better with a smaller model trained on far more tokens than with a giant model trained briefly. The headline lesson — stated conceptually, without leaning on specific ratios that vary by setup — is that many famous large models of the past were under-trained: their parameter counts were writing cheques their training data didn't cash. A smaller, well-fed model could match or beat them.

There's a twist that matters for anyone actually deploying models, though. Compute-optimal is about the training budget. But most models are trained once and then run millions of times, so a lot of the industry deliberately trains past the compute-optimal point — feeding a smaller model even more data than "optimal" — because a smaller model that's expensive to train but cheap to run is a great trade when you're serving it forever. This is exactly why today's compact models are so strong: they've been trained on far more tokens per parameter than older giants, front-loading training cost to buy cheap, capable inference. The takeaway for reading spec sheets: a parameter count with no sense of how much data went in tells you almost nothing about quality. "7B" from a lavishly-trained recent model and "7B" from an under-trained older one are the same size and not remotely the same product.

Why bigger isn't automatically better

Here's the skeptical core. Parameter count measures capacity — how much the model could store — the way a bookshelf's size measures how many books it could hold. An empty ten-metre shelf holds nothing useful. What fills the shelf is training, and the count says nothing about how well that went.

Concretely, several things beat raw size:

Data quality. A model trained on carefully curated, deduplicated, high-quality text learns more per parameter than one trained on scraped sludge. Garbage in, garbage weights.
Training compute and duration. A large model trained too briefly is under-baked; its extra parameters sit half-random. For years the field under-trained big models until researchers showed that smaller models fed far more data often win. Capacity you don't fill is wasted.
Architecture. How the parameters are wired — attention patterns, layer design — changes how efficiently they're used. See how transformers work for what that wiring does.
Post-training. The raw pretrained pile of numbers is not the helpful assistant you talk to. Fine-tuning and preference alignment reshape a fraction of the weights and dramatically change usefulness — with zero change to the count.

The empirical reality is that a well-executed 30B model routinely outperforms a sloppy 70B one, and today's small models beat the giant models of a few years ago despite a fraction of the parameters. The count tells you the ceiling of what's possible, not what was achieved. Treat "N billion parameters" as marketing shorthand for "roughly this size class," never as a quality score. If you're actually choosing a model, benchmarks and hands-on testing matter far more than the headline number — which is the whole point of how to choose an LLM for your app and which AI chatbot.

Dense vs. sparse: total vs. active parameters

Everything so far quietly assumed a dense model — one where every parameter participates in processing every token. In a dense model, the parameter count means one clean thing: this many numbers are multiplied for every token, so the count sets both memory and per-token compute together. Most models you've used are dense, and for them the simple story holds.

But there's a second family: sparse models, where only a fraction of the parameters activate for any given input. "Sparse" here means the activation is sparse — the parameters are all present and stored, but most of them sit idle on any particular token. This single design choice is what splits the once-unified parameter number into two different numbers that answer two different questions, and it's the reason you increasingly see models quoted as, for example, "a 100B model with 12B active."

The distinction matters because dense and sparse models with the same headline count behave nothing alike on cost. A dense 100B model does 100B parameters' worth of arithmetic per token. A sparse 100B model that activates 12B does roughly 12B parameters' worth of arithmetic per token — cheaper to run — while still needing enough memory to hold all 100B in case any of them is needed. You get much of the storage-capacity benefit of a large model at a fraction of the per-token compute. The dominant way to build sparse models is Mixture-of-Experts, which is worth understanding in its own right.

Total vs. active parameters (the MoE twist)

One more wrinkle that breaks the simple story. Some models use a Mixture-of-Experts (MoE) design — the main sparse architecture in practice — where the parameters are split into many "expert" sub-networks, and only a few experts activate for any given token. A small router network learns which experts to send each token to. Such a model might quote a huge total parameter count but only use a small slice per token. The full mechanics, and why serving MoE models is its own engineering challenge, are covered in the complete guide to mixture of experts.

For example, a model might advertise 400 billion total parameters but activate only ~40 billion for each token it processes. This decouples the two things the count used to tell you at once:

Total parameters still set the memory footprint — you have to load all the experts, even the idle ones.
Active parameters set the compute cost per token — you only pay arithmetic for the experts that fire.

So an MoE model can be cheap to run (few active parameters) while being heavy to store (many total). When you see a headline number for a big model, it's worth asking whether it's total or active, because they answer different questions. This is a deliberate architectural trick to get the storage-capacity benefits of size without paying full compute on every token — and it's a clean demonstration that "parameter count" was always a bundle of two separate ideas: how much the model holds, and how much work it does.

Reading a model card like a pro

All of this becomes practical the moment you look at a real model listing. A "model card" — the page a provider publishes for a model on a hub or in its docs — is dense with numbers that now decode cleanly. Here's how to read one skeptically, in the order the numbers should trigger questions.

Start with the parameter count, but immediately ask "dense or sparse?" A dense "8B" and a sparse "8x7B" (an MoE with eight experts) are wildly different objects: the second has far more total parameters but activates only a couple of experts per token. If the card quotes both a total and an "active" figure, it's an MoE — read the active number for compute cost and the total number for memory.

Then find the precision and format. Look for whether weights are published in 16-bit, 8-bit, or a quantized 4-bit format, and in what file format. This, not the parameter count, tells you the download size and the memory you need to load it. A card that lists multiple quantized variants is handing you the memory-versus-quality dial directly.

Look for the training data scale, if disclosed. A stated token count (or even a vague "trained on trillions of tokens") tells you whether the model is well-fed for its size — the parameters-versus-tokens question from earlier. Many cards omit this; its absence is itself informative.

Check the context window separately. This governs KV cache memory and has nothing to do with parameter count. A small model with a very long context window can need surprising amounts of runtime memory. Understand what a context window is before you assume a small model is automatically light to run.

Distinguish base from instruct/chat. A "base" model is the raw pretrained pile of numbers; an "instruct" or "chat" variant has been post-trained to be a helpful assistant. Same parameter count, very different usability — the difference is in how a fraction of the weights were reshaped, not in how many there are.

Only then look at benchmarks — and distrust them a little. Numbers on standard tests are gameable and often don't reflect your actual use. Treat them as a coarse filter, then test the model on your own tasks. This is the entire thesis of how to choose an LLM for your app and, for everyday users, which AI chatbot you should use: the headline parameter count is where you start narrowing, never where you decide.

Read a card this way and the parameter count takes its proper place — one field among several, useful for sizing and budgeting, silent on quality. That's the whole skill: knowing which questions the number answers and which it doesn't.

FAQ

Q: What exactly is one parameter in an AI model? A parameter is a single learned number — a plain decimal value like 0.34 or -2.1 — that the model adjusted during training and then froze. Most parameters are "weights," which multiply values as data flows through the network. A model with 70 billion parameters literally contains 70 billion such numbers. No single one means anything alone; knowledge emerges from running data through all of them together.

Q: Are "weights" and "parameters" the same thing? Nearly. "Parameters" is the umbrella term for all learned numbers, which includes weights (multipliers) and biases (offsets). Since weights vastly outnumber biases, people use the two words interchangeably in casual conversation. When someone says "the model's weights," they usually mean the entire set of learned numbers — the actual downloadable content of the model.

Q: Does more parameters mean a smarter model? No. Parameter count measures capacity — how much the model could store — not how well it was trained. Data quality, training compute, architecture, and post-training often matter more. A well-trained smaller model routinely beats a poorly-trained larger one, and modern small models outperform the giants of a few years ago. Use benchmarks and real testing to judge quality, not the headline number.

Q: How do I know if I can run a model on my hardware? Estimate the memory: multiply the parameter count by the bytes per parameter (2 bytes at 16-bit, 0.5 byte at 4-bit), then add headroom for the context window and activations. A 7B model at 4-bit needs roughly 4-5 GB and runs on many laptops; a 70B model at 16-bit needs ~140 GB and requires multiple GPUs or cloud hardware. The parameter count is your first sanity check.

Q: What's the difference between a 16-bit and a 4-bit version of the same model? Same number of parameters, fewer bytes each. Quantization stores each weight at lower precision — essentially rounding the numbers — so the file shrinks (a 70B model drops from ~140 GB to ~35 GB) while keeping all 70 billion parameters. Quality degrades gradually as compression increases, but modern 4-bit methods are good enough that everyday use barely suffers.

Q: Why do some models quote a huge parameter count but claim to be cheap to run? Those are usually Mixture-of-Experts models. They split parameters into many expert sub-networks and activate only a few per token. The large total count sets the memory needed to load the model, but the small active count sets the compute per token. So the model is heavy to store but light to run — total and active parameters answer different questions.

Q: What's the difference between parameters and training tokens? Parameters are the model's capacity — how many learned numbers it has to store patterns in. Training tokens are how much text it was shown while learning. They're independent: a big model fed too little data leaves much of its capacity unused, while a small model fed enormous data eventually saturates. Quality depends on balancing the two, not maxing either. This is why two models with the same parameter count can differ enormously — one may have been trained on far more data per parameter than the other.

Q: Can I delete or edit a specific fact by changing a model's weights? Not cleanly. Facts aren't stored in individual weights; they're distributed across many, entangled with everything else the model knows. Zeroing out a weight doesn't remove a fact — it slightly perturbs countless behaviours at once. Editing what a model knows is an active, unsolved research area precisely because knowledge is smeared across the parameters rather than filed in tidy rows. In practice, people steer behaviour through fine-tuning, prompting, or retrieval, not by hand-editing weights.

Q: My model file fits in my GPU memory, so why do I still run out of memory? Because parameters aren't the only thing using memory at runtime. Loading the weights is just the baseline; actually running the model also needs memory for activations (the intermediate numbers of each forward pass) and, critically, the KV cache, which grows with how long your conversation or document is. On long inputs the KV cache can rival the size of the weights. Always budget headroom above the raw weight figure — a model that "fits" empty can overflow mid-generation on a long prompt.

The bottom line

"N billion parameters" is the AI industry's favourite unit, and now you know what it measures: the quantity of learned numbers packed into the model, which sets its memory footprint and roughly its running cost. It's a real, useful figure — you need it to size hardware and budget compute. But it's a measure of how much machinery, not how good the output is. Capacity is not achievement. A model earns its quality from data, training, architecture, and alignment; the parameter count just tells you how big the container was. Read it as a size label, quote it as a size label, and reach for benchmarks and hands-on testing when you actually want to know whether a model is any good.

Table of contents