Every laptop now advertises local AI, and every model page brags about how well it runs on consumer hardware, but almost none of them tell you the two numbers that actually decide whether a model runs on your machine: how much memory it needs, and how fast it will produce text. The good news is you do not need to guess, and you certainly do not need to download twenty gigabytes just to find out it stalls. A handful of simple formulas turn a model’s size, its quantization, and your context length into a memory figure, and a second formula turns your machine’s memory bandwidth into a realistic tokens-per-second range. This guide walks through those formulas, hands you the spreadsheet columns to build, and shows worked examples across real 2026 laptops, so you can size a model before you commit the download.

First, What Actually Fits in Your Memory: Running a model locally comes down to two questions in order. Does it fit, and if it fits, is it fast enough to be worth using? Fit is memory. A model’s weights take up roughly its parameter count times the bytes each parameter uses after quantization, plus a working buffer for the context window, and that total has to sit inside your VRAM on a discrete GPU or your unified memory on Apple Silicon. Speed is bandwidth. Because generating each token means reading the whole model out of memory, your rough tokens-per-second is your memory bandwidth divided by the size the model occupies. Get those two estimates first, and every “will it run” question answers itself before you install anything.

The honest reason people end up disappointed with local models is that “can my laptop run it” has two very different failure modes, and marketing only ever addresses one. A model can technically load and still be useless because it crawls at one token per second, or it can be plenty fast but silently spill into slow system memory the moment you paste a long document. So the useful question is not “will it run” but “will it fit, and will it stay fast at the context length I actually use.” Community threads make this concrete: people report 24GB cards running a 27B model beautifully until they hit a real coding task and run out of context mid-session. If you are still deciding whether to run anything locally at all, our local LLM versus cloud AI checklist is the step before this one. Once you have decided to try local, the rest of this guide is about predicting both fit and speed with arithmetic instead of trial and error.

Start with the weights, because they are the biggest and most predictable cost. A model’s weight memory is close to its parameter count multiplied by the bytes each parameter takes after quantization. A 7-billion-parameter model at 4-bit quantization uses roughly four gigabytes for weights, an 8B lands near five, and a 70B at the same 4-bit setting needs somewhere around forty. On top of the weights you add a working buffer for activations and overhead, commonly a gigabyte or two, plus the context memory covered in a moment. A widely used shortcut is that one billion parameters costs about 0.6 gigabytes at 4-bit, which lets you eyeball any model in your head. The tooling most local runners sit on, whether that is Ollama or LM Studio, is built on the llama.cpp engine, and it reports these memory figures when a model loads, so you can check your estimate against reality.

Quantization is the single biggest lever you control, because it sets the bytes-per-parameter in that formula. Full 16-bit weights use two bytes per parameter, 8-bit uses about one, and the popular 4-bit formats use roughly half a byte, with the common Q4_K_M variant landing near 0.57 bytes and Q5_K_M near 0.68. In plain terms, moving a 13B model from 16-bit to 4-bit takes it from around twenty-six gigabytes to under eight, which is the difference between needing a workstation and fitting on a laptop. The quality cost of 4-bit is real but usually modest for everyday use, and it is the setting most people run day to day. These weights ship in the GGUF format that local tools expect, and Hugging Face documents how GGUF files package a quantized model, including how the quantization type is named in the filename so you know what you are downloading.

Here is the cost that ambushes people. The context window, the amount of text the model holds in view, is stored in a key-value cache that grows with both the model and the length of the conversation, and it sits in the same memory as the weights. For a 7B model, a 4,000-token context adds only a few hundred megabytes, but a 32,000-token context can add a couple of gigabytes, and a 128,000-token context can demand ten gigabytes or more on its own. That is why a model that loads fine can fall over the moment you feed it a long file: the weights fit, but the weights plus a large context do not. When you build your estimate, treat context as a real line item, not a rounding error, and size it for the longest prompt you actually plan to use. Hugging Face’s guide to optimizing LLM inference explains why the KV cache grows the way it does and what reduces it.

Once a model fits, speed is what decides whether you keep using it, and the surprise is that raw processing power barely matters for text generation. Producing each token requires reading the entire active model out of memory, so your ceiling is memory bandwidth divided by the size the model occupies. A useful back-of-envelope version circulating in local AI communities: typical DDR5 system memory moves around eighty gigabytes per second, so an 80B model quantized to 8-bit running from system RAM tops out near one token per second, because eighty divided by eighty is one. Graphics memory is far faster, from a couple hundred gigabytes per second on modest cards to well over a thousand on high-end ones, which is exactly why a model that fits entirely in VRAM feels quick while the same model spilling into system RAM feels broken. A 70B model kept fully in a large GPU can generate thirty tokens a second or more, while the same model offloading part of itself to the processor and system memory drops to single digits.

Now assemble the estimate. Build one row per model you are considering, with these columns, and the last three tell you everything.

If “Fits?” says no, either raise the quantization to a smaller format, shorten your context, or pick a smaller model. If it fits but the speed estimate is low, the model is spilling out of fast memory, and the fix is the same set of levers.

The formulas get real when you drop in actual machines. Here is how a few common 2026 setups shake out, using the arithmetic above and the numbers people report running these configurations.

If those numbers are steering a hardware purchase, our checklist for buying an NPU laptop and our Copilot+ PC versus regular laptop guide cover which specs matter, and our business laptop features checklist lists the rest of what to weigh beyond memory.

One class of model breaks the tidy rules above in your favor, and it is worth knowing because it changes what a modest laptop can do. Mixture-of-experts models have a large total parameter count but only activate a small fraction of it per token. A 30B mixture model might use only about three billion parameters for any given token, so while you still need the memory to hold the whole thing, the speed formula uses the small active size, not the large total. In practice people report a 30B mixture model running at thirty tokens a second or more on a machine with only eight gigabytes of VRAM helped by system RAM, and someone even reported a 35B model reaching nearly eighteen tokens per second on a plain processor with no graphics card in the loop. If your memory is generous but your bandwidth is ordinary, a mixture model is often the way to run something genuinely capable at usable speed. Just remember the fit rule still applies: you need room for the full parameter count, even though only a slice runs each step.

The last column of your mental spreadsheet is the one the specs never mention: does it actually run without a connection. The honest answer is yes, with one asterisk. Once the model file is downloaded, a local runner like LM Studio, Ollama, or a raw llama.cpp build executes entirely on your machine with the network switched off, which is the whole point for privacy and travel. The asterisk is that anything wrapped around the model may still reach out: a browser front end, a plugin that fetches web pages, a voice layer, or an app that phones home for updates. So whether something is truly offline depends on your whole setup, not the model alone. If your reason for going local is keeping data on the device, verify it by pulling the network cable and watching the model still answer, and read our note on local versus cloud AI and writing privacy before you trust any tool with sensitive text. A model that fits, runs fast, and truly stays on the device is the trifecta worth aiming for.

For a solid general-purpose model, 8GB of VRAM comfortably runs 7B to 8B models at 4-bit, 16GB opens up 13B to 14B, and 24GB reaches into the 27B to 32B range. Below 8GB you are limited to small models or heavy quantization, and long context can push any of these over their limit.

Partly. A model can run from system RAM when it does not fit in VRAM, but system memory bandwidth is far lower, so speed drops sharply, often to a few tokens per second for a large dense model. Extra RAM helps you load bigger models and is genuinely useful for mixture-of-experts models, but it does not match the speed of fitting everything in graphics memory.

Almost always because part of it is running from slow system memory instead of fast graphics memory, or because a long context has ballooned the KV cache. Check whether the whole model plus your context fits inside VRAM or unified memory, and if not, drop to a smaller quantization or a shorter context.

A 4-bit format such as Q4_K_M is the usual starting point, because it roughly halves memory against 8-bit with a modest quality cost. If you have memory to spare and want a little more accuracy, Q5 or Q8 are reasonable steps up. Go below 4-bit only when you must fit a model that otherwise will not.

It can load one if you have enough combined memory, often 48GB or more, but on a typical laptop it will run from system RAM at only a few tokens per second, which most people find too slow for real work. A capable mixture-of-experts model usually gives a far better experience on the same hardware.

• Estimate weight memory as parameters times bytes per parameter: about 0.57 bytes at Q4_K_M, 1.0 at Q8, 2.0 at 16-bit.
• Add a working buffer of a gigabyte or two, then add the KV cache for the longest context you actually use.
• Confirm the total fits inside your VRAM or unified memory before you download anything.
• Estimate speed as memory bandwidth divided by the memory the model occupies, and expect a steep drop when a model spills into system RAM.
• Treat context length as a real memory cost, since a long prompt can add several gigabytes on its own.
• Consider a mixture-of-experts model when your memory is large but your bandwidth is ordinary.
• Verify “offline” by disconnecting the network and checking the model still answers, and watch for wrappers that phone home.

Running AI on your own laptop stopped being exotic in 2026, but the marketing still hides the two numbers that decide whether it is a good experience or a frustrating one. Weight memory plus context has to fit, and memory bandwidth divided by that footprint has to clear the speed you can tolerate. Put those into a few spreadsheet cells, drop in the model you are eyeing and the machine you own, and you will know the answer before the download bar finishes, which is a far better use of an afternoon than watching a model load only to crawl. Size it first, then run it, and local AI becomes something you plan rather than something you gamble on.