The model is the learned set of numbers that produces answers. A model file stores those numbers; a runtime is the app that loads the file and runs it. Start with a task you can test, such as summarizing a document or explaining some code.
What actually runs on your computer?
Local AI means the model does its computation on hardware you control. A downloaded model can produce text without sending each prompt to a hosted model service. Downloading files and using connected features may still involve the internet; check the app and the model you selected.
There are three separate pieces: your computer, a model file, and a runtime. Think of the file as the learned material and the runtime as the program that uses it. A chat app may bundle a runtime and add a conversation interface. Downloading a file alone does not install a chat app.
A language model generates a response a token at a time using patterns learned during training and the material in its current context. It can produce helpful explanations and plausible mistakes. Start with a small task whose answer you can judge yourself.
Read a model name without guessing
In a name such as “8B Instruct Q4”, 8B is roughly eight billion parameters, Instruct usually identifies a model adapted to follow instructions, and Q4 describes a weight precision. None of these labels alone proves quality or compatibility. The publisher’s model card describes intended uses, limitations and licensing.
Some mixture-of-experts models report both stored and active parameters. The active number describes the subset used for parts of the computation. An all-on-device memory plan still needs the stored weights. This is why a small active count can accompany a large download.
Different files for the same model may use different precision or packaging. GGUF is a file format used by llama.cpp and apps built around it. MLX is a framework with a separate model ecosystem. Choose a download your app explicitly supports, then record the exact file name.
Make a model shortlist
Use the model directory to shortlist candidates, then read their publisher records. Choose for a task you can test rather than a size label alone.
- Write one task and a way to check it: for example, extract three dates from a short document and compare them with the original.
- Find an instruction-following model suited to that task. Check the stored size, license and supported inputs.
- Check that your chosen runtime supports the exact architecture and downloadable format. Then use the calculator to check its memory needs.
Put the idea to work
Open a model page. Find its stored parameters and supported tasks. The B in 9B means billion parameters, not gigabytes or a quality score.
Check your understanding
Does a model with more parameters always give better answers?
Reveal the explanation
No. Training, model design, precision and your task all matter. Compare answers on a few examples you actually care about before choosing a larger model.
Keep this in mind: A model, its downloadable file and the app that runs it are separate choices. Record all three when comparing results.