Local LLM guides
Understand the basics, fix a frustrating problem, or go deeper into the technical details. Start with the memory guide below.
How much VRAM do you actually need?
Calculate model weights, KV cache and runtime overhead before choosing hardware for local AI.
Read guideSolve a local LLM problem
Why is my local LLM so slow?
Troubleshoot slow Ollama, LM Studio and llama.cpp inference. Separate loading, prompt processing and generation before changing hardware.
Why your local LLM runs out of memory in longer chats
Fix local LLM out-of-memory errors as context grows. Budget the KV cache, reduce concurrency and test cache quantization without losing key information.
Smaller Q8 or larger Q4: which local model should you run?
Choose between a smaller high-precision LLM and a larger quantized model with a practical test for quality, memory, context and response time.
Why the same model gives different answers across local AI apps
Compare Ollama, LM Studio and llama.cpp fairly. Check the model artifact, chat template, sampling, thinking mode and context before blaming the model.
Local RAG giving bad answers? Check retrieval first.
Fix local document Q&A with a practical RAG checklist: PDF extraction, chunks, embeddings, hybrid search, reranking and a small evaluation set.
Understand the fundamentals
What is quantization?
Understand Q4, Q5, Q8 and FP16, and how lower precision trades model size for possible quality changes.
How context uses memory
See how context length, the KV cache and concurrent requests affect local LLM memory requirements.
Mac or GPU workstation for local AI?
Compare unified memory and discrete GPU memory, runtime compatibility and upgrade paths for local models.
Two GPUs do not become one bigger GPU.
Understand the conditions under which multiple GPUs can run a larger local model, and what combined VRAM leaves out.
Buying a used RTX 3090
A practical checklist for assessing a used 24 GB GPU for local AI, including memory, thermals, power and seller evidence.