What is local AI?
Local AI means running AI models on hardware you control, such as a laptop, a desktop or a cluster in your office, instead of sending requests to a cloud provider. The model weights live on your machines, and your prompts and outputs never leave them.
Most local AI today uses open-weight models: models whose trained weights are published so anyone can download and run them. Examples include DeepSeek, Qwen, Kimi, GLM, Llama and OpenAI’s gpt-oss.
You run them with inference software such as exo, Ollama, LM Studio, llama.cpp, MLX or vLLM. Most of these expose an API in the same format as cloud providers, so existing apps and agents can switch with a configuration change.
The trade-offs are simple. You pay for hardware up front instead of per token. You get privacy, predictable cost and no rate limits. You give up the very newest closed models, and you are responsible for the machines.
Sources
Updated