Can I run a large language model on my own computer?
Yes. A recent laptop with 16 GB of memory can run small models of about 7 to 14 billion parameters. A desktop with 64 to 128 GB can run models in the 70 to 120 billion range. The largest open models, from about 700 billion to over 2 trillion parameters, need hundreds of gigabytes or more and usually a cluster of machines.
Memory is the main limit. The whole model has to fit in fast memory (GPU memory, or unified memory on a Mac), with room left for the conversation context.
Speed depends mostly on memory bandwidth. A model that fits but runs at two words per second is not useful for daily work, so check both size and speed before you buy.
The quickest start is a desktop app such as exo, LM Studio or Ollama. You pick a model, it downloads, and you can chat with it or call it through a local API.
Sources
Updated