How to Run an AI Model on Your Own Computer
A practical guide to running open-weight AI models locally using Ollama or LM Studio — no API key, no usage limits, and your data stays on your machine.

Running an AI model locally means it lives on your own computer. It reads your questions and writes its answers there, without sending anything to another company's servers. No API key, no usage limits, and no data leaving your device.
This has become much easier in the last two years. A mid-range laptop can now comfortably run a capable model with 7 or 8 billion parameters, and getting started can take just a command or two.
This guide covers the two most popular tools: Ollama, which you use from the command line, and LM Studio, a normal desktop app. It also explains how to pick a model that suits your computer.
When running a model yourself makes sense
It isn't always the best choice. Online assistants are quicker to start with, run much bigger models and need no setup. Running your own model makes sense when:
- Privacy matters. If you work with client data, sensitive documents or anything under a confidentiality agreement, your questions never leave your machine.
- You need to work offline. Once the model is downloaded, you don't need an internet connection.
- You don't want usage limits. You can send as many requests as your computer can handle, with no per-use charges.
- You want to experiment. You can try specialised versions of models, switch between them instantly and compare several on the same question.
If none of those apply and you mainly want a good assistant for everyday tasks, an online tool is almost certainly the more practical choice.
What you need: hardware
The main limit is memory (RAM), or your graphics card's own memory (VRAM) if you have a separate graphics card. The model has to fit in memory. If it doesn't, it'll be very slow or won't load at all.
Memory guide:
- 8 GB RAM: enough for a compressed model with 3–4 billion parameters. Usable, but replies may feel slow.
- 16 GB RAM: comfortable for 7–8 billion parameter models. This is the most common starting point, and a good 8-billion-parameter model handles most writing, coding and everyday questions well.
- 32 GB RAM or more: opens up models with 13–30 billion parameters, which give noticeably better answers on complex tasks.
Graphics card (optional, but faster): A separate graphics card with 8 GB or more of its own memory speeds things up a lot. Ollama and LM Studio use an NVIDIA card automatically. Macs with Apple's M-series chips share one pool of memory between the processor and graphics, so most of a 16 GB Mac's memory can go to the model. That makes these Macs particularly good for running models.
Without a graphics card, models still run, just more slowly. On a modern processor alone, a 7-billion-parameter model typically writes a few words per second. That's fine for jobs you leave running, but slow for chatting.
Storage: Model files range from about 2 GB for small compressed models to 40 GB or more for large ones. Make sure you have space before downloading.
Route 1: Ollama (command line)
Ollama is the most widely used tool for running models on your own computer. It downloads and starts models for you, and lets other apps on your computer use them too.
Step 1: Install Ollama
Download Ollama from ollama.com (opens in a new tab) for macOS, Windows or Linux. Open the installer and follow the prompts. On Linux, the install script is:
curl -fsSL https://ollama.com/install.sh | shOnce installed, Ollama runs as a background service. On Linux, if it isn't already running, start it with:
ollama serveStep 2: Download and run a model
Pick a model from Ollama's model library (opens in a new tab). A good starting point on 16 GB of RAM is llama3.2 (a 3B model, fast and lightweight) or llama3.1:8b (more capable). Run:
ollama run llama3.2Ollama downloads the model the first time and drops you into a chat. Type your message and press Enter. Type /bye to exit.
Step 3: Check it worked
If you see a >>> prompt and the model replies to your first message, it's running. You can check which models you've downloaded with:
ollama listStep 4: Use Ollama from other apps
Other apps on your computer can talk to Ollama at the address http://localhost:11434. Many apps can use it this way, including Open WebUI (a chat window that runs in your browser), VS Code extensions and coding agents. In Open WebUI, for example, you just enter that address in the settings.
Troubleshooting:
- "Error: model not found": check the exact model name on ollama.com/library. Names are case-sensitive.
- Slow replies: the model may be running without your graphics card. If you have one, make sure its drivers are up to date.
- Out of memory errors: try a smaller model, or a more compressed version of the same one (for example
llama3.1:8b-instruct-q4_K_M).
Route 2: LM Studio (desktop app)
LM Studio is a normal desktop app for downloading and running models, with no command line needed. It can also let your own code use the model, in the same format as OpenAI's API.
Step 1: Install LM Studio
Download from lmstudio.ai (opens in a new tab) and install it. On a Mac it needs Apple Silicon (an M-series chip); check the download page for the current system requirements on Windows and Linux.
Step 2: Download a model
Open LM Studio and go to the Discover tab (the magnifying glass icon). Search for a model by name: llama, gemma, phi and qwen are all good starting points. Click the download icon next to one. Versions marked Q4_K_M are a good balance of size, speed and quality for most computers.
Step 3: Load the model and chat
Go to the Chat tab, pick your model from the menu at the top and start chatting. LM Studio shows how fast the model is writing in the status bar at the bottom.
Step 4: Check it worked
If the model loads and replies in the Chat tab, it's working. If you write code, you can also start LM Studio's local server from the Developer tab and use the model from your own programs.
Troubleshooting:
- Model won't load: check how much memory is free. If the model (its size is shown in the Discover tab) is bigger than your free memory, it won't load. Try a smaller or more compressed version.
- Not using your graphics card: when loading the model, raise the setting that controls how much of it goes on the graphics card, and make sure your graphics drivers are up to date.
- Very slow on a Mac: pick a version marked MLX if there is one. These are built for Apple's M-series chips.
Picking a model for your hardware
| Memory | Model size (parameters) | Example models |
|---|---|---|
| 8 GB | 3–4 billion | Llama 3.2 3B, Phi-4-mini, Gemma 3 4B |
| 16 GB | 7–8 billion | Llama 3.1 8B, Mistral 7B, Qwen2.5 7B |
| 32 GB | 13–14 billion | Qwen2.5 14B, Phi-4 (14B) |
| 64 GB+ | 30–70 billion | Llama 3.3 70B (compressed) |
New models come out often, so check the Ollama model library (opens in a new tab) or LM Studio's Discover tab for what's current. Q4_K_M is a safe default. It's a common level of compression, or quantization: the file is roughly a third the size of the original model, with only a small loss in quality.
What models on your computer can't do
These are general-purpose models. They can write, summarise, answer questions and write code. They can't:
- Look things up online or know about recent events, unless you connect a tool that fetches information
- Read your files, unless you paste them in or connect an app that reads them
- Match the biggest online models from OpenAI, Anthropic and Google on the hardest reasoning tasks, though the gap has narrowed a lot
When you need the very best answers, an online model is still the better choice. For private work, working offline or processing lots of documents, running your own model is now genuinely practical.
For more tools that run on your own computer, see the best local AI tools and the open-source vs closed AI models comparison.

Stephen Gbolagade
Software Engineer and Technical Writer at Ploba
Writes Ploba's guides to AI tools, agents and protocols, explaining how they work in plain language.
Keep learning
Plain-language explanations, technical guides and practical comparisons.
