Ploba logo

Find the right AI tool for the job: apps, agents, APIs, MCP servers and datasets, explained plainly.

Get the AI Digest

Discover

  • Explore
  • Top Charts
  • Categories
  • Agents
  • MCP servers
  • Developer tools

Learn & community

  • Learn AI
  • AI Glossary
  • Blog
  • Hackathons
  • People
  • AI Digest

For makers

  • List your tool
  • Docs
  • Claim a listing
  • Advertise
  • Write for us

Company

  • About
  • Careers
  • Changelog
  • Trust
  • Security
  • Contact

© 2026 Ploba. All rights reserved.

  • Privacy
  • Terms
  • Cookies
  • Advertising
  • All policies
llms.txt
All articles
Technical Tutorials

How to Run an AI Model on Your Own Computer

A practical guide to running open-weight AI models locally using Ollama or LM Studio — no API key, no usage limits, and your data stays on your machine.

Stephen Gbolagade

Nov 17, 2026 · 6 min read

Summarize with
ChatGPTClaudePerplexity

Programmer working at a desk with a laptop and external monitor
Image: Photo by Cláudio Emanuel on Pexels

Running an AI model locally means it lives on your own computer. It reads your questions and writes its answers there, without sending anything to another company's servers. No API key, no usage limits, and no data leaving your device.

This has become much easier in the last two years. A mid-range laptop can now comfortably run a capable model with 7 or 8 billion parameters, and getting started can take just a command or two.

This guide covers the two most popular tools: Ollama, which you use from the command line, and LM Studio, a normal desktop app. It also explains how to pick a model that suits your computer.

When running a model yourself makes sense

It isn't always the best choice. Online assistants are quicker to start with, run much bigger models and need no setup. Running your own model makes sense when:

  • Privacy matters. If you work with client data, sensitive documents or anything under a confidentiality agreement, your questions never leave your machine.
  • You need to work offline. Once the model is downloaded, you don't need an internet connection.
  • You don't want usage limits. You can send as many requests as your computer can handle, with no per-use charges.
  • You want to experiment. You can try specialised versions of models, switch between them instantly and compare several on the same question.

If none of those apply and you mainly want a good assistant for everyday tasks, an online tool is almost certainly the more practical choice.

What you need: hardware

The main limit is memory (RAM), or your graphics card's own memory (VRAM) if you have a separate graphics card. The model has to fit in memory. If it doesn't, it'll be very slow or won't load at all.

Memory guide:

  • 8 GB RAM: enough for a compressed model with 3–4 billion parameters. Usable, but replies may feel slow.
  • 16 GB RAM: comfortable for 7–8 billion parameter models. This is the most common starting point, and a good 8-billion-parameter model handles most writing, coding and everyday questions well.
  • 32 GB RAM or more: opens up models with 13–30 billion parameters, which give noticeably better answers on complex tasks.

Graphics card (optional, but faster): A separate graphics card with 8 GB or more of its own memory speeds things up a lot. Ollama and LM Studio use an NVIDIA card automatically. Macs with Apple's M-series chips share one pool of memory between the processor and graphics, so most of a 16 GB Mac's memory can go to the model. That makes these Macs particularly good for running models.

Without a graphics card, models still run, just more slowly. On a modern processor alone, a 7-billion-parameter model typically writes a few words per second. That's fine for jobs you leave running, but slow for chatting.

Storage: Model files range from about 2 GB for small compressed models to 40 GB or more for large ones. Make sure you have space before downloading.

Route 1: Ollama (command line)

Ollama is the most widely used tool for running models on your own computer. It downloads and starts models for you, and lets other apps on your computer use them too.

Step 1: Install Ollama

Download Ollama from ollama.com (opens in a new tab) for macOS, Windows or Linux. Open the installer and follow the prompts. On Linux, the install script is:

Shell
curl -fsSL https://ollama.com/install.sh | sh

Once installed, Ollama runs as a background service. On Linux, if it isn't already running, start it with:

Shell
ollama serve

Step 2: Download and run a model

Pick a model from Ollama's model library (opens in a new tab). A good starting point on 16 GB of RAM is llama3.2 (a 3B model, fast and lightweight) or llama3.1:8b (more capable). Run:

Shell
ollama run llama3.2

Ollama downloads the model the first time and drops you into a chat. Type your message and press Enter. Type /bye to exit.

Step 3: Check it worked

If you see a >>> prompt and the model replies to your first message, it's running. You can check which models you've downloaded with:

Shell
ollama list

Step 4: Use Ollama from other apps

Other apps on your computer can talk to Ollama at the address http://localhost:11434. Many apps can use it this way, including Open WebUI (a chat window that runs in your browser), VS Code extensions and coding agents. In Open WebUI, for example, you just enter that address in the settings.

Troubleshooting:

  • "Error: model not found": check the exact model name on ollama.com/library. Names are case-sensitive.
  • Slow replies: the model may be running without your graphics card. If you have one, make sure its drivers are up to date.
  • Out of memory errors: try a smaller model, or a more compressed version of the same one (for example llama3.1:8b-instruct-q4_K_M).

Route 2: LM Studio (desktop app)

LM Studio is a normal desktop app for downloading and running models, with no command line needed. It can also let your own code use the model, in the same format as OpenAI's API.

Step 1: Install LM Studio

Download from lmstudio.ai (opens in a new tab) and install it. On a Mac it needs Apple Silicon (an M-series chip); check the download page for the current system requirements on Windows and Linux.

Step 2: Download a model

Open LM Studio and go to the Discover tab (the magnifying glass icon). Search for a model by name: llama, gemma, phi and qwen are all good starting points. Click the download icon next to one. Versions marked Q4_K_M are a good balance of size, speed and quality for most computers.

Step 3: Load the model and chat

Go to the Chat tab, pick your model from the menu at the top and start chatting. LM Studio shows how fast the model is writing in the status bar at the bottom.

Step 4: Check it worked

If the model loads and replies in the Chat tab, it's working. If you write code, you can also start LM Studio's local server from the Developer tab and use the model from your own programs.

Troubleshooting:

  • Model won't load: check how much memory is free. If the model (its size is shown in the Discover tab) is bigger than your free memory, it won't load. Try a smaller or more compressed version.
  • Not using your graphics card: when loading the model, raise the setting that controls how much of it goes on the graphics card, and make sure your graphics drivers are up to date.
  • Very slow on a Mac: pick a version marked MLX if there is one. These are built for Apple's M-series chips.

Picking a model for your hardware

Scroll sideways to see the whole table.

MemoryModel size (parameters)Example models
8 GB3–4 billionLlama 3.2 3B, Phi-4-mini, Gemma 3 4B
16 GB7–8 billionLlama 3.1 8B, Mistral 7B, Qwen2.5 7B
32 GB13–14 billionQwen2.5 14B, Phi-4 (14B)
64 GB+30–70 billionLlama 3.3 70B (compressed)

New models come out often, so check the Ollama model library (opens in a new tab) or LM Studio's Discover tab for what's current. Q4_K_M is a safe default. It's a common level of compression, or quantization: the file is roughly a third the size of the original model, with only a small loss in quality.

What models on your computer can't do

These are general-purpose models. They can write, summarise, answer questions and write code. They can't:

  • Look things up online or know about recent events, unless you connect a tool that fetches information
  • Read your files, unless you paste them in or connect an app that reads them
  • Match the biggest online models from OpenAI, Anthropic and Google on the hardest reasoning tasks, though the gap has narrowed a lot

When you need the very best answers, an online model is still the better choice. For private work, working offline or processing lots of documents, running your own model is now genuinely practical.

For more tools that run on your own computer, see the best local AI tools and the open-source vs closed AI models comparison.

About the author

Stephen Gbolagade

Software Engineer and Technical Writer at Ploba

Writes Ploba's guides to AI tools, agents and protocols, explaining how they work in plain language.

More from Stephen

Keep learning

Plain-language explanations, technical guides and practical comparisons.

More articlesAI terms glossary

On this page

  • When running a model yourself makes sense
  • What you need: hardware
  • Route 1: Ollama (command line)
  • Route 2: LM Studio (desktop app)
  • Picking a model for your hardware
  • What models on your computer can't do

More in Technical Tutorials

See all
  • Ethernet cables plugged into the ports of a network switchTechnical Tutorials · Nov 6, 2026How to Add an MCP Server to Claude, Cursor and VS CodeStep-by-step setup for MCP servers in Claude Desktop, Claude Code, Cursor and VS Code, with config examples, checks and fixes for common errors.