How to Run AI Models Locally: A Beginner's Guide

Most people meet AI through a website: you type into a box, and a distant server does the thinking. Running a model locally flips that around. The model lives on your own computer, so your prompts never leave the machine. This guide explains what that means and the three easiest ways to try it.

What “local LLM” actually means

A large language model (LLM) is the technology behind chatbots. A “local” LLM is simply an open-weight model — one whose files you’re allowed to download — running on your own hardware instead of a company’s cloud. Nothing is streamed to a server. The trade-off is that your laptop, not a data centre, does the work, so smaller models run comfortably while very large ones need a powerful machine.

Why you’d want it

• Privacy. Sensitive notes, client work or personal writing stay on your device. Ollama, for example, states plainly that with local use your data can run entirely offline.
• Offline access. Once a model is downloaded, no internet connection is required.
• No subscription. The core tools below are free to run locally; you pay only in disk space and electricity.

The honest downside: local models are generally smaller and less capable than the biggest cloud services, and speed depends on your hardware. For everyday drafting, summarising and coding help, they are often more than good enough.

Option 1: Ollama (the simplest engine)

Ollama is the most popular way to run open models locally and works on macOS, Windows and Linux. After installing it, you download and chat with a model from a single command, for example:

1. Install Ollama from the official site.
2. In a terminal, run ollama run llama3.2 (or another model name).
3. Wait for the download to finish, then type your prompt.

Ollama is command-line first, which some beginners find bare, but it is also the engine that many friendlier apps quietly rely on.

Option 2: LM Studio (a friendly desktop app)

If a terminal isn’t your thing, LM Studio gives you a proper graphical application for macOS, Windows and Linux. You browse and download models inside the app, then chat with them in a familiar window. It works fully offline once your model is downloaded. It’s the gentlest on-ramp for people who want to click rather than type commands.

Option 3: Raycast (local models in your workflow)

If you already use Raycast on a Mac, version 1.99 added built-in local model support through Ollama. As the Raycast changelog describes, you install Ollama, then enable local models in Raycast’s AI settings — giving access to a wide range of open models directly from the launcher. Note that Raycast flags AI Extensions with local models as experimental, because some tool-calling features aren’t yet fully supported, so treat that part as work-in-progress.

How to choose

• Just want the easiest start? LM Studio.
• Comfortable with a terminal, or want the broadest tool support? Ollama.
• A Raycast user on Mac? Turn on local models there and skip a separate app.

Start with a small model to check that everything runs smoothly, then move up in size if your machine can handle it. Whichever route you pick, you’ll end up with a genuinely private assistant that costs nothing per query and keeps working when the Wi-Fi doesn’t.

Sources: Ollama official site, Raycast v1.99.0 changelog — Local Models