How to run a local model on a Mac without knowing machine learning
Community Discussion · General

How to run a local model on a Mac without knowing machine learning

PM YuanPM YuanOct 62026/10/06 311 views

As someone who doesn't understand machine learning at all, I tried running a local model on a Mac a few days ago. I used Apple's own MLX, not a cloud API.

Whether it's worth using depends on what you want.

If you want to run a model on your own computer with data never leaving the machine, then MLX is currently the most direct path on a Mac. Installation isn't complicated—two or three command-line steps and you get results. But if you want something ready out of the box that you can use with a few mouse clicks, it's not for you; it's a tool for developers, not a product for end users. I work in healthcare, and the point about data not leaving the machine carries weight for me.

From my testing, going from zero to the model spitting out its first sentence took about twenty minutes, half of which was spent waiting for the download.

Why MLX and not Core ML. Both are Apple's, and beginners get tripped up here most easily.

Core ML MLX
What it's for Stuff a trained model into an App so it runs on iPhone, Mac Write, compute, and run models directly on a Mac
Do you need to write code Only App developers touch it Yes, Python or command line
Is it open source The framework itself is Apple proprietary; companion conversion tools are open source Fully open source, MIT license
Does it cost money Free, but you need an Apple developer account Free, no other barriers

My own judgment is that Core ML handles deployment, MLX handles experimentation. If you want to put a model into an App for users, go Core ML. If you want to run an open-source model locally, tweak parameters, and see the results, go MLX. The source material says the same thing—MLX is the most direct path from open-source models to experimenting on Apple silicon.

In Apple's WWDC talk on MLX, there's a line to the effect that they showed how simple it is to do inference, training, and quantization with MLX, via both code and terminal commands. I took the terminal route this time.

Here's the full process I went through.

1. First confirm your Mac's chip. Click the Apple icon in the top-left corner, go to About This Mac, and look at the "Chip" line. If it says M1, M2, M3, etc., it's Apple silicon and can run it. If it's Intel, this article ends here—MLX only supports Apple silicon.

2. Open Terminal. Press Command + Space, type Terminal, and hit Enter. A window appears with a blinking cursor. This is the terminal; you type here and the computer executes.

3. Install MLX's language model toolkit. In the terminal, type pip install mlx-lm and hit Enter. Lots of lines will scroll by on screen—that's normal. If the last line has no red error, it's installed.

4. Download and run a model. Type mlx_lm.generate --model, followed by a model name, then add --prompt "Hello, introduce yourself", and hit Enter.

The first run will download the model weights first, and a progress bar appears on screen with the percentage creeping up. The one I downloaded wasn't small; I waited about ten minutes. After the download finishes, the terminal first prints your question, then spits out the model's generated text line by line.

5. If you see text streaming out, it worked. The model is stored locally, and running the same model again won't require re-downloading.

Now here come the pitfalls.

The first pitfall: pip not found. Typing pip gives a command not found message. The fix is to install Homebrew first, then brew install python, then come back and install mlx-lm. I got stuck on this for a while—the error message looks scary, but it's really just the path not being set up.

The second pitfall: typing the model name wrong. The name after --model is fixed; you can't make one up. If you get it wrong, it'll look for a nonexistent repository and report a 404. My approach is to first find a model name someone else has run in the community, copy it directly, and don't type it by hand.

The third pitfall: memory. The model is loaded into memory to run, and a Mac's unified memory is shared between CPU and GPU.

If you pick a particularly large model, and halfway through the machine starts lagging and the fan spins like crazy, that's insufficient memory. Beginners should pick a smaller model to get it working first; don't jump straight to the biggest one.

MLX's computation is lazy. When you write code, it doesn't necessarily compute immediately—it waits until the result is actually needed. For beginners this means one thing: if the program seems unresponsive, it's not necessarily frozen, it might just be waiting. Don't rush to press Ctrl+C.

Throughout the whole process, downloading is much slower than computing. Once the model starts generating, the speed is smoother than I imagined. The source material mentions it's optimized for unified memory on Apple silicon, and my experience matches that.

Once you've got it working, don't rush to switch to a bigger model. Try rephrasing the same sentence several times and see if the answers are stable; then try a quantized version—the same model compressed smaller—and see how speed and quality trade off. Training and fine-tuning are a separate topic; get the inference step down pat first.

2 replies

?
Ctrl + Enter to reply
Momo-chan

Twenty-some minutes, half of it waiting on downloads — that kills the timeliness of trending topics. Ops folks can't afford to wait.

Mo Mo
Mo MoOct 6
Reply to Momo-chan

The slow download is only the first time. OP said the second run with the same model doesn't need to re-download—do you switch model names every day or something?