🧠 Your iPhone Is an OpenAI-Compatible LLM Server Now

*TL;DR: [Local LLM Server](https://apps.apple.com/us/app/local-llm-server/id6757007308?ct=kevinmd-202607) turns an iPhone, iPad, or Mac into an LLM server: Apple's on-device Foundation Models exposed over an OpenAI- and Ollama-compatible HTTP API. Any existing client library points at your phone's LAN address and just works. Free with ads; [Pro is $1.99](https://apps.apple.com/us/app/local-llm-server-pro/id6758336829?ct=kevinmd-202607) without them.*

Apple Intelligence ships a ~3B-parameter language model on every recent iPhone, running locally on the neural engine. It's genuinely capable for structured tasks—classification, extraction, summaries—and it costs nothing per token. The catch: Apple exposes it as a Swift framework. If your tooling speaks HTTP—Python scripts, LangChain, DSPy, a curl one-liner, literally the entire LLM ecosystem—there's no door in.

Local LLM Server is that door. Open the app, pick a port, hit Start Server, and the device serves at `http://<your-lan-ip>:8080` with the two API shapes everything already understands: OpenAI's (`/v1/chat/completions` and friends) and Ollama's. Your OpenAI client library doesn't know it's talking to a phone.

<img src="/images/local-llm-server-running-apple-intelligence.png" alt="Local LLM Server running on an iPhone with a Powered by Apple Intelligence label and a LAN address" width="300" />

*The phone is the server.*

## What "compatible" actually covers

The parts I'm most pleased with, because they're the parts most wrappers skip:

- **Structured outputs** — `response_format` with a JSON schema drives Apple's guided generation, so you get valid JSON back, not JSON-shaped prose.
- **Tool / function calling** — the standard OpenAI client-executed loop.
- **Real token counts** — `usage` comes from the actual tokenizer, with a `context_length_exceeded` preflight instead of a mystery truncation.
- **Embeddings** — `/v1/embeddings` served from Apple's contextual embedding model, also on-device.
- **A browser playground** at `GET /`, optional Bearer auth, LAN or localhost-only binding, and App Intents so Shortcuts can start and stop the server.

<img src="/images/local-llm-server-chat-clients.png" alt="OpenAI and Ollama-compatible chat clients prompting the local Apple Intelligence model through Local LLM Server" width="520" />

*The compatibility test I actually care about: existing chat clients talking to the phone.*

There's a 90-test suite behind the API surface. One user wired the server into DSPy for prompt optimization against the on-device model, which is exactly the kind of thing I hoped people would do and did not want to explain was impossible.

## Why bother, when you could call a cloud API?

Three answers, in my order of caring: it's *private* (prompts never leave the device—for some data that's not a preference, it's the requirement); it's *free at the margin* (bulk classification jobs cost zero tokens); and it's *always with you* (the server is in your pocket, on a plane, in a dead zone). The honest counterweight: it's a 3B model. It will not write your novel. For extraction, tagging, structured summaries, and glue-code intelligence, it's shockingly serviceable—and those are most of what scripts actually need.

<img src="/images/local-llm-server-on-device-proof.png" alt="Server Information screen showing on-device processing and no cloud data transfer" width="520" />

*The important line: on-device processing, no cloud round-trip.*

## Free vs Pro

The free app is ad-supported; [Local LLM Server Pro](https://apps.apple.com/us/app/local-llm-server-pro/id6758336829?ct=kevinmd-202607) is $1.99, one-time, no ads. Same server, same API. Pro briefly hit #5 on the US paid Developer Tools chart—a thin, volatile chart, so I won't pretend that's fame, but it did tell me this audience would rather pay two dollars than look at a banner, which matches how I feel about banners.

<img src="/images/local-llm-server-pro-app-store-rank.png" alt="App Store page showing Local LLM Server Pro at #5 in Developer Tools" width="300" />

*Tiny chart, still a real signal: Local LLM Server Pro at #5 in Developer Tools.*

Requires iOS/iPadOS/macOS 26+ on Apple Intelligence-capable hardware (that's where the on-device model lives). If you've got a spare device on your desk, it's a free local inference endpoint you didn't know you owned.