AI routing

Every model call in Storyline goes through one funnel, AIService, and every model choice goes through one resolver inside it. Policy, preference and availability are decided there, so no feature picks a provider on its own and nothing bypasses the project’s policy.

The providers

Provider	Where it runs	Key	Notes
Apple Intelligence	This device	None	Apple’s on-device model through the Foundation Models framework, on iOS and macOS 26 or later
MLX	This device	None	mlx-community models from Hugging Face. The default, Llama 3.1 8B Instruct at 4-bit, needs Apple silicon and 8 GB or more of memory
Ollama	Your Mac	None	Your own install at localhost:11434
A nearby device	Another of your devices on your Wi-Fi	None	A model that device serves, such as a Mac’s Ollama models
Anthropic, OpenAI	Cloud	Yours	The native APIs
OpenAI-compatible	Wherever the endpoint is	Yours	Any OpenAI-shaped endpoint, which covers Gemini, Grok and others

Your keys live in the device Keychain and never leave it; endpoint settings shared with family arrive without them. A cloud request goes straight from your device to the provider over TLS, and Storyline is never in the middle.

MLX weights download once, when you ask, and a model only becomes selectable once its weights are installed. The default is a full-attention model on purpose: it can reuse a cached prompt prefix across turns, which sliding-window models such as Gemma 3 and Qwen 3 cannot. The prefix cache that builds on this is on the roadmap.

Three access policies

Each request carries an AIAccessPolicy from its project.

Policy	Allowed	Used for
.anyProvider	Local or cloud	Projects that are not privileged, by default
.clusterOnly	This device or a trusted device of yours on the local network; third-party cloud refused	Work scoped to your home network
.localOnly	This device only	Every privileged project, enforced in code

The fail-closed rule

A privileged project forces .localOnly, and .localOnly means the device in front of you.

• The check runs when a model is resolved, before any request is built. A refusal throws a privilege violation, so there is nothing to send.
• Custom endpoints are refused along with cloud providers. A custom endpoint can be any HTTPS address, and it is also how a model on another machine is adopted, so only Apple Intelligence, MLX and Ollama on loopback can serve privileged work.
• Nearby devices are refused, even your own, and a privileged project’s automations run locally instead of being dispatched.
• If the on-device embedder is unavailable, a privileged note stays unindexed rather than going to a cloud embedder.
• With no local model available, the request fails with a clear message. There is no cloud fallback.
• The quick Continue, Polish and Tighten chips hide when the policy would refuse their agent’s models, when no compatible model is available, or when you are a viewer on a shared project.

The cloud is not switched off by a setting you might forget. It is refused by the code that chooses the model.

How a request picks where to run

request (lane, project, prompt)
  → policy floor from the project        .localOnly · .clusterOnly · .anyProvider
  → your routing preference              Best per task · Prefer on-device
  → walk the providers the policy allows, skipping any that are unavailable
  → fit: if the model cannot hold the context, move to one that can, on-device first
  → nothing left → refuse with a reason

1. Policy floor. Set by the project, and never weakened by anything below it.
2. Preference. “How work is routed”, in AI and Models: Prefer on-device (a local model, then a nearby one, then the cloud), or Best per task, the default, which picks the best model the policy allows. Privileged projects stay on the device either way.
3. Availability. A provider that cannot serve the request steps aside: no Apple Intelligence on this device, no weights installed, no key, or no nearby device online.
4. Fit. If the model cannot hold the request’s context, the resolver escalates to one that can, on-device first.

Nearby models

A model served by another of your devices on the local network joins the routable pool automatically. Nothing is persisted or synced: the pool is rebuilt from the live advertisements of devices on your iCloud account. Traffic between your devices is encrypted (see Security), and privileged projects refuse nearby models.

Agents that plan

• The co-author’s verbs plan before they draft, and the heavier Outline-to-draft verb also checks its draft and applies one revision pass. Each step is best-effort, so a failed step still leaves you the plain draft, and each runs under the project’s policy.
• MasterAgent fans specialists out in parallel and finishes with a synthesis pass. Before you commit, it estimates the token total and shows what would run locally and what in the cloud.
• An agent can only use the tools it is granted. Project tools run on the device, inside the project; web tools go through the egress funnel.

Work that is always local

Job	How
Dictation	WhisperKit (Core ML) by default, transcribing each phrase at the pause, or Apple Speech with no download. The Whisper model downloads once (about 150 MB); audio and transcripts never leave the device
Dictation Polish and cleanup	The on-device model under .localOnly, so both are safe in privileged projects. Cleanup is off by default and falls back to the raw transcript on any failure
Query generalisation	The rewrite that broadens a web search runs on the device, before you approve it
Screening	The leak guard and the search-pattern guard compute on the device
Embeddings for protected content	NLEmbedding, pinned to .localOnly