The fm CLI: Apple Foundation Models from the macOS Terminal

Using the newly introduced fm CLI macOS 27 tool allows developers to access Apple’s local and cloud intelligence engines directly from their shell environment. By running fm serve OpenAI compatible endpoints locally, macOS developers can expose Apple Silicon-accelerated foundation models to third-party tools, development scripts, and API wrappers without modifications. However, deploying these models in production workloads requires managing the routing thresholds between on-device inference and Apple’s Private Cloud Compute (PCC).

Key Takeaways

• Local HTTP Serving: Converts local Apple foundation models into an OpenAI-compatible Chat Completions API server.
• Unified Model Routing: Integrates local model endpoints with LangChain and Cursor by updating standard base URLs.
• Private Cloud Compute Gateway: Routes complex requests dynamically to PCC servers for high-parameter inference.
• Low-Latency Inference: Achieves optimal token-per-second generation speeds by compiling models to the Apple Neural Engine.
• Unified Sandbox Restrictions: Bypasses local network restrictions to allow secure inter-process communication on localhost.

---

The “Why”: Developer Access to Unified Apple Silicon Models

Historically, developers who wanted to experiment with Apple’s generative model family had to write native Swift code using high-level frameworks like App Intents or Core ML. Running model tests from scripts, terminal prompts, or cross-platform codebases (e.g., Python, Node.js) required building custom Swift command-line tools. This separation isolated Apple’s on-device intelligence from the broader developer tooling ecosystem.

+-----------------------------------------------------------+
|               Developer Tool (Cursor / LangChain)         |
+-----------------------------------------------------------+
                               |
                               v (OpenAI Chat Completions API)
+-----------------------------------------------------------+
|              fm serve daemon (localhost:8080)             |
+-----------------------------------------------------------+
                               |
                               v (Decides routing based on complexity)
        +-----------------------------------------------------+
        |                                                     |
        v (Local Inference < 3B)                             v (Complex Task > 3B)
+---------------------------------+                 +---------------------------------+
|     Apple Neural Engine (ANE)   |                 |      Private Cloud Compute      |
+---------------------------------+                 +---------------------------------+

Figure 1: Architecture diagram showing how the fm CLI routes local developer tool requests.

In macOS 27, Apple addresses this gap with the fm CLI. Pre-installed with Xcode developer utilities, this CLI provides commands to download, configure, and inspect Apple Foundation Models.

The standout feature is fm serve, which launches a local HTTP server exposing an OpenAI-compatible Chat Completions endpoint. This allows tools like Cursor, LangChain, or simple curl commands to query Apple’s local models. When a request exceeds the local model’s parameter capacity, the CLI automatically routes it to Apple’s Private Cloud Compute (PCC), ensuring high-quality responses while maintaining user privacy.

---

The Implementation: Deploying fm serve and Building a Swift Client

Below, we detail how to launch the local fm serve gateway and write a Swift script to call this local endpoint, including handling Private Cloud Compute routing options.

1. Starting the fm serve Gateway

To launch the OpenAI-compatible endpoint in your terminal:

# Launch fm serve on localhost using the unified 3B foundation model
fm serve --port 8080 --model unified-3b --allow-origin "*"

To enable automatic routing to Private Cloud Compute for complex tasks:

# Enable PCC routing for queries requiring higher parameter depth
fm serve --port 8080 --model unified-3b --pcc-routing auto

2. The Swift Model Client Implementation

Below is a production-grade Swift command-line tool that interfaces with the local fm serve endpoint, handling response streaming and network routing.

import Foundation
import OSLog

private let logger = Logger(subsystem: "in.iosdev.intelligence", category: "FoundationModelClient")

public struct ChatMessage: Codable {
    public let role: String
    public let content: String
    
    public init(role: String, content: String) {
        self.role = role
        self.content = content
    }
}

public struct ChatCompletionRequest: Codable {
    public let model: String
    public let messages: [ChatMessage]
    public let stream: Bool
    public let temperature: Double
    
    public init(model: String, messages: [ChatMessage], stream: Bool = false, temperature: Double = 0.7) {
        self.model = model
        self.messages = messages
        self.stream = stream
        self.temperature = temperature
    }
}

public struct ChatCompletionResponse: Codable {
    public struct Choice: Codable {
        public struct Message: Codable {
            public let role: String
            public let content: String
        }
        public let message: Message
    }
    public let choices: [Choice]
}

public final class FoundationModelClient {
    private let serverURL: URL
    
    public init(port: Int = 8080) {
        guard let url = URL(string: "http://localhost:\(port)/v1/chat/completions") else {
            fatalError("Invalid local address configured.")
        }
        self.serverURL = url
    }
    
    public func promptModel(text: String) async throws -> String {
        let payload = ChatCompletionRequest(
            model: "unified-3b",
            messages: [
                ChatMessage(role: "user", content: text)
            ]
        )
        
        var request = URLRequest(url: serverURL)
        request.httpMethod = "POST"
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        request.httpBody = try JSONEncoder().encode(payload)
        
        logger.debug("Dispatching request to local fm endpoint: \(self.serverURL.absoluteString)")
        
        let (data, response) = try await URLSession.shared.data(for: request)
        
        guard let httpResponse = response as? HTTPURLResponse, httpResponse.statusCode == 200 else {
            logger.error("Local fm endpoint returned invalid status code.")
            throw NSError(domain: "in.iosdev.intelligence", code: 1, userInfo: [NSLocalizedDescriptionKey: "Invalid API response"])
        }
        
        let completion = try JSONDecoder().decode(ChatCompletionResponse.self, from: data)
        guard let resultText = completion.choices.first?.message.content else {
            throw NSError(domain: "in.iosdev.intelligence", code: 2, userInfo: [NSLocalizedDescriptionKey: "Empty completion returned"])
        }
        
        return resultText
    }
}

3. Execution Script

Here is how to call the model client within an asynchronous script context.

@main
struct ScriptEntry {
    static func main() async {
        let client = FoundationModelClient()
        
        do {
            let prompt = "Explain Swift 6 structural concurrency and actor isolation boundaries."
            print("Sending prompt: '\(prompt)'")
            
            let response = try await client.promptModel(text: prompt)
            print("\nResponse from Apple Foundation Model:\n")
            print(response)
        } catch {
            print("Error executing model query: \(error.localizedDescription)")
        }
    }
}

---

The Verdict: Evaluating the fm CLI Strategy

When to Use

• Local Tool Integration: When using Cursor, LangChain, or custom Python scripts that expect an OpenAI-compatible endpoint.
• Privacy-First Workflows: For offline-capable applications that process sensitive text structures without sending data to third-party APIs.
• Low-Resource Prototyping: When testing LLM-driven features on macOS without paying for cloud API keys or dealing with network latency.

When NOT to Use

• Non-Apple Deployments: The fm CLI relies on macOS 27 and Apple Silicon hardware; it cannot run on Linux servers or Intel Macs.
• Massive Context Windows: For tasks requiring large context windows (100k+ tokens), the local 3B model will run out of memory or suffer from performance degradation.

The Hidden Cost

• The PCC Routing Tax: While local inference runs on-device for free, routing requests to Private Cloud Compute consumes system compute slots. During peak demand, Apple limits concurrent requests for Small Business Program members, leading to timeouts or fallback latency.
• RAM Allocation Overhead: Running fm serve locks down ~4GB of unified system memory to keep weights resident on the ANE. On 8GB or 16GB Macs, this memory footprint can trigger swap-file thrashing, slowing down Xcode compilations and other development tools.

---

Internal Links

• Core AI Deep Dive: Deploying On-Device Models with the New Framework — Read how local models use direct memory mapping to run efficiently on the ANE.
• Local RAG & Natural Language: Building Context-Aware On-Device Search — Integrate local foundation models with database frameworks for semantic search.

External Links

• Consult the Apple Developer Portal for Foundation Models documentation.
• Learn more about on-device security guarantees in Apple’s Private Cloud Compute Guide.