4  Pointing pi at a local model

NoteLearning objective

You can register an LM Studio model with pi through ~/.pi/agent/models.json, list it, and understand the compatibility flags that local OpenAI-compatible servers need.

4.1 The idea

pi already knows how to speak the OpenAI Chat Completions protocol, and LM Studio serves that protocol at http://localhost:1234/v1. So “use a local model instead of Claude” is really just: define a provider in models.json that points pi at that local URL (uniform resource locator).

4.2 Minimal configuration

Create ~/.pi/agent/models.json:

{
  "providers": {
    "lmstudio": {
      "baseUrl": "http://localhost:1234/v1",
      "api": "openai-completions",
      "apiKey": "lmstudio",
      "compat": { "supportsDeveloperRole": false, "supportsReasoningEffort": false },
      "models": [
        { "id": "qwen3-coder-30b-a3b-instruct", "reasoning": false },
        { "id": "openai/gpt-oss-20b",           "reasoning": true  }
      ]
    }
  }
}

Field by field:

  • baseUrl: LM Studio’s OpenAI-compatible endpoint. Always /v1.
  • api: "openai-completions": the protocol pi speaks to this provider.
  • apiKey: "lmstudio": required by pi, but LM Studio ignores its value. Any non-empty string works.
  • compat: local servers do not implement every OpenAI extension. For reasoning models such as gpt-oss, set supportsDeveloperRole: false (pi then sends the system prompt as a system message rather than the newer developer role) and supportsReasoningEffort: false. These can sit at the provider level (all models) or per model.
  • models[].id: must match the identifier the server advertises exactly (lms ps, or curl http://localhost:1234/v1/models).
  • reasoning: true for models that “think” (e.g. gpt-oss); false otherwise.

For local models that is genuinely all you need: name, contextWindow, maxTokens, and cost are optional and default sensibly (cost defaults to zero, which is correct for a model running on your own hardware). A fuller entry looks like:

{
  "id": "openai/gpt-oss-20b",
  "name": "GPT-OSS 20B (LM Studio, local)",
  "reasoning": true,
  "input": ["text"],
  "contextWindow": 131072,
  "maxTokens": 32000,
  "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
}
ImportantcontextWindow is pi’s bookkeeping, not LM Studio’s limit

The contextWindow you declare here only tells pi when to compact the conversation. The actual served context is the window the model was loaded with in LM Studio (default 8192; see Section 2.3). If you declare 131072 but leave the model loaded at 8192, long sessions still truncate. Reload the model with a larger context to match.

4.3 Adding the MLX provider

The same providers map holds one block per backend (Chapter 2). Add mlx next to lmstudio; both speak openai-completions and want the same compat flags. models.json is re-read whenever you open /model, so no restart is needed.

{
  "providers": {
    "lmstudio": { "...": "as above" },
    "mlx": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "mlx",
      "compat": { "supportsDeveloperRole": false, "supportsReasoningEffort": false },
      "models": [
        { "id": "mlx-community/Qwen2.5-Coder-7B-Instruct-4bit", "reasoning": false }
      ]
    }
  }
}

Each id must match what the server advertises: for mlx it is the Hugging Face repo (curl http://localhost:8080/v1/models); for lmstudio it is the loaded model’s identifier (lms ps). Launch against either provider:

pi --provider lmstudio --model 'qwen3-coder-30b-a3b-instruct'   # MLX, tool-calling
pi --offline --list-models mlx                                  # what mlx serves
ImportantThe mlx provider is for generation, not pi’s tools

mlx_lm.server does not return structured tool_calls (Section 2.2), so a run with --provider mlx will not execute pi’s tools, so keep it for plain generation. For pi’s agent loop, use the lmstudio provider. See Chapter 8.

4.4 Confirm pi sees the model

models.json is re-read every time you open /model, so no restart is needed. List your provider’s models:

pi --offline --list-models lmstudio
provider  model                         context  max-out  thinking  images
lmstudio  qwen3-coder-30b-a3b-instruct  131.1K   32K      no        no
lmstudio  openai/gpt-oss-20b            131.1K   32K      yes       no

--offline skips pi’s startup network calls, appropriate when you are only using local models.

4.5 Choosing and switching models

  • At launch: --provider lmstudio --model 'qwen3-coder-30b-a3b-instruct'.
  • Mid-session in the terminal user interface (TUI): type /model (or Ctrl+P to cycle a shortlist).
  • The --model flag accepts patterns and provider/id form, e.g. --model lmstudio/qwen3-coder-30b-a3b-instruct.
TipWhich local model should I pick?

Favor models with genuine tool-calling support (the qwen3-coder, gpt-oss, and llama3.3 families all work). Pick the size your random-access memory (RAM) and patience allow: a 13 GB model is snappy; a 40+ GB model is capable but slower to load. Gemma-family models currently lack tool-calling templates and will not drive pi’s agent loop. See Chapter 8.

TipExercises
  1. Add a third model you have downloaded (lms ls) to models.json and confirm it appears in pi --list-models lmstudio.
  2. Remove the compat block, re-run a gpt-oss prompt, and observe what breaks. Then put it back.
  3. Set a deliberately tiny contextWindow (e.g. 2048) and watch when pi decides to compact during a longer task.
  1. Add a new entry to the lmstudio provider’s models array whose id matches the identifier from lms ls exactly, then list the provider again (no restart needed, since models.json is re-read whenever you open /model):

    { "id": "qwen2.5-coder-7b-instruct", "reasoning": false }
    pi --offline --list-models lmstudio

    The new model appears as a row in the table alongside the existing ones.

  2. Removing compat drops supportsDeveloperRole: false and supportsReasoningEffort: false. pi then sends the system prompt with the newer developer role and may pass a reasoning-effort parameter, which local OpenAI-compatible endpoints such as LM Studio’s do not implement. The result is errors or degraded behaviour for the gpt-oss reasoning model. Restoring the compat block fixes it.

  3. With contextWindow set to 2048, pi treats the conversation budget as tiny and decides to compact (summarize earlier turns) very early in a longer task, since contextWindow is pi’s own bookkeeping for when to compact, not the served limit. Note the served context is still governed by the window the model was loaded with in LM Studio, so a too-small value there causes truncation independently of this setting.