Coding with the LLM API (Experimental)

Experimental

This guide describes an experimental capability. The self-hosted coding model and its integration with third-party coding agents are under active development and may change without notice. Expect rough edges and limited throughput. It is currently available in the RoW region only (not yet available in China). Please share feedback with the CAIP team.

Index

Jump to	What you'll find
Overview	Model capabilities, context window, and availability
Choosing a model: self-hosted vs. managed	When to prefer the CAIP-hosted coding model
Prerequisites	Required API key and base URL
GitHub Copilot CLI	BYOK setup with the CAIP endpoint
opencode	Custom OpenAI-compatible provider setup
GitHub Copilot Chat (VS Code)	LiteLLM provider setup in VS Code
Tips & limitations	Throughput, capability, and routing caveats
Troubleshooting and Support	Support path for self-hosted model issues

Overview

CAIP now hosts qwen3.6-35b-a3b, a self-hosted Qwen3.6-35B-A3B Mixture-of-Experts (MoE) model tuned for coding and agentic tool use. Unlike the lightweight qwen3-chat model, it provides the capabilities that modern agentic coding tools such as GitHub Copilot CLI, opencode, and GitHub Copilot Chat (VS Code) rely on — tool / function calling, streaming, and a 128K-token context window — and it additionally supports vision (image input).

Because it runs entirely inside the CAIP cluster, you get the same benefits as every other self-hosted model: zero additional cost, full data sovereignty, and no data egress.

Property	Value
Model name	`qwen3.6-35b-a3b`
Reasoning variant	`qwen3.6-35b-a3b-think` (extended thinking enabled)
Underlying model	Qwen/Qwen3.6-35B-A3B (FP8)
Context window	128K tokens
Tool / function calling	Supported
Vision (image input)	Supported
Streaming	Supported

Self-Hosted: available 24/7, but capacity-limited

The coding model runs on a fixed, dedicated GPU allocation inside the CAIP cluster. It is available 24/7, but unlike managed cloud-vendor models (Azure OpenAI, AWS Bedrock) it has limited throughput and concurrency. Under heavy parallel use you may experience queuing or slower responses. Keep this in mind when configuring agents that fan out many requests in parallel.

Reasoning (thinking) variant

By default, qwen3.6-35b-a3b runs with thinking disabled for clean, fast responses. If you want the model to reason step-by-step before answering (useful for harder problems), use the qwen3.6-35b-a3b-think model name instead — it points at the same deployment with extended thinking enabled. Responses then include the model's reasoning in the reasoning_content field.

Choosing a model: self-hosted vs. managed

Coding agents like Copilot CLI and opencode can be pointed at any model the LLM API exposes — including the managed cloud-vendor models (Azure OpenAI, AWS Bedrock). For day-to-day coding, however, we recommend the self-hosted qwen3.6-35b-a3b:

Save on cost. The self-hosted model runs on a dedicated GPU allocation inside CAIP at no additional usage cost, making it a cost-effective alternative to GitHub Copilot for most coding work.
Avoid premium managed models for agentic coding. Agentic coding tools generate a very high volume of tokens — repeated file reads, tool calls, and long contexts add up quickly. Driving a premium managed model such as Claude Opus through the LLM API for coding can run up significant token costs. Reserve frontier managed models for the occasional task that genuinely needs their reasoning, not for routine agentic loops.

Prerelease

Coding support on the self-hosted qwen3.6-35b-a3b model is currently a prerelease feature. Expect changes as it matures.

More self-hosted coding models coming

The self-hosted coding line-up will grow. Planned additions after qwen3.6-35b-a3b:

DeepSeek V4 Flash
GLM 5.2

Prerequisites

You need:

A CAIP LLM API key. See Authentication for how to obtain one.
The base URL for the LLM API: https://llm.api.caip.bmw.cloud/v1

These tools speak the OpenAI-compatible API, so they connect to CAIP by pointing at the base URL above and authenticating with your API key.

Keep your API key safe

Never commit your API key to source control or paste it into shared files. The examples below read it from an LLM_API_KEY environment variable. Export it once in your shell:

export LLM_API_KEY="<your-api-key>"

GitHub Copilot CLI

GitHub Copilot CLI can be pointed at any OpenAI-compatible provider through environment variables ("bring your own model").

1. Install

npm install -g @github/copilot

2. Configure the CAIP provider

Copilot CLI reads its custom-provider configuration from environment variables:

export COPILOT_PROVIDER_TYPE=openai
export COPILOT_PROVIDER_BASE_URL=https://llm.api.caip.bmw.cloud/v1
export COPILOT_PROVIDER_API_KEY=$LLM_API_KEY
export COPILOT_MODEL=qwen3.6-35b-a3b

Variable	Value
`COPILOT_PROVIDER_TYPE`	`openai` (works with any OpenAI-compatible endpoint)
`COPILOT_PROVIDER_BASE_URL`	CAIP base URL, ending in `/v1`
`COPILOT_PROVIDER_API_KEY`	Your CAIP API key
`COPILOT_MODEL`	`qwen3.6-35b-a3b` (or `qwen3.6-35b-a3b-think`)

3. Run

copilot

Copilot CLI now routes its requests to the CAIP self-hosted model. Run copilot help providers for more details on provider configuration.

Sending images to the model

In non-interactive mode, attach an image (or a native document) to your prompt with --attachment. Copilot CLI forwards it to the model natively — no extra configuration required:

copilot -p "Describe this image" \
  --attachment ./screenshot.png \
  --model qwen3.6-35b-a3b --allow-all

Why this model works

Copilot CLI requires a model that supports tool calling and streaming, with a context window of at least 128K tokens. qwen3.6-35b-a3b meets all three requirements, which is why it works as a drop-in coding backend.

opencode

opencode is an open-source terminal coding agent that supports any OpenAI-compatible endpoint as a custom provider.

1. Install

curl -fsSL https://opencode.ai/install | bash

Or install it with npm:

npm install -g opencode-ai

2. Configure the CAIP provider

Add a custom provider to your opencode config — either globally at ~/.config/opencode/opencode.json or per-project at opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "caip": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "CAIP LLM API",
      "options": {
        "baseURL": "https://llm.api.caip.bmw.cloud/v1",
        "apiKey": "{env:LLM_API_KEY}"
      },
      "models": {
        "qwen3.6-35b-a3b": {
          "name": "Qwen3.6 35B A3B (CAIP)",
          "attachment": true,
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          },
          "limit": {
            "context": 131072,
            "output": 65536
          }
        },
        "qwen3.6-35b-a3b-think": {
          "name": "Qwen3.6 35B A3B Thinking (CAIP)",
          "attachment": true,
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        }
      }
    }
  }
}

caip is the custom provider ID — any string works.
npm: @ai-sdk/openai-compatible selects the OpenAI-compatible adapter.
options.baseURL points at the CAIP endpoint; options.apiKey reads your key from the LLM_API_KEY environment variable.
models maps each callable model name to a display name (and optional context/output limits).
attachment together with modalities.input: ["text", "image"] tell opencode the model accepts image input. These are required for vision to work — without them, opencode strips attached images and the model never receives them. (Tool calling is auto-detected for OpenAI-compatible providers, so no extra flag is needed for it.)

Sending images to the model

opencode only forwards an attached image to a custom provider when the model declares image support via modalities. With the config above you can attach an image and ask about it, for example:

opencode run "Describe this image" -f ./screenshot.png -m caip/qwen3.6-35b-a3b

3. Select the model

Start opencode and pick the model with the /models command:

opencode

/models

Choose Qwen3.6 35B A3B (CAIP) from the list. Alternatively, pin it as the default in your config:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "caip/qwen3.6-35b-a3b"
}

GitHub Copilot Chat (VS Code)

If you prefer the chat experience built into VS Code, the LiteLLM Provider for GitHub Copilot Chat extension adds CAIP models to the Copilot Chat model picker. Because the CAIP LLM API is a LiteLLM-compatible gateway, the extension can talk to it directly — the models then work across ask, edit, and agent modes just like any built-in model.

Requirements

VS Code 1.108.0 or newer with the GitHub Copilot Chat extension installed.

1. Install

Install LiteLLM Provider for GitHub Copilot Chat from the Marketplace, or from the command line:

code --install-extension vivswan.litellm-vscode-chat

2. Add the CAIP server

Open the Chat view's model picker → Manage Models… → LiteLLM (or run Manage LiteLLM Provider from the Command Palette), then Add Server with:

Field	Value
Label	Any name, e.g. `CAIP`
Base URL	`https://llm.api.caip.bmw.cloud`
API key	Your CAIP API key

The API key is stored in VS Code's secret storage.

Base URL has no /v1 suffix

Unlike the Copilot CLI and opencode examples above, this extension appends /v1/... to the base URL itself (it calls /v1/model/info and /v1/chat/completions). Enter the root URL without /v1:

✅ https://llm.api.caip.bmw.cloud
❌ https://llm.api.caip.bmw.cloud/v1

Using the /v1 form produces a doubled /v1/v1/... path and the model list comes back empty.

When prompted, select the models to add — qwen3.6-35b-a3b (and qwen3.6-35b-a3b-think). The status bar shows ✓ LiteLLM (N) once the server is reachable; run LiteLLM: Test Connection from the Command Palette if you want to verify it explicitly.

3. Use it in chat

Open Copilot Chat, click the model picker, and choose qwen3.6-35b-a3b. It now serves your chat, inline-chat, and agent-mode requests.

Sending images to the model

The extension supports multimodal input. Attach an image to your chat message (drag it in or use the attach button) and the model receives it natively — no extra configuration required.

Tips & limitations

Throughput is limited. This is a single, fixed GPU deployment. Avoid configurations that issue many parallel requests; you may hit queuing under load.
Experimental. Behavior, model names, and capabilities may change. Some OpenAI request parameters may behave differently than on the managed endpoints.
Model capability. A 35B-parameter MoE model is capable for everyday coding tasks but will not match frontier models (e.g. GPT-5, Claude Opus) on the hardest reasoning problems. Use qwen3.6-35b-a3b-think for tougher tasks.
Verify routing. Check the model field in API responses to confirm your request was served by the self-hosted model.

Troubleshooting and Support

See Self-Hosted Models → Troubleshooting, or contact the CAIP team via the ORbit ITSM group:

FT_orbit-2nd (RoW)

Index​

Overview​

Reasoning (thinking) variant​

Choosing a model: self-hosted vs. managed​

Prerequisites​

GitHub Copilot CLI​

1. Install​

2. Configure the CAIP provider​

3. Run​

opencode​

1. Install​

2. Configure the CAIP provider​

3. Select the model​

GitHub Copilot Chat (VS Code)​

1. Install​

2. Add the CAIP server​

3. Use it in chat​

Tips & limitations​

Troubleshooting and Support​

Index

Overview

Reasoning (thinking) variant

Choosing a model: self-hosted vs. managed

Prerequisites

GitHub Copilot CLI

1. Install

2. Configure the CAIP provider

3. Run

opencode

1. Install

2. Configure the CAIP provider

3. Select the model

GitHub Copilot Chat (VS Code)

1. Install

2. Add the CAIP server

3. Use it in chat

Tips & limitations

Troubleshooting and Support