Skip to main content

Deploying Agents to Production ( Work in Progress )

Learn how to containerize and deploy your CAIP agents via CAIP Apps for production use.

Overview​

This guide covers:

  • ✅ Creating Docker containers for your agents
  • ✅ Building and testing images locally
  • ✅ Pushing images to CAIP platform ECR registry
  • ✅ Deploying agents as containerized services
  • ✅ Configuring environment variables securely
  • ✅ Calling deployed agent endpoints
  • ✅ Production best practices

Prerequisites​

Before deploying, ensure you have:

  • ✅ A working agent (tested locally)
  • ✅ Docker installed (Get Docker)
  • ✅ CAIP account with API access
  • ✅ CAIP API Key
  • ✅ CAIP Space ID
  • ✅ CAIP Agent ID (created via CAIP Portal)

Step 1: Prepare Your Project​

Project Structure​

my-agent/
├── Dockerfile           # Container definition
├── requirements.txt     # Python dependencies
├── .env                # Local environment variables (DO NOT commit)
├── .dockerignore       # Files to exclude from Docker image
├── main.py             # FastAPI app entry point
├── src/                # (optional) your source modules and packages
│   └── ...
└── README.md           # Documentation
tip

This shows a minimal layout. Real projects often have deeper structures — e.g. src/agents/, src/tools/, src/utils/, shared config modules, etc. Organise your code however makes sense and adjust the COPY statements and CMD in your Dockerfile accordingly.

Dependency Files​

requirements.txt
--index-url https://packages.orbit.bmwgroup.net/artifactory/api/pypi/connected-ai-platform-pypi-local-public/simple
--extra-index-url https://pypi.org/simple

caip-agents-sdk>=<x.y.z>
fastapi>=<x.y.z>
uvicorn>=<x.y.z>
python-dotenv>=<x.y.z>
pydantic>=<x.y.z>

Create .dockerignore​

Create a .dockerignore to keep your image lean:

# Python cache
__pycache__/
*.py[cod]
*.pyo
*.pyd

# Virtual environments
.venv/
venv/
env/

# Build artifacts
*.egg-info/
dist/
build/

# Secrets — never bake into the image
.env

# IDE / OS
.DS_Store
.idea/
.vscode/
*.swp

# Tests
tests/

# Git
.git/
.gitignore

Step 2: Create Dockerfile​

FROM --platform=linux/amd64 python:3.12-slim

WORKDIR /app

# Create non-root user/group
RUN groupadd -g 1000 app && \
    useradd -u 1000 -g 1000 -m -s /bin/bash app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt \
    --index-url https://packages.orbit.bmwgroup.net/artifactory/api/pypi/connected-ai-platform-pypi-local-public/simple \
    --extra-index-url https://pypi.org/simple

COPY app/ ./app/

# Set ownership
RUN chown -R 1000:1000 /app

USER 1000:1000

EXPOSE 8501

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8501"]

Further Dockerfile Examples
main.py
from fastapi import FastAPI
from pydantic import BaseModel
from caip_agents_sdk import CAIPAgentsClient

app = FastAPI()
client = CAIPAgentsClient()

class ChatRequest(BaseModel):
    message: str

class ChatResponse(BaseModel):
    response: str

@app.get("/health")
async def health_check():
    """Health check endpoint."""
    return {"status": "healthy"}

@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest):
    """Chat with the agent."""
    agent = client.create_agent("pydantic_ai", "<your-agent-id>")
    await agent.initialize()
    response = await agent.run(request.message)
    return ChatResponse(response=response)

Step 3: Build and Test Locally​

Build Docker Image​

Build for your platform:

# Build the image
docker build --platform linux/amd64 -t <image-name>:<version> .

# Verify image was created
docker images | grep <image-name>

Test Locally​

# Run with environment variables
docker run \
  -e CAIP_API_KEY=$CAIP_API_KEY \
  -e CAIP_SPACE_ID=$CAIP_SPACE_ID \
  -e CAIP_AGENT_ID=$CAIP_AGENT_ID \
  -e CAIP_REGION=${CAIP_REGION:-ROW} \
  -p 8501:8501 \
  <image-name>:<version>

# Test health endpoint
curl http://localhost:8501/health

# Test chat endpoint
curl -X POST http://localhost:8501/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, agent!"}'

Using .env file (Recommended):

docker run --env-file .env -p 8501:8501 <image-name>:<version>
info

For detailed CAIP Apps documentation, see CAIP Apps Guide

Step 4: Internalize Image in CAIP Platform​

Before creating an app, your Docker image must be internalized into the CAIP platform ECR.

Prerequisites:

  • A GitHub repository with your agent code
  • The CAIP GitHub App installed on your repository — this is required for the platform to pull and internalize your image

Install the GitHub App: https://bmw.ghe.com/apps/connected-ai/connected-ai-platform

Compliant Images

Your image must be compliant with the platform requirements. Non-compliant images will fail the internalization process.

Trigger internalization via the Apps API:

POST /v1/spaces/{space_id}/images
{
  "image_name": "my-agent",
  "image_tag": "1.0.0",
  "docker_context": ".",
  "docker_file": "./Dockerfile",
  "repository_owner": "<github-org>",
  "repository_name": "<repo-name>"
}

Verify the image is ready before proceeding:

GET /v1/spaces/{space_id}/images

Your image_name and image_tag should appear in the results once internalization is complete.

info

For the full API details and example curl requests, see the CAIP Apps Guide.

Step 5: Create App in CAIP Platform​

An app defines the image configuration. Creating an app does not deploy it — deployment is triggered separately in Step 6.

POST /v1/spaces/{space_id}/apps
{
  "app_name": "<app-name>",
  "image_name": "my-agent",
  "image_tag": "1.0.0"
}

Verify the app was created:

GET /v1/spaces/{space_id}/apps/list

Step 6: Deploy Your Agent​

Once the app exists, create a deployment. A deployment is the running instance of your app — it's what makes the agent accessible on the platform.

One app can have multiple deployments with different configurations (e.g. different env vars for staging vs production).

Required Environment Variables​

Your agent needs these environment variables at runtime:

VariableDescriptionRequired
CAIP_API_KEYUnified API key for both LLM API and Agents API access✅
CAIP_SPACE_IDYour CAIP Space ID✅
CAIP_AGENT_IDAgent ID (from CAIP Portal)✅
CAIP_REGIONRegion-aware routing target (ROW or CN)Optional (ROW default)
PYTHONUNBUFFEREDSet to 1 for real-time log outputRecommended

Get your API key from CAIP API Key Authentication.

Security Best Practice

Never hardcode secrets in your Docker image or code. Pass API keys through env_vars or secrets in the deployment config.

Create Deployment​

POST /v1/spaces/{space_id}/apps/{app_name}/deployments
{
  "deploy_name": "<deploy-name>",
  "additional_config": {
    "env_vars": [
      { "name": "CAIP_SPACE_ID", "value": "<your-space-id>" },
      { "name": "CAIP_AGENT_ID", "value": "<your-agent-id>" },
      { "name": "CAIP_REGION", "value": "ROW" },
      { "name": "PYTHONUNBUFFERED", "value": "1" }
    ],
    "secrets": [
      { "env_name": "CAIP_API_KEY", "secret_name": "<your-caip-api-key-secret>" }
    ]
  }
}

additional_config options:

FieldDescription
env_varsList of {name, value} pairs injected as environment variables. Use for non-sensitive config (space IDs, feature flags, etc.)
secretsList of {env_name, secret_name} pairs. The platform mounts the named secret as the given env var. Use for API keys and credentials.
Secrets Support Status

You can use the secrets field in deployments, but secret lifecycle operations are not yet fully defined. You must create or manage secrets separately — this is not part of the Apps API.

Access Environment Variables in Your Code​

import os
from caip_agents_sdk import CAIPAgentsClient

# The SDK reads env vars automatically
client = CAIPAgentsClient()

# Or access them explicitly
api_key = os.getenv("CAIP_API_KEY")
space_id = os.getenv("CAIP_SPACE_ID")
agent_id = os.getenv("CAIP_AGENT_ID")
region = os.getenv("CAIP_REGION", "ROW")

Step 7: Verify and Test​

List Deployments​

curl -X 'GET' 'https://apps.int.caip.api.orbit.eu-central-1.aws.cloud.bmw/v1/spaces/<your-space-id>/apps/<app-name>/deployments/list?env=test'

Test Your Deployment​

The list deployments response above will include the URL of your deployed app. Use that URL to test your agent:

# Health check
curl https://<your-deployment-url>/health

# Chat endpoint
curl -X POST "https://<your-deployment-url>/chat" \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, agent!"}'

Production Best Practices​

1. Use Health Checks​

from fastapi import FastAPI

app = FastAPI()

@app.get("/health")
async def health_check():
    return {"status": "healthy", "service": "<your-service-name>"}

2. Implement Logging​

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

3. Handle Errors Gracefully​

@agent.tool_plain
async def safe_operation(query: str) -> str:
    try:
        return await some_operation(query)
    except ValidationError as e:
        logger.error(f"Validation error: {e}")
        return "Invalid input provided"
    except Exception as e:
        logger.error(f"Unexpected error: {e}", exc_info=True)
        return "An error occurred. Please try again."

4. Version Your Images​

Use semantic versioning for your Docker images:

docker tag <image-name> <image-name>:1.0.0
docker tag <image-name> <image-name>:1.0
docker tag <image-name> <image-name>:latest

Troubleshooting​

IssueCauseSolution
Image internalization failsGitHub App not installed or image not compliantInstall the CAIP GitHub App and check image requirements
Deployment stuck in "pending"Image not found or internalization incompleteVerify image via GET /v1/spaces/{space_id}/images
App returns 500 errorsMissing environment variablesCheck that all required env vars and secrets are set in the deployment config
Agent not respondingApplication crashed or missing configCheck deployment logs, verify health endpoint works

Next Steps​