Skip to main content

Running a Workflow using CAIP Workflows API

Overview

This guide walks through an end-to-end example of defining and running a pipeline with the CAIP Workflows API. The same workflow version runs across every environment — only the configuration bound to each stage changes.

Runnable examples

Every call below has a working, forkable counterpart in the example repository — run the whole flow with a single command instead of copying curl snippets.

Steps to Run a Workflow​

  1. Create a workflow.
  2. Push a workflow version (the task DAG).
  3. Create configurations for your environments.
  4. Assign the version and a configuration to a stage.
  5. Trigger a run and monitor it.

To interact with the API you can use any of the following:

  • Command line via curl
  • Swagger UI at https://workflows.api.caip.bmw.cloud/docs
  • An SDK / API client of your choice
Onboarding

You must be onboarded to the platform and have a spaceId with the necessary permissions (Git repository access, Ray cluster) before running workflows. If you haven't done so, start with the onboarding guide.

All endpoints are space-scoped under /v1/spaces/{spaceId}. The examples below use space_12345 and the workflow ml-training-pipeline.

1. Create a Workflow​

A workflow is a lightweight, idempotent container for versions, configurations, and runs.

PUT /v1/spaces/{spaceId}/workflows/{workflowId}

Example request
curl -X PUT "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "name": "ML Training Pipeline" }'

2. Push a Workflow Version​

A version is the task DAG. Each task references a workflow template via templateRef and supplies values through a flat arguments map. Argument values may embed ${config.KEY} references, resolved at run time.

PUT /v1/spaces/{spaceId}/workflows/{workflowId}/versions/{versionId}

Example workflow version
{
  "tasks": [
    {
      "name": "fetch-git-code",
      "templateRef": { "name": "fetch-git-code", "version": "v1" },
      "arguments": {
        "git-url": "git@bmw.ghe.com:example/example-repo.git",
        "git-branch": "main"
      },
      "dependencies": []
    },
    {
      "name": "ray-training",
      "templateRef": { "name": "submit-ray-job", "version": "v1" },
      "arguments": {
        "entrypoint": "code/train.py",
        "output-bucket": "s3://${config.BUCKET}/output",
        "worker-replicas": "${config.RAY_REPLICAS}"
      },
      "dependencies": ["fetch-git-code"]
    }
  ]
}
Template versions

templateRef.version is v1 here — a floating alias that always resolves to the latest v1.x.x release of the template. Pin a full vMAJOR.MINOR.PATCH (e.g. v1.2.3) instead when you need fully reproducible re-runs. See Workflow Templates.

Validate first

To catch problems (such as a dependency that points at a non-existent task) before persisting, send the same tasks payload to POST /v1/spaces/{spaceId}/workflows/{workflowId}/versions:validate. Validation is informational and returns a valid flag plus a list of errors.

3. Create Configurations​

A configuration is a flat key/value map that supplies the values for the ${config.KEY} references above. Create one per environment.

PUT /v1/spaces/{spaceId}/workflows/{workflowId}/configurations/{configurationId}

Create dev-config and prod-config
# Development values
curl -X PUT "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline/configurations/dev-config" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "data": { "BUCKET": "dev-bucket", "RAY_REPLICAS": "2" } }'

# Production values
curl -X PUT "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline/configurations/prod-config" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "data": { "BUCKET": "prod-bucket", "RAY_REPLICAS": "4" } }'

4. Assign the Version to a Stage​

A stage assignment binds a version and a configuration to an environment. Do this once per stage.

PUT /v1/spaces/{spaceId}/workflows/{workflowId}/stages/{stage}

Bind version 1.0.0 to int and prod
curl -X PUT "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline/stages/int" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "versionId": "1.0.0", "configurationId": "dev-config" }'

curl -X PUT "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline/stages/prod" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "versionId": "1.0.0", "configurationId": "prod-config" }'

5. Trigger a Run​

versionId and stage are query parameters. On a stage that already has a version and configuration bound, no request body is needed — the API resolves every ${config.KEY} reference and submits the workflow to Argo.

POST /v1/spaces/{spaceId}/workflows/{workflowId}/runs?versionId={versionId}&stage={stage}

Trigger a production run
curl -X POST "https://workflows.api.caip.bmw.cloud/v1/spaces/space_12345/workflows/ml-training-pipeline/runs?versionId=1.0.0&stage=prod" \
  -H "Authorization: Bearer <your-jwt-token>"

The response includes a resolvedConfiguration snapshot of the exact values used, kept for audit:

{
  "runId": "ml-training-pipeline-xyz123",
  "stage": "prod",
  "versionId": "1.0.0",
  "configurationId": "prod-config",
  "resolvedConfiguration": { "BUCKET": "prod-bucket", "RAY_REPLICAS": "4" },
  "status": "Running"
}
One-off overrides

For a quick experiment you can pass an inline body without touching the stage binding. Inline values win per key and fall back to the stage-bound configuration for everything else:

curl -X POST ".../runs?versionId=1.0.0&stage=prod" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -H "Content-Type: application/json" \
  -d '{ "configuration": { "LOG_LEVEL": "DEBUG" } }'

If any ${config.KEY} reference is left unresolved, the run is rejected with 422 Unprocessable Entity listing the missing references.

6. Monitor and Manage the Run​

  • Status: GET /v1/spaces/{spaceId}/workflows/{workflowId}/runs/{runId}
  • List / filter: GET /v1/spaces/{spaceId}/workflows/{workflowId}/runs (filter by status, versionId, stage)
  • Cancel: POST /v1/spaces/{spaceId}/workflows/{workflowId}/runs/{runId}:cancel
  • Retry failed tasks: POST /v1/spaces/{spaceId}/workflows/{workflowId}/runs/{runId}:retry
Important Notes
  • Replace the placeholders (spaceId, git-url, image versions, etc.) with values for your own use case.
  • Ensure the necessary permissions are in place — Git repository access and Ray cluster availability.
  • Set the MLFLOW_TRACKING_URI environment variable in your Ray training script to log training metrics and parameters to MLflow.
  • Webhook and schedule triggers always use the stage-bound configuration and cannot pass inline values.

📖 Full endpoint reference for the Workflows API →