Migration Support
The Problem
"We have an existing Kubeflow Pipeline (KFP) — a DAG of containerized components chained via
.after()and input/output artifacts, defined with the KFP SDK, deployed into a specific Kubeflow namespace, and triggered either via a recurring run or our own CI cron job. Environment differences (dev vs. prod bucket names, replica counts, credentials) are currently hardcoded per copy of the pipeline. We want this same logic running as a CAIP 2.0 Workflow — one versioned DAG, environment-specific values resolved from a config at run time, and webhook triggers in addition to cron — without a ground-up rewrite. Separately, some of our team is still on the retired AI-Research cluster orcode.connected.bmwand needs a supported path forward."
CAIP has evolved through three stages (distributed Kubeflow → Managed Kubeflow → CAIP 2.0 Service Offerings). CAIP 2.0 is the current, actively developed target; this recipe focuses on getting a Managed Kubeflow (CAIP 1.5) workload onto it, and links to the (already-completed) migrations off older, retired infrastructure.
Ingredients
- CAIP Workflows (CAIP 2.0 target) — versioned DAG, configurations, stages, and triggers
- Your existing Kubeflow Pipeline (KFP SDK DAG) — mapped, not rewritten
- The concept-mapping table below
The Recipe
Target: CAIP 2.0 — Migrating a Kubeflow Pipeline to CAIP Workflows
The KFP concepts you already have map directly onto CAIP Workflows concepts — you're translating the DAG, not rewriting the logic:
| Kubeflow Pipelines (CAIP 1.5) | CAIP Workflows (CAIP 2.0) |
|---|---|
| Pipeline (KFP SDK DAG) | Workflow version — an immutable task DAG (see Workflows & Versions) |
Pipeline component / container step, chained via .after() / artifacts | Task in the version's DAG, run as an Argo Workflow step (KubeRay is used specifically for distributed training tasks) |
| Recurring run (cron) | Trigger — schedule or webhook (see Triggers) |
| Hardcoded per-environment values / separate pipeline copies per env | One version, with a configuration (${config.KEY}) bound per stage (test, int, e2e, prod) (see Configurations, Stages) |
| Kubeflow namespace | CAIP Workflows space (SPACE_ID) |
Start from Workflows: Getting Started and the end-to-end API walkthrough to rebuild your pipeline as a workflow version; the example repository has a runnable 07-full-lifecycle example covering workflow → config → version → stage → trigger → run end to end. If your pipeline includes a Spark/PySpark step, see the Data Processing & Orchestration page for that specific journey.
A dedicated, guided walkthrough for migrating a specific Kubeflow pipeline to the Workflow API is currently being developed by the CAIP team. Until it's published, use the concept mapping above together with the Workflows getting-started guide, or reach out via Support for hands-on migration help.
Already Migrated: Retired Infrastructure
The AI-Research cluster and the legacy code.connected.bmw GitHub instance have both been retired; migration for these is already complete and documented as reference only:
- Platform Evolution & Migration overview — current status of all completed migrations.
- Migrate to Managed Kubeflow — general steps, plus dedicated guides for migrating from AI-Research and from dedicated clusters.
- GitHub migration guides — moving from
code.connected.bmwto ATC GitHub or GHE.
Related Recipes
- Rebuilding your ETL/orchestration steps? See Data Processing & Orchestration.
- Re-registering and re-serving your model on the new stage? See Model Training & Model Registry and Model & LLM Inference.