Kubeflow Flavours on CAIP
Kubeflow is a core component of the Connected AI Platform (CAIP), enabling teams to run end-to-end ML workflows — from data preparation to model deployment.
Overview
Over time, CAIP has supported multiple ways of providing access to Kubeflow, depending on the maturity and needs of individual teams. These include:
- AI-Research for rapid prototyping in TEST
- Dedicated Clusters for production, team-managed setups
- Managed Kubeflow — the new and recommended standard
This page explains the differences between these offerings, their current lifecycle status, and what to use moving forward.
✅ Managed Kubeflow (Recommended)
Managed Kubeflow is the default and most future-proof way to use Kubeflow on CAIP. It is fully operated by the platform team and designed to support both experimentation and production use cases in a secure and scalable way.
Key benefits:
- Fully integrated into the CAIP Self-Service Portal
- Fast, reliable onboarding through service requests
- Namespace-based isolation instead of per-cluster setups
- Production infrastructure only, no separate environments like INT or E2E
- Built-in staging concept for pipelines and models (within a single infra)
- Optimized for performance, observability, and security
- Backed by standard platform SLAs (e.g. Connected Drive-grade)
- Supports seamless go-live without any migration between environments
If you're starting a new ML use case today, you should use Managed Kubeflow.
⚡ AI-Research (Legacy - For Rapid Prototyping)
The AI-Research environment was designed to help teams prototype quickly and get started with Kubeflow without complex onboarding. It runs on a shared TEST cluster, with each use case receiving a dedicated namespace.
Intent & characteristics:
- Hosted in a shared TEST cluster, not suitable for production
- Each use case received a dedicated namespace
- Designed for low-friction experimentation, not long-term usage
Limitations:
- Onboarding was manual and error-prone
- Performance and stability issues due to shared cluster usage
- Long wait times for notebooks and pipeline runs
- No observability, SLAs, or proper support model
- Infrastructure itself was TEST-only (e.g., shared cloud rooms, non-HA)
⚠️ Use cases in AI-Research were not allowed to go productive. Teams had to migrate to Dedicated Clusters to go live.
❗ AI-Research will be discontinued. A migration plan and end-of-life (EOL) timeline will be communicated soon. All existing use cases must transition to Managed Kubeflow.
🏗️ Dedicated Clusters (Legacy - For Production Use)
Before Managed Kubeflow, the default path to production involved ordering a dedicated Kubeflow cluster. Each cluster was provisioned by the platform team based on a customer request.
How it worked:
- The customer ordered the cluster via onboarding request
- It was provisioned by the platform team, then handed over
- The use case team was fully responsible for operations and lifecycle
- Platform supported operations in principle — but shared responsibility was unclear, often leading to confusion
Challenges:
- Manual setup with long lead times
- No standard for observability or monitoring
- Limited or no SLAs
- Fragmented support experience
- Difficult to scale or evolve with platform upgrades
❗ Dedicated Clusters will be deprecated. They remain operational for now, but are considered feature-finished. No new clusters will be created. A migration plan and end-of-life (EOL) timeline will be communicated in 2026. Teams should begin considering to migrate to Managed Kubeflow.
📊 Comparison Overview
| Feature | AI-Research | Dedicated Clusters | Managed Kubeflow |
|---|---|---|---|
| Portal Integration | ❌ | ❌ | ✅ Fully integrated |
| Environment Model | Shared TEST cluster | Custom (TEST, INT, E2E, PROD) | PROD infra only, with internal staging |
| Infrastructure Lifecycle | TEST only (incl. cloud) | Provisioned by platform, handed off | Fully platform-managed |
| Go-Live Readiness | ❌ Migration required | ✅ Go-live possible | ✅ Ready by default |
| Use Case Isolation | Namespace per use case | One cluster per use case | Namespace per use case |
| Onboarding Process | Manual & error-prone | Manual, request-based | Self-service via SRs |
| Support / SLA | ❌ None | ❌ Unclear ownership, Untracked SLAs | ✅ Connected Drive SLAs |
| Observability / Monitoring | ❌ None | ⚠️ Team-specific if any | ✅ Built-in |
| Performance / Reliability | ⚠️ Shared, low perf. | ⚠️ Varies by team | ✅ Optimized, secure, fast |
| Current Status | ❌ EOL planned | ❌ EOL planned | ✅ Active & supported |
🧭 What Should You Use?
- For all new use cases, Managed Kubeflow is the platform standard and should be used exclusively.
- If you're currently using AI-Research or Dedicated Clusters, you will be required to migrate. The platform team will share detailed instructions and timelines to support a smooth transition.
- Managed Kubeflow now covers the full lifecycle — from prototyping to production — with no need to migrate between environments.
For onboarding, please raise a Service Request. For migration support, contact the platform team.