Skip to main content

CAIP Workflows

CAIP Workflows is a FastAPI service that lets you define, version, promote, and run ML and data pipelines on EKS. It abstracts the underlying infrastructure — Argo Workflows as the execution engine and KubeRay for distributed training — behind a space-scoped REST API.

All workflow definitions live in MongoDB as the single source of truth; Argo workflow specs are generated on demand only when a run is created.

Features​

  • Versioned workflows assembled from reusable pipeline templates
  • Stages (test, int, e2e, prod) to promote a version per environment
  • Configurations (${config.KEY}) to parameterise one version per stage
  • Triggers (webhook & schedule) for automatic runs
  • GitHub integration to fetch source code directly into a run
  • Ray jobs for distributed execution
  • Per-space Grafana for run metrics and logs

Where to start​

API reference​

The full endpoint contract lives in the CAIP Workflows API Reference. A live Swagger UI is served at /docs on any running instance.