What's New? - CAIP - 24-PI3
🚀 What's New in CAIP after 24-PI3?
24-PI3 (May 7 – July 1, 2024) made CAIP projects more configurable, broadened Python 3.11 notebook support, and gave teams clearer ways to inspect Glue workloads and pipeline metrics.
🖥️ CAIP Offboard v1 / v1.5: Classic & Managed Kubeflow
✨ New Features
📓 Notebooks & Development
Python 3.11 notebook images became available for Glue and SciPy workloads as well as PyTorch and TensorFlow development, giving teams a broader choice of current runtimes. The CAIP SDK introduced a configuration-driven CLI for multiple pipelines and environments, image build and push, lifecycle hooks, proxy configuration, and environment-variable placeholders. Teams could also use a Dockerfile that sits separately from its build context for ECR image builds, avoiding unnecessary project restructuring. Richer configuration objects made those settings easier to reuse across project code.
🔀 Pipelines & Serving
Glue pipeline runs gained the ability to reuse existing Glue jobs, helping teams continue running workflows without consuming additional account job capacity. A Spark UI access example and tutorial also made Glue Spark jobs easier to inspect when teams needed operational detail.
📈 Training & Experiment Tracking
The run-metrics component added named, labelled Gauge and Counter metrics, enabling teams to publish the measurements that matter most to their experiments and workflows.
📦 Repositories in This Release
jupyter-notebooks-images— Python 3.11 Glue, SciPy, PyTorch, and TensorFlow images.caip-sdk— configuration-driven CLI, flexible ECR image builds, and project configuration support.kubeflow-components— Glue job reuse, Spark UI guidance, and run metrics.
This section covers customer-visible changes for CAIP Offboard v1/v1.5 from May 7 – July 1, 2024 (24-PI3). For guides and reference material, see the CAIP Documentation.