Getting Started
note
What is CAIP Inference?
CAIP Inference API abstracts the complexity of managing AI infrastructure resources in a distributed cloud environment. Built on Ray framework and MLFlow, it exposes independent CRUD operations for two inference modes: Live Inference for real-time model serving, and Batch Inference for asynchronous job processing.
Features
This API currently features both types of inference tasks that can be done by the users:
Live Inference
- Create a live inference application by specifying MLFlow models
- List your currently available live inference applications
- Check a live inference application's status
- Perform a live inference prediction
- Delete a live inference application
Batch Inference
- Create a batch inference job for currently available models
- List your created and ongoing batch inference jobs
- Check a batch inference job's status
- Delete a batch inference job
Quickstart
Get started quickly with these hands-on guides:
Live Inference Usage
Learn how to use the Inference API to execute Live Inference jobs.
Start Tutorial →Batch Inference Usage
Learn how to use the Inference API to execute Batch Inference jobs.
Start Tutorial →note