CAIP Onboard
CAIP Onboard lets you deploy AI models to run directly on the vehicle — on-device (in-vehicle) inference — rather than in the cloud. It provides a standardized, ready-to-use, and scalable AI infrastructure for non-safety digital-service use cases, delivered on the CDC IP-Next compute platform.
CAIP Onboard is a distinct capability that is independent of the CAIP evolution stages (it is not part of Managed Kubeflow (CAIP1.5) or CAIP Offboard v2: All-New CAIP). Whereas those run models offboard (in the cloud), CAIP Onboard runs them onboard (in the car).
This page is a high-level overview. The full engineering documentation (QNN SDK details, CDC script development, deployment steps, data types, etc.) lives in Confluence: CAIP Onboard (CONNECTAI).
Why onboard?
Running a model in the vehicle — instead of offboard in the cloud — is the right choice when you need:
- Data privacy — data stays on the vehicle and never leaves it.
- Low latency — inference happens locally, without a round-trip to the backend.
- Independence from remote updates — new models can be deployed any time, decoupled from ECU software (SW) update cycles, reducing time-to-market.
- Direct access to vehicle data — ultimately all data collectable via CDC IP-Next.
Because in-vehicle resources are limited, only use cases that genuinely need onboard deployment should run on vehicle hardware. Key questions to determine fit:
- What are the data-privacy and latency requirements?
- Is the input data large? Onboard processing is limited.
- Is the function safety-relevant? Not supported by CAIP Onboard.
- Is it a customer function — is a fallback available?
- During which PWF (vehicle) state does the data arrive? Execution is only possible during PWF Fahren (driving).
- Does the model stay within the max size (300 MB)?
How it works
At a high level, a model goes through compilation and signing, is packaged with a CDC job that implements the input/output logic, and is then deployed to the vehicle target. The diagram below shows how these pieces fit together, from the offboard platform through to the vehicle:
The end-to-end flow moves through four stages:
Every model deployed to the car is double-signed: first by the CAIP Offboard platform, then by BMW's internal Signing Service (SiSe). Only the model hash is sent for signing, not the full model.
Deployment options
Depending on the target hardware and whether hardware acceleration is available, a model is deployed in one of three ways. The diagram below contrasts these paths; the table beneath it lists the specifics:
| Option | Target | HW acceleration | Frameworks |
|---|---|---|---|
| 1 — IP-Next with HW acceleration | IP-Next | Qualcomm HTP / TPU (via QNN) | ONNX, TensorFlow, PyTorch, TFLite (converted to QNN) |
| 2 — IP-Next without HW acceleration | IP-Next | None (model + framework compiled to WebAssembly CDC job) | TFLite MCU |
| 3 — IDC-Evo without HW acceleration | IDC-Evo | None (compiled to WebAssembly CDC job) | TFLite MCU |
Target hardware (resources available to the model)
| Feature / Resource | IP-Next | IDC-Evo |
|---|---|---|
| RAM | ~2150 MB (excl. CDC) | ~900 MB (excl. CDC) |
| Flash | ~2150 MB (excl. CDC) | ~1000 MB (excl. CDC) |
| CPU | ~7.4k DMIPS (excl. CDC) | ~5k DMIPS (excl. CDC) |
| GPU | 37.5 GFLOPS (excl. CDC) | N/A |
| Tensor Processing Unit (TPU) | 9.5 TOPS (excl. CDC) | N/A |
| Max model size | 300 MB | 20 MB |
| Inference hardware | TPU (HTP); theoretically also GPU/CPU | CPU |
Model compatibility
CAIP Onboard uses the Qualcomm Neural Network (QNN) SDK for hardware-accelerated deployment, which shapes what models run well:
- ✅ Well supported: neural-network architectures such as MLPs, CNNs, and Transformers — pipeline-like models with controlled value ranges (normalization). Convolution / MLP / LayerNorm / activation ops are supported.
- ⚠️ Avoid: models needing high dynamic range (e.g. exponentials) or control flow (recurrent architectures).
- ❌ Not supported: tree-based architectures (e.g. XGBoost, Random Forest).
Quantization is required for onboard execution and can affect model accuracy — expect several iterations, and validate on actual IP-Next hardware with representative data samples.
Building a use case
To implement the custom input/output logic around the model, a CDC job is developed in C++ or TypeScript. The use-case owner defines the job requirements and registers the use case via the data-ticket process; job development starts only after Gate 1 approval (technical feasibility confirmed by Data Governance).
Information to gather for the job:
- Data source on IP-Next — SOME/IP, NPDU tunnel (CAN FD, FlexRay), or Nautilus topics.
- Which signals are needed.
- Where to send the outputs (e.g. CDH) and how often.
Vision & roadmap
CAIP Onboard's goal is to increase the number of AI models deployed to the car and reduce time-to-market by decoupling model deployment from ECU software updates. A key differentiator is a simulation environment to test models before they go into the vehicle.
Roadmap highlights include:
- CAIP monitoring — live in-vehicle monitoring of onboard models for faster debugging.
- CAIP Onboard Simulator — test and iterate on models before deploying to real hardware.
For the full vision and roadmap, see CAIP Vision & Roadmap (CDCDEV).