Skip to main content

Deploying and Serving Your Model with KServe and WebEAM M2M Authentication

Managed Kubeflow supports WebEAM machine-to-machine (M2M) authentication for KServe endpoints exposed through the private Kong ingress. Use this option when services or automation workflows need to call a private model endpoint with a WebEAM M2M bearer token.

Note: This feature is enabled only for deployments on Managed Kubeflow. Dedicated clusters do not have this Kong private M2M setup enabled.

Warning: WebEAM M2M authentication is supported only with Kong ingress. Do not use the Traefik ingress examples for this authentication mode.

Realm support: Current WebEAM M2M validation uses the Alpha realm configuration, not the Legacy realm, because the Legacy realm will be deprecated by the end of 2026.

Step 1: Update the WebEAM M2M Credential Secret​

For each customer namespace, the platform creates an AWS Secrets Manager placeholder for the WebEAM M2M client credentials used by Kong. The naming format is:

/orbit/kong/oidc/wen-m2m-<namespace>/prod/<region>/kong-private

The default secret value is ChangeMe. Before using the endpoint, log in with orbit-use for the prod environment and assume the namespace-scoped caip-developer role, then replace the default value with your WebEAM M2M client credentials.

orbit-use cloud <orbit-space> prod <orbit-teamspace> --role caip-developer
export AWS_DEFAULT_PROFILE=orbit-use-<orbit-space>-prod-<orbit-teamspace>-caip-developer

aws secretsmanager update-secret \
  --region <region> \
  --secret-id "/orbit/kong/oidc/wen-m2m-<namespace>/prod/<region>/kong-private" \
  --secret-string '{"client_id":"<your-webeam-m2m-client-id>","client_secret":"<your-webeam-m2m-client-secret>"}'

Do not commit WebEAM M2M credentials to Git or include them in support tickets.

Step 2: Deploy the InferenceService with Kong Private Ingress​

Create the InferenceService as usual, but set the inference type to kong-private.

Create inferenceservice-kong-private.yml:

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: sklearn-iris-kong-private
  namespace: mcaip-kflw
  annotations:
    sidecar.istio.io/inject: "false"
  labels:
    inferenceType: kong-private
spec:
  predictor:
    serviceAccountName: default-editor
    model:
      modelFormat:
        name: sklearn
      storageUri: s3://cd4ml-mcaip-kflw-eu-central-1-prod-mcaip-kflw/test/model/model.joblib

Apply the InferenceService configuration:

kubectl apply -f inferenceservice-kong-private.yml

Step 3: Create the Kong Private Ingress and Bind the Namespace Plugin​

The namespace-specific Kong plugin is created by the platform. Its naming format is:

kong-private-m2m-<namespace>-plugin

Create an ingress-kong-private.yml file to expose your InferenceService through Kong private ingress and bind the namespace plugin.

⚠️ Important

  • The ingressClassName must be kong-private.
  • Add the konghq.com/plugins annotation to bind the namespace M2M plugin.
  • Use cert-manager.io/cluster-issuer: kong-private-bmwca for Kong private TLS certificates.
  • The Kong private hostname uses the api.orbit private domain format, for example: <model-name>.<namespace>.<env>.<product>.api.orbit.<region>.aws.cloud.bmw

Create ingress-kong-private.yml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sklearn-iris-kong-private
  namespace: mcaip-kflw
  annotations:
    konghq.com/plugins: kong-private-m2m-mcaip-kflw-plugin
    konghq.com/strip-path: "false"
    konghq.com/preserve-host: "true"
    cert-manager.io/cluster-issuer: kong-private-bmwca
spec:
  ingressClassName: kong-private
  rules:
    - host: sklearn-iris-kong-private.mcaip-kflw.prod.mcaip-kflw.api.orbit.eu-central-1.aws.cloud.bmw
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: istio-ingress
                port:
                  number: 80
  tls:
    - hosts:
        - sklearn-iris-kong-private.mcaip-kflw.prod.mcaip-kflw.api.orbit.eu-central-1.aws.cloud.bmw
      secretName: sklearn-iris-kong-private-tls

Apply the ingress configuration:

kubectl apply -f ingress-kong-private.yml

Step 4: Test the Endpoint with an M2M Token​

To call a private KServe endpoint, first obtain a WebEAM M2M access token for the configured client. Then send the token in the Authorization header:

curl -v -i -X POST \
  -H "Authorization: Bearer <m2m-access-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {
        "name": "input-0",
        "shape": [2,4],
        "datatype": "FP32",
        "data": [
          [6.8,2.8,4.8,1.4],
          [6.0,3.4,4.5,1.6]
        ]
      }
    ]
  }' \
  https://sklearn-iris-kong-private.mcaip-kflw.prod.mcaip-kflw.api.orbit.eu-central-1.aws.cloud.bmw/v2/models/sklearn-iris-kong-private/infer

Kong validates the token with WebEAM by using the namespace-specific M2M credentials from AWS Secrets Manager. Istio also validates WebEAM M2M JWTs at the ingress level.

Troubleshooting Kong M2M Authentication​

If calls to the private endpoint fail, check the following points:

  • The AWS Secrets Manager value no longer contains the placeholder value ChangeMe.
  • The secret name matches your namespace, environment, and region.
  • The JSON keys are named exactly client_id and client_secret.
  • The InferenceService uses inferenceType: kong-private.
  • The Kong-exposed ingress references kong-private-m2m-<namespace>-plugin.
  • The request uses Authorization: Bearer <m2m-access-token>.
  • The token issuer matches the WebEAM Alpha realm configured for the platform.
  • The token is not expired and has been issued for the expected M2M client.

If the configuration looks correct but authentication still fails, contact the Connected AI Platform support team and include the namespace, environment, region, endpoint host, and timestamp of the failed request. Do not include client secrets or access tokens in support tickets.