Skip to main content

Add/Remove CDH Datasets

No Pull Request needed

On Managed Kubeflow, adding or removing access to a CDH dataset is not a PR-based self-service action. There is no product.yaml to edit and no dedicated cluster/AWS account per product — every use case simply gets one namespace on a shared, platform-managed cluster. Dataset access is granted entirely on the CDH side.

Prerequisite​

This guide assumes your namespace already has a CDH Environment linked to your Managed Kubeflow Workspace, as described in Step 3 of the Onboarding Guide. This connection (role chaining between your Managed Kubeflow Workspace and the caip-role-managed-cdh role on the CDH environment account) only needs to be set up once, via the CDH Connection setup guide and a service request.

Why there is no CAIP-side step​

With the previous, dedicated-cluster architecture, CDH datasets had to be wired up individually per namespace by adding s3_arn/kms_key_arn entries to a product.yaml file. Managed Kubeflow uses a different, role-chaining based IAM setup: your workspace's roles assume a single, fixed role (caip-role-managed-cdh) on the linked CDH environment account. That role is already permitted to access all S3, Glue, KMS, and Athena resources available in that CDH environment.

This means that once a CDH Environment is linked to your namespace, any dataset access granted to that environment on the CDH side automatically becomes available to your Managed Kubeflow Workspace — no additional CAIP configuration or Pull Request is required.

Add a Dataset​

  1. On the CDH Data Catalog, find the dataset you want to consume.
  2. Raise a Data Access Request against the CDH Environment that is linked to your Managed Kubeflow Workspace (see Prerequisite).
  3. Once the request is approved on the CDH side, the dataset is accessible from your Managed Kubeflow Workspace — see Accessing CDH Data for example notebooks using role chaining.

Remove a Dataset​

Revoke or let expire the corresponding data access grant on the CDH side (via the CDH Data Catalog or the CDH portal). There is nothing to change on the CAIP side.

Don't have a CDH Environment linked yet?​

If your namespace doesn't have a CDH Environment linked to your Managed Kubeflow Workspace yet, follow Step 3 of the Onboarding Guide first.