📣
TiDB Cloud Premium is now in public preview. Unlimited growth, instant elasticity, advanced security for enterprise workloads. Try it out →
​
Sign InStart for Free
AI
Developer
Best Practices
API
​
Sign InStart for Free

Set Up Private Endpoint for Changefeeds

This document describes how to create a private endpoint for changefeeds in your TiDB Cloud Premium instances, enabling you to securely stream data to self-hosted Kafka, Amazon MSK Provisioned clusters, or MySQL through private connectivity.

Prerequisites

  • Check permissions for private endpoint creation
  • Set up your network connection

Permissions

Only users with one of the following roles in your organization can create private endpoints for changefeeds:

  • Organization Owner
  • Instance Manager for the corresponding instance

For more information about roles in TiDB Cloud, see User roles.

Network

Private endpoints leverage the Private Link technology from cloud providers, enabling resources in your VPC to connect to services in other VPCs through private IP addresses, as if those services were hosted directly within your VPC.

    If your changefeed downstream service is hosted on AWS, collect the following information based on your connection type:

    If your changefeed downstream service is hosted on Alibaba Cloud, collect the following information:

    • The name of the Private Endpoint Service for your downstream service
    • The availability zones (AZs) where your downstream service is deployed

    To grant TiDB Cloud VPC access, you must add the TiDB Cloud's Alibaba Cloud account ID to the allowlist of your endpoint service.

    If the Private Endpoint Service is not available for your downstream service, follow Step 2. Expose the Kafka cluster as Private Link Service to set up the load balancer and the Private Link Service.

    Step 1. Open the Networking page for your instance

    1. Log in to the TiDB Cloud console.

    2. On the My TiDB page, click the name of your target TiDB Cloud Premium instance to go to its overview page.

    3. In the left navigation pane, click Settings > Networking.

    Step 2. Configure the private endpoint for changefeeds

    The configuration steps vary depending on the cloud provider where your instance is deployed.

      On AWS, choose a connection type based on the downstream service:

      • If your downstream service is exposed through an AWS endpoint service, such as self-hosted Kafka or MySQL, select AWS Endpoint Service.
      • If your downstream service is an Amazon MSK Provisioned cluster, select Amazon MSK Provisioned.

      AWS Endpoint Service

      1. On the Networking page, click Create Private Endpoint in the AWS Private Endpoint for External Services section.

      2. In the Create Private Endpoint for External Services dialog, enter a name for the private endpoint.

      3. Follow the reminder to authorize the AWS Principal of TiDB Cloud to create an endpoint.

      4. Enter the Endpoint Service Name that you collected in the Network section.

      5. Select the Number of AZs. Ensure that the number of AZs and the AZ IDs match your Kafka deployment.

      6. If this private endpoint is created for Apache Kafka, enable the Advertised Listener for Kafka option.

      7. Configure the advertised listener for Kafka using either the TiDB Managed domain or the Custom domain.

        • To use the TiDB Managed domain for advertised listeners, enter a unique string in the Domain Pattern field, and then click Generate. TiDB Cloud will generate broker addresses with subdomains for each availability zone.
        • To use your own Custom domain for advertised listeners, switch the domain type to Custom, enter the root domain in the Custom Domain field, click Check, and then specify the broker subdomains for each availability zone.

      8. Click Create to validate the configurations and create the private endpoint.

      Amazon MSK Provisioned

      1. On the Networking page, click Create Private Endpoint in the AWS Private Endpoint for External Services section.
      2. In the Create Private Endpoint for External Services dialog, enter a name for the private endpoint.
      3. Enter the MSK Cluster ARN of your Amazon MSK Provisioned cluster. To learn about how to create an Amazon MSK Provisioned cluster for changefeeds, see Set Up an Amazon MSK Provisioned Cluster via AWS PrivateLink.
      4. Click Create to validate the configurations and create the private endpoint.

      1. On the Networking page, click Create Private Endpoint in the Alibaba Cloud Private Endpoint for Changefeed section.

      2. In the Create Private Endpoint for Changefeed dialog, enter a name for the private endpoint.

      3. Follow the reminder to add TiDB Cloud's Alibaba Cloud account ID to the allowlist of your endpoint service to grant TiDB Cloud VPC access. For more information, see managing account IDs in the allowlist of an endpoint service.

      4. Enter the Endpoint Service Name that you collected in the Network section.

      5. Select the Number of AZs. Ensure that the number of AZs and the AZ IDs match your Kafka deployment.

      6. If this private endpoint is created for Apache Kafka, enable the Advertised Listener for Kafka option.

      7. Configure the advertised listener for Kafka using either the TiDB Managed domain or the Custom domain.

        • To use the TiDB Managed domain for advertised listeners, enter a unique string in the Domain Pattern field, and then click Generate. TiDB will generate broker addresses with subdomains for each availability zone.
        • To use your own Custom domain for advertised listeners, switch the domain type to Custom, enter the root domain in the Custom Domain field, click Check, and then specify the broker subdomains for each availability zone.

      8. Click Create to validate the configurations and create the private endpoint.

      Was this page helpful?