Configure a sink connector

Step 1: Download binary files

Download the latest version of the Confluent connector zip/jar binary from Artifactory{:target=“_blank”}.

Step 2: Setting up a sink connector

Prerequisites

Confluent connector binary files
A running cluster

The following procedures assume the use of the Confluent Cloud Platform’s Kafka Connect framework. Refer to the Confluent documentation{:target=“_blank”} for screen-by-screen configuration details.

Upload the sink connector

To add a sink connector:

Go to Connectors in your Confluent Cloud cluster. If you have one or more existing connectors, select Add Connector.
In the Connector Plugins section, select Add Plugin.
A custom connector plugin is needed to create a source connector. In the Add Custom Connector Plugin page, enter the required Connector plugin details.
- Connector plugin name: Provide a meaningful name for your connector.
- Custom plugin description: Provide a description for your connector.
- Connector class: Enter the Java class or alias for the connector. See the connector.class in the sink connector properties table for more details about the value.
- Connector type: Click Sink.
Select Select connector archive. To upload the archive file, select the local .zip file that you downloaded from Artifactory.
To exclude sensitive properties from being logged, select Add sensitive property. For example, by adding app.secret, the client secret is excluded from the log. Remember to identify and handle sensitive information to ensure security and compliance.
Check the box that indicates that you are responsible for the connector and then select Submit. Once you submit the archive, the source connector is deployed to Confluent Cloud.

Step 3: Configure and launch the sink connector

To configure and launch the sink connector:

When the main Connectors page displays, select Filter by: Deployment and select custom. The uploaded custom connector is displayed.
Select the way you want to provide Kafka Cluster credentials. Select Use an existing API key.
Select Continue.
To configure a source connector, you need to specify the necessary settings as key-value pair or add the following JSON configuration:
```
  {
        "env": "[ENVIRONMENT]",
        "app.id": "[APP\_ID]",
        "app.secret": "[APP\_SECRET]",
        "app.name": "[APP\_NAME]",
        "events": "[EVENT\_NAMESPACE]-[EVENT\_NAME]-[EVENT\_VERSION]",
        "topics": "[TOPIC\_NAME]"
    }
```
Note: The parameters within square brackets must be substituted with the relevant values. For details regarding these values, refer to the Sink Connector Configuration Properties.
Add Trimble Identity and Events publisher endpoints to the safe list.

Example:

https://id.trimble.com
Staging: preprd-events-ws-publisher.events.trimblecloud.com
Prod: events-ws-publisher.events.trimblecloud.com
Select the number of Tasks and select Continue.

The tasks in Kafka Connect serve as consumer threads, responsible for receiving partitions and reading data from them. Suppose you have ten partitions and set the tasks.max configuration to 5. In this case, each task will handle two partitions, effectively tracking their respective offsets.

However, it is important to note that if you set tasks.max to a value higher than the total partition count, Kafka Connect will launch a number of tasks equal to the total partitions, while the remaining tasks will remain idle and unused.

Note: Therefore, it is strongly recommended to configure the tasks.max parameter to be equal to the number of partitions present in your topics. This ensures optimal utilization of resources and efficient distribution of workload among the tasks.
The status of your connector should read Provisioning. Select the card to get the provisioning details.
Once the connector completes provisioning, the status changes to running.