Source connector configuration properties
The source connector consumes events from the Events Service and puts them into Kafka. Source connectors require configuration parameters to specify the connection details, data retrieval logic, and other properties specific to the source system. Additionally, it is common to implement monitoring capabilities to track the connector’s health, performance, and data ingestion status.
The following required parameters are specific to the source connector:
| Name | Description |
|---|---|
connector.class | |
| com.trimble.tcp.events.kafka.connect.EventsServiceSourceConnectorv | |
task.max | |
| The number of connector instances. A source connector can have an arbitrary number of tasks. | |
app.id | |
| The consumer or publisher application identifier. You can fetch this value from the Trimble Cloud Console. | |
app.secret | |
| The consumer or publisher application secret. You can fetch this value from the Trimble Cloud Console. | |
app.name | |
| The consumer or publisher application name. You can fetch this value from the Trimble Cloud Console. | |
events | |
A comma-separated list of the published or subscribed events. The format of each entry is [namespace]-[name]-[version], e.g. com.trimble.tcp.profiles-profiles-v2.0. For details on Namespace, name and version, see Available Events.{:target=“_blank”} |
The following required parameters are specific to the source connectors deployed on Confluent Kafka Connect:
| Name | Description |
|---|---|
kafka.api.mode | |
| The mode that authorizes the connector to Kafka. See the Confluent documentation for more information. | |
kafka.api.key | |
| The API key used for authorizing the connector to Kafka. See the Confluent documentation for more information. | |
kafka.api.secret | |
| The API secret used for authorizing the connector to Kafka. See the Confluent documentation for more information. | |
confluent.custom.plugin.id | |
| The plugin identifier given by Confluent after uploading the connector package. See the Confluent documentation for more information. | |
Confluent.connection.endpoints | |
The list of allowed endpoints used by Confluent to limit the traffic from the connector. You have to set the following value: id.trimble.com:443:TCP;preprd-events-ws-consumer.events.trimblecloud.com:443:TCP;events-ws-consumer.events.trimblecloud.com:443:TCP. |
The following optional parameters are specific to the source connector:
| Name | Description |
|---|---|
| poll.max.batch.size | The maximum number of events to include in a single batch when polling for new data. This setting limits the amount of data buffered internally in the connector. The default value is 1000. |
| env | The event environment. The supported values are dev, stage, and prod. The default value is prod. |
| connection.timeout | The connection timeout to the Events Service in milliseconds. The default value is 60000ms. |
| connection.retries | The number of connection retries to the Events Service. The default value is 10. |
| copy.queue.size | The maximum queue size to use when copying data between systems. This setting limits the amount of data buffered internally in the connector. The default value is 16000. |
| poll.tps | The maximum number of consume request per second. The default value is 10 |