|
Resolve External Library Requirements |
Scroll |
Librdkafka.so provides the "C" language API for Producer, Consumer and Admin clients and is required by the Engines to connect to Kafka. The required version level of the Kafka external library is 0.80 or higher. This library can be installed using a distribution provided package if available or built from source which can be downloaded from github https://github.com/edenhill/librdkafka.
Note, Precisely recommends using the most current version of this open source library whenever possible. The popularity of both managed Kafka clusters and Kafka compatible message brokers like Azure EventHub present tuning opportunities that the librdkafka developers are using to provide specialized defaults based on user experience.
Kcat, a command line tool from the developers of librdkafka that uses the same API should also be installed https://github.com/edenhill/kcat. It is a prerequisite and should be used to test and diagnose all installation specific configurations and connection issues. Once Kcat is working the same configuration parms can be used for Connect CDC SQData. Kcat can also be used to confirm topic content and Engine execution by acting as a local Kafka consumer, providing end to end validation of data replication.
libcurl is required to support communication with the Confluent Schema Registry by the Engines when using AVRO formatted Kafka topics. Libcurl can be installed using distribution provided package if available or built from source which can be downloaded from https://curl.se/download.html. You can read more about AVRO and the Confluent Platform at https://www.confluent.io/download/.
Other functionality beyond simply the Kafka interface may also be utilized:
•SSL - While disabled by default, SSL can be used for Encryption and Authentication by the Connect CDC SQData Kafka client. See https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka and for detailed configuration instructions, http://kafka.apache.org/documentation.html#security_ssl
•A variety of Kakfa dashboard tools, both open source and proprietary can be used to monitor the flow of data into Kafka.
