|
DB2 to Kafka AVRO with Confluent |
Scroll |
Replicate DB2 changed data (CDC) for the IVP_HR database, EMPLOYEE and DEPARTMENT tables into unique AVRO formatted Kafka Topics with default partitioning based on table keys. All changes to the source tables are Replicated and no filtering is supported. The values for TOPIC and SUBJECT used here are arbitrary but were selected based on the source Table Names, source application and schema source, in this example, the EMPLOYEE and DEPARTMENT Tables, a DB2 "IVP_HR" Database and SQData respectively. Operation of the Producer will be optimized using one worker thread.
Example
----------------------------------------------------------------------
-- Name: DB2TOKAF: Z/OS DB2 To Kafka AVRO on Linux
-- Client/Project: client/project
----------------------------------------------------------------------
-- Change Log:
----------------------------------------------------------------------
-- 2019-07-01 INITIAL RELEASE using Kafka Replicator Engine
--
----------------------------------------------------------------------
-- Replicate Source/Target
----------------------------------------------------------------------
REPLICATE
DB2 cdc://<src_host_name>:<src_sqdaemon_port>/<publisher_name>/<replicator_engine>
TO
AVRO CONFLUENT 'kafka:///<prefix>_*_<suffix>/key'
WITH 1 WORKER
;
----------------------------------------------------------------------
-- Processing Option References
----------------------------------------------------------------------
OPTIONS
STRIP TRAILING SPACES
CONFLUENT REPOSITORY 'http://confluent.repo.<domain>.com:8081'
;
----------------------------------------------------------------------
-- Source References
----------------------------------------------------------------------
MAPPINGS
SOURCE 'IVP_HR.EMPLOYEE'
TOPIC 'EMPLOYEE'
SUBJECT 'SQDATA_DB2_HR_EMPLOYEE'
ALIAS 'EMP'
,SOURCE 'IVP_HR.DEPARTMENT'
TOPIC 'DEPARTMENT'
SUBJECT 'SQDATA_DB2_HR_DEPARTMENT';
ALIAS 'DEPT'
;
