Connect CDC SQData

Static Mappings are the most common type and are explicitly specified in the Replicator script. The meaning and implication of the mapping items depend on the nature of the source and target.

A MAPPINGS statement with a SOURCE entry for each source Table or IMS DBD is:

Syntax

MAPPINGS

SOURCE <table_name_1> | <dbd_name_1>

        [TOPIC <topic_name_1>]

        [SUBJECT <subject_1>]

        [ALIAS <alias_1>]

        [NAMESPACE <name_space_1>]

        [ROOTKEYLEN <key_length_1>]

 

[,SOURCE <table_name_2>  | <dbd_name_2

        TOPIC <topic_name_2>

        SUBJECT <subject_2>

        ALIAS <alias_2>

        NAMESPACE <name_space_2>] [, ...]

        ROOTKEYLEN <key_length_2>

;

 

Keyword and Parameter Descriptions

<table_name> | <dbd_name> Fully qualified Source Table or IMSDBD name. When a MAPPINGS statement is used, a SOURCE is required for every Table or IMSDBD configured for capture and unless a MAPPINGS EXIT is specified that handles exceptions or OPTIONS IGNORE NOMAPPINGS has been specified.

TOPIC <topic_name>  Optional full or partial Kafka topic name to be used for the SOURCE that will be substituted in the REPLICATE target_url.

SUBJECT <subject>  Required when using Confluent to retrieve/maintain the AVRO schema of the SOURCE in the Registry. Typically same as TOPIC with "-value" appended, i.e. <topic_name>-value.

ALIAS <alias>  Optional alias assigned to the SOURCE table_name, that will appear in the JSON/AVRO (header/payload)  as the "alias":"<alias_name>" pair.

NAMESPACE<name_space>  Optional name_space assigned to the SOURCE table_name, that will appear in the JSON/AVRO (header/payload) as the "namespace":"<name_space>" pair.

ROOTKEYLEN < key length> IMS CDC Sources only. Optional shorter ONLY root key length can be specified to override the default automatic key length in the IMS CDC data.

 

Example 1

Two HR tables are captured, their AVRO schemas are managed by Confluent and both will be written to the same Kafka Topic.

 

MAPPINGS

   SOURCE 'IVP_HR.EMPLOYEE'

          TOPIC IVP_HR_EMPLOYEE

          SUBJECT IVP_HR_EMPLOYEE-value;

           ALIAS 'EMPLOYEE'

  ,SOURCE 'IVP_HR.DEPARTMENT'

          TOPIC IVP_HR_DEPARTMENT

          SUBJECT IVP_HR_DEPARTMENT-value;

           ALIAS 'DEPARTMENT'

;

 

Example 2

Two IMS DBD's are captured and will be Distributed to two sets of Parallel Apply Engine Kafka Consumers. The root key length of the first DBD will be overridden to limit the number of parallel Apply Engines.

MAPPINGS

  SOURCE 'IVPHRDBD'

          SUBJECT '<HRDBD-value>

          ALIAS 'IVPHRCDC' ROOTKEYLEN <key_length_1>

  ,SOURCE 'IVPDPDBD'

          SUBJECT '<DPDBD-value>

          ALIAS 'IVPDPCDC'

Notes:

1.Avro Container targets use a file rotation method controlled by a size and or a delay. For hadoop (HDFS), if an OPTIONS statement does not define a size and delay, by default a delay of 1 hour is applied. A delay of one hour means that from the time a record is first written to a target file, the file will be rotated after one hour has passed, if the file has not been rotated for other reason.

2.As previously described, if Static MAPPINGS have been provided but one or more SOURCE <object names> have not been specified the default behavior of the Replicator will enable continuous operation when a new source objects is encountered for the first time and where the Replicator script has not been revised to provide for non-default options, i.e: Topic Names, AVRO Confluent Subjects, Aliases, etc. This default behavior can be prevented by including a MAPPINGS Exit (see Dynamic Mappings below) where the exit merely terminates execution of the Replicator.