|
Add Source Data Structures |
Scroll |
This Quickstart assumes that full before and after data is being placed in a Kafka topic for subsequent downstream processing. The Apply Engine simplifies creation of the topic with a single "REPLICATE" Command. Replicate uses the source data structure "DESCRIPTION" for both source and target automatically generating either AVRO Schemas and Payloads or pure JSON payloads. While Precisely highly recommends using Replicate, some applications and "legacy" Kafka Consumers may require a subset or super-set of source schema components. In those cases Precisely recommends creation of a Relational table like SQL statement or DDL containing the "Column Names" corresponding to each of the source data fields or columns required. The Apply Engine will automatically generate either AVRO Schemas and Payloads or pure JSON payloads from the provided DDL.
Notes:
1.Apply Engines automatically handle both Code page translation and Data Type transformation as well as provide for the handling of invalid source data when applicable.
2.The source data structures, typically in the form of DDL for Relational source data or COBOL copybooks for IMS or VSAM source data are placed into the Data Type specific directories created previously.
Example 1 - DB2
BEGIN GROUP DB2_SOURCE;
DESCRIPTION DB2SQL ./DB2DDL/EMP.ddl AS EMPLOYEE
KEY IS EMP_NO;
DESCRIPTION DB2SQL ./DB2DDL/DEPT.ddl AS DEPARTMENT
KEY IS DEPT_NO;
END GROUP;
The content of the Db2 source DDL for the EMPLOYEE table in the EMP.ddl file:
CREATE TABLE EMPLOYEE
( EMP_NO CHAR(6) NOT NULL
,FIRST_NAME VARCHAR(12) NOT NULL
,MIDINIT CHAR(1) NOT NULL
,LAST_NAME VARCHAR(15) NOT NULL
,WORKDEPT CHAR(3)
,PHONE_NO CHAR(4)
,HIRE_DATE DATE
,JOB CHAR(8)
,ED_LEVEL SMALLINT
,SEX CHAR(1)
,BIRTH_DATE DATE
,SALARY DECIMAL(9, 2)
,BONUS DECIMAL(9, 2)
,COMM DECIMAL(9, 2)
);
Example 2 - IMS
BEGIN GROUP IMS_DBD;
DESCRIPTION IMSDBD ./IMSDBD/HREMPLDB.dbd AS HREMPLDB;
END GROUP;
BEGIN GROUP IMS_SEG;
DESCRIPTION COBOL ./IMSSEG/EMPLOYEE.cob AS EMPLOYEE
FOR SEGMENT EMPLOYEE
IN DATABASE HREMPLDB;
DESCRIPTION COBOL ./IMSSEG/ANNULREV.cob AS ANNULREV
FOR SEGMENT ANNULREV
IN DATABASE HREMPLDB;
END GROUP;
The content of the IMS source COBOL copybook for the EMPLOYEE segment in the EMP.cob file:
05 EMPLOYEE. 00000010
08 KFLDYK. 00000020
10 EMP-NO PIC X(6). 00000030
10 FIRST-NAME PIC X(12). 00000040
10 MIDINIT PIC X(1). 00000050
10 LAST-NAME PIC X(15). 00000060
10 WORK-DEPT PIC X(3). 00000070
10 PHONE-NO PIC X(4). 00000080
10 HIRE-DATE PIC 9(8). 00000090
10 JOB PIC X(8). 00000120
10 ED-LEVEL PIC 9(2). 00000130
10 SEX PIC X(1). 00000140
10 BIRTHDATE PIC 9(8). 00000150
10 SALARY PIC 9(7)V9(2) COMP-3. 00000160
10 BONUS PIC 9(7)V9(2) COMP-3. 00000170
10 COMM PIC 9(7)V9(2) COMP-3. 00000180
10 FILLER PIC X(10). 00000190
Example 3 - VSAM
BEGIN GROUP VSAM_REC;
DESCRIPTION COBOL ./COBOL/EMPLOYEE.cob AS EMPLOYEE
DESCRIPTION COBOL ./COBOL/ANNULREV.cob AS REVIEW
END GROUP;
A VSAM source COBOL copybook for the EMPLOYEE record might look identical to the IMS segment example above.
The DESCRIPTION alias MUST match the VSAMDD or short system name for VSAM files defined in CICS or the RECALIAS runtime parm from the Keyed File compare capture. In our example the VSAM files are EMPLOYEE and REVIEW.
