Change Data Capture for SAP ECC via Syniti Replicate

Overview

Syniti Replicate provides an incremental replication from SAP ECC systems. This replication option is called Mirroring, allows our users to create a transactional replication using the Change Data Capture (CDC) capabilities from SAP. Ensure that you have completed the Prerequisites steps before proceeding to configure Mirroring replications for an SAP ECC system. Transparent Tables data extraction objects are supported in Change Data Capture.

SAP Environment Compatibility ECC

Transport is compatible with SAP installations of ECC6 SAP ECC Basis 750 and higher.

User Authorization for Mirroring or CDC

The following Authorization Objects must be assigned to the user performing the generation steps.

The following Authorization Objects must be assigned to the user using the Consumption APIs Read Capabilities.

The following Authorization Objects must be assigned to the user using the Consumption APIs Update Capabilities.

Note

Refer to Executing the Consumption APIs for more information on how different components of Consumption APIs are used.

Setup SAP ECC for Mirroring

Download and Install Syniti Replicate

Refer to Installing Syniti Replicate for more information on the Syniti Replicate installation process.

Install Mirroring Transport for ECC

Install the provided transport in the SAP Application Server. The transport files are listed in this article. To install them, please contact the Customer Basis team.

Transport Contents ECC (Mirroring)

Executing the Generation Program

The Syniti Replicate Mirroring solution uses a mix of Transportable Objects (Transparent Tables) and Data Base Objects (Sequences and Triggers) to implement the change data capture solution. The Transparent Tables can be separated into the following four groups:

• Control Tables:
  The Control tables are provided with the installation packaged and are used to keep up an inventory of what Log Transparent Tables (/BOA/GEN_TR_OBJS) and Database Objects (/BOA/GEN_DB_OBJS) were generated, as well as the mapping between Standard SAP Tables in scope for Mirroring and their corresponding Log Tables (/BOA/TGT_OBJECTS).

• Master Log Tables:
  The Master Log Table (/BOA/MASTER_LOG) is also provided with the installation package and is used to track all Mirroring activities.

• Error Log Tables:
  The Error Log Table (/BOA/ERROR_LOG) is also provided with the installation package and is populated with any error message raised during the execution of Triggers during the Mirroring activities.

• Log Tables:
  Log Tables are not provided with the installation package and must be generated in the source system's development environment. Log tables are Transportable Objects and need to be placed in Transport Requests and promoted through the landscape to the desired production environment.

No Database Object is provided with the installation package. All Database Objects must be generated directly in each environment of the source system.

To generate both Log Tables and Database Objects, report /BOA/GENERATOR (transaction code /BOA/GENERATOR) is available.

Master Log Table Database Objects - Sequence

As explained in the previous section, the Master Log Tables is provided with the installation package and does not need to be generated using /BOA/GENERATOR. However, the Database Objects associated with it must be created. In this case, only a Sequence will be created in the database as no Triggers are required for Master Log Table.

To execute this step, all that is required is to select the Master Table radio button under Generation Control in the selection screen and execute the report as shown in Image A.

After the report is executed, Sequence /BOA/MASTER_LOG_SEQ will be created, and the execution log will be displayed. In the following example, the Master Log Table Sequence had previously been created, therefore in the execution log the message states that a new one cannot be created. When the program is executed in an environment where the Master Log Table Sequence does not exist, a successful message would be displayed.

Log Table Transportable Objects – Transparent Table

The Transportable Objects need to be created for the tables in scope for Mirroring/CDC.

To execute this step using /BOA/GENERATOR, the Log Table radio button must be selected under Generation Control section, and the Transportable Objects radio button must be selected under Transportable or Database Objects section. On the main section, the following fields must be entered:

• Transport Request: Transport Request to be used to promote the Log Tables to other environments;

• Package: Development Package in which the Log Table should be created;

• Prefix: Used to construct the name of the Log Table - SDR<TABLE_NAME>;

• Table Name: Name of the table for which a Log Table should be created.

Image B: Log Tables Transportable Objects Generation shows an example of how create Log Table ZSDRLFA1 for table LFA1 in package ZSDRGEN.

During the execution of the report for the creation of Log Tables Transportable Objects, the Package assignment and Transport Request assignment must be confirmed.

Once the report is executed, Table SDR<TABLE_NAME> will be created and the execution log will be displayed.

Log Table Database Objects – Triggers & Sequence

The Database Objects need to be created for the tables in scope for Mirroring/CDC.

To execute this step using /BOA/GENERATOR, the Log Table radio button must be selected under Generation Control section, and the Database Objects radio button must be selected under Transportable or Database Objects section. On the main section, the following fields must be provided:

• Prefix: Used to construct the name of the Log Table Sequence and Triggers- SDR<TABLE_NAME>_[SEQ, _INS, _UPD, DEL];

• Table Name: Name of the table for which a Log Table should be created.

Image C: Log Tables Database Objects Generation shows an example of how create Sequence ZSDRLFA1_SEQ and Triggers ZSDRLFA1_INS, ZSDRLFA1_UPD, ZSDRLFA1_DEL for table LFA1.

Once the report is executed, the following objects will be created, and the execution log will be displayed.

• Sequence: SDR<TABLE_NAME>_SEQ;

• Triggers:

  • SDR<TABLE_NAME>_INS;

  • SDR<TABLE_NAME>_UPD;

  • SDR<TABLE_NAME>_DEL;

It is possible to execute /BOA/GENERATOR to drop the created triggers for a table. To perform this action, radio button Log Table must be selected under Generation Control section, and radio button Database Objects must be selected under Transportable or Database Objects section as well as the check box Drop Triggers Only. On the main section, the following fields must be provided:

• Prefix: Used to construct the name of the Log Table Triggers- SDR<TABLE_NAME>_[INS, UPD, DEL];

• Table Name: Name of the table for which a Log Table should be created.

Image D: Log Tables Database Objects Generation - Drop Triggers shows an example of how create Sequence ZSDRLFA1_SEQ and Triggers ZSDRLFA1_INS, ZSDRLFA1_UPD, ZSDRLFA1_DEL for table LFA1.

Once the report is executed, the Triggers will be dropped, and the execution log will be displayed.

Steps for Replicating CDC Enabled Tables

Extracting data from SAP ECC Systems using Syniti Replicate requires software to be installed on the application server running Syniti Replicate.

Configure the Enable Transactional Replication Wizard

This section assumes you have selected the Source Connection wizard option to launch the Enable Transactional Replication wizard. To open the wizard from the Management Center, right-click a connection in the Metadata Explorer, and then select Transactional Setup > Enable.

In the Enable Transactional Replication wizard:

1. Select the Triggers option.

2. On the Triggers Based Log Setting page, complete the required fields using the following table:

Define Replications

Refer to the Define the Replication section for more information to define Mirroring Replications.

Syniti SAP RFC .NET Data Provider Connection Properties

Remove Components Used in a Triggers-Based Replication

In this scenario, follow these steps:

1. Disable the Mirroring replication

2. Delete the replication from the Replication Monitor. The following message will be displayed:

   

3. Click Yes. A Confirmation dialog box is displayed to remove your Triggers from your source database.

   

4. Click Yes.

5. If you are removing a group of replications, select Yes All.

Uninstall Transactional Components

Select and right-click the source connection and select Transactional Setup > Disable.

This wizard offers two options to disable the Transactional setup:

• Disable Transaction Replication: Choose this option to disable transactional replication from this connection and leave all settings on the database unchanged.

• Complete Uninstall: The database’s transactional setup will be uninstalled. If other connections are using the same server as source for transactional replication, they will no longer work correctly.

Uninstall components on SAP 

Remove the Database Object on SAP

1. Go to transaction SE80, select /BOA/GENERATOR program and execute it.

   

2. On Generator Control, select Log Table.

3. On Transportable or Database object section, select Database Object and select the Drop Triggers Only checkbox.

4. Complete the information with you Transport Request, Package, Prefix used when you enable Mirroring, and enter the table name.

   

5. Once you execute this action, a confirmation page will be displayed.

   

Remove Custom Tables for Mirroring

Syniti Replicate creates a Zxxx table in SAP for Mirroring activities, and it must be removed manually by following these steps:

1. Go to Transaction SE14, and remove the temporal table created for replications, in this example we enable mirroring replication for table AUSP. The custom table created for Mirroring is called ZSDRAUSP

   

2. To remove a custom table in SAP you have to make sure it is empty. Select the option processing Type - Direct, check the option “Delete Data” and click on “Activate and Adjust Database” button.

   

3. Go back to transaction SE80, select the Package used for Mirroring, in this example we used “Z001”, select the table and Delete it.

   

Executing the Consumption APIs

Overview

The Syniti Replicate Mirroring mode realizes the Consumption API using the Remote Function Modules. The Consumption API consists of the following Remote Function Modules:

• /BOA/SDRM_READ_TABLE: Read API

• /BOA/SDRM_UPDATE_MASTER_TABLE: Update Master Log Table

• /BOA/SDRM_UPDATE_LOG_TABLE: Update Log Tables

The following sections depicts how the different components of the Consumption API can be utilized:

Read API

The Read API (/BOA/SDRM_READ_TABLE) can be used to read any table in the system, provided that the user executing it has the required authorizations. This includes the tables delivered with the solution as well as the ones generated using /BOA/GENERATOR.

Input parameters for the Read API:

• QUERY_TABLE: Name of the Table to be read;

• DELIMITER: Field separator for output data;

• NO_DATA: Flag that controls if DATA output table is filled;

• ROWSKIPS: Number of rows to be skipped while extracting data;

• ROWCOUNT: Number of rows to be selected while extracting data;

• OPTIONS: Table of free text fields to define the extraction where clause;

• FIELDS: Table of structured field. Structure field FIELDNAME can be used to specify a projection clause.

Output parameters for Read API:

• TABLE_ROWS: Total number of rows in the QUERY_TABLE. It is only populated when ROWCOUNT is negative.

• FIELDS: Return the details of the fields of QUERY_TABLE;

• DATA: Return the extracted data in a table of text fields.

Reading Master Log Table

To read the Master Log Table, the QUERY_TABLE input field must be populated with the /BOA/MASTER_LOG value. The other input fields are optional. The example below reads 100 entries (ROWCOUNT = 100) from the Master Log Table skipping the first 10 entries (ROWSKIPS = 10).

Once the API is executed, tables FIELDS and DATA are populated.

Reading Log Table

To read a Log Table, the QUERY_TABLE input field must be populated with the Log Table Name value (for example, ZSDRBUT000). The other input fields are optional. The example below reads all entries from the Log Table ZSDRBUT000 using semi colon as the data delimiter (DELIMITER = ‘;’).

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Target Objects

To read the Generated Target Objects Table, the QUERY_TABLE input field must be populated with the /BOA/TGT_OBJECTS value. The other input fields are optional. The example below uses the OPTIONS table to read only the Target Objects for Log Table ZSDRLFA1 (OPTIONS[] = TNAME = ‘ZSDRLFA1’).

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Transportable Objects

To read the Generated Transportable Objects Table, the QUERY_TABLE input field must be populated with the /BOA/GEN_TR_OBJS value. The other input fields are optional.

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Database Objects Table

To read the Generated Database Objects Table, the QUERY_TABLE input field must be populated with the /BOA/GEN_DB_OBJS value. The other input fields are optional.

Once the API is executed, tables FIELDS and DATA are populated.

Reading Error Table

To read the Error Log Table, the QUERY_TABLE input field must be populated with the /BOA/ERROR_LOG value. The other input fields are optional.

Once the API is executed, tables FIELDS and DATA are populated.

Update Master Log Table API

The Update Master Table API (/BOA/SDRM_UPDATE_MASTER_TABLE) can be used to update the FLAG field of the Master Log Table /BOA/MASTER_LOG, provided that the user executing it has the required authorizations.

Input parameters for the Update Master Log Table API:

• OLD_FLAG: Current value of field FLAG – Used to select entries in /BOA/MASTER_LOG;

• NEW_FLAG: Value to be used to update field FLAG;

• SCHEMA_NAME: Selection parameter for querying /BOA/MASTER_LOG;

• TABLE_NAME: Selection parameter for querying /BOA/MASTER_LOG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying /BOA/MASTER_LOG – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Master Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Updating Master Log Table

To update the Master Log Table, the OLD_FLAG and NEW_FLAG input fields must be populated. The other input fields are optional. It is recommended to populate either SCHEMA_NAME and TABLE_NAME or TID table to limit the records being updated. The following example updates FLAG field from /BOA/MASTER_LOG from 0 to 1, and for TIDs 1 to 10.

Once the API is executed, table MESSAGES is populated.

Update Log Tables API

The Update Log Table API (/BOA/SDRM_UPDATE_LOG_TABLE) can be used to update the FLAG field of the Log Tables, provided that the user executing it has the required authorizations.

Input parameters for the Update Log Table API:

• LOG_TABLE_NAME: Name of the Log Table to be updated;

• OLD_FLAG: Current value of field FLAG – Used to select entries from LOG_TABLE_NAME;

• NEW_FLAG: Value to be used to update field FLAG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying LOG_TABLE_NAME – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Updating Log Table

To update the Log Table, the LOG_TABLE_NAME, OLD_FLAG, and NEW_FLAG input fields must be populated.

Once the API is executed, table MESSAGES is populated.

Delete Master Log Table API

The Delete Master Table API (/BOA/SDRM_DELETE_MASTER_TABLE) can be used to delete entries of the Master Log Table /BOA/MASTER_LOG, provided that the user executing it has the required authorizations.

Input parameters for the Delete Master Log Table API:

• SCHEMA_NAME: Selection parameter for querying /BOA/MASTER_LOG;

• TABLE_NAME: Selection parameter for querying /BOA/MASTER_LOG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying /BOA/MASTER_LOG – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Master Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Deleting Master Log Table

To delete the Master Log Table the input fields are optional. It is recommended to populate either SCHEMA_NAME and TABLE_NAME or TID table to limit the records being updated.

Once the API is executed, table MESSAGES is populated.

Delete Log Tables API

The Delete Log Table API (/BOA/SDRM_DELETE_LOG_TABLE) can be used to delete entries from Log Tables, provided that the user executing it has the required authorizations.

Input parameters for the Delete Log Table API:

• LOG_TABLE_NAME: Name of the Log Table to be updated;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying LOG_TABLE_NAME – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Deleting Log Table

To delete the Log Table entries, the LOG_TABLE_NAME input field must be populated. The following example deletes entries for TIDs 1 to 10 from ZSDRBUT000.

Once the API is executed, table MESSAGES is populated.