Change Data Capture for SAP S/4HANA via Syniti Replicate

Overview

Syniti Replicate provides an incremental replication from SAP S/4HANA systems, this replication option is called Mirroring, and allows our users to create a transactional replication using the CDC capabilities from SAP. Ensure that you have completed the Prerequisites steps before proceeding to configure Refresh replication for an SAP S/4HANA system.

SAP Environment Compatibility S/4HANA

Transport is compatible with SAP installations of S/4HANA 1909 and higher.

User Authorization for Mirroring (CDC)

The following Authorization Objects must be assigned to the user performing the generation steps.

The following Authorization Objects must be assigned to the user using the Consumption APIs Read Capabilities.

The following Authorization Objects must be assigned to the user using the Consumption APIs Update Capabilities.

Note

Refer to Executing the Consumption APIs in SAP S/4HANA for more information on how different components of Consumption APIs are used.

Setup SAP S/4HANA for Mirroring

Download and Install Syniti Replicate

Refer to Installing Syniti Replicate for more information on the Syniti Replicate installation process.

Install Mirroring Transport for S/4HANA

Install the provided transport in the SAP Application Server. The transport files are listed below. To install them, please contact the Customer Basis team.

Transport Contents S/4HANA (Mirroring)

Executing the Generation Program

The Syniti Replicate Mirroring solution uses a mix of Transportable Objects (Transparent Tables) and Data Base Objects (Sequences and Triggers) to implement the change data capture solution. The Transparent Tables can be separated into the following four groups:

• Control Tables:
  The Control tables are provided with the installation packaged and are used to keep up an inventory of what Log Transparent Tables (/BS4/GEN_TR_OBJS) and Database Objects (/BS4/GEN_DB_OBJS) were generated, as well as the mapping between Standard SAP Tables in scope for Mirroring and their corresponding Log Tables (/BS4/TGT_OBJECTS).

• Master Log Tables:
  The Master Log Table (/BS4/MASTER_LOG) is also provided with the installation package and is used to track all Mirroring activities.

• Error Log Tables:
  The Error Log Table (/BS4/ERROR_LOG) is also provided with the installation package and is populated with any error message raised during the execution of Triggers during the Mirroring activities.

• Log Tables:
  Log Tables are not provided with the installation package and must be generated in the source system's development environment. Log tables are Transportable Objects and need to be placed in Transport Requests and promoted through the landscape to the desired production environment.

No Database Object is provided with the installation package. All Database Objects must be generated directly in each environment of the source system.

To generate both Log Tables and Database Objects, report /BS4/GENERATOR (transaction code /BS4/GENERATOR) is available.

Master Log Table Database Objects - Sequence

The Master Log Tables is provided with the installation packaged and does not need to be generated using /BS4/GENERATOR. However, the Database Objects associated with it must be created. In this case, only a Sequence will be created in the database as no Triggers are required for Master Log Table.

To execute this step, select the Master Table radio button under Generation Control in the selection screen and execute the report as shown in Image A.

Transaction SE80

Object Navigator Execution

After the report is executed, Sequence /BS4/MASTER_LOG_SEQ will be created, and the execution log will be displayed. In the following example, the Master Log Table Sequence had previously been created, therefore in the execution log the message states that a new one cannot be created. When the program is executed in an environment where the Master Log Table Sequence does not exist, a successful message would be displayed.

Log Table Transportable Objects – Transparent Table

The Transportable Objects need to be created for the tables in scope for Mirroring/CDC.

To execute this step using /BS4/GENERATOR, the Log Table radio button must be selected under Generation Control section, and the Transportable Objects radio button must be selected under Transportable or Database Objects section. On the main section, the following fields must be entered:

• Transport Request: Transport Request to be used to promote the Log Tables to other environments;

• Package: Development Package in which the Log Table should be created;

• Prefix: Used to construct the name of the Log Table - SDR<TABLE_NAME>;

• Table Name: Name of the table for which a Log Table should be created.

Image B: Log Tables Transportable Objects Generation shows an example of how create Log Table ZSDRLFA1 for table LFA1 in package ZSDRGEN.

Image B - Log Tables Transportable Objects Generation

During the execution of the report for the creation of Log Tables Transportable Objects, the Package assignment and Transport Request assignment must be confirmed.

Log Tables Transportable Objects Generation - Package Assignment

Log Tables Transportable Objects Generation - Transport Request Assignment

Once the report is executed, Table SDR<TABLE_NAME> will be created and the execution log will be displayed.

Image C: Log Tables Transportable Objects Generation Log

Log Table Database Objects – Triggers & Sequence

The Database Objects need to be created for the tables in scope for Mirroring/CDC.

To execute this step using /BS4/GENERATOR, the Log Table radio button must be selected under Generation Control section, and the Database Objects radio button must be selected under Transportable or Database Objects section. On the main section, the following fields must be provided:

• Prefix: Used to construct the name of the Log Table Sequence and Triggers- SDR<TABLE_NAME>_[SEQ, _INS, _UPD, DEL];

• Table Name: Name of the table for which a Log Table should be created.

Image C: Log Tables Database Objects Generation shows an example of how create Sequence ZSDRLFA1_SEQ and Triggers ZSDRLFA1_INS, ZSDRLFA1_UPD, ZSDRLFA1_DEL for table LFA1.

Log Tables Database Objects Generation

Once the report is executed, the following objects will be created, and the execution log will be displayed.

• Sequence: SDR<TABLE_NAME>_SEQ

• Triggers:

  • SDR<TABLE_NAME>_INS

  • SDR<TABLE_NAME>_UPD

  • SDR<TABLE_NAME>_DEL

It is possible to execute /BS4/GENERATOR to drop the created triggers for a table. To perform this action, radio button Log Table must be selected under Generation Control section, and radio button Database Objects must be selected under Transportable or Database Objects section as well as the check box Drop Triggers Only. On the main section, the following fields must be provided:

• Prefix: Used to construct the name of the Log Table Triggers- SDR<TABLE_NAME>_[INS, UPD, DEL];

• Table Name: Name of the table for which a Log Table should be created.

Image D: Log Tabes Database Objects Generation - Drop Triggers shows an example of how create Sequence ZSDRLFA1_SEQ and Triggers ZSDRLFA1_INS, ZSDRLFA1_UPD, ZSDRLFA1_DEL for table LFA1.

Image D: Log Tables Database Objects Generation - Drop Triggers

Once the report is executed, the Triggers will be dropped, and the execution log will be displayed.

Log Tables Database Objects Generation - Drop Triggers Log

Steps for Replicating CDC Enabled Tables

Extracting data from SAP S/4HANA Systems using Syniti Replicate requires software to be installed on the application server running Syniti Replicate.

Configure the Enable Transactional Replication Wizard

Note

Skip this section if you are setting up a Refresh replication. Continue here if you are setting up a Mirroring replication from SAP S/4HANA.

This section assumes you have selected the Source Connection wizard option to launch the Enable Transactional Replication wizard. To open the wizard from the Management Center, right-click a connection in the Metadata Explorer, and then select Transactional Setup > Enable.

In the Enable Transactional Replication wizard:

1. Select the Triggers option.

   

2. On the Triggers Based Log Setting page, complete the required fields using the following table:

   Field	Description
   User and Password	These fields are automatically filled from the connection string properties.
   Tablespace	Before selecting the Master Table, make sure to assign the proper tablespace name for ECC systems: /BS4/
   Master Table	Once selected the Tablespace name, you will be able to select or create a new Mater Table.
   Retention Time	The amount of time in hours that a transaction is kept in the log tables. The default value is 72 hours. When the amount of time a transaction resides in the log exceeds the retention time, the transaction is permanently removed from the log tables. Tuning the retention time provides control over the size of the log tables. It is also possible to instruct Syniti Replicate to remove all the processed transactions at the end of each mirroring interval. Tuning the retention time provides control over the size of the log tables.
   Delete Block Size	Based on the retention time, Syniti Replicate deletes items from the log. This field specifies the maximum number of records to delete from the Syniti log tables with a single SQL statement. The default value is 10,000 records. You do not typically need to edit this value.
   Lower-case Trigger Identifiers	Check this option if your database installation uses lower-case trigger identifiers. Always inactive for SAP ECC sources.
   Trigger Order	Always inactive for SAP ECC sources.
   Uncommitted Transactions Recovery Option	This section can be used to indicate how uncommitted transactions should be handled during replication. None: No gap conditions handled. Uncommitted transactions may cause data to be skipped in replication. Wait: Set a number of mirroring intervals to hold all replications on the connection to wait when a gap is found due to an uncommitted transaction. All replications in the connection will hold and wait for the number of cycles specified to see if the gap is filled. If after the number of cycles, a transaction is still not committed, it will be skipped. Continue and Process when Committed: Instead of pausing all replications in case of a gap, replication proceeds with all currently committed transactions. During the next mirroring cycle, the trigger log table is checked for earlier transactions that now have been committed, and any identified transactions are processed. Skip Uncommitted Transactions Older Than: If wait or continue are selected, this property sets a limit on the amount of time to wait for uncommitted transactions. For instance, a value of 15 minutes means that, no matter which option you choose, transactions opened and not committed for more than 15 minutes will be skipped.

Define Replications

Refer to the Define the Replication section for more information to define Mirroring Replications.

Syniti SAP RFC .NET Data Provider Connection Properties

Remove Components Used in a Triggers-Based Replication

In this scenario, follow these steps:

1. Disable the Mirroring replication

2. Delete the replication from the Replication Monitor. The following message will be displayed:

   

3. Click Yes. A Confirmation dialog box is displayed to remove your Triggers from your source database.

   

4. Click Yes.

5. If you are removing a group of replications, select Yes All.

Uninstall Transactional Components

Select and right-click the source connection and select Transactional Setup > Disable.

This wizard offers two options to disable the Transactional setup:

• Disable Transaction Replication: Choose this option to disable transactional replication from this connection and leave all settings on the database unchanged.

• Complete Uninstall: The database’s transactional setup will be uninstalled. If other connections are using the same server as source for transactional replication, they will no longer work correctly.

Uninstall components on SAP 

Remove the Database Object on SAP

1. Go to transaction SE80, select /BOA/GENERATOR program and execute it.

   

2. On Generator Control, select Log Table.

3. On Transportable or Database object section, select Database Object and select the Drop Triggers Only checkbox.

4. Complete the information with you Transport Request, Package, Prefix used when you enable Mirroring, and enter the table name.

   

5. Once you execute this action, a confirmation page will be displayed.

   

Remove Custom Tables for Mirroring

Syniti Replicate creates a Zxxx table in SAP for Mirroring activities, and it must be removed manually by following these steps:

1. Go to Transaction SE14, and remove the temporal table created for replications, in this example we enable mirroring replication for table AUSP. The custom table created for Mirroring is called ZSDRAUSP

   

2. To remove a custom table in SAP you have to make sure it is empty. Select the option processing Type - Direct, check the option “Delete Data” and click on “Activate and Adjust Database” button.

   

3. Go back to transaction SE80, select the Package used for Mirroring, in this example we used “Z001”, select the table and Delete it.

   

Executing the Consumption APIs in SAP S/4HANA

Overview

The Syniti Replicate Mirroring mode realizes the Consumption API using Remote Function Modules. The Consumption API consists of the following Remote Function Modules:

• /BS4/SDRM_READ_TABLE: Read API

• /BS4/SDRM_UPDATE_MASTER_TABLE: Update Master Log Table

• /BS4/SDRM_UPDATE_LOG_TABLE: Update Log Tables

The following sections depicts how the different components of the Consumption API can be utilized:

Read API

The Read API (/BS4/SDRM_READ_TABLE) can be used to read any table in the system, provided that the user executing it has the required authorizations. This includes the tables delivered with the solution as well as the ones generated using /BS4/GENERATOR.

The input parameters for the Read API are the following:

• QUERY_TABLE: Name of the Table to be read;

• DELIMITER: Field separator for output data;

• NO_DATA: Flag that controls if DATA output table is filled;

• ROWSKIPS: Number of rows to be skipped while extracting data;

• ROWCOUNT: Number of rows to be selected while extracting data;

• OPTIONS: Table of free text fields to define the extraction where clause;

• FIELDS: Table of structured field. Structure field FIELDNAME can be used to specify a projection clause.

The output parameters for Read API are the following:

• TABLE_ROWS: Total number of rows in the QUERY_TABLE. It is only populated when ROWCOUNT is negative.

• FIELDS: Return the details of the fields of QUERY_TABLE;

• DATA: Return the extracted data in a table of text fields.

Reading Master Log Table

To read the Master Log Table, the QUERY_TABLE input field must be populated with the /BS4/MASTER_LOG value. The other input fields are optional. The example below reads 100 entries (ROWCOUNT = 100) from the Master Log Table skipping the first 10 entries (ROWSKIPS = 10).

Read API - Master Log Table

Once the API is executed, tables FIELDS and DATA are populated.

Read API - Master Log Table – Output

Read API - Master Log Table - Fields Table

Read API - Master Log Table - Data Table

Reading Log Table

To read a Log Table, the QUERY_TABLE input field must be populated with the Log Table Name value (for example, ZSDRBUT000). The other input fields are optional. The example below reads all entries from the Log Table ZSDRBUT000 using semi colon as the data delimiter (DELIMITER = ‘;’).

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Target Objects

To read the Generated Target Objects Table, the QUERY_TABLE input field must be populated with the /BS4/TGT_OBJECTS value. The other input fields are optional. The example below uses the OPTIONS table to read only the Target Objects for Log Table ZSDRLFA1 (OPTIONS[] = TNAME = ‘ZSDRLFA1’).

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Transportable Objects

To read the Generated Transportable Objects Table, the QUERY_TABLE input field must be populated with the /BS4/GEN_TR_OBJS value. The other input fields are optional.

Read API - Generated Transportable Objects

Once the API is executed, tables FIELDS and DATA are populated.

Reading Generated Database Objects Table

To read the Generated Database Objects Table, the QUERY_TABLE input field must be populated with the /BS4/GEN_DB_OBJS value. The other input fields are optional.

Read API - Generated Database Objects

Once the API is executed, tables FIELDS and DATA are populated.

Reading Error Table

To read the Error Log Table, the QUERY_TABLE input field must be populated with the /BS4/ERROR_LOG value. The other input fields are optional.

Read API - Error Log

Once the API is executed, tables FIELDS and DATA are populated.

Update Master Log Table API

The Update Master Table API (/BS4/SDRM_UPDATE_MASTER_TABLE) can be used to update the FLAG field of the Master Log Table /BS4/MASTER_LOG, provided that the user executing it has the required authorizations.

Input parameters for the Update Master Log Table API:

• OLD_FLAG: Current value of field FLAG – Used to select entries in /BS4/MASTER_LOG;

• NEW_FLAG: Value to be used to update field FLAG;

• SCHEMA_NAME: Selection parameter for querying /BS4/MASTER_LOG;

• TABLE_NAME: Selection parameter for querying /BS4/MASTER_LOG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying /BS4/MASTER_LOG – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Master Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Updating Master Log Table

To update the Master Log Table, the OLD_FLAG and NEW_FLAG input fields must be populated. The other input fields are optional. It is recommended to populate either SCHEMA_NAME and TABLE_NAME or TID table to limit the records being updated. The following example updates FLAG field from /BS4/MASTER_LOG from 0 to 1 for TIDs 1 to 10.

Update Master Log Table API

Once the API is executed, table MESSAGES is populated.

Update Log Tables API

The Update Log Table API (/BS4/SDRM_UPDATE_LOG_TABLE) can be used to update the FLAG field of the Log Tables, provided that the user executing it has the required authorizations.

Input parameters for the Update Log Table API:

• LOG_TABLE_NAME: Name of the Log Table to be updated;

• OLD_FLAG: Current value of field FLAG – Used to select entries from LOG_TABLE_NAME;

• NEW_FLAG: Value to be used to update field FLAG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying LOG_TABLE_NAME – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Updating Log Table

To update the Log Table, the LOG_TABLE_NAME, OLD_FLAG, and NEW_FLAG input fields must be populated.

Once the API is executed, table MESSAGES is populated.

Delete Master Log Table API

The Delete Master Table API (/BS4/SDRM_DELETE_MASTER_TABLE) can be used to delete entries of the Master Log Table /BS4/MASTER_LOG, provided that the user executing it has the required authorizations.

Input parameters for the Delete Master Log Table API:

• SCHEMA_NAME: Selection parameter for querying /BS4/MASTER_LOG;

• TABLE_NAME: Selection parameter for querying /BS4/MASTER_LOG;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying /BS4/MASTER_LOG – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Master Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Deleting Master Log Table

To delete the Master Log Table the input fields are optional. It is recommended to populate either SCHEMA_NAME and TABLE_NAME or TID table to limit the records being updated.

Delete Master Log Table API

Once the API is executed, table MESSAGES is populated.

Delete Log Tables API

The Delete Log Table API (/BS4/SDRM_DELETE_LOG_TABLE) can be used to delete entries from Log Tables, provided that the user executing it has the required authorizations.

Input parameters for the Delete Log Table API:

• LOG_TABLE_NAME: Name of the Log Table to be updated;

• TOP: Maximum number of rows to be updated;

• TID: Selection parameter for querying LOG_TABLE_NAME – multiple values can be defined;

  • SIGN:

    • I – Include;

    • E – Exclude;

  • OPTION:

    • EQ – Equal;

    • NE – Not Equal;

    • GT – Greater Than;

    • LT – Lower Than;

    • GE – Greater or Equal;

    • LE – Lower or Equal;

    • BT – Between;

    • CP – Contains Pattern;

  • LOW: Value for single value options and from value for double value options;

  • HIGH: To value for double value options.

Output parameters for Update Log API:

• MESSAGES: Table of messages raised during the execution of the API.

Deleting Log Table

To delete the Log Table entries, the LOG_TABLE_NAME input field must be populated. The following example deletes entries for TIDs 1 to 10 from ZSDRBUT000.

Delete Log Table API

Once the API is executed, table MESSAGES is populated.