What is the upper limit of records to de-duplicate a list?

Answered

DSP User

• December 21, 2016 13:58

I need to de-duplicate a contact list. There are 6.8 million records in the table and the transaction keeps timing out, even with subsets of the data. I know that I can break the table into smaller segments, but what is the upper limit of records on the Objects page (Common > Analyze > Analyze > Duplicates icon)? This page is where a set of data is registered for duplicate detection (formerly dDup).

1

• Best answer

  

  DSP User

  • December 21, 2016 14:02

  From several sources, the best practice seems to be using "Merge-Match" in SAP DS. If DS isn't available, break the full data set into manageable subsets and register each subset as a separate "Duplicate Object" batch, in this case, a list of people with similar names. You could break the list into manageable chunks by looking at the country associated with the person, i.e., Adam Smith in the UK is not likely to be the same person as Adam Smith in the US.

  0

Please sign in to leave a comment.