Does dspCompose have a limit to the number of records that can be processed?

Answered

DSP User

• September 14, 2017 08:33

Does dspCompose have a limit to the number of records that can be processed? At the framework level, there’s a limit of 100,000 records for an Excel import, but it seems like that limit is not applied to the Excel import process for dspCompose. Is the limit based on the system’s performance and connection timeout, or is there a setting used to avoid huge dspCompose background processes?

1

• Best answer

  

  DSP Expert

  • September 27, 2017 14:45

  CBermingham: There is not a setting to limit the number of records per request in dspCompose. The practical limit will vary between templates, based on the number of fields and validations and the strength of the server. You could add config and validation to your templates to enforce a policy-driven limit by template.

  0
• 

  DSP Expert

  • September 22, 2017 11:11

  TEmpson: The number of records for the Excel import is configured under Admin > Configuration > Parameters >  Page Parameters. 100,000 is a good balance for normal use. If you have a powerful server, you can explore with more. When importing Excel via Collects, there is a commit size (normally 10,000) to keep down memory usage. I do not think that same option is available under the Excel Integration, which is another reason for using 100,000.

  0
• 

  DSP Expert

  • September 27, 2017 14:52

  JWoods: Hi Pasquale. In my experience, dspCompose does struggle with timeouts on validations with any large record sets, particularly from the built-in config option for listbox validations (with medium to large field groupings). I agree with Chris that you should definitely add config and validation to enforce record limits.

  0

Please sign in to leave a comment.