Memory Settings
Memory settings are optional. If not specified, then the defaults are as follows:
<memory><inputBufferSize units="MB" value="1" /><outputBufferSize units="MB" value="4" /><blockSize units="KB" value="16" /><cacheLimit units="GB" value="0" /><threshold units="GB" value="0" /><compression level="1" /><encryption enabled="false" keySize="128" /></memory>
Valid units are KB, MB, GB, and TB, with a kilobyte (KB) being 1024 bytes.
inputBufferSize: All data added to the Syniti Match API engine is initially stored in the input buffer. The processing threads remove this data from the input buffer for processing. For maximum efficiency, it's important that the input buffer is of sufficient size and is always kept as full as possible, so that the processing threads are never kept waiting for data to process. The input buffer is allocated in full when the configuration is applied, and a high buffer size might be unnecessary and wasteful of resources.
outputBufferSize: Similarly, all results are written to the output buffer. The application must remove results from the buffer to prevent it from becoming full, otherwise processing will be affected when the processing threads are waiting for space in the output buffer.
blockSize: Every item of data - once it's been removed from the input buffer and acquired by a processing thread - is stored internally in blocks along with other items of data.
cacheLimit: When a block is full, it's added to the fast cache. When the cache becomes full (if the cacheLimit is not 0) then archiving will begin. Blocks that haven't been accessed for some time will be archived; such blocks can be compressed (to reduce memory usage) and/or encrypted (to increase the security of the data held in memory). It should not normally be necessary to change the blockSize from the default of 16 KB. Note that each data source has both its own cache and its own archive; so an overlap of two data sources, for example, means that there will be two caches, both with a maximum size determined by cacheLimit (cacheLimit will not be divided by 2).
threshold: When the memory usage of the running process exceeds the threshold, blocks will be moved from memory to the temporary disk paging file. Any archived blocks are first moved; if there are none, then blocks will be moved from the cache. Refer to Disk Settings, below, for further details.
compression: The compression level can be 0 for disabled, or 1 (fastest compression) to 9 (slowest/best compression). The greater the compression level, the greater the impact on performance; compression should therefore only be enabled only when necessary.
encryption: The encryption key size can be 128, 192, or 256. Encryption of memory-resident data should not normally be required, but can be enabled if necessary. If enabled, performance will unfortunately be impacted. Note that a random encryption key is generated each time a Syniti Match API engine is created and retained in memory until the process terminates; it is not persisted to disk.
Disk Settings
Disk settings are optional. If not specified, then the defaults are as follows:
<disk><location>C:\Temp</location><limit units="TB" value="1" /><compression level="0" /><encryption enabled="true" keySize="256" /></disk>
location: Specifies the directory in which a temporary disk paging file will be created, should the process's memory usage exceed the threshold (refer to Memory Settings, above). A uniquely-named file will be created at the specified location. Whether the process completes or crashes, the file will always be deleted.
limit: A nonzero limit can be specified, in which case the process will be aborted should the disk file's size exceed the limit. A limit of 0 will allow the file to grow indefinitely in size, but this should normally be avoided. Valid units are KB, MB, GB, and TB.
compression: The compression level can be 0 for disabled, or 1 (fastest compression) to 9 (slowest/best compression). The greater the compression level, the greater the impact on performance; compression should therefore only be enabled only when necessary.
encryption: The encryption key size can be 128, 192, or 256. A key size of 256, for maximum security, is highly recommended. Note that a random encryption key is generated each time a Syniti Match API engine is created and retained in memory until the process terminates; it is not persisted to disk.