W1.9 - The findIT S2 Web Service – Software Support

matchIT Web Index

Both the findIT S2 Administration Program and Web Site make use of the web methods that the findIT S2 service has to offer. You can view the methods that are available by browsing to the web service directly on http://{your local domain}/findITS2Service/Service1.asmx. You will be presented with the following screen, which displays a list of the web methods available, which will be described in detail next.

AdminBulkGenerateKeys (List of Data source IDs, Encrypted Admin Password)

This is the method that is used by the administration program when clicking the ‘Generate Keys’ button. It accepts a list of IDs that are IDs of data sources that do NOT need their keys generating. The method will therefore generate the keys for all data sources specified in the configuration other that the ones specified in the argument passed to the AdminBulkGenerateKeys() method.

There are some important points to note about this method. The first is that the process is multithreaded to provide the best performance. By default, the number of threads to use is set to 4, which is controlled through the ‘KeyGenMaxThreads’ app setting in the Web.config file of the web service. If this setting is removed or left blank, this method will automatically use the same number of threads as there are processors on the hardware. The second point to mention is that, for the purpose of stability, this method keeps a log of its progress in the FIS2_Log table for the data source in question. If for some reason this method errors and terminates, then the last logged progress is used to re-initiate the key generation process when calling PerformPendingKeyUpdates(), which is discussed in more detail below. PerformPendingKeyUpdates() is actually called periodically by the Keys Synchronisation Windows Service, so there are regular checks for the event of a key generation failure.

CancelAdminBulkGenerateKeys()

This is the method that is used by the administration program when clicking the ‘Cancel Generate Keys’ button. It will cause the AdminBulkGenerateKeys() method to terminate if it is running.

CancelIndexKeysTable()

This is the method that is used by the administration program when clicking the ‘Cancel Rebuild Indexes’ button. It will cause the CreateCompoundIndexes() method to terminate if it is running.

CompareRecords()

This method can be used when wanting to compare records in a batch fashion. Records are submitted to the method as an XML string in the following format -

</records>

These records will then be compare to each other, with each pair being compared exactly once (i.e. in the above example 1 vs 2, 1 vs 3 and 2 vs 3). The results returned depend on the batch datasource settings in the findITConfiguration.xml file (described in the web service configuration section). An example is shown below.

</results>

Note that each comparison node refers to a comparison result (which is only returned if the threshold score is reached) and contains the unique refs of each record, plus a breakdown of the score components for the match (which again is configurable in the batch datasource settings in the configuration file).

ConfigurationErrorList()

This method simply returns a list of any current errors with the configuration, that typically occur on trying to load the XML configuration file.

DeleteAllDatabaseObjects()

This is the method that is used by the administration program when clicking the ‘Delete DBOs’ button. It will delete any database objects created by findIT S2 in all databases in the current configuration.

DeleteKeys(Unique record reference, Data source ID)

This method deletes the entry in the keys table for the specified record in the specified data source. The method will check to see if the keys exist, returning an integer result code of -5 (keys don’t exist) if they do not exist, or on success, an integer result code of 1 (method succeeded) is returned.

EnableService(Boolean status)

This method sets the status of the configuration class to whatever is provided as the argument. It is used in the administration program to disable the configuration status when clicking the ‘Disable’ button. Typically, subsequent calls to web methods of the web service will re-enable the configuration providing the configuration xml file has not changed – To prevent this from happening the configuration file either needs to change or the configuration file path / hash values stored under the findIT S2 registry key need to be blanked (The latter is done when clicking the disable button in the administration program).

FindMultiRecords(XML Search String)

This method is essentially the same as the FindRecords() method described in more detail below. The main difference is that no specific data source has to be specified in the query sent to FindMultiRecords - all data sources are queried concurrently. However, if the user wishes to only query a select number of data sources, it is possible to specify a specific data source attribute (shown below) with a comma separated list of data sources to query.

FindRecords(XML Search String)

This is the method that the sample website uses to query the data source(s) and retrieve matching results to what was input. The method accepts an XML string in the following format

The attributes in the node above are examples of the fields that can be used – In fact, any field that can be mapped in the findITConfiguration.xml file can be submitted as an attribute to be used in a query to the web service. As an example, if a search on fullname only was being done, the search string would look like the following –

The ‘specificdatasource’ attribute is used to specify a single particular datasource to use for the query. For example, if you had three data sources set up, with IDs of 1, 2 and 3 respectively, but only want to query datasource 2, you would set the attribute specificdatasource=”2” in your input string. In the FindRecords Method, a specific data source must always be specified. To query all data sources (or a specific range) use the FindMultiRecords() method.

Another two attributes which go hand in hand that can be included in the input string are ‘uniqueref’ and ‘uniquerefcolumns’. The former contains a unique value to search on and the latter contains the datasources/columns to search. The uniquerefcolumns value should be in the format datasourceID/table/column: datasourceID/table/column: datasourceID/table/column etc, where datasourceID is the ID of the datasource to query, table is the name of the table that the column exists in, and column is the name of the column to query. Note that multiple data sources can be specified by using a comma separator. If the uniqueref attribute is supplied without the uniquerefcolumns attribute, a result code of -6 (InsufficientUniqueColumnInformation) will be returned in the XML results, described below.

If no attributes are specified, the result will be a warning that no appropriate search key could be determined for the search. The results themselves are returned as XML also, in the following format

<?xml version="1.0" encoding="ISO-8859-1"?>

<resultcode>(integer result code)</resultcode>

<data>(XML input string)</data>

<searchinputfields>(search inputs used)</searchinputfields>

<dataProcessed>(Processed data nodes)</dataProcessed>

<count>(Number of results returned)</count>

(Match Nodes)

</matches>

<elapse>(Total process time (ms))</elapse>

</results>

A match node (nodes that exist between the ‘matches’ node) contains data for a match, in the following format

Where datasourceID is the ID of the datasource that the match exists in, id is the unique id of the record in the main table of data, fullname is the full name of the of the record, address1 is the first address line, town is town, region is region, postcode is postcode and score is the matching score that the match has achieved.

A processed data node (that exists between) the ‘dataProcessed’ node) is a node that contains cleaned / normalised elements generated from the input data for the query. The following is an example of a Salutation Generated for a search on fullname=”John Smith”

The search input fields node contains the names of the inputs that were used for this specific query to determine which match keys to use.

Note – More or less attributes can be specified to be returned in the <match> xml node by amending the ‘outputfields’ node in the web service XML configuration file. This is described in section 4 – ‘Amending the XML file’.

If an error occurs during the process, the XML is returned in the following format

<?xml version="1.0" encoding="iso-8859-1"?>

<resultcode>(integer result code)</resultcode>

<data>(XML input string)</data>

<searchinputfields>(search inputs used)</searchinputfields>

<count>(Number of results returned)</count>

<warning>(Warning message)</warning>

(Match Nodes)

</matches>

<elapse>(Total process time (ms))</elapse>

</results>

Which as you can see is in exactly the same format as that of a successful search, only with an extra ‘warnings’ node containing the error / warning message. You can test this method and view the XML output by clicking on the link for the method and submitting an XML string in the form <contact fullname=”” /> with the full name of somebody that you know exists in the database. Note – in order to get a successful result, you will need to make sure the configuration is loaded, which can be done with the LoadConfiguration() method described below.

GetBulkGenerateProgress()

This method is called at regular intervals during the Generate Keys process in the administration program to retrieve two variables – the total number of records to be processed, and the total number of records processed so far (so 2 integers). The two integers retrieved are used to display output to the job list box and to give a relevant value to the progress bar.

GetDatasourcesWithKeys()

This method returns a list of the datasources in the configuration that have populated keys tables in them. This is used by the Administration Program when clicking on the ‘Generate Keys’ button to get a list of the data sources with keys and prompt the user with a message box asking if they want to overwrite the keys for each datasource (during which process a list of data sources to NOT generate keys for is made, and passed to the AdminBulkGenerateKeys() method described above).

GetIndexingProgress()

This method is called at regular intervals during the Create Compound Indexes process in the administration program to retrieve two variables – the total number of indexes to be created, and the total number of indexes processed so far (so two integers). The two integers retrieved are used to display output to the job list box and to give a relevant value to the progress bar.

GetLastError()

This method returns a string in the form of the last error that occurred in the Web service, specifically with the AdminBulkGenerateKeys() method. It is used in the Administration program to display the error on screen in the event of a failure during the ‘Generate Keys’ process.

GetOutputFields()

This method returns a list of fields (or xml attributes) that are available in each result returned from the FindRecords() method.

IndexKeysTable(Encrypted Admin Password)

This method creates indexes on the keys table in each data source in the configuration from the search key combinations that are described in the configuration for each datasource. For example, if a search key is defined to use ‘mkName1’ and ‘mkPostOut’, a compound index will be created in the keys table using both of these columns, plus individual indexes for each too.

InsertKeys(Unique record reference, Data source ID)

This method inserts the entry in the keys table for the specified record in the specified data source. The method will check to see if the keys exist, returning an integer result code of -4 (keys already exist) if they do exist. On success, an integer result code of 1 (method succeeded) is returned.

IsServiceEnabled()

This method simply returns a Boolean (true or false) value reflecting the status of the configuration – If the configuration is active (true) then the result is ‘true’, otherwise the result is false. This method can be used to check if the configuration is available before calling other methods such as FindRecords().

LoadConfiguration(File path to XML Configuration File)

This function loads / instantiates the configuration class from an XML configuration file located from the path passed as an argument to the function. It returns a Boolean value, true on success, false on failure. Any errors that occur in the event of a failure can be retrieved using the ConfigurationErrorList() method. Calling this method also deploys all the necessary database objects to each data source to implement dynamic key updates, which are carried out by the findITS2UpdateKeys Windows Service which periodically calls the PerformPendingKeyUpdates() method, described below.

PerformPendingKeyUpdates()

This function simply checks for and performs any pending key updates that exist for all data sources loaded in the current configuration. All updates to data being pointed at by the web service are logged in an updates table in each data source. This method reads those tables for entries that may exist, updates any that do and then deletes the entries that it has performed from the updates table.

Another key part of this method is that, before running updates, it runs a check for each data source to see whether a key generation is in progress, or whether a key generation was in progress that had failed. In both these cases, updates will not be performed, and specifically in the case of the latter, it will re-initiate the key generation process for the data source in question on the remaining records that do not have keys generated for them.

ReturnKeys(Unique record reference, Data source ID)

This method returns the keys for the specified record in the specified data source in the form of an XML string with the keys as attributes.

UpdateKeys(Unique record reference, Data source ID)

This method updates the entry in the keys table for the specified record in the specified data source. The method will check to see if the keys exist, returning an integer result code of -5 (keys don’t exist) if they do not exist, or on success, an integer result code of 1 (method succeeded) is returned.

ValidateAdmin(Encrypted Admin Password)

This method checks that the user has the correct level of administrative privileges to open up the findIT S2 Admin tool. To use a password, you will need to enter a password of your choice into the findIT S2 admin tool encryption tool and store this in the

C:\{findITS2 Path}\website\findITS2Service\web.config file

Under the node : <add key="adminPassword" value="" />

By default this is left blank.

matchIT Web Index