I've been lately asked about Automate, so I've decided to start with a simple post, reusing some work that my colleague Karl Boseley has done in our project.
The online help it contains all the details so I am not going to try to explain everything, just show you a basic example in a real project.
We have all the objects split by process area (we created an interface for process area), but usually the load is done by subset of objects that can be load in sequence. For us, it would be useful to group those objects by small groups and run Transform (which have objects from different process areas). This will save some time so we created an interface for those load groups.
Let's start with the real example;
Go to Automate > Interfaces.
Figure 1: Examples of process area interfaces configured in DSP
Figure 2: Examples of load group interfaces configured in dsp
Group -: Each of the data objects belongs to a loading group so that data can be processed sequentially. For example, Customers would be loaded in an earlier group than open sales orders. As illustrated above, we also have interfaces configured for processing all objects within a single process area.
- For loading groups, we use a naming convention of [Wave_Name]_[Group][Number].
- For transforming process area, we use a naming convention is [Wave_Name]_[ProcessArea].
Order -: Numbering that can be used to process the transform groups in a specific order, the lowest number will be processed first.
Interface ID -: Automatically generated by DSP, internal numbering.
Interface -: Provide a name for your interface. We use the following naming conventions -:
- Transform Individual Process Area -: [ProcessArea] (e.g. COM for Company, CUS for Customers)
- Transform load group -: ‘Transform’
Data Source ID -: The dsw data source for the objects being transformed – for example dsw[ProcessArea]_W1. When transforming a load group, we use dswAutomate_W1. This is because a load group contains objects from multiple different dsw databases, so if the interface requires any custom views or stored procedures in the future, we can add them to dswAutomate (which is a data base in SQL).
Queue ID -: Select ‘Background Events’.
Schedule Type -: There are two standard schedule type options to choose from.
- Runtime -: Your interface will run on the days specified in section 1.2, at the time entered in the following ‘Runtime’ field.
- Frequency -: Running an interface on a Frequency schedule allows users to determine how often an interface runs, e.g., every 10 minutes or once a month
The vertical view information is split across four different tabs.
Figure 3: Vertical view of an interface in DSP
General -: From this tab, users can manually ‘Execute’ the interface. ‘Monitor’ links to a page where error information can be viewed if the interface happens to fail to execute. The ‘Last Run Date’ & ‘Last Run Completed’ fields are useful in providing users information about how long the interface should take to run.
Figure 4: Vertical view - General Information
Contact Info -: We do not use but it can be set up for workflows to alert business users / developers when an interface has failed, successfully completed, or both.
Figure 5: Vertical view - Contact information
Schedule Info -: Details of when the interface should run are configured on this screen. The example below shows an interface that is set to run once a week on Sunday (runtime is specified in the horizontal view earlier).
Figure 6: Vertical view - Schedule Information
Logging -: Debug mode can be activated from this tab.
To access the Events for an interface, navigate by using the icon. New events can be added by clicking on the ‘Add’ button’. Existing events can be modified by clicking on the ‘Edit’ icon.
Figure 7: Horizontal view of the 'Interface Events' page
Event ID -: Auto generated by DSP, Internal Numbering.
Priority -: The events contained within the interface will be processed in sequence, priority determines the order.
Event Type -: For Transform automation, then ‘Event Type’ should always be ‘WebApp’. Other options include ‘Workflow’ (can be used to notify users of task completion) & Stored Procedure (schedule execution of a custom SQL procedure)
Page ID -: We are using the option Transform: Targets. Note that there are a number of other Transform options to select from, such as ‘Objects’ (higher level), ‘Target Source’ (lower level).
Event -: When saving the interface event, the user is switched to the vertical view automatically and will be able to select the event from a listbox. When using Event Type ‘webapp’ and one of the Transform pages, the only available option is ‘Process’.
Loop -: Not used
Comment -: Enter the name of the object or target that is being transformed into this field.
Active -: It is possible to activate / deactivate individual events within an interface using this checkbox. On the previous screen, the ‘Active’ checkbox can be used to deactivate the entire interface if necessary.
Note: After these fields have been entered, the record should be saved and DSP will switch to the vertical view. Select ‘Process’ from the ‘Event’ drop down listbox and click ‘Save’ again.
Figure 8: Interface Event vertical view in DSP
Parameters -: This icon links to a page where the user can enter the parameters necessary so that DSP understands which Object / Target needs to be processed. Transform is done at ‘Target’ level which requires a single parameter to be entered -: WaveProcessAreaObjectTargetID. This value can be found easily in either of the following ways -:
- Via SQL select statement -:
SELECT [WaveProcessAreaObjectTargetID] FROM [cmap].[dbo].[tttarget]
WHERE [WaveName] = '<#value#>' AND -- WaveName
[ProcessArea] = '#value#' AND -- ProcessArea
[Object] = '#value#' AND -- Object
[Target] = '#value#' --Target Table
- Go to Transform in DSP. Select the target that needs to be transformed. Hover the mouse over the heading ‘Targets’ at the top of the screen. The relevant GUID will be displayed in the lower left pane.
Figure 9: Finding the WaveProcessAreaObjectTargetID in Transform
Once the WaveProcessAreaObjectTargetID value is known, it can be entered in the Interface Event Parameters screen.
Figure 10: Interface Event Parameter screen
Only enter one parameter per interface event. Create a new interface event for each separate target that needs to be transformed.
Validations -: Pre or post event validations are not used.
Interfaces can be manually run to test the interface, rerun the interface due to errors in the last run, or if the interface needs to be run before the next scheduled time.
To manually run an interface:
- Select Interfaces in the Navigation
- Click Vertical View for the interface. Run options are:
- Debug – Immediately runs the interface in the foreground with full logging information. Debug Mode must be enabled under Interfaces > Vertical View > Logging tab.
- Execute – Immediately runs the interface in the foreground.
- Submit – Submits the interface to run in the background.
In most cases, events run sequentially according to the event’s priority, with the lowest priority executing first. However, a user may need to force sequential events if the preceding event submitted a background job, as performed in Collect. The validation would have to determine if the background job had finished before the next event would start.
Automate is delivered with various history pages and charts that show the history of an interface. These pages display when the interface ran and if it had any errors. Open the Detail pages for more detailed logging information.
The data is saved based on the History Retention Days setting on the Parameters page.
To view the history of an interface in Automate:
- Select Interfaces in the Navigation
- Click the History icon; the Interface History page displays.
- Select History in the Navigation
- Click the Chart icon to see open the History (Chart)
- Click the numbered icon in the HISTORY column to open the History (Detail)
- Click the numbered icon in the FAILED column to open the History Failed (Detail)
The Monitor page displays and permits interaction with the interface events that are currently running or that have failed.
To access the Monitor page in Automate for an interface:
- Select Monitor in the Navigation
- Select Interfaces in the Navigation
- Click the Vertical View icon for the desired interface.
- Click the Monitor icon; the Monitor page displays.
- Click the Vertical View icon for the desired interface to view the following action icons:
- Events—Navigates to the Monitor (Events) page where details about a single event are displayed.
With interfaces that have several events, it is common for the events to take several hours to run. It is possible for an event somewhere in the process to fail, requiring the interface to be restarted. Advanced users (who thoroughly understand the interface and its events) can re-start the interface from the failed event instead of re-starting the interface from the first event.
NOTE: The following process only works if there are no parameters being passed to the event that failed or any of the subsequent events. The user must also understand the interface event well enough to know whether the events that did succeed would need to be re-run for some reason. On the chance that the interface job might fail, validations can be added to each event to determine whether or not they need to run.
To re-start an interface at a specific event:
- Disable the events before the event that failed by unchecking Active. Refer to Activate Interfaces, Events and Validations for more information.
- Submit the interface again.Refer to Manually Run an Interface for more information.
I think this is enough to get you started. I will post another about most common issues.
I hope you have found this post interesting.
Please do not hesitate to ask any questions.
Please sign in to leave a comment.