Identifying and evaluating design options

There are at least two design options for a possible solution, as described below:

Option 1: Data Page with REST Connector source

• The auto-complete control sources a node-level Data Page that, in turn, sources a REST connector.
• The node-level Data Page refreshes once per week.

Option 2: Data Page with Local Data Source

• A Job Scheduler executes the REST connector calls, persisting the retrieved data in a local table.
• The auto-complete control sources a node-level data page that, in turn, sources a Report Definition that queries the locally persisted data.
• The node-level data page can refresh often (such as once per day) because its data queries are local.
• The Job Scheduler is configured to run once per week on a single node.

Option 1 works and is simple to implement. Option 2 also works but is more complex to implement compared to Option 1.

Option 1 requires that each WebTier node updates itself by calling the very slow REST connector, which might cause an unpleasant user experience on the initial load. However, with Option 2, the very slow REST connector is only called once and runs on a background processing node. Option 2 WebTier users never see this initial delay because the WebTier node that their browser talks to has a node-level DP that has queried fast-to-load local live data.

These two design options provide a trade-off between time to implement and maintain or an improved user experience. The recommended option depends on which factors are more important. Option 1 may be acceptable and a more desirable option due to its simplicity. Option 2 guarantees an improved initial user experience, which may be preferable.

Option 3:

Option 3 is similar to Option 1, but leverages the Load data page method to request the data page load asynchronously at some point earlier in the process, to give sufficient time for complete loading before the data is needed. At worst, this happens on Login. Preemptive loading may be accomplished through a smart shape or similar.

This option provides the following results:

•    A kind of background processing (uses a background requestor)
•    Optimal memory management (just as many node-scoped data page as Option 1)
•    No need to persist the remote data in Pega
•    Makes one trip to the remote data store per node; Option 2 makes one trip per cluster

Option 4: 

A variant of Option 2, but runs the Job Scheduler on all nodes to have it automatically trigger loading the node-scoped data page at the same time every week for each node (perhaps after flushing any data page still deemed fresh, which can be performed with an activity). 

This option provides the following results:

•    Requires the same number of remote trips as Option 1
•    Removes the need to persist a local copy
•    Provides the same user experience as Option 2 (assuming that the data pages are loaded before a user needs the data)

For Options 3 and 4, the desired design for a solution is a trade-off between time to implement and maintain, with little difference in user experience between the two options as the initial loading problem is solved. Other contributing factors that may weigh into the evaluation are potential rule reuse and future evolution of the application.