Identifying and evaluating design options

The solution has at least two design options, as described below:

Option 1: Data Page with REST Connector source

• The auto-complete control sources a node-level Data Page that, in turn, sources a REST connector.
• The node-level Data Page refreshes once every week.

Option 2: Data Page with Live Data Source

• A Job Scheduler runs the REST connector calls and persists the retrieved data in a local table.
• The auto-complete control sources a node-level data page that, in turn, sources a Report Definition that queries the locally persisted data.
• The node-level data page can refresh often (such as once per day), because its data queries are local.
• The Job Scheduler is configured to run once every week on a single node.

Option 1 works and is simple to implement. Option 2 also works but is more complex to implement.

Option 1 requires that each WebTier node updates itself by calling the very slow REST connector, which might cause an unpleasant user experience on the initial load. However, with Option 2, the very slow REST connector is only called once, and runs on a background processing node. Option 2 WebTier users never see this initial delay because the WebTier node that their browser talks to has a node-level DP that has queried fast-to-load local live data.

These two design options provide a trade-off between time to implement and maintain, and improved user experience. The recommended option depends on which factors are more important. Option 1 may be acceptable and more desirable because of its simplicity. Option 2 guarantees an improved initial user experience, which may be preferable.

Option 3:

Option 3 is similar to Option 1, but makes use of the Load data page method to request the data page load asynchronously at some point earlier in the process. This gives sufficient time for complete loading before the data is needed. At worst, this happens on Login. Preemptive loading can be accomplished by using a smart shape, or similar.

This option provides the following results:

•    A kind of background processing (uses a background requestor).
•    Optimal memory management (just as many node-scoped data pages as Option 1).
•    No need to persist the remote data in the Pega application.
•    Makes one trip to the remote data store for each node; Option 2 makes one trip for each cluster.

Option 4: 

A variant of Option 2, but runs the Job Scheduler on all nodes to have it automatically trigger loading the node-scoped data page at the same time every week for each node (perhaps after flushing any data page that is still deemed fresh, which can be performed with an activity). 

This option provides the following results:

•    Requires the same number of remote trips as Option 1.
•    Removes the need to persist a local copy.
•    Provides the same user experience as Option 2 (assuming that the data pages are loaded before a user needs the data).

For Options 3 and 4, the wished-for design for a solution is a trade-off between time to implement and maintain, with little difference in user experience between the two options, because the initial loading problem is solved. Other contributing factors that might weigh into the evaluation are potential rule reuse and future evolution of the application.