Background processing options

Pega Platform supports several options for background processing. You can use Queue Processors, Standard and Advanced agents, Job Scheduler, Service level agreements (SLAs), and the Wait shape to design background processing in your application.

Queue Processor allow you to focus on configuring the specific operations to perform. When using Queue Processor, Pega Platform provides built-in capabilities for error handling, queuing and dequeing, and commits. . Queue Processors are often used in an application that stems from a common framework or used by the Pega Platform itself.

By default, Queue Processors run in the security context of the ASYNCPROCESSOR requestor type When configuring the Queue-For-Processing method in an Activity, or the Run in Background step in a Stage, is it possible to specify an alternate Access Group. It is also possible for the activity that the Queue Processor runs to change the Access Group. An example is the Rule-Test-Suite pzInitiateTestSuiteRun activity executed by the pzInitiateTestSuiteRun Queue Processor.

Queues are multithreading and shared across all nodes. Each queue processor can process messages across 20 partitions, which means that Queue Processor rules can support up to 20 separate processing threads simultaneously with no conflict. also throughput can be improved by leveraging multiple Queue Processors on separate nodes to process the items in a queue.

Use standard queue processors for simple queue management or dedicated queue processors for customized or delayed processing of messages. If we define Queue Processor as delayed  then define date and time while calling via method Queue-For-Processing or a Run in Background smart shape.

Standard agents are generally preferred when you have items queued for processing. Standard agents allow you to focus on configuring the specific operations to perform. When using standard agents, Pega Platform provides built-in capabilities for error handling, queuing and dequeing, and commits.

By default, standard agents run in the security context of the person who queued the task. This approach can be advantageous in a situation where users with different access groups leverage the same agent. Standard agents are often used in an application with many implementations that stem from a common framework or in default agents provided by Pega Platform. The Access Group setting on an Agents rule only applies to Advanced Agents which are not queued. To always run a standard agent in a given security context, you need to switch the queued Access Group by overriding the System-Default-EstablishContext activity and invoke the setActiveAccessGroup() java method within that activity.

Queues are shared across all nodes. The throughput can be improved by leveraging multiple standard agents on separate nodes to process the items in a queue.

Use Job Schedulers when there is no requirement to queue a reoccurring task. Unlike Queue Processors, the Job Scheduler not only must decide which records to process, it also must establish each record’s step page context before performing work on that record. For example, if you need to generate statistics every midnight for reporting purposes, the output of a report definition can determine the list of items to process. The Job Scheduler must then operate on each item in the list.

Unlike Queue Processors, a Job Scheduler needs to decide whether a record needs to be locked. It also must decide whether it needs to commit records that have been updated using Obj-Save. If, say, a Job Scheduler creates a case, or opens a case with a lock and causes it to move to a new assignment or complete its life cycle, it would not be necessary for the Job Scheduler to issue a Commit.                    

Use Advanced agents when there is no requirement to queue and perform a reoccurring task. Advanced agents can also be used when there is a need for more complex queue processing. When advanced agents perform processing on items that are not queued, the advanced agent must determine the work that is to be performed. For example, if you need to generate statistics every midnight for reporting purposes,. the output of a report definition can determine the list of items to process.

In situations where an advanced agent uses queuing, all queuing operations must be handled in the agent activity.

When running on a multinode configuration, configure agent schedules so that the advanced agents coordinate their efforts. To coordinate agents, select the advanced settings Run this agent on only one node at a time and Delay next run of agent across the cluster by specified time period.

Using SLAs is a viable alternative to using an agent in some situations. The escalation activity in an SLA provides a method for you to invoke agent functionality without creating a new agent. For example, if you need to provide a solution to conditionally add a subcase at a specific time in the future, then adding a parallel step in the main case incorporating an assignment with an SLA and escalation activity can perform this action.

An SLA must always be initiated in the context of a case. Any delay in SLA process impacts the timeliness of executing the escalation activity.

The SLA Assignment Ready setting allows you to control when the assignment is ready to be processed by the SLA agent. For example, you can create an assignment today, but configure it to process tomorrow. An operator can still access the assignment if there is direct access to the assignment through a worklist or workbasket.

The Wait shape provides a viable solution in place of creating a new agent or using an SLA. The Wait shape can only be applied against a case within a flow in a step, and wait for a single event (timed or case status), before allowing the case to advance. Single-event triggers applied against a case represents the most suitable use case for the Wait shape; the desired case functionality at the designated time or status follows the Wait shape execution.

Within the provided FSG Booking application there is a good example of where a Timer Wait Shape could be used. The Timer can be used in a loop-back polling situation, where a user may want to have an operation executed immediately within the loop-back. In this example, a user may want to poll for the current weather forecast instead of waiting for the next automated retrieval to occur. As shown, this loop-back can be implemented in parallel to a user task such as flagging weather preparation set up and tear down task completion. It would be overly complex to update a Queue Processor’s record to fire as soon as possible, then have to wait several seconds to see the result. As stated earlier, an SLA should not be used for polling or periodic update situations.

Question: As part of an underwriting process, the application must generate a risk factor for a loan and insert the risk factor into the Loan case. The risk factor generation is an intensive calculation that requires several minutes to run. The calculation slows down the environment. You would like to have all risk factor calculations run automatically between the hours of 10:00 P.M. and 6:00 A.M. to avoid the slowdown during daytime working hours. Design a solution to support this.

Answer: Use a Delayed Dedicated Queue Processor. Set the Date Time for Processing to 10:00PM

OR

Create a standard agent to perform the calculation. Include a step in the flow to queue the case for the agent. Pause the case processing and wait for the agent to complete processing.

This solution delays the loan process and waits for the agent to resume the flow. It can take advantage of other claims processing agents if enabled on other nodes which may reduce the time it take stop process all of the loan risk assessments.

Question: You need to automate a claim adjudication process in which files containing claims are parsed, verified, adjudicated. Claims which pass those initial steps are automatically created for further processing. A single file containing up to 1,000 claims is received daily before 5:00 P.M. Claim verification is simple and takes a few milliseconds but claim adjudication might take up to five minutes.

Answer: In an activity invoke the Queue-For-Processing method against each claim.

OR

Create a standard agent to perform the calculation. Include a step in the flow to queue the case for the agent. Pause the case processing and wait for the agent to complete processing

Using the File service activity to only verify claims and then offload the task to the agent is preferred because it does not significantly impact the intake process. It can also take advantage of multinode processing if available. Furthermore, the modular design of the tasks would allow for reuse and extensibility if required in the future. However, if you use the same file service activity for claim adjudication, it impacts the time required to process the file. Processing is only available on a single node and there is little control over the time frame while the file service runs. Extensibility and error handling might also be more challenging. Consideration must be given to the time an agent requires to perform the task. For example, the time required to process the claims by a single agent is 5,000 minutes (83.33 hours). This is not suitable for a single agent running on a single node to complete the task. A system with the agent enabled on eight nodes could perform the task in the off-hours. If only a single node is available, an alternative solution is to split the file up into smaller parts, which are then scheduled for different agents (assuming there is enough CPU available for each agent to perform its task).

Question: ABC Company is a distributor of discount wines and uses Pega platform for order tracking. There are up to 100 orders per day, with up to 40 different line items in each order specifying the product and quantity. There are up to 5,000 varieties of wines that continuously change over time as new wines are added to and dropped from the list. ABC Company want to extend the functionality of the order tracking application to determine recent hot-selling items by recording the top 10 items ordered by volume each day. This information is populated in a table and used to ease historical reporting.

Answer: Use Job Schedulers.

OR

An advanced agent runs after the close of business each day, and it performs the following tasks:
• Opens all order cases for that day and tabulates the order volume for each item type
• Determines the top 10 items ordered and records these in the historical reporting table

The agent activity should leverage a report to easily retrieve and sort the number of items ordered in a day. When recording values in the historical table, a commit and error handling step must be included in the activity.