Background process management

Activities provide flexibility to automate system processing tasks. For example, an organization wants to offer a 20 percent discount on products that will expire in seven days. A job scheduler runs an activity once a day to perform this task. To enable this process, you first configure a report that filters products that will expire seven days from the date of the report. Then, you create a job scheduler activity to run the report, apply the discount to the expiring products, and update the product prices in the database.

The following video illustrates the activity that begins each time the job scheduler wakes up.

Video transcript

The job scheduler wakes up at 12:00 A.M. and generates a report that will contain a list of products that will expire in seven days. The report retrieves the products that will expire in seven days. The job scheduler locks the products and applies the discount to the expiring products. The job scheduler commits the updates to the database. When the updates are complete, the job scheduler waits and wakes up again in 24 hours.

Queue processors and job schedulers handle read/write transactions on database objects differently. As shown in the video, a job scheduler includes steps in the activity to identify the items expiring in seven days, apply the discount, and then save and commit the change. Job schedulers use activity to manage any read/write operations they perform on database objects. Queue processors automatically manage the read/write transactions on database objects. If a queue processor needs to update an object, the activity should not contain any steps to perform the read/write objects to the database as part of the process.

Queue processors and job schedulers are background jobs that use an activity to implement processing logic. These activities expect to finish processing in less than a few seconds. If an activity takes longer to complete, use the following best practices to ensure that work items are not affected during the shutdown process.

Create small work items

Break down large work items into smaller work items. Queue them to a queue processor, and process them individually. If you have background tasks that break into multiple tasks with different performance capabilities, consider dividing them and scaling each independently.

Use checkpoints

To ensure that activity logic can gracefully handle restarts without corrupting data or introducing inconsistencies into your application, use checkpoints by saving the state of jobs in persistent storage or as messages in a queue that is appropriate to the use case.

As shown in the following image, consider a scenario where a work object can have the following statuses: New, In-Progress, and Processed. You can read all of the work object records to be processed, loop through them one by one, mark the status of the current work object as In-Progress, continue with the business logic, and when processing is complete, change the status to Processed.

Break business logic from transaction logic

When configuring job scheduler activities, separate business logic from transaction logic. For example, a company sends congratulatory letters to employees celebrating work anniversaries. The business logic identifies employees with work anniversaries. The transaction logic executes the process to send the letter. Logic separation enables you to adjust processing if the activity cannot obtain a lock. To separate the logic, you can create one activity for the business logic and a second activity for the transaction logic.

Queue processors and job schedulers can fail to process items correctly for several reasons. For example, the processes cannot obtain a lock on an entry because the processes lack security access or another user or process already has a lock on the object.

Queue processors queue objects and then process them. As shown in the following image, if the process fails and the system cannot commit a queue entry, the system puts the entry into failure status and indicates the process is broken. The system also reverses any changes the process initiated.

In Admin Studio, you use the Queue processor landing page to trace and monitor Queue Processor rules in your application. If there is an error, users with the SysAdmin4 role can troubleshoot issues with the queue processor by tracing the running processes and examining the items in the Broken queue.

The job scheduler triggers items to process at preconfigured times. The job scheduler does not queue items. To troubleshoot issues with job scheduler processes, you can run a trace on the failing processes. You can also configure the Performance tool to manage statistic aggregation for job scheduler rules. 

Check your knowledge with the following interaction: