Skip to main content

Using model-based topic detection to route emails

Archived

3 Tasks

20 mins

 Visible to: All users
Beginner Pega Customer Service 8.4 English
This content is now archived and is no longer updated. Progress is not calculated. Pega Cloud instances are disabled, and badges are no longer awarded.

Scenario

U+ Bank currently has a rule-based email routing system in place — all emails with the topic detected as address change are routed to the Account Maintenance department, and emails with the topic detected as dispute transaction are routed to the Transaction Disputes department.

The current topic detection is not extremely intelligent, as longer emails with many keyword multiple topics are detected with the same confidence factor.

U+ Bank wants the email routing system to select the most prominent topic in an email if the system detects multiple topics. Emails with a negative sentiment must go to a specialized department to optimize the customer experience.

As a data scientist, you are tasked with the creation and training of a new text categorization model that outputs a confidence score for each detected topic. The topic with the highest confidence score drives the email routing.

You must also configure a model to detect the sentiment expressed in the emails and route these emails to the appropriate department.

Use the following credentials to log in to the exercise system:

Role Username Password
System Architect CASysAdmin rules
Caution: This challenge builds on top of the previous challenge. Hence when you click on Initialize Pega or Reset Instance in this challenge, you first need to complete the previous challenge: Using rule-based topic detection to route emails.

Your assignment consists of the following tasks:

Task 1: Create a text categorization model

Switch to the model-based topic detection method, which is AI-driven. Create a new text categorization model, MySupport_TM, and train the model with the TrainingData.csv training set.

Task 2: Configure the text analyzer

Configure the text analyzer to detect sentiment and route emails with a negative sentiment to the General service requests department.

Task 3: Verify your work

Verify your work by sending emails with specific topics and sentiments to U+ Customer Service. Test the emails and observe the analysis of these emails.

Challenge Walkthrough

Detailed Tasks

1 Create a text categorization model

  1. Log in as System Architect with user name CASysAdmin and password rules.
  2. At the top of the screen, click Application: Customer Service > Channels and interfaces.
  3. Scroll down and click MySupport to open the email channel configuration.
  4. On the Behavior tab, in the Text Analyzer section, click Open text analyzer rule.
  5. In the upper right, click Check out.
  6. In the Text categorization section, select Use model-based topics if available.
    Topic preference
  1. Note: This option ensures that the topic detection looks at available models first and then applies the rule-based topic detection only if there are no models.
  2. In the Topic model field, enter MySupport_TM.
  3. Click the gear icon to create the model.
    Topic model
    1. In the Class field, enter or select Data-NLP-Taxonomy.
    2. In the Add to ruleset field, select PegaCS-Sample.
    3. Click Create and Open.
    4. Click Save.
  4. Click the Open in Prediction Studio link.
    Switch to
  5. In the navigation pane on the left, click Settings > Prediction Studio settings.
  6. In the Storage section, enter or select defaultstore as the Analytics repository. Ignore the warning.
    Warning
  7. Click Save.
  8. In the upper left, click the arrow to return to the model.
    Return
  9. Click Save, and then click Add language.
  10. In the Add language dialog box, in the Language list, select English.
  11. Select Use machine learning to begin the topic detection model creation wizard.
    Machine learning
  12. Click Start.
  13. Select MySupport as taxonomy, inspect the list of topics, and then click Next.
    Taxonomy
  14. Download the TrainingData.zip file and extract the TrainingData.csv file.
  15. Click Choose file, and then select TrainingData.csv.
  16. Click Preview to see a list of training records.
    Preview
  17. Close the window, and then click Next.
  18. In the Construct training and test sets using section, select User defined sampling.
    User defined sampling
  19. Click Next.
  20. Select one model, Maximum Entropy, and then click Next.
  21. Expand the Maximum Entropy row to view some statistics about the model created.
    Statistics
  22. Click Next.
  23. Click Add English, and then click Save.

2 Configure the Text Analyzer

  1.  In the lower left, click Back to Pega Dev Studio.
  2. Close the MySupport_TM tab.
  3. On the Select Analysis tab of the text analyzer, verify that the Topic model is MySupport_TM.
  4. Select Enable sentiment detection. The default Lexicon and Sentiment model are automatically set.
  5. On the Advanced tab, in the Topic settings section, select top 3 topics.
    Topic settings
  6. In the top right, click Check in and enter appropriate check-in comments.
  7. Close the text analyzer and open the Behavior tab of the MySupport email channel.
  8. In the Intelligent Routing section, click Add condition.
  9. In the Action list, select Route to work queue.
  10. In the Value list, select GeneralService.
  11. In the When list, select NLP > Sentiment.
  12. Set the Sentiment equal to Negative.
  13. Drag the new When rule to the top of the list to ensure that all emails with a negative sentiment are routed to the GeneralService department first.
  14. To the right of the Account Address Change when rule, click the Gear icon.
  15. Set the Confidence threshold to 0.3.
  16. To the right of the Dispute Transaction when rule, click the Gear icon.
  17. Set the Confidence threshold to 0.3.
  18. Click Save.
  19. Click Test and test the Moving to a new house email.
Subject: Moving to a new house
Body:
Dear U+,
My customer number is: 123456789.
I would like to inform you that I have a new address: 222 West Las Colinas Blvd., Irving, TX 75039, USA.
Also, I have a new email: [email protected]
Cheers, Sara
Note: The topics detected have a model-generated confidence score lower than 1, the sentiment is detected as neutral and the email is routed to the AccountMaintenance department.
  1. Click Back, and then test the following email:
Subject: Wrong again
Body:
Dear U+,
I am a very unhappy customer of yours. I always had good experiences. But last month, I have noticed several invalid transactions on my invoice. How can this happen!
Cheers, Sara
Note: The negative sentiment in the email is detected with a model-generated confidence score, and the email is routed to the GeneralService department.

3 Confirm your work

  1. Compose a text that addresses both an address change and a credit card dispute and contains both positive and negative sentiments and test it.
  2. Alter the text and observe how changes in the text influence the confidence scores for topic detection and sentiment.

Available in the following mission:

Return to mission

We'd prefer it if you saw us at our best.

Pega Academy has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice