Testing strategies en masse with simulations

Scenario

U+ Bank is currently doing cross-sell on the web by showing various credit cards to its customers.

They would like to implement a change in their eligibility criteria and their offering. To understand the impact of these changes, U+ would like to run some Next-Best-Action simulations before they implement them.

U+ is currently using the real-time container (RTC) TopOffers, for the web channel, which is mapped to the Business Issue-level structure: Sales->Credit Cards.

Use the following credentials to log in to the exercise system:

Your assignment consists of the following tasks:

Task 1: Check the configurations

Check the NBA Designer configurations U+ Bank currently uses to present customers with relevant offers.

Task 2: Execute a baseline simulation

Create and execute a baseline simulation before implementing any changes. Use the Customer Data as input and output the results into a Visual Business Director data source, called Baseline. Inspect the result once the simulation is executed.

Task 3: Alter eligibility criteria

Alter the eligibility criteria (Age) of the CreditCards Group to offer credit cards to customers over 21 instead of over 18.

Task 4: Execute a new simulation

Assess the impact of the change by executing a new simulation. Output the results in the Simulation data source. Examine the results using the Delta view in Visual Business Director.

Task 5: Disable an action and execute a new simulation

Disable the Standard credit card offer and re-execute the simulation. Examine the results using the Delta view in Visual Business Director.

Challenge Walkthrough

Detailed Tasks

1 Check the configurations

1. Log in as Decisioning Analyst with user name CDHAnalyst and password rules.
2. In the navigation pane on the left, click Next-Best-Action > Designer.
3. Click through the tabs to see the configurations made to present customers with relevant offers.

Note: NBA Designer generates the main trigger strategy, Trigger_NBA_Sales_CreditCards, which is executed for all incoming RTC requests. This strategy does not contain the outcome set. For Visual Business Director simulation purposes, you need to have an outcome set for visualizing the KPIs. The strategy Trigger_NBA_Sales_CreditCards_Behavior was created for this purpose. This strategy references the NBA Designer-generated trigger strategy, which considers all decisions as Impressions. This is the decision strategy you simulate.

2 Execute a baseline simulation

1. In the navigation pane on the left, click Data > Data Flows.
2. Search and open the PrepareCustomerData_REF data flow to prepare the data set used for simulations. This data is based on a Monte Carlo dataset, which is generated.

Note: The Monte Carlo data set generates a mock data set. Therefore, different simulation runs will have different results.

3. From the Actions menu, select Run.

Note: Notice that the prospect data is populated once the test run is complete.

4. Close the data flow Test run window.
5. In the navigation pane on the left, click Intelligence > Strategies.
6. Search for the Trigger_NBA_Sales_CreditCards_Behavior strategy.
7. Click Trigger_NBA_Sales_CreditCards_Behavior.
8. Click Actions > Simulate to open the simulation configuration panel.
9. In the Audience area, click Configure.
   
10. In the Audience pane,  to the right of the Customer Data data set, click Add.
11. Click Apply.
12. In the Assign output destinations section, click Create New to create a new output.
    
13. In the new window, enter the following information:
    1. Name: Baseline
    2.  Type: Visual Business Director.
14. Click Done to close the window. 
15. Click Submit and run to execute the simulation.
    
16. Once the Status has changed to Completed, in the Assigned reports section, click Baseline to open the Visual Business Director.
17. At the top of the 3D view, click Total accepted, and then select the Total Impressions KPI.
18. Click OK.
19. Click Credit Cards to view the three available cards. 
    
    Note: The distribution of the 10,000 customer data test is recorded in the three CreditCards groups.

3 Alter eligibility criteria

1. In Customer Decision Hub, in the navigation pane on the left, click Next-Best-Action > Designer.
2. On the Next-Best-Action Designer Home page, click Engagement policy.
3. In the Business structure section, click CreditCards.
4. Click Edit to modify the conditions.
5. In the Eligibility area, modify the eligibility criteria to Age > 21
   
6. Click Save to save the changes.

4 Execute a new simulation

1. In the navigation pane on the left, click Intelligence > Strategies.
2. On the Strategies landing page, search for the Trigger_NBA_Sales_CreditCards_Behavior strategy.
3. Click Trigger_NBA_Sales_CreditCards_Behavior to open the decision strategy details.
4. In the decision strategy, click Actions > Simulate.
5. In the Audience area, click Configure.
   1. In the Audience pane, to the right of the Customer Data data set, click Add.
   2. Click Apply to close the pane.
6. In the Assign output destinations area, click Create New to create a new output.
7. In the Create new output window, enter the following information:
   1. Name: Simulation
   2. Type: Visual Business Director.
   3. Click Done to close the window
8. Select the Overwrite outputs from previous simulations check box to ensure that every time you re-simulate, the previous results are cleared.
9. Click Submit and run to start the simulation.
10. Once the Status changes to Completed, in the Assigned reports section, click Simulation to open the Visual Business Director.
11. At the top of the 3D view, click Total accepted to open the Select KPI dialog box.
12. In the Select KPI dialog box, from the list, select Total Impressions.
13. Click OK.
14. On the 3D view, click Credit Cards to view the three cards. 
    Note: The distribution of the 10,000 customer data test records into the three CreditCards Groups. Do you see a difference between these results and the previous simulation?

15. In the pane on the right, configure the following information.

    1. In the Reference data source list, select Baseline. 
    2. In the Data mode section, select Delta
       
16. Verify that, after the change, there are approximately 190 fewer Rewards Plus Cards offered (because you increased the age limit from 18 to 21) and that approximately 160 fewer Standard Cards are offered.

Note: The Monte Carlo data set generates a mock data set;different simulation runs will have different results.

5 Disable an action and execute a new simulation

1. In Customer Decision Hub, in the navigation pane on the left, click  Next-Best-Action > Designer to disable the Standard card offer.
2. On the Next-Best-Action Designer Home page, click Engagement policy.
3. In the Business structure section, click CreditCards.
4. Click Edit.
5. Scroll down and click Standard card.
6. At the top, click Delete to disable the Standard card offer.
7. Close the Standard card offer details, and then click Save to save the changes.
8. In the navigation pane on the left, click Simulation Testing.
9. Click Trigger_NBCustomerDa-2 to open the second execution.
   
10. Click Restart to execute the simulation with the change.
11. Click the Simulation data source.
12. Check that the Total Impressions in Delta mode decreases as a result of removing the Standard card from the list of Actions.

Note: Ensure that Baseline is selected as the Reference data source.

Available in the following mission:

• Cross-Sell on the Web Extended v1