B2E UX quality metrics

As companies invest more in their user experience organizations, they will need to establish ways to measure the results of their UX efforts. UX organizations should use a combination of quantitative and qualitative metrics to measure results.

• Quantitative metrics are numerically driven and provide concrete information about user behavior. These metrics may include metrics such as task success (how many end users can complete a task), task time (how long a task takes to complete), error rate (how many errors occur during the task completion process).
• Qualitative metrics measure the quality of interactions with customers, and may or may not be numerically driven. These metrics may include various employee/consumer expectations, qualitative feedback, and consumer sentiment metrics.

As an example, Pega used a combination of qualitative and quantitative metrics to improve Agile Studio.

• Initial research uncovered that employees used spreadsheets to track projects, which was time-consuming to edit and relied on one extra file.
• Pega Design ran a series of qualitative interviews to see how a new design for a digital tool might fit into employees' current processes. Then, the team surveyed employees to understand how well projects with dependencies were run at the time (on a scale of 1-10), and how often employees worked on such projects.
• The resulting analysis, prioritization of results, and prototype usability testing helped the team develop and implement an initial design for Agile Studio.

Today there are many frameworks used to measure the performance of user experience organizations at companies.

The AARRR Funnel 

One notable framework is known as AARRR, or "the Pirate Funnel." Developed by venture capitalist and 500 Startups founder Dave McClure, this framework measures startup performance using key milestones:

• Acquisition (getting users)
• Activation (user sign-ups, downloads)
• Retention (number of users that use the product repeatedly)
• Referral (promotions, recommending to friends or colleagues)
• Revenue (generating revenue for the company)

Mainly used in scaling a business quickly, this framework allowed companies to find the weakest point of its performance from its users' perspective. It was one of the first growth frameworks to prioritize user experience to drive company decisions explicitly. The Pirate Funnel is criticized for its aggressive focus on growth above other metrics. Still, it serves as an example of prioritizing the user needs to achieve and contextualize company goals.

Pega's OOOO Metrics 

Pega believes that enterprise organizations have slightly different measurement needs for their applications. Pega created a set of four metrics to evaluate the performance of UX/design organizations.

The OOOO Metrics include: 

• Onboarding – The time it takes to train someone to use an application. Pega products are designed to reduce training time.
• Operations – The amount of accurate work a user can achieve. The goals may be slightly different for each solution; sometimes, Pega designs for the maximum number of pieces of work, and sometimes it is the accuracy that is more important than productivity. (Productivity is measurable as well.)
• Observation –  The ability to monitor application usage and drive behavioral insights from actual users' usage.
• Optimization – The ability to update and improve product-based on gathered insights. Optimization drives most of the work in Pega Express, Agile Workbench, and other contextual authoring pieces of the Pega Platform™. These built-in Pega Platform features give Pega clients a huge advantage in iteratively improving their applications.