Monitor application quality

Showing and measuring the quality of an application on a regular basis is essential to building robust applications. Quality metrics consists of two types of metrics:

• Functional (Logic/Design related)
• Non-functional (System/Infrastructure) metrics.

No list of defined metrics exists. Every product and company can develop its own metrics to measure quality.

A rule is considered covered if it is invoked at least once during the testing session. The following image depicts the 26 rule types that the feature supports:

The Test coverage report includes coverage information about only the supported rule types (listed above).

Check your knowledge with the following interaction:

The following image shows how test coverage projects the metrics for an application that has 1,000 rules, from which 600 rules belong to supported rule types:

When a test coverage report is generated in Developer Studio, it looks like the image shown below. Rules that belong to the supported rule types are shown as "Executable rules". Counts of "Covered rules" and "Uncovered rules" along with the complete list of rules belonging to each type are also shown.

Test coverage is generated at the following two levels:

• User level: A user-level coverage report can be used to generate a local test coverage report by a single user in a single session. If the user wants to generate a test report against a set of test cases in the application, and does not want the report to be considered for the purpose of quality metrics, the User level option can be used. This is entirely a user-specific report.
• Application level: An application-level coverage report can be used to generate a test coverage report for the entire application, with the testing performed by multiple people across multiple sessions. This is useful when regression testing an entire application. This metric can be found in the application quality metrics dashboard for the application.

Check your knowledge with the following interaction:

The Test coverage feature also offers the following capabilities:

• Merge test coverage reports: It is possible to merge most recent test reports of different applications or test reports of multiple built-on applications into a single test report to obtain a consolidated overview of the test coverage of the entire application stack. This is especially helpful when multiple development teams are working on a single base application by building multiple built-on development applications on top of it.
• Test coverage of recently updated rules: If a team is developing new rules for an existing application from last two sprints and has authored test cases for these new rules, they can use this feature to find the coverage of all the new rules against a specific test session. To perform this, the team must supply a Date filter to consider only those rules which have been created or updated after a certain date.
• Test coverage history: History of both the User-level and Application-level test coverage reports can be accessed to understand the progress over a period of time.