2.3.5. Test Types and Test Levels

It is possible to perform any of the test types mentioned above at any test level. To illustrate, examples of functional, non-functional, white-box, and change-related tests will be given across all test levels, for a banking application, starting with functional tests:

• For component testing, tests are designed based on how a component should calculate compound interest.

• For component integration testing, tests are designed based on how account information captured at the user interface is passed to the business logic.

• For system testing, tests are designed based on how account holders can apply for a line of credit on their checking accounts.

• For system integration testing, tests are designed based on how the system uses an external microservice to check an account holder’s credit score.

• For acceptance testing, tests are designed based on how the banker handles approving or declining a credit application.

The following are examples of non-functional tests:

• For component testing, performance tests are designed to evaluate the number of CPU cycles required to perform a complex total interest calculation.

• For component integration testing, security tests are designed for buffer overflow vulnerabilities due to data passed from the user interface to the business logic.

• For system testing, portability tests are designed to check whether the presentation layer works on all supported browsers and mobile devices.

• For system integration testing, reliability tests are designed to evaluate system robustness if the credit score microservice fails to respond.

• For acceptance testing, usability tests are designed to evaluate the accessibility of the banker’s credit processing interface for people with disabilities.

The following are examples of white-box tests:

• For component testing, tests are designed to achieve complete statement and decision coverage (see section 4.3) for all components that perform financial calculations.

• For component integration testing, tests are designed to exercise how each screen in the browser interface passes data to the next screen and to the business logic.

• For system testing, tests are designed to cover sequences of web pages that can occur during a credit line application.  For system integration testing, tests are designed to exercise all possible inquiry types sent to the credit score microservice.

• For acceptance testing, tests are designed to cover all supported financial data file structures and value ranges for bank-to-bank transfers.

Finally, the following are examples for change-related tests:

• For component testing, automated regression tests are built for each component and included within the continuous integration framework.

• For component integration testing, tests are designed to confirm fixes to interface-related defects as the fixes are checked into the code repository.

• For system testing, all tests for a given workflow are re-executed if any screen on that workflow changes.

• For system integration testing, tests of the application interacting with the credit scoring microservice are re-executed daily as part of continuous deployment of that microservice.

• For acceptance testing, all previously-failed tests are re-executed after a defect found in acceptance testing is fixed.

While this section provides examples of every test type across every level, it is not necessary, for all software, to have every test type represented across every level. However, it is important to run applicable test types at each level, especially the earliest level where the test type occurs.