2.2.4. Acceptance Testing
Objectives of acceptance testing
Acceptance testing, like system testing, typically focuses on the behavior and capabilities of a whole system or product. Objectives of acceptance testing include:
Establishing confidence in the quality of the system as a whole
Validating that the system is complete and will work as expected
Verifying that functional and non-functional behaviors of the system are as specified
Acceptance testing may produce information to assess the system’s readiness for deployment and use by the customer (end-user). Defects may be found during acceptance testing, but finding defects is often not an objective, and finding a significant number of defects during acceptance testing may in some cases be considered a major project risk. Acceptance testing may also satisfy legal or regulatory requirements or standards.
Common forms of acceptance testing include the following:
User acceptance testing
Operational acceptance testing
Contractual and regulatory acceptance testing
Alpha and beta testing.
Each is described in the following four subsections.
User acceptance testing (UAT)
User acceptance testing of the system is typically focused on validating the fitness for use of the system by intended users in a real or simulated operational environment. The main objective is building confidence that the users can use the system to meet their needs, fulfill requirements, and perform business processes with minimum difficulty, cost, and risk.
Operational acceptance testing (OAT)
The acceptance testing of the system by operations or systems administration staff is usually performed in a (simulated) production environment. The tests focus on operational aspects, and may include:
Testing of backup and restore
Installing, uninstalling and upgrading
Disaster recovery
User management
Maintenance tasks Data load and migration tasks
Checks for security vulnerabilities
Performance testing
The main objective of operational acceptance testing is building confidence that the operators or system administrators can keep the system working properly for the users in the operational environment, even under exceptional or difficult conditions.
Contractual and regulatory acceptance testing
Contractual acceptance testing is performed against a contract’s acceptance criteria for producing custom-developed software. Acceptance criteria should be defined when the parties agree to the contract. Contractual acceptance testing is often performed by users or by independent testers.
Regulatory acceptance testing is performed against any regulations that must be adhered to, such as government, legal, or safety regulations. Regulatory acceptance testing is often performed by users or by independent testers, sometimes with the results being witnessed or audited by regulatory agencies.
The main objective of contractual and regulatory acceptance testing is building confidence that contractual or regulatory compliance has been achieved.
Alpha and beta testing
Alpha and beta testing are typically used by developers of commercial off-the-shelf (COTS) software who want to get feedback from potential or existing users, customers, and/or operators before the software product is put on the market. Alpha testing is performed at the developing organization’s site, not by the development team, but by potential or existing customers, and/or operators or an independent test team. Beta testing is performed by potential or existing customers, and/or operators at their own locations. Beta testing may come after alpha testing, or may occur without any preceding alpha testing having occurred.
One objective of alpha and beta testing is building confidence among potential or existing customers, and/or operators that they can use the system under normal, everyday conditions, and in the operational environment(s) to achieve their objectives with minimum difficulty, cost, and risk. Another objective may be the detection of defects related to the conditions and environment(s) in which the system will be used, especially when those conditions and environment(s) are difficult to replicate by the development team.
Test basis
Examples of work products that can be used as a test basis for any form of acceptance testing include:
Business processes
User or business requirements
Regulations, legal contracts and standards
Use cases and/or user stories
System requirements
System or user documentation
Installation procedures
Risk analysis reports
In addition, as a test basis for deriving test cases for operational acceptance testing, one or more of the following work products can be used:
Backup and restore procedures
Disaster recovery procedures
Non-functional requirements
Operations documentation
Deployment and installation instructions
Performance targets
Database packages
Security standards or regulations
Typical test objects
Typical test objects for any form of acceptance testing include:
System under test
System configuration and configuration data
Business processes for a fully integrated system
Recovery systems and hot sites (for business continuity and disaster recovery testing)
Operational and maintenance processes
Forms
Reports
Existing and converted production data
Typical defects and failures
Examples of typical defects for any form of acceptance testing include:
System workflows do not meet business or user requirements
Business rules are not implemented correctly
System does not satisfy contractual or regulatory requirements
Non-functional failures such as security vulnerabilities, inadequate performance efficiency under high loads, or improper operation on a supported platform
Specific approaches and responsibilities
Acceptance testing is often the responsibility of the customers, business users, product owners, or operators of a system, and other stakeholders may be involved as well.
Acceptance testing is often thought of as the last test level in a sequential development lifecycle, but it may also occur at other times, for example:
Acceptance testing of a COTS software product may occur when it is installed or integrated
Acceptance testing of a new functional enhancement may occur before system testing
In iterative development, project teams can employ various forms of acceptance testing during and at the end of each iteration, such as those focused on verifying a new feature against its acceptance criteria and those focused on validating that a new feature satisfies the users’ needs. In addition, alpha tests and beta tests may occur, either at the end of each iteration, after the completion of each iteration, or after a series of iterations. User acceptance tests, operational acceptance tests, regulatory acceptance tests, and contractual acceptance tests also may occur, either at the close of each iteration, after the completion of each iteration, or after a series of iterations.
Last updated