5.1.1 Independent Testing

Testing tasks may be done by people in a specific testing role, or by people in another role (e.g., customers). A certain degree of independence often makes the tester more effective at finding defects due to differences between the author’s and the tester’s cognitive biases (see section 1.5). Independence is not, however, a replacement for familiarity, and developers can efficiently find many defects in their own code.

Degrees of independence in testing include the following (from low level of independence to high level):

• No independent testers; the only form of testing available is developers testing their own code

• Independent developers or testers within the development teams or the project team; this could be developers testing their colleagues’ products

• Independent test team or group within the organization, reporting to project management or executive management

• Independent testers from the business organization or user community, or with specializations in specific test types such as usability, security, performance, regulatory/compliance, or portability

• Independent testers external to the organization, either working on-site (in-house) or off-site (outsourcing)

For most types of projects, it is usually best to have multiple test levels, with some of these levels handled by independent testers. Developers should participate in testing, especially at the lower levels, so as to exercise control over the quality of their own work.

The way in which independence of testing is implemented varies depending on the software development lifecycle model. For example, in Agile development, testers may be part of a development team. In some organizations using Agile methods, these testers may be considered part of a larger independent test team as well. In addition, in such organizations, product owners may perform acceptance testing to validate user stories at the end of each iteration.

Potential benefits of test independence include:

• Independent testers are likely to recognize different kinds of failures compared to developers because of their different backgrounds, technical perspectives, and biases

• An independent tester can verify, challenge, or disprove assumptions made by stakeholders during specification and implementation of the system

• Independent testers of a vendor can report in an upright and objective manner about the system under test without (political) pressure of the company that hired them

Potential drawbacks of test independence include:

• Isolation from the development team, may lead to a lack of collaboration, delays in providing feedback to the development team, or an adversarial relationship with the development team

• Developers may lose a sense of responsibility for quality

• Independent testers may be seen as a bottleneck

• Independent testers may lack some important information (e.g., about the test object)

Many organizations are able to successfully achieve the benefits of test independence while avoiding the drawbacks.