User Acceptance Testing (UAT) is a challenging phase of the testing life cycle and one of the most critical for the development organisation and customer alike.
UAT is commonly the final stage of testing that allows the customer to accept the software that’s been developed. The development team can celebrate a successful delivery and the customer can start using new software and realise the benefits it brings. Ensuring the UAT phase goes as expected is therefore important to everyone involved. While it’s usually the final phase of testing, it might actually be followed by other forms of testing.
Defining User Acceptance Testing (UAT)
At the point of conducting UAT the software is agreed to be working from a functional perspective, as it’s already been through numerous earlier testing phases, and now needs to be validated from the business user perspective.
The UAT phase typically takes place after the test team have completed System testing and before the Operations team complete Operational Acceptance Testing (OAT), hence why UAT isn’t always the last test phase, in readiness for putting the software live.
Essentially, UAT is testing that the end user performs to ensure the software allows them to carry out the daily working tasks they need to perform with the software. They test that the software can do these tasks accurately and in a way that is easy to understand and efficient.
UAT in a traditional / waterfall development environment
Where the phases of software development are performed in a sequential manner (aka Waterfall) UAT will be conducted late in the project. It will typically be a single phase with bug fixes and retesting completed as part of the planned UAT phase.
UAT in an Agile development environment
When a more agile approach to software development is followed it’s possible to perform UAT several times. If the customer can accept a staged delivery of the complete system, by taking releases that deliver some of the agreed functionality each time a release is done (increments), then UAT can be performed against each release.
In this way the customer can provide feedback as to how easy to use and efficient the software is and how well it allows them to do their daily tasks. This feedback can then be incorporated into new releases. This is typical of the Scrum software development model.
Testing
When we talk about performing UAT we mean the business user running a series of tests that have been agreed in advance and that will show them the new software does what they want.
The tests are called Acceptance Tests and are derived from the Requirements, User Stories or Use Cases that were written down at the start of the project. UAT isn’t the only form of Acceptance Tests.
Writing the Tests
While the idea is that the business users will write the acceptance tests this is often impractical. They will have the domain knowledge of the business but not the domain knowledge of testing, meaning they may simply not know how to write congruent, well structures test sets. Therefore, the test team will usually try to work with the business user at the start of the project to capture the acceptance tests they will want to run. The amount of tests that will be run during UAT will be at least the number of Use Cases and possibly more if there are many variants or system conditions the business user want to validate.
In addition, there may be a number of business user groups, perhaps representing various departments, with their own tests to run. The test team will have to consider how to manage the sets of tests, assign them to the groups to run and then collate the results to see how the UAT phase is running overall.
Managing the Tests
The easiest way to manage the tests is via a test management tool designed for that very purpose. Atlassian JIRA is such a tool which allows the acceptance tests to be written, modified or added to as needed and then assigned to various groups or individuals for ownership or for running.
When the tests are run the test team can create specific Test Runs with collections of test cases in. Once they’re run a report is automatically generated so the team can see the current status of testing, checking how many test cases assed or failed, bugs raised, etc.
Challenge – User Participation
Despite the objective of UAT being that business users specify and conduct the testing it can often be difficult to get them to actually do this. As mentioned, part of the reason is that the testing process and practice they need to apply is so far removed from their day to day jobs as to be somewhat of a mystery.
A key responsibility of the test team it to make it easy for the business users to be part of the UAT and ultimately capable of owning it. Using a tool such as JIRA helps as it provides a common tool for all involved to use. This means a preparatory step, before any tests are written, is making sure the business users are set-up on the test management tools and adequately trained.
Other Types of Acceptance Testing
In addition to UAT there are other types of Acceptance testing that are performed either before or after the UAT phase. These are also important to manage to the same degree as UAT and the approach to UAT can also be applied to these further Acceptance testing phases.
Alpha and Beta Phases
Once the System Testing is completed the application is moved into an environment within the development organisation and is set-up to use data derived by the same to allow the demonstration and first exposure of the application to the customer in a controlled environment. The next step is Beta Testing.
Operational Acceptance Testing (OAT)
This testing is part of the Acceptance test type and follows the User Acceptance Testing phase. At this point the software is agreed to be working from the business perspective and the testing now is to ensure it can be deployed and made operationally ready for use as live software. Forms of OAT include Installation Testing and Disaster Recovery.
Challenge – Getting OAT Agreed
It’s not uncommon on projects to see the need for OAT get missed and often blended in to UAT. They are not the same – UAT and OAT exist to achieve very different objectives and are the concern of very different teams. When planning your test programmes make sure the need for the different test phases is understood early on.
Mark.
