Timeframes for Acceptance Testing must be realistic, planned, and signed-off by management and endorsed by stakeholders.

Everyone involved in Acceptance Testing should be relieved of their normal duties to a level that allows them to meet their testing commitments.

Suppliers of the system must thoroughly test the system and guarantee it’s stability and quality before handing it over for Acceptance Testing.

Train all Acceptance Testing participants in the use of the system and the testing procedures.

Inform management and stakeholders of progress regularly, and keep the Acceptance Test Plan up-to-date.

Never overlook the end-user who has the best knowledge of the business area under test and how to most successfully improve the business processes.
Give the testing staff ready access to technical support personnel who maintain the test environment, including network administrators, database administrators, and software specialists.

There are key steps in Acceptance Testing that must be considered no matter the size or complexity of the Project. The extent of an Acceptance Test may range from testing a small change to an existing system, to implementing a large new package or custom-built system. Whatever the extent, there are fundamental steps to be followed.

It is important the person responsible for Acceptance Testing formalises the testing process and arranges for the necessary personnel and resources to be available. This requires preparation and planning, and quite often training as well.

Phase 1 Planning

• Prepare a Test Plan
• Review the Test Plan with participants and stakeholders
• Arrange sign off of the Plan

Phase 2 Preparing Tests, Test Data, and Training

• Prepare the tests (Test Cases)
• Prepare the Test Scripts (or Test Scenarios)
• Prepare the Test Data
• Conduct training for the participants
• Establish the Test Environment
• Confirm the availability of personnel and resources

Phase 3 Executing and Controlling

• Run the tests (Test Scripts and Test Cases)
• Record the results
• Log problems and monitor their resolution
• Fix problems and re-test

Phase 4 Closure

• Arrange for formal acceptance of the system
• Hand over the system, the documentation and test records

Before any system is handed over to users for Acceptance Testing, the system suppliers and/or developers have tested it thoroughly at various levels.  These tests are usually technically-focused and only one part of the Acceptance Testing process.  The IT department will also have input into Acceptance Testing based upon technical Acceptance Criteria and report to the Project Manager.

Unfortunately, the computer professionals will not understand your business as well as you do.  Computer personnel will view the system from a technical rather than business-user perspective.  Given they had a hand in constructing the system, they may have developed misconceptions of how it should operate.

For these reasons it is best for end-users to Acceptance Test their own system.

When the system is ‘web-based’ or has web components, testing effort should be increased to allow for full useability testing and for testing against different browser versions and platforms. Web pages are subject to many types of web interfaces and require more extensive useability testing than standard structured and controlled graphical user interface screens. Web-based systems have a number of characteristics that necessitate additional care during testing:

• There is less (or no) control over the environment in which the system will run. Generally, the environment for an Intranet based system can be controlled to some extent, whereas for a system on the World Wide Web, there is no control;
• The system may need to run using different browsers and under a number of different versions;
• There is little opportunity to train end-users and their actions and logic are not easily predicted;
• There is no capability of controlling the sequence of events that an end-user may invoke.

Consequently, testing of web pages is often less structured, allowing the testers free reign in their (test) use of the system, possibly with an observer or recording mechanism to monitor events. Test Plan preparation will need to allow for this.

The expected duration for an Acceptance Test is also derived from the system’s value in function points.  For each 1000 function points, usually allow one calendar month of Acceptance Testing.

Precise duration is more evident once the detailed Plan and “bottom up” estimates have been prepared, and in light of the number of participants and their availability.

In general, the Acceptance Test Plan preparation phase is usually about 2% of the overall effort.  As an example, a system that requires 4000 hours of development will require in the order of 80 hours of resources dedicated to Test planning.

To gain an appreciation of the resources that must be assigned to Acceptance Testing, there are several ways to estimate the effort required for individual stages of the Test process.  In addition, web-based systems present their own ‘peculiarities’ and are assessed using separate criteria.  Any and every project should be subjected to some degree of Acceptance Testing.

No matter how thorough the testing by the system suppliers, given the nature of the technology it is likely the new system will still encounter ‘unexpected’ problems.  For end-users, it is far better to identify and fix these problems during Acceptance Testing before the development team disbands, than to leave it until later when the time taken to rectify problems (say, under warranty) will be much longer.  The costs involved in rectifying problems at this stage may also be greater and outside the scope of the original budget.

The consequences of completely ignoring, or not giving enough consideration, to Acceptance Testing can range from minor inconvenience to total disaster when the system goes ‘live’.  To make things even worse, the external contractors responsible for supplying the system may have formally handed it over and finalised their contractual obligations.  Any modifications will cost more money!

Preparation of the Acceptance Test Plan should start as soon as possible after the development of a specification of business requirements to help clarify gaps in the proposed specification.  It then continues with project commencement and runs in parallel with system development.  The logistics of conducting Acceptance Tests may affect the system design and development process and have significant impact on the system requirements.

Because of their importance in any supplier contract, Acceptance Criteria should be considered during preparation of any Request for Tender or Quotation and in contract negotiations.  Include them with any other special requirements in the requirements documentation.

Key stakeholders must be involved in the Acceptance Testing process and will usually include any of the following:

• Representatives of the Business Units that will either use or be beneficiaries of the system outputs;
• Business Units that will be responsible for operation of the system; and/or
• Business Units that will be responsible for maintenance of the system.

The Business Unit that ‘owns’ the system usually has responsibility for making sure that Acceptance Testing is carried out.  This means the Manager of that Business Unit (or a delegated officer) holds ultimate responsibility for the Testing process, including:

• Preparation of an Acceptance Test Plan and Test Cases;
• Management of the Testing activities; and
• Ensuring sign-off and official acceptance of the system.

In addition, there is a need for Testing of a more technical nature, often conducted by the system supplier, with additional advice and guidance provided by the Information Management Branch in your Division.

There are many individuals and groups that must become involved in the Acceptance Testing process, under the direction of the Acceptance Test Manager.  The roles and responsibilities of all those involved in Acceptance Testing are discussed in Part 2 of the Kit.

Test participants should be told of the importance of their involvement and, wherever possible, be included in Test planning to increase their commitment to a successful outcome.  Also, make sure they are given realistic time frames for Testing. They may need to be relieved of their normal day-to-day responsibilities and duties to allow for this.

Usually in any Business Unit that operates computer systems, there are a number of experienced and dedicated system end-users who have an in-depth understanding of the business processes involved.  The benefits these individuals can bring to a Project are significant.  Their involvement in Acceptance Testing is highly desirable, if not essential, especially in the business areas with which they are familiar.