Best Practices for Effective Factory Acceptance Testing

Best Practices for Effective Factory Acceptance Testing

When writing code for automated systems, it is necessary to demonstrate that the code is written to the customer’s specifications. At Matrix Technologies, Inc., we begin our projects by listening to the customer’s needs and writing a functional document describing how we will address those needs. Upon client agreement, we code the entire automated process according to those specifications. We then write device and process simulations to facilitate thorough integrated testing of the entire system.  Before we can field test or place a system into service, we must first complete a process called Factory Acceptance Testing (FAT).

Factory Acceptance Testing is the process of validating that our engineers wrote the code in the way that the customer expected it to function. To complete the FAT, the end user will often visit our location for a one- or two-day demonstration of software functionality. During the FAT, we demonstrate the process and isolate problems.

This article briefly describes best practices for preparing for and conducting an effective Factory Acceptance Test.

FAT Test Specifications and Preparation

Accurate simulation conditions for Factory Acceptance Testing are essential when implementing new computer process control and information systems. The FAT will usually follow a test script that is typical of how the operator will use the system, with expected versus actual results logged at each step of the test script.

The testing process begins with the development of a Test Specification, which is a document that lays the groundwork for planning and executing all stages of testing. These specifications include:

• Scope of Testing. By defining the scope and purpose of the test, all parties understand what is to be tested and the expectations or outcome of the test.
• Responsibilities, Schedule, and Manpower Requirements. Identify who is responsible for each step of the procedure and set a schedule for the testing. Gantt charts can be very helpful in this regard.
• Equipment Requirements. Identify the types of equipment needed and who will be responsible for procuring them.
• Simulation Requirements. Simulation requirements can include special I/O simulation drivers, internal code simulation, or connections into databases for simulation data.
• Testing Procedures. The procedures should be laid out in a step-by-step fashion to aid the user in performing the test effectively and efficiently. Each step in the test procedure includes a description of how the test is performed, the expected results, an area to record the test results, and a signature or initialization line for the authorized approval that the test was acceptable.

Cost Considerations

Once your test specifications have been defined, the equipment can be set up and configured for testing. Some cost-impact items to consider include the cost of procuring the equipment, labor and material costs relative to any control, power or communications wiring, and the labor and area to stage the equipment. Keep in mind that while small setups may be very inexpensive, as systems increase in size, so does the impact of planning, space, labor, and costs.

There are other cost considerations as well. Often, the equipment has been purchased with project funds; however, sometimes this equipment needs shipped on-site or to a panel shop for fabrication. If so, consider purchasing spares and utilizing them for testing. If testing interfaces to legacy systems, you may want to consider end-user spares. Also, particularly if performing prototyping, consider using vendor-supplied equipment to verify functionality before proceeding.

A demonstration might also require special I/O simulation drivers, internal code simulation, or connections into databases for simulation data. Special HMI test screens can be developed as well to simulate hardwired devices in an office environment as opposed to trying to hardwire I/O within the test setup, which can be time-consuming and costly.

Optimizing the FAT

Optimal Personnel Involvement

We recommend involving plant personnel from varying responsibilities such as the project engineers, operations managers, operators, and the technical personnel responsible for long-term maintenance of the system. While the presence of multiple roles is helpful to catching any potential problems, there is also the risk of losing focus and getting off-task. As described above, the Test Specification can help keep the FAT on schedule and people on task.

System Update Form

As testing progresses, individuals will find various items they would like changed. To document comments, recommendations, or changes to the system throughout the FAT, we recommend using a System Update Form. Potential notes may include incorrect results, a change in control philosophy, or even something as simple as the color or formatting of an object on the HMI screen. The items on the form can be reviewed at the end of the day to allow the programmers to make the modifications before the next day of testing resumes. Once the changes are made, review the items with the authorized authorities and have them sign off that each item was addressed.

The Final Step: Operator Testing

The FAT should conclude with the user conducting a scripted test of the complete sequence of operations with programmers remaining as hands-off as possible. The script should be run through several times until a sequence can be run from start to finish with no problems encountered. Exception conditions should also be considered and run through to verify the sequence can adjust and continue operations as would be expected in the plant environment. Once the FAT is complete, have the designated authorities sign off that the system is approved for shipment.

 Factory Acceptance Testing: Efficient and Cost-Effective

With today’s emphasis on seamless integration and trouble-free startups, thorough software testing is essential for minimizing production downtime at startup. Factory Acceptance Testing provides an efficient and cost-effective way to debug code before placing equipment in field commissioning, where any changes to code would increase costs and delay production. By carefully planning and scheduling the process, identifying the required equipment and simulation methods, and following procedures, the FAT will utilize an accurate simulation, which is essential for implementing new computer process control and information systems.

Matrix Technologies is one of the largest independent process design, industrial automation engineering, and manufacturing operations management companies in North America. To learn more about our industrial automation services and factory acceptance testing, contact Ronald England, Vice President and Director of Project Management.

© Matrix Technologies, Inc.