This post was authored by Tim Green, manager of commissioning services at Endress+Hauser and formerly U.S. operations manager for the field services division at MAVERICK Technologies, a Rockwell Automation company.

When a company launches a major project—something on the order of a new plant, process unit, or automation system upgrade—a schedule is produced. For far too many companies, the schedule is merely symbolic, as they do not believe it will be strictly adhered to as the project progresses.

Such an attitude is simply wrong, and can end up being a self-fulfilling and expensive prophecy. Schedule delays add costs, because labor resources spend more time than planned. But maybe worse, the company loses income every day its new plant or equipment is not running.

Projects can stick to a schedule if construction is managed properly. This kind of thing does not happen of its own volition, but instead depends on properly executing three major tasks:

1. Effective planning
2. Active management
3. Proactive testing

Many articles have been written on project planning, so for this discussion we will assume that planning has been well executed, and concentrate on the second and especially the third factors.

Figure 1. Projects that fall behind schedule can cause significant cost increases beginning with capital expenses by virtue of the additional time needed. Most companies understand this cost, but are not as fast to recognize that operational costs also begin to accumulate and are not offset by production, because the plant has not started operation. When all these costs are combined, the cost of a delayed startup grows for every day past the desired schedule.

Active management

Once a project moves from planning to actual construction, it has to be actively managed to stay on course and on schedule. The key word is actively. Without proper supervision, projects can quickly fall behind. Why do schedules get off track?

Too many companies look at construction as a black box. They understand it begins and ends, but there is little sense of what goes on in the middle, so management is passive. Those charged with project management often simply expect everything to happen on its own. Without people who can dive in and get a grip on each project task, companies feel helpless to drive or even manage the process.

The project management team must work with contractors and monitor the schedule constantly, but must avoid becoming micromanagers. There is a fine balance between being effective and being a nuisance, and each situation is different. It can take some time for novice managers to develop such a sense, but it is critical for effective project management.

A key first step in active management is opening the black box and breaking the construction timeline into subsections, so each can be monitored and tested along the way. Some things do have to happen in sequence. A flowmeter cannot be installed before there is piping, but it should be possible to look ahead while the piping is being fabricated. Are the flowmeter and all its necessary fittings on hand so it can be installed without delay when the piping is done? Does it require any additional support structure to be fabricated along with the piping?

Steps that can run simultaneously should do so, because this makes the schedule collapse on itself, and it reduces overall project time. Those steps are not likely to happen simultaneously without getting into the construction black box.

Testing has to happen at each step as soon as possible to ensure every element is functioning properly before a particular contractor moves on. Few companies realize the benefits of this approach until they see it firsthand. Integrating testing with construction drives the schedule and keeps everyone, employees and contractors, focused. Individual contractors are held accountable for correcting any problems they caused and for fixing items left undone, all while they are still on site.

Proactive testing

Maintaining an aggressive project schedule requires testing at each step of the process to verify performance and fulfillment of specifications. Given the criticality of testing, it needs to be divided into three phases or stages, corresponding to the relevant stage of construction (table 1). In a large-scale project, all three phases can often take place simultaneously in different areas.

Prestatic inspection is designed to identify installation issues early in the process. It also helps quantify the percentage of completion of a specific portion of the project. This is critical to keeping parallel activity moving to fulfill the schedule. Without it, the timeline often stretches out, because the desired overlap is eliminated.

Static checks deal with wiring issues during electrical construction. These checks are performed before any of the equipment is energized, primarily checking wiring continuity and correct wire tagging. For motors, thermal protection and correct rotational direction may be verified. Like prestatic inspection, this step is critical to verifying a contractor’s performance and percentage of completion.

Predynamic testing is the first stage when equipment is energized. It confirms functionality of instruments, valves, and motors—and verifies operability from the appropriate controller. Process variables are simulated to verify scaling; valves are given full-stroke tests; motors are bumped; variable frequency drive operation is confirmed, and so forth. This is the last phase of functional testing before full dynamic testing.

Each test is performed as early as possible, corresponding to the stage of construction, and all testing is documented and incorporated into the schedules.

Commissioning and startup

As the final stages of construction are taking place, the project is almost ready for full dynamic testing. Using the heel-to-toe process of testing at each phase of the project means there is no lengthy period at the end when all elements have to be tested at once.

Full dynamic testing is the final stage when equipment is energized and controller logic is exercised. This testing ensures adequate interlock protection is in place for a safe process, and operational logic functions are in accordance with the needs of the specific process. This live logic testing confirms operation as defined by the control narratives.

Now the instrumentation and controls (I&C) team gets to see the culmination of all the efforts, as the last elements are put in place and late-stage design modifications are made. But there is still much to be done as commissioning and startup begin. At this point, ownership transfers from the I&C provider to plant personnel. The I&C team members of the larger startup team support the process experts as they execute their full dynamic test plans.

• The plant and the automation solutions provider typically verify functionality and interlocks by performing water runs or some other full functionality simulation.
• Complete control system functionality is verified.
• Final loop tuning is performed.

Resource flexibility

Considering all the project activities that must take place in a short period, any company trying to carry out such a comprehensive range of tasks has to have huge resource flexibility. Over the weeks and months of a project, there will be times of relative inactivity, and other periods when many things have to happen together, such as when a major phase is nearing completion.

Keeping things moving as quickly as possible while maintaining peak personnel efficiency demands constant adjustment of not only headcount on a site, but also of personnel skill sets. The ability to have the right number and right type of people on the clock, no more and no less, requires a pool of highly qualified engineers and technicians ready to move as needed. Major automation solutions providers have this kind of flexibility, and this capability is critical to realizing all the gains possible from more aggressive scheduling, while still controlling costs.

A major upgrade project can require more than 100 people to be on site simultaneously for the startup and commissioning of just the electrical and automation system portions of the project, typically requiring the plant owner to engage outside assistance to meet the schedule.

Hot cutover techniques

For many production facilities, the costs of interrupting production to perform an automation system upgrade are prohibitive. However, in some cases the need to implement such an upgrade is also highly compelling. The solution to this dilemma is performing a hot cutover while the plant remains in production. This stage of a project requires very careful planning and coordination between the plant operations team and the automation solutions provider.

A hot cutover moves one control loop at a time from the old to the new automation system. Each loop needs to be verified and tuned in the process. All parts of the new system hardware must be thoroughly tested and need to perform flawlessly so there are no unfortunate surprises at any step. Both automation systems have to run simultaneously, so all digital communication networks must be capable of supporting increased traffic.

Also, engineers and technicians must test control code in the new system to make sure it operates correctly during and after the transition. They also need to compare output image tables of the new controller with image tables in the live existing system for parity. Once they have confirmed 100 percent parity, the cutover can begin.

This kind of transition can be smoothly carried out if done carefully by experienced engineers and technicians. Skilled automation solutions providers have performed many such transitions during large distributed control system upgrades and migrations without interrupting production. Such a track record depends on having talented people and the know-how to perform a wide range of critical functions.

Closeout and documentation

For new systems or those upgraded during an outage, a full dynamic test is the final check before putting the new unit or automation system into production. At this point, every hour counts, because everyone wants to close out the project and realize its income.

Using traditional project management methods, full dynamic tests can be slow and painful as numerous bugs and problems crop up. Inadequate testing at earlier stages leaves undiscovered problems, so the test moves by fits and starts as those bugs are identified and fixed. Many expensive people may be left standing around while a technician corrects the rotation of a pump or figures out why a valve will not respond to a control system command. In many cases, test results prove inconclusive, and engineers must burn up time with troubleshooting and fixes.

By contrast, the right testing and other procedures ensure this final stage moves quickly and deterministically toward a positive conclusion, because:

• All systems and subsystems have already been verified at each critical stage
• Commissioning and startup personnel do not have to wait for components to be fixed
• The process gets to run continuously without stops and starts
• Safety incidents and damage to equipment are avoided
• Plant personnel see a reliable representation of the live process for a conclusive test

Meanwhile, all the documentation compiled throughout the planning and construction process can be easily assembled, so it is accurate, complete, and ready for turnover.

As a final step, the automation solutions provider sits down with the plant project team to discuss collective reactions to the project. This debriefing is something of a celebration, but also a time to gather observations and lessons learned to pass on to operations and maintenance people. Tribal knowledge should be collected and documented to support future projects.

Conclusion

Quality automation solutions providers perform project management, startup, and commissioning work, bringing a high level of experience and technical skill to projects. Some companies consider hiring outside help on such an extravagant scale beyond a project’s budget, believing it cheaper to use only internal resources. However, the reality is few, if any, companies these days have the necessary resources available. If internal people are pulled into a construction project, what other tasks are not going to be done?

Consider the situation as a mathematical proposition: A project of whatever size is going to require some basic number of man-hours to get all the functions done (figure 2). A company can bring in a large number of people to do it in a short time, or it can have a small group of people doing the project on a longer timeline. Either way, the project cost is essentially the same, but if the plant starts up sooner, the company enjoys the income it generates sooner. Moreover, when a project can be managed as described here, the company can realize major efficiency gains.

An effective automation solutions provider improves efficiency by having exactly as many people on a site as the situation demands to push the schedule as far as practical. The number might be 30 this week, but 110 the next. Such flexibility is impossible with internal resources.

When a qualified and experienced automation solutions provider, capable of bringing all the necessary resources to bear on a project, can guide project planning and manage construction and testing, companies receive the highest assurance the project will meet all performance expectations, will conclude on schedule, and will be at or under budget. The costs of such services are typically recovered quickly through improved performance and increased production realized through an earlier startup.

Figure 2. Any project requires a certain level of effort (in man-hours) over a specific period of time. The upper graph illustrates a project operating under a theoretical best-case scenario given the practical constraints of labor efficiency for the work being performed. If all the resources necessary can be brought to bear, the work can be completed in this period of time.
The middle graph indicates what happens in most situations where companies try to carry out projects with only internal resources or minimal outside help. The number of actual man-hours does not necessarily increase, but the total amount of time required to get the job done at the lower effort level pushes out the startup time, delaying the beginning of production and income generation.
The lower graph illustrates what can happen when greater resources are available and can be combined with aggressive project management. Under these circumstances, even the theoretical best schedule can often be beaten without increasing overall costs significantly. The biggest benefit is production begins on time, or even early, allowing the company to realize production income earlier.

About the Author
Tim Green is manager of commissioning services at Endress+Hauser and formerly U.S. operations manager for the field services division at MAVERICK Technologies. Previously, he worked at a Fortune 100 automation provider where he had held positions at the North American and global levels. Tim began working in the electrical field in the U.S. Navy in San Diego, Calif. During his career he has worked as an industrial electrician, instrument technician, PLC programmer, engineering manager, and technical sales professional.

Connect with Tim

A version of this article also was published at InTech magazine.