Commissioning: How to Break the System in Order to Achieve Success

Although some are more aesthetically pleasing than others, from an outsider’s perspective, Data Centers are “gray” boxes, filled with “black” boxes…  But to the insiders, data centers are beautifully crafted facilities managing the world’s data and anxious to be put to the test!  How do you put a data center to the test?   There’s only one way – you commission it!  You attempt to “break” the system to test the design and determine its vulnerabilities.  And, once all the “bugs” have been terminated, you will have achieved success.

Step 1:  Find the Right “Team”

Step 1 is pretty simple – find the right “Team.” When you find the right design team, General Contractor, Subcontractors, Commissioning Agent, and Vendors, you automatically start out with an A- on your “Commissioning Exam”.

Step 2:  Be a Proactive Thinker

Although generic, this one is so important that I felt the need to create its own step.  Think ahead of the commissioning “bugs”…  Establish a well thought-out plan and follow through.

Step 3:  Design Reviews

Involve the Commissioning Agent (CxA) in design reviews.  Start out early and have them be involved with Conceptual, 75%, and 100% construction document reviews.  Let them gain an understanding for how the system works and what the intent of the system design shall be.  Ultimately, the CxA ends up adding value to the facility design.

Step 4:  Equipment Procurement

Let the CxA review the equipment submittals along with the Design Engineer and General Contractor with the goal gaining system familiarity.  Make sure they fully understand the equipment and its capabilities.

Step 5:  Factory Witness Testing (FWT)

Ensure there’s a plan for who’s attending each FWT (Owner, Design Team, CxA, General Contractor, Subcontractor, etc.).  Has the Vendor created a FWT script?  Has the “Team” reviewed the Vendor FWT script prior to FWT?  Identify who’s creating FWT Reports to recognize deficiencies and who’s tracking to completion.  These are simple proactive steps to ensure a successful and thorough FWT is achieved.

Step 6:  Onsite Equipment Verification

Validate the equipment arrived onsite with no deficiencies.  Do the quantity counts match what was ordered?   Are there any dings or dents?

Step 7:  Step Back and Catch any “Bugs” Falling Through the Sieve   

If there’s anything that can devastate commissioning success, it’s letting the small “bugs” fall through the cracks.  Take a step back to ensure that any deficiencies noted during the submittal, FWT and equipment verification process are being tracked and dealt with accordingly.

Step 8:  Electrical Testing

Who’s testing breakers and who’s tracking deficiencies?  Although debatable, I like the  scope for Electrical Testing and Commissioning to be completed by the same company.  It eliminates room for errors and allows the CxA to get their hands dirty with the system prior to jumping into onsite commissioning activities.

Step 9:  Equipment Startup

A qualified CxA will require vendor startup scripts to be completed prior to delving into equipment systems.  The importance of having each equipment vendor validate their gear is essential – they are the experts, let them show teams what they know.  Afterwards, it’s even more critical to take it to the next step and have the Contractors (General Contractor and associated MEP subs) sign off on their startup activities.  Diagnose who’s tracking the documentation and move onto Step 10.

Step 10:  Lineup Your Operations (Ops)/Facilities Maintenance Staff

By this stage, if you have an Ops staff on board to run the facility then you’re an All Star!  If you’ve had an Ops staff on board prior to this stage and they’ve been involved with design reviews, equipment procurement, FWT’s, startup activities, etc., then you’re a Hall-of-Famer!   If you DON’T have an Ops Team on board now, get it done ASAP.  It’s critical that the staff running the data center understand the “ins” and “outs” of their facility.  There’s no better way to do this than during Functional and Integrated System Testing.  They will get the opportunity to validate the system’s capabilities (i.e. controls scheme for Switchgear and how it interfaces with Utility and Generators, UPS System capabilities, Mechanical Plant capabilities, Automatic Transfers vs. Manual Transfers, and the list goes on and on…).  They will thank you for the involvement!

Step 11:  Make Sure You’ve Lined Up Your Operations / Facilities Maintenance Team ….

Reinforcing step 10!

Step 12:  Functional Commissioning

If you’ve successfully completed Step 1: Find the right “Team”, it will make life very easy for the Owner and not to mention assist in a successful Functional Commissioning effort.  During Functional Commissioning, it’s important for the “Team to think outside the box and test every imaginable scenario to ensure each individual system is ready to be integrated as one overall building system (a.k.a. Integrated System Testing or IST).

It’s important to ensure the CxA has completed the Functional Commissioning Scripts.  Also, have the Team review, field walk, and provide comments to the Functional Commissioning Scripts.   Once this is done, go have some fun!  This is when the team starts challenging or “breaking” the system(s).

Establish a daily meeting to review activities.  Having everyone on the same page = success.  Make sure there’s a plan for who’s tracking deficiencies.  I like the Commissioning Agent to lead this effort because they will be the party who ultimately signs off on any deficiencies.

Have the Ops staff “irritate” the CxA and stand over their shoulder for the duration of this testing; a valuable experience for Ops.  Challenge the Ops staff to come up with testing scripts they would like to run. “Think outside the box!”  During the IST, ask the Ops staff what will happen during this test.  This is a good check to see how well they’ve reviewed the Cx Script.  Finally, have the Ops staff be thinking of MOPs (Maintenance Operation Procedures) they want to test during Integrated Systems Testing.

Step 13:  Integrated Systems Testing (IST)

If you’ve completed Steps 1 through 12 successfully, then things will seem to take care of themselves.  IST’s are the most important step in commissioning.   It’s the step where you find out if the machine is fine-tuned or needs tuning…

Anything can happen during IST’s.  Outside of achieving commissioning success, some important goals should be:  test a total electrical utility outage if the Utility Company will allow…a “Pull The Plug” test.  Remember, if you don’t do it now, the Utility Company will do it for you eventually!  Validate all notification alarms are being sent to Operations Teams via email, pagers, text, etc.; let the Ops Team be involved, and even run, Integrated System Testing Procedures so they learn how to operate the system; roam the data hall to validate if there are any “hot spots” or unneeded air leakage from floor or ceiling; etc. (the list is endless, but all are important).

Step 14:  Document and Run the Building for Success

Last but definitely not least, make sure all documentation is provided once complete.  This may take several months, but it’s very important to have accurate final commissioning and as-built documentation so the operations team can effectively run the data center.  We also recommend video recording the IST…this will become an invaluable “visual” training tool in the future.

Commissioning is like panning for gold nuggets.  Think of the gold nugget as a commissioning “bug.”   Be proactive, use a small sieve and find all those “bugs.”  This will lead to your achieving success and finding gold!