...

Log each walkthrough in the IOC log, and indicate if DCM was emailed (or called, as the case may be). No email or call is needed if no issues are found. Whether or not follow-up email was sent, please select 'Yes' for the "Email?" option when making the log entry (examples near bottom of page).

Schedule:

WalkOn normal business days, walk-throughs should be performed, everyday, once on first shift, and twice on subsequent shifts. On non-business days (weekends and holidays), walk-throughs should be performed twice on all shifts:

(Table 1)

Walk-throughs are performed, primarily, by the IOC secondary working from MATH inspecting MATH, and the IOC primary inspecting TEL . In the event that the secondary is unable to perform a walk-through for whatever reason, the IOC primary will assume this task.

...

Walkthroughs should be performed as scheduled, (above, Table 1, and here; Table 2): 

(Table 2)

It is recommended that walk-throughs consist of two distinct walkthroughs per datacenter. The first walk should focus "up"; observe and note the condition of the ceiling, the airspace above the racks and equipment, the walls and columns, and any utilities located 'up' (i.e. cabling, piping, etc.) When this first walk-through has been completed, a second should be performed, focusing this time on "down";  observe and note the condition of the racks themselves, the floor, walls, doors, and equipment. During both walks, the general condition should be observed; smells, sounds, temperature, etc.

...

Residue from a previous leak (fig. 6). Be on the lookout for such (fresh) leaks (fig. 6.2). Fluids of ANY colour are an issue!:

(Figure 6)

Image Added

(Figure 6.2)

Image Added

(Figure 7)

Check Clean Mats at the entrances to the data center. When excessively dirty (as pictured above, fig. 7 ), please remove and discard the top layer, revealing a fresh layer underneath. This helps to prevent dirt from being tracked into the datacenter, and ultimately, into the equipment.

Datacenter specific considerations:

...

In addition to the general concerns outlined above, when performing a walk-through of MATH G190 and G109, please also inspect the Cooling Distribution Units (CDUs) located in each datacenter for any alarms (fig. 7): 

Image RemovedImage Added

(Figure 78)

CDUs are labeled and shown in approximate location in red. Green Xs indicate doors that should be checked to verify they are closed and secured. Purple stars indicate doors that should remain closed, but unlocked. Yellow stars indicate doorways that should remain open for ventilation. 

CDUs come in several flavours. Examples are shown below, with notes:

Coolcentric

Without PDU (fig. 89):

(Figure 89)

Two Coolcentric CDUs with PDU (fig. 910):

(Figure 910)

This CDU (fig. 1011) is experiencing an alarm (fig. 1112, circled), with close-up view (fig. 1213):

(Figure 1011)

(Figure 1112)                                                                           (Figure 1213)

CoolIT

(Figure 1314)

CoolIT CDUs are equipped with touchscreens that may go into screen-saver mode. Lightly touch the screen to check its status.

...

This CDU is experiencing an alarm (fig. 1415):

(Figure 1415)

Also check the following spaces:

• • • The areas and hallways around B60, B22, B18, and B10. Leaks or water here could indicate an issue above (in the datacenter).
    • Freight elevator. Check that there are no packages. If any packages are found, place them on the flat cart outside the elevator in G190 in B92 (the room that opens to the freight elevator on B level) and send an email to DCM.
    • Loading dock exterior door is locked.

• B60
• TEL
• HASSHAAS
• LAMB?

Reporting:

Emergencies involving life-threats (fire, injuries, etc) should be reported via 911!

Issues involving large volumes of water

...

in the datacenter or electrical issues (sparking/arcing/smoke from feed panels or energized equipment) should be reported first to PUPD dispatch, followed immediately by contact to DCM via phone.

Issues of excessive heat or water should be reported immediately to DCM via phone. 

Relatively minor issues of open doors, CRAC noises, CDU errors, or other non-critical issues should be reported to DCM via email.

Email DCM after every walkthrough, even if nothing is found (specify that in the email). Log each walkthrough in the IOC log, and indicate if DCM was emailed (or called, as the case may be). Whether or not follow-up email was sent, please select 'Yes' for the "Email?" option when making the log entry.

...