Content Comparison

8:00 AM to 5:00 PM, Monday through Friday:           

Outside of 8:00 AM to 5:00 PM, Monday through Friday:          

Network Incident On-Call Process

For all SQUARED UP alarms and trouble calls follow the on-call triage process:

For alarms originating from StruxureWare, DCM (Data Center Management) personnel should be notified in addition to proceeding with the Data Networking (ITIS) notification steps outlined below.

1. Verify that the building that is showing the alarm has electrical power. Pass this information to the on-call and in the follow up email.
   

   To determine if a device is down on Squared Up due to power outage please start checking the middle screen that networking maintains.

   • If it is on this list do not call.
   • If a building near the building listed on the power outage list is on Squared Up do not call. (Please only call if it is still showing on Squared Up once the power outage time listed is over.)
   (You can look on Purdue Campus Map https://www.purdue.edu/campus_map/ to tell if where buildings are located)

2. For alarms originating from StruxureWare, DCM personnel should be notified in addition to proceeding with the Data Networking notification steps outlined below.

3. Call Data Networking (ITIS) Primary On-Call phone (765-494-1591). If no answer, voice mail should be left on the phone and you should wait 5 minutes before proceeding.
   
   
4. A Footprints ticket needs to be made and assigned to ITIS Networking. Put the ticket number in the follow up email. Contact info is the on-calls name.
   
   
   
5. An email should be sent to itns-pdnhlog-ext@lists.purdue.edu. Justin McIntyre would like to be cc'd in the email also: mcintyrj@purdue.edu
   You do not need to send another email when you try to reach someone. If you leave a message, include this sentence in the body of the email: "We will escalate the on-call process if there is no response within 5 minutes."
   
   
6. Call Data Networking (ITIS) Secondary On-Call phone (765-494-1530). If no answer, voice mail should be left on the phone and you should wait 5 minutes before proceeding.
   
   

7. Call Justin McIntyre at 630-675-7640. If no answer, call Richard Letts 206-790-5837. Wait 5 minutes before proceeding to Step 7.

   
   

8. In the event that none of the individuals from Steps 3 through 6 above has responded, repeat those steps until contact is made. 

Special Notes:

• Data Networking personnel who are contacted by IOC staff are responsible for providing issue status updates to IOC in a timely fashion.  As a general rule this means that such feedback should be provided once you start investigating an issue, whenever an ETA for resolution has been determined and again when the issue has been resolved.  Additional updates are always welcome as well, especially for extended outages.
• If Squared Up, itself, ever goes down for more than 20 mins make sure to let Data Networks know that we cannot monitor their devices while it is down. (If it is the Xymon part down do NOT call Data Networks as it does not pertain to their devices, only when "Network Devices" section is down). IT Service Management (ITSMO) 765-496-6390/ itcr-itsmo@lists.purdue.edu will be the on-call if Squared Up is down.  (If it is the Xymon part down do NOT call Data Networks as it does not pertain to their devices.)
  
• Battery Alarms in Squared Up: Email Data Networks only. Do NOT call. Make FP ticket, assign to Networking and contact can be left blank or jpublic. Ex's: anything with "APC" in it.
  
  
  Ignore the following list of buildings only outside of normal business hours:
  

  Cases where it is outside of business hours (as is also defined below), please do not call the data networks on-call staff for Squared Up alarms in any of the locations listed in the table down below.  Essentially if the hostname value of the alarm contains a value from the “Squared Up Value” column of the list down below, it should be ignored.  These values are typically fairly static but should any updates need to occur we’ll forward you an updated list.

  • • Business Hours – Monday through Friday – 7:00 AM to 5:00 PM
    • (Official Purdue Holidays and Weekends Adjoining those Holiday Dates are Considered to be Non-Business Hours, Superseding the Monday through Friday Standard)
    • Example: 5/25/19 through 5/27/19 (Memorial Day and the weekend adjoining it) would not be considered business hours

  
  

  Building Short Name	Building Long Name	SquaredUp Value
  -	Any APC Devices	"-apc******-"
  -	Any TRP Devices	"-trp******-"
  2550	State Farm	"2550-"
  844S	844 South River Road	"844s-"
  AC22	Field Research Facility (ACRE Farm)	"ac22-"
  AC35	Pest Lab and Storage Facility (ACRE Farm)	"ac35-"
  AC41	Grain Drying Complex - Grain Auger (ACRE Farm)	"ac41-"
  AC42	Scales House (ACRE Farm)	"ac42-"
  AC43	USDA Soybean Research Lab (ACRE Farm)	"ac43-"
  AC44	USDA Rainulator Building Soil Erosion (ACRE Farm)	"ac44-"
  AC45	Var Test Facility (ACRE Farm)	"ac45-"
  AC46	Headquarters and Shop (ACRE Farm)	"ac46-"
  AC51	Weather Facility (ACRE Farm)	"ac51-"
  AC54	Crop Diagnostic Training Center (ACRE Farm)	"ac54-"
  AF01	Aquaculture (ASREC Farm)	"af01-"
  AFC	Anderson Flagship Center	"afc-"
  AIDC	Agricultural Information Distribution Center	"aidc-"
  ASB	Airport Service Building	"asb-"
  B201	Swine Evaluation Headquarters (ASREC Farm)	"b201-"
  B401	Poultry and Hatchery Facility (ASREC Farm)	"b401-"
  B501	Sheep Research and Teaching Facility (ASREC Farm)	"b501-"
  B602	Feed Mill (ASREC Farm)	"b602-"
  B701	Swine Office Metabolism Facility (ASREC Farm)	"b701-"
  B713	Environmental Research Facility (ASREC Farm)	"b713-"
  B801	Farm Operations Shop and Headquarters (ASREC Farm)	"b801-"
  B901	Teaching Center and Classroom (ASREC Farm)	"b901-"
  BBCH	Purdue Baseball Clubhouse	"bbch-"
  BBPB	Purdue Baseball Press Box	"bbpb-"
  BECK	Beck Agricultural Center (ACRE Farm)	"beck-"
  BTV	Boiler Television Building	"btv-"
  CB10	Beef Building (ASREC Farm)	"cb10-"
  COAL	Coal Handling Control/Fire Pump Building	"coal-"
  FSHR	TAP Fishers Remote Site	"fshr"
  GCMB	Golf Course Maintenance Barn	"gcmb-"
  GMF	Grounds Maintenance Facility	"gmf-"
  GMGF	Grounds Maintenance Greenhouse Facility	"gmgf-"
  ICSC	Indiana Corn and Soybean Innovation Center (ACRE Farm)	"icsc-"
  IDOT	Indiana Department of Transportation	"idot-"
  INDY	Indianapolis External Site	"indy-"
  INOK	Investments Warehouse	"inok-"
  INSS	Intramural Storage Shed	"inss-"
  KKM	TAP Kokomo Site	"kkm-"
  LMSB	Laboratory Material Storage Building	"lmsb-"
  NA	TAP New Albany Site	"na-"
  NACC	Native American Educational and Cultural Center	"nacc-"
  PAGE	Thomas A. Page Pavilion	"page-"
  PWB	Purdue West Annex - Building B	"pwb-"
  PWC	Purdue West - Building C	"pwc-"
  RALR	Stadium Area - Visiting Team Locker Room	"ralr-"
  SBCH	Purdue Softball Clubhouse	"sbch-"
  SBPB	Purdue Softball Press Box	"sbpb-"
  SCHO	Schowe House	"scho-"
  SD02	Dairy Research Unit (ASREC Farm)	"sd02-"
  SIA	Subaru of Indiana Automotive	"sia-"
  SOCC	Purdue Women’s Soccer Building	"socc-"
  SPUR	Spurgeon Golf Training Center	"spur-"
  SWNA	State Wide New Albany	"swna-"
  SWSB	State Wide South Bend	"swsb-"
  TM02	Throckmorton Pesticide Building	"tm02-"
  TM08	Throckmorton Meigs Building	"tm08-"
  TM11	Throckmorton Fruit Barn	"tm11-"
  TM36	Throckmorton Farm Crop Barn	"tm36-"
  TPB	Rankin Track Press Box	"tpb-"
  TURF	Intercollegiate Athletic Sports Turf Building	"turf-"
  UNPD	University Police Department	"unpd-"
  USDA	USDA Building 1 (ASREC Farm)	"usda-"
  VCPR	Veterinary Center for Paralysis Research	"vcpr-"
  VLAB	Veterinary Laboratory Animal Building	"vlab-"
  VOIN	Voinoff (Samuel) Golf Pavilion	"voin-"
  VTCH	Vision Technology 1 Building	"vtch-"
  VPRB	Veterinary Pathobiology Research Building	"vprb-"
  WH9	Well House 9	"wh9-"
  WRIT	John S Wright Forestry Center	"writ-"

  
  

Information to Provide When Reporting Network Issues

For Squared Up alarms report the following information from Squared Up when reporting issue:

1. Date/Time that alarm started (Example: October 26, 10:00 AM)
2. Affected Device Name (Example: mrdh-285n-c2950-01)
3. Last Ping (Example: 2017-11-01 11:14:43)

For network issues reported by I-Light / GigaPOP or the ITaP help desk, campus personnel, students, or visitors, report the following information:

1. Date/Time that issue began or was first noticed (Example: October 26, 10:00 AM)
2. Affected service (Examples: wireless/PAL, data PIC(s), or I-Light / GigaPOP call back)
3. Location where problem is occurring (Examples: wireless on 2nd floor of Armstrong, all data PICs in the Forestry building, or I-Light / GigaPOP). Whenever possible obtain a specific building and nearest room number.
4. Name and phone number (or at least email) of person experiencing/reporting problem

More details can be found on the attached document below:

ITIS-Data Network Incident Resolution Process.docx

• Check calendar and Log for any special instructions regarding these systems.
• Firmware/Software update pop ups - Send e-mail to ITIS Data Center Management.
• Humidity alarms (low or high) in StruxureWare should be reported by email only - no phone call needed.
  
• Dew Point Alarms These alarms should be reported by email only.
  
• Battery alarms in StruxureWare for UPSs (devices with "apc" or "trp" in the hostname) should be reported by email only - no phone call needed.
  
• Red alarms (except data network alarms : ERHT 5A/5B, LAMB 20, LYNN B168, or TEL 210 and the exceptions listed above) - Call Todd Turner (68214).
  
  • Data Network alarms for the buildings LAMB, LYNN, ERHT, and TEL need to be reported to Data Networks. (I believe this changed to just DCM now - I will follow up)
  • “Device status may be inaccurate because an attempt to transfer a device definition file (DDF) failed" alarm. Right click on the device and select "Request device scan" (per the email received by IOC on 05/06/2018).
    • Call Todd Turner (68214) if the step above doesn't clear the alarm. If Todd does not answer or respond through email after 20 mins, call Patrick at 765-427-3020(C),  6-1752(W), 765-421-6069(H), if he does not answer cal Jon Miller at 765-414-7646. 
      
• Communication lost, Connection, or Timeout errors
1. If ALL devices in a room are down, call right away
2. Ping the IP Address. If it pings it should clear. If it does not ping or clear after 10 minutes then move to step 3.
3. Contact Data Center Management via phone & email.  
• Physical Facilities will call to report PMs on the CRAC & generator units listed below.Send email to ITI Data Center Management at iti-dcm@purdue.edu when they start and stop.  
  This protocol also pertains to when they come in on the weekends for a key. Ask what they are working on and send email to ITI Data Center Management.
1. Include the building, room, device name, and Physical Facilities technician name in the e-mail.  
2. This is only for the equipment listed below.  In all other cases dealing with Physical Facilities requests for permission to do work call Todd Turner at 496-8214.

Generator Test TEL Nodes

• ERHT
• LAMB
• LYNN
• TEL

Crac units Data Centers

• FREH G2 CRAC #1,2,3,4
• FREH G57 CRAC 1
• FREH G60 CRAC 1, ACG-2
• HAAS CRAC 1,2,3
• MATH B60 CRAC 1,2,3
• MATH G72 Chiller
• MATH G109 CRAC 5,6,13
• MATH G190 CRAC 1,12,32

TEL Nodes

• ERHT 5 CRAC 1,2
• LAMB 20 ACG-20,21
• LYNN G168 ACG-40, 41
• TEL 210 CRAC 1,2,3

Anchor
cameras cameras

Cameras

StruxureWare also includes monitoring functionality for the cameras in the data centers. This view in StruxureWare should be open at all times on the large screen.

• Please monitor the cameras from time to time to make sure nothing suspicious is going on. Check the Shift Log, Change and Release Management Workspace Calendar and your email for scheduled work. If it appears that the occupants do not belong in the room or are removing things during the night, notify PUPD (48221) to investigate. You will also need to call Todd Turner if you do call PUPD, or if you feel unsure.
• HAAS is considered a lights-out facility with several groups who have card access. The building fire panel is located in the datacenter. The Fire & Safety group works Midnight to 8am doing panel tests across campus, and you will see them or the PUFD in the room from time-to-time.