EOP Example — Cooling Failure Response

Document ID: EOP-CLG-002

Revision: 1.0

Status: Example

Activation: Cooling Degradation / Failure

1. Purpose

Provide a structured response framework for cooling-system degradation or failure events affecting critical infrastructure operations, with focus on environmental stability, operational overview, thermal risk control, communication and escalation.

2. Scope

Applies to abnormal environmental or cooling-system conditions affecting technical rooms, data halls or supporting infrastructure, including CRAC/CRAH systems, chilled-water systems, DX systems, airflow containment, environmental monitoring and related cooling infrastructure.

This example does not replace HVAC repair procedures, manufacturer instructions, refrigerant-handling requirements, electrical safety rules, fire procedures, local regulations or site-specific emergency response plans.

3. Activation Criteria

This EOP may be activated when one or more of the following conditions are observed:

Unexpected temperature increase in a critical technical area
Loss of one or more cooling units affecting operational margin
Environmental alarms outside approved operating thresholds
Unexpected airflow loss or containment failure
Cooling water leak or condensate issue affecting operations
Loss of cooling redundancy
Thermal instability with unknown root cause
Cooling infrastructure alarms affecting critical rooms

Emergency principle: Maintain thermal control and operational overview before attempting detailed fault resolution.

4. Immediate Response Priorities

Priority	Focus Area	Objective
1	Life Safety	Confirm no immediate hazard to personnel exists.
2	Environmental Overview	Determine affected rooms, thermal trend and cooling status.
3	Cooling Stability	Preserve remaining cooling capacity and airflow control.
4	Impact Control	Reduce risk of secondary infrastructure impact.
5	Communication	Inform responsible stakeholders and maintain updates.
6	Incident Logging	Maintain timeline and operational record.

5. Initial Response Procedure

Step	Area/Tag	Action	Verification	Owner
1	OPS-BRIDGE	Declare operational incident and assign response lead.	Single responsible coordinator established.	Operations Lead
2	DATAHALL / TECH AREA	Confirm personnel safety and identify any immediate hazard.	No active life-safety emergency identified.	Operations / Security
3	BMS / DCIM	Review environmental alarms, room temperatures and cooling-system status.	Affected cooling zones and thermal trend identified.	Facility Operations
4	CRAC/CRAH / CLG-ZONE	Identify affected cooling equipment and determine current operating state.	Failed, degraded or unavailable cooling components identified.	HVAC Specialist
5	DATAHALL	Verify airflow paths, containment integrity and visible room conditions.	No obvious blockage, displaced containment or uncontrolled airflow issue observed.	Facility Operations
6	CLG-ZONE	Determine remaining cooling redundancy and environmental margin.	Current operational risk level understood and communicated.	Operations Lead
7	COMMUNICATION	Notify required stakeholders according to incident communication plan.	Stakeholder communication completed and logged.	Incident Lead
8	INCIDENT LOG	Start or maintain incident timeline and operational log.	Observations, alarms, actions and decisions continuously recorded.	Assigned Recorder
9	OPS-BRIDGE	Establish environmental review interval and escalation threshold.	Response cadence agreed and communicated.	Incident Lead

6. Stabilization Measures

Stabilization actions should focus on maintaining acceptable thermal conditions while avoiding uncontrolled changes to the environment.

Preserve remaining cooling capacity
Minimize unnecessary room access or airflow disturbance
Protect containment integrity where applicable
Increase monitoring frequency during unstable conditions
Escalate early if thermal margin decreases
Coordinate technical troubleshooting through the incident lead

Any compensating action involving cooling setpoints, airflow paths, electrical isolation or infrastructure configuration should follow approved site procedures and authorized decision-making.

Control rule: Avoid uncontrolled environmental changes during a cooling incident. Stability is often more valuable than aggressive adjustment.

7. Escalation Triggers

Immediate escalation is required if one or more of the following conditions occur:

Temperature continues increasing despite stabilization efforts
Additional cooling units fail or become unavailable
Environmental conditions exceed approved operating limits
Water leak or condensate issue threatens equipment areas
Monitoring visibility is lost or becomes unreliable
IT equipment alarms or thermal shutdown events begin
Power systems, UPS systems or network infrastructure become affected
Room conditions can no longer be predicted with confidence

8. Communication Structure

Cooling incidents should be communicated using clear operational status updates based on measured conditions and observed trends.

Use factual environmental data where available
Avoid assumptions about root cause until confirmed
Separate technical troubleshooting from stakeholder communication
Maintain regular update intervals during unstable conditions
Record all major environmental changes and escalation decisions

Example update structure:

Time → Current Temperature / Trend → Affected Area → Current Cooling Status → Risk Level → Next Update

9. Recovery and Return to Normal

Recovery should focus on restoring a known stable environmental state, not only restarting failed equipment.

Verify cooling systems returned to approved operating condition
Review environmental trend stability before incident closure
Verify airflow and containment integrity restored
Confirm no unresolved alarm or degraded condition remains
Review monitoring visibility and sensor status
Complete incident documentation and operational review

10. Related Procedures

Cooling System Visual Inspection Procedure (SOP)
CRAC/CRAH Planned Maintenance (MOP)
Water Leak Response (EOP)
Utility Power Loss Response (EOP)
Site HVAC Repair Procedure (Site-Specific)

11. Operational Notes

Cooling incidents often evolve gradually compared to electrical failures. Early detection, environmental awareness and controlled escalation are therefore extremely important.

Thermal conditions should be evaluated using measured trends, infrastructure awareness and operational judgment — not isolated alarm points alone.

Real emergency response procedures must align with facility design, environmental requirements, manufacturer instructions, local safety rules, monitoring strategy and approved operational governance.

Type: EOP Example

Category: Cooling / Environmental Incident Response

Criticality: Critical

Status: Public example — not for direct production use

Key principles: maintain environmental overview, protect thermal margin, preserve cooling stability, escalate early and maintain disciplined communication throughout the incident.