Get Rootly's Incident Communications Playbook

Don't let an incident catch you off guard - download our new Incident Comms Playbook for effective incident comms strategies!

By submitting this form, you agree to the Privacy Policy and Terms of Use and agree to sharing your information with Rootly and Google.

Back to Blog
Back to Blog

January 9, 2025

7 mins

RescueOps - Ep. 4: Situation Awareness and Real-Time Tracking

Whether scaling a mountain or troubleshooting an outage, situational awareness and real-time tracking can help your team build resilience and minimize costly delays.

 Claire Leverne
Written by
Claire Leverne
RescueOps - Ep. 4: Situation Awareness and Real-Time Tracking
Table of contents

In this series, Claire Laverne—outdoor rescue expert and SRE—shares insights that SREs can draw from rescue operations. Check out the previous parts of the series for more context:

Know Thy Maps

I love maps. I love the colors, the line weights, the precision of information, and the breadth of knowledge that can be conveyed without a written word. Most of all, I love that—no matter how much I study a map—when my boots are planted in the same location, the landscape always surprises me. It’s possible to have extensive information about a place and still not know its essence: the crisp mountain air contrasting with the sun’s warm rays, the robins’ warbles, and the faintly sweet smell of buttercups.

There is yet a more practical reason I pore over maps: safety. Maps are a guide to a destination, but the destination is only a point—not the point. The map is about the journey between the start and endpoint. It provides an understanding of the path, the obstacles, the aids, and the interests along the way.

A map instills in us a quality called “situational awareness,” which we can draw upon when our environment changes unexpectedly and circumstances call for adaptation.

If a storm rolls in, where can I seek shelter? Where might I find a low point in the terrain to avoid lightning? If the day is far hotter than expected and I’ve depleted my water, where can I find a stream? If I get hurt, where might I find a phone signal or the nearest road?

Careful planning fosters an increased sense of situational awareness when we’re in the field. It’s a tool that allows us to wisely adapt in real-time to unexpected changes.

Understanding Situational Awareness

Situational awareness is integral to search and rescue operations and a rescuer’s ability to make decisions in real-time. Before rescuers go into the field, they’re briefed on the scenario, maps of the terrain are scoured, and communication channels are established for real-time feedback to Incident Command (typically via radio comms and GPS tracking through platforms like Gaia).

This planning is a necessary precursor to starting the search, but it also becomes critical once rescuers are in the field, making individual decisions as they encounter the terrain and unexpected obstacles. Without careful planning and situational awareness, operations can quickly become disorganized, leading to delays that might cost lives.

Situational awareness is equally important for an engineering team’s incident response. It involves having a real-time understanding of the environment, including system performance, security threats, and user impact.

Just as rescuers need to know the physical conditions on the ground, engineers must monitor their distributed landscapes to detect anomalies, understand the scope of incidents, and anticipate potential repercussions.

Effective situational awareness in tech ensures that incidents are managed proactively, minimizing downtime and maintaining service reliability.

The Risk: Delays are Deadly

We must avoid delays at all costs in incident management. Delays can be fatal to incident resolution, as they force us to make reactive decisions.

Whether rallying to find a lost hiker or restoring a system outage, both scenarios require activating complex response systems. Every complex system operates through feedback loops. For example, in Search and Rescue (SAR), resources are deployed based on new intelligence from a witness.

These resources may begin their search but relay conflicting information back to Incident Command (IC), prompting changes in search tactics. Each decision builds on previous feedback. When significant delays enter this feedback cycle, decisions may become irrelevant or even counterproductive, reducing operational efficiency.

Consequently, we may end up making delayed decisions about a scenario that no longer represents the current situation. This failure to identify the root cause of the disruption can lead to reactive decisions that create new problems. To summarize: this is a costly scenario.

In the business world, delays can severely impact operations and erode user trust. Prolonged downtime leads to financial losses, reputational damage, and decreased customer satisfaction. By applying feedback loop principles, engineering teams must act swiftly to identify, diagnose, and resolve incidents.

Streamlined communication channels, automated alerting systems, and well-defined playbooks help minimize response times, ensuring that issues are addressed before they escalate into more severe problems.

Real-Time Tracking

Situational awareness requires meticulous planning and research to set a strong foundation for success in the field. However, once the mission is underway and conditions become unpredictable, real-time tracking becomes the cornerstone of effective incident management.

The ability to receive and interpret live data about the ongoing situation allows response teams to avoid delays, respond to new information promptly, and adapt to evolving circumstances. In a word, they become resilient.

Real-time tracking ensures that all team members are continuously updated on each other’s locations and statuses. This minimizes the risk of miscommunication and overlapping efforts, allowing for a more synchronized and efficient operation. Centralized platforms like Gaia or GIS—which integrate GPS data with other data layers—facilitate better information flow and enhance overall coordination and search-grid strategies.

Parallel observability tools and strategies in tech incident management follow the same principle: working with the most up-to-date data and responding in real-time offers the best chance to resolve outages quickly, without fighting unnecessary fires. Cut out delays—they’re bad-news-bears and only lead to unexpected or unhelpful system behaviors.

Conclusion: Situational Awareness and Real-Time Tracking Depend on Each Other

Situational awareness and real-time tracking are indispensable components of modern rescue and tech incident operations.

They are codependent tools: situational awareness requires real-time tracking to stay relevant; real-time tracking requires situational awareness to know where the problem started in the first place and how it can be resolved. Strategic application of these tools will mitigate delays in any operation—be it rescue or a system outage—and steer a hardworking team into productive solutions and effort. Integrating careful planning and real-time intelligence will not only enhance efficiency, but improve safety and transform iterative work into fruitful results. 

Practical advice for adventurers: unless you’re already familiar with the terrain, I always recommend spending more time planning for the trip than being on the trip. Pore over maps from multiple sources (it may even provide ideas for additional detours!). Note every fork in the trail so you’re never surprised when confronted with one in-the-wild. Always, carry a paper map. Learn to gauge topo lines for the rate of elevation climb and use that to help budget your water load. 

Make good choices, and remember to pack snacks!