← all cheat sheets
FUNDAMENTALS · MECHANISM WALKTHROUGH

Metro Fiber Circuit Down
Detection to Reroute — or Not

"the link is down" and "the link stopped passing traffic" aren't always the same event — this is the mechanism that has to tell them apart before anything can actually fail over.
FIBER CUT INTERFACE DOWN (L1) BFD / HEALTH PROBE PATH DECLARED DOWN ROUTE WITHDRAWN RECONVERGE TO BACKUP FAILBACK (LATER)
01 The Full Sequence — Worked Example
1

The Failure

A fiber cut occurs between Branch-12 and the metro provider's aggregation point. Branch-12 has two WAN paths: this primary metro fiber circuit, and a secondary broadband/LTE circuit, both terminated into an SD-WAN edge running continuous path monitoring.

Failure
2

Physical Layer Detection — Interface Down

The branch router's WAN interface loses carrier signal within milliseconds and transitions to a "down" state. This is the fastest possible detection layer — pure hardware, no protocol involved yet.

L1
3

Why Link-Down Alone Isn't Enough

Not every WAN failure is this clean. A gray failure — the physical link stays up (carrier present) but traffic upstream is silently dropped, e.g. a provider-side routing issue — produces no interface-down event at all. Relying purely on physical link state would leave the router believing this path is still healthy while every packet sent over it vanishes.

Gray Failure
4

Active Health Probing — BFD / SD-WAN Path Checks

A lightweight hello protocol — BFD (Bidirectional Forwarding Detection) or an SD-WAN-native health probe — runs continuously over the link, often every 50-300ms, independent of the routing protocol's own hello timers. This is what actually catches a gray failure: the probes simply stop getting replies, regardless of whether the physical interface still reports "up."

BFD
5

Checkpoint — Failure Formally Declared

Once a configured number of consecutive probe intervals are missed (a detection multiplier, e.g. 3 missed at 150ms = ~450ms), the path is formally declared down — typically within a few hundred milliseconds to a couple of seconds. This is well before a human notices anything, and crucially, works the same whether the underlying cause was a hard link failure or a gray one.

Checkpoint
6

Routing Reacts

If dynamic routing is used between branch and HQ (e.g. eBGP per transport), the route learned over the failed path is withdrawn. On an SD-WAN overlay, the affected underlay member is simply marked unusable for path selection — either way, the failed path stops being a candidate for new traffic decisions.

Route Withdraw
7

Reconvergence to the Backup Path

A route lookup for HQ/internet-bound traffic now resolves to the surviving path — the broadband/LTE circuit — either via a pre-provisioned lower-preference static route now promoted, or SD-WAN's path selection algorithm automatically choosing the next best available member. No human intervention is required for this step.

Reconverge
8

What Happens to Traffic That Was Mid-Flight

Any packet already in transit over the failed link at the moment of failure is simply lost — rerouting doesn't retrieve it. Existing TCP sessions bound to that specific path (e.g. an active RDP session) typically break and must reconnect; some SD-WAN platforms with per-flow tracking can reroute successive packets of the same flow onto the new path transparently, but this depends heavily on platform capability, not something to assume universally.

In-Flight
9

Traffic Resumes on the Backup

New sessions, and any reconnecting old ones, now traverse the broadband/LTE circuit. If an IPsec/SD-WAN tunnel wasn't already active over this backup transport, it establishes now — adding a small additional delay on top of the raw path-selection time.

Resumed
10

"Rerouted" Doesn't Mean "Unaffected"

The backup path typically has materially different characteristics — lower bandwidth, higher latency, sometimes metered/capped data on LTE. Users regain connectivity but may notice real performance degradation; a fast failover is not the same claim as a transparent one.

Degraded
11

If There's No Backup Path at All

Without a secondary WAN transport, the branch loses connectivity to HQ and the internet entirely until the fiber is physically repaired. Critically, purely local traffic — same-VLAN and inter-VLAN communication within the branch — keeps working fine, since none of that depends on the WAN. The outage's blast radius is scoped exactly to anything requiring HQ or internet reachability.

No Backup
12

Alerting and Provider Engagement

Monitoring (NMS/SD-WAN controller) detects the interface-down and BFD-down events and alerts the NOC. A fault ticket is raised with the metro fiber provider referencing the circuit ID; physical repair — often requiring a field technician to splice the cut — restores the primary circuit.

Alert
13

Failback — Not Always Immediate

Once the primary circuit comes back up and passes health checks for a stabilization period, traffic can shift back if policy prefers the primary — but immediate failback isn't always configured, deliberately, to avoid "flapping" traffic back and forth if the repaired link is still intermittently unstable.

Failback
02 How to Explain This in an Interview
03 Follow-Up / Gotcha Questions
Q What's a gray failure, concretely, and why can't interface link-state alone catch it?
A A gray failure is one where the physical/logical link stays up — carrier detected, interface reports "up" — but traffic is silently dropped somewhere upstream, e.g. a provider misconfiguration or partial hardware fault. Interface state only reflects the local port's own signal, not whether traffic actually makes it end-to-end, which is exactly the gap active health probing (BFD) closes.
Q Why is BFD faster than relying on routing protocol hello timers alone?
A Routing protocol hellos (e.g. OSPF's default 10s, BGP's default 60s keepalive) are designed for general topology maintenance, not sub-second failure detection, and lowering them aggressively strains CPU on every neighbor. BFD is a purpose-built, lightweight protocol that can safely run at tens of milliseconds and simply notifies the routing protocol to react immediately once it detects loss — decoupling fast detection from the routing protocol's own overhead.
Q Does failover always mean zero data loss?
A No — any packet already committed to the failed path at the moment of failure is lost, and TCP sessions tied to that specific path typically have to reconnect from scratch unless the platform has flow-aware rerouting. Fast detection minimizes the window of impact; it doesn't eliminate loss for traffic already in flight.
Q Why might failback to the primary link not happen immediately even once it tests healthy?
A A freshly repaired link can be intermittently flaky before it's truly stable — automatically failing back the instant it passes one health check risks flapping traffic back and forth if it drops again shortly after. Many designs require the primary to stay healthy for a defined stabilization window, or require manual promotion, before committing production traffic back to it.
Q If the branch has no secondary WAN circuit at all, what's the actual scope of the outage?
A Everything requiring HQ or internet reachability is down — email, cloud apps, remote access, VoIP to other sites — but purely local activity keeps functioning: printing to a local printer, same-VLAN or inter-VLAN traffic between local hosts, and any application hosted entirely on local branch infrastructure.
Q How would you actually diagnose whether an outage is a hard failure or a gray failure during troubleshooting?
A Check interface/carrier state first — if it shows up but BFD/health checks are failing, that's the signature of a gray failure and points toward an upstream provider issue rather than a local physical fault. Confirming with the provider's own circuit status, or testing reachability to a known-good hop just beyond the local CPE, helps narrow down where along the path the actual blackholing is occurring.
04 Quick-Fire Glossary
TermMeaning
Gray FailureA link that stays physically up but silently fails to pass traffic correctly
BFDBidirectional Forwarding Detection — a fast, lightweight hello protocol for sub-second failure detection
Detection MultiplierThe number of consecutive missed health-probe intervals required before declaring a path down
Route WithdrawalRemoving a route learned over a now-failed path from the routing table
SD-WAN Member / UnderlayOne of the physical transports (e.g. MPLS, broadband) an SD-WAN overlay can route traffic across
Failover / FailbackSwitching to a backup path on failure, and later switching back once the primary recovers
Dampening / Stabilization WindowA delay requiring sustained health before trusting a recovered link again, to prevent flapping
Circuit IDThe provider's unique reference for a specific WAN circuit, used when raising fault tickets
NOCNetwork Operations Center — the team monitoring and responding to infrastructure alerts
Blast RadiusThe actual scope of what's impacted by a given failure, as distinct from what merely sounds impacted