Interview prompt
A sudden traffic spike brings the product down. Walk through your first minutes, the evidence you need, the mitigations you choose, and what happens after recovery.
The first strong observation
The first objective is not a perfect architecture; it is restoring the most important user journey without making the incident worse. Stabilize, observe, isolate, mitigate, and only then move into permanent fixes.
Evidence to inspect first
- ◆Request rate, errors, and latency by endpoint.
- ◆Saturation across app, database, connection pools, queues, and dependencies.
- ◆Recent deploys, feature flags, and traffic-source changes.
- ◆Whether retries or one expensive endpoint amplify the load.
A strong step-by-step approach
Declare and coordinate
Open the incident channel, assign an incident commander, freeze unrelated deploys, and establish one shared timeline. Clear ownership prevents parallel fixes from making recovery harder.
Protect the system
Rate-limit abusive or non-critical traffic, disable expensive features, serve stale data when safe, and shed work the system cannot complete. Preserve the core journey first.
Find the saturated dependency
Use service-level telemetry to distinguish app CPU, database locks, connection exhaustion, cache misses, queue backlog, and third-party failure. Scaling the wrong tier can increase pressure downstream.
Apply reversible mitigation
Scale the constrained stateless tier, warm or expand cache, reduce retry storms, pause consumers, or route traffic away. Prefer changes that are fast to undo and have a measurable hypothesis.
Recover deliberately
Restore features and traffic gradually while watching error budgets, queue age, database health, and cache hit rate. A green homepage alone does not mean the system has recovered.
Prevent recurrence
Write a blameless post-incident review with contributing factors, missing alerts, capacity assumptions, and owned actions such as load tests, autoscaling policy, backpressure, and runbooks.
Structure your answer in this order
- 01Name the user impact and incident objective
- 02Coordinate and stop additional change
- 03Use telemetry to isolate saturation
- 04Choose reversible mitigations
- 05Verify recovery and create prevention actions
Weak-answer signals
- ×Saying only 'add more servers' without identifying the constrained tier.
- ×Letting retries multiply pressure on an unhealthy dependency.
- ×Making multiple irreversible changes with no timeline or hypothesis.
Likely follow-up questions
- ?What would you disable first?
- ?What if the database, not the app tier, is saturated?
- ?How do you avoid a retry storm during recovery?
Quick FAQ
What is the interviewer evaluating in this scenario?
The interviewer is not looking for one technology name. They are evaluating how you gather evidence, limit risk, choose a safe action, and verify the outcome under uncertainty.
How long should the answer be?
Use the first two minutes to state your priority and initial hypothesis, then move through signals, diagnosis, mitigation, and verification. Go deeper when the interviewer asks follow-up questions.