Research guide
How to Debug Flaky E2E Tests: A Practical Workflow
Flaky tests turn CI into noise. This guide walks a practical debugging workflow: reproduce the failure, categorize the root cause, inspect traces, fix the race not the symptom, and prove the fix.
Reproduce before you theorize
Run the failing test in a loop, up to twenty-five times, and break on the first failure. Capture the full output, screenshot, and trace from the failing run.
If it passes twenty-five times locally, the flake is likely environment-specific, and reproducing the CI environment becomes the task. If it fails, you now have a concrete starting point.
Categorize the failure
Every flaky failure is one of three things: a race or timing problem, an outdated test that no longer matches the app’s intent, or a real bug the app introduced. The fix differs for each.
Misdiagnosing an outdated test as a race sends a team adding waits to a test that should have been rewritten.
The seven root causes
The common causes are timing and race conditions, brittle selectors, shared test state, environment instability, animation interference, parallelism conflicts, and locator drift after UI changes.
Each has a distinct signature. Timing failures break under load, shared-state failures pass solo and fail in parallel, and animation failures are tied to transitions on the page.
Inspect the artifacts
The trace, screenshot, video, and console logs tell the story the assertion error does not. Compare the expected state with what actually rendered, and look for layout shifts, late network responses, or elements that never appeared.
The failing step plus the trace usually narrows the root cause to one of the seven categories.
Fix the root cause, not the symptom
Replace fixed sleeps with web-first assertions that wait for state, use semantic locators, isolate test data per test and per worker, mock external dependencies, and disable animations in the test environment.
Never paper over a flake with arbitrary delays, weakened assertions, or retry loops around assertions. Those convert a quality signal into noise that teams eventually stop reading.
Prove the fix and watch the suite
Run the test repeatedly after the fix and confirm the failure class is gone. Then watch the suite-wide flake rate, because one fixed flake often reveals the next.
For deciding what a failure means when it appears, use the failure-triage guide, and for the special case that only fails in CI, use the CI-debugging guide.
Key takeaways
- Reproduce in a loop and categorize the failure before you change anything.
- Fix the race condition, never paper over it with sleeps or retries.
- Use web-first assertions and semantic locators to remove whole classes of flake.