Research guide
How to Build a Smoke Test Suite That Runs After Every Deploy
Smoke tests answer one question after every deploy: is this build stable enough for users? This guide covers the 10–15 high-signal checks, how to run them, and how to keep them fast and trustworthy.
What a smoke test is for
A smoke suite is a fast liveness check for the deployed build: the app loads, login works, the core action completes, and key integrations respond. It is not full regression coverage.
Its job is to catch the broken deploy within minutes, while the rollback is still cheap.
The 10–15 high-signal checks
Pick the checks that would have caught your last few incidents: homepage loads without console errors, login works, the primary action completes, search responds, checkout is reachable, and the critical API returns a healthy status.
Choose checks by what actually breaks deploys in your app, not by what is easy to write.
Run it against the live deploy
Run the smoke suite after the deploy completes, against the live environment or the preview URL, not a stale staging box. The point is to test the build that users will hit.
Keep the run to minutes so it can gate the release. A smoke suite that takes an hour gets skipped.
Keep it stable so it can be trusted
A smoke suite is only useful if teams trust it. Use resilient locators, mock external dependencies that do not belong to the app, use deterministic data, and make the health checks retry-able.
A flaky smoke suite becomes noise, and teams stop watching it exactly when it matters.
Wire it into the pipeline
Trigger the suite on every deploy, notify on failure, and block or warn based on confidence. For critical apps, block the deploy on a failed smoke check.
For the deeper coverage that runs against the exact build about to ship, see the staging and preview guide.
Grow it deliberately
Add a smoke check only when it would have caught a real incident. This keeps the suite small, fast, and directly tied to the failures that have actually hurt you.
Decide the full coverage set separately, using the what-to-test guide.
Add one agentic journey for the path most likely to move
A purely scripted smoke suite drifts as product and design reshape the buyer, signup, or recovery path. The script can keep passing because it is still validating yesterday’s route, which is exactly the failure mode a post-deploy smoke check exists to catch.
The pragmatic pattern is to keep deterministic checks for stable rules and add one intent-based browser journey for the single flow your team is actively changing. Describe the user outcome in plain language, run it against the live deploy, and verify the expected result rather than just the page load. When the coded check and the agentic journey disagree, the journey shows whether a real customer can still complete the path.
Key takeaways
- A smoke suite is a fast liveness check for the deployed build, not full regression.
- Run 10–15 high-signal checks against the live deploy within minutes.
- Keep it stable or it becomes noise and teams stop watching it.