Research guide

Your AI Browser Tests Can Pass Because of Leftover State: A Release QA Gate for Scenario Isolation

A natural-language E2E test looks simple: describe the user, describe the action, describe the expected result. But if the scenario does not describe the starting state, the test inherits whatever state already exists in the browser or backend. A checkout test can pass because a previous scenario left the cart populated. A password change test can pass because the browser is still authenticated from an earlier login. The release gate turns green, but the committed evidence is not that the workflow works. The evidence is that the workflow worked under whichever conditions happened to exist before the test began.

Why a Clean Scenario Is Not Automatic

Traditional Playwright tests are usually written as scripts. The test author controls the exact steps, including whichever setup or cleanup is required before the main action. A scripted test may still be flawed if setup is forgotten, but the missing setup is visible in code.

Natural-language E2E tests invert that relationship. The author describes an outcome, and the execution layer decides how to reach it. If the scenario begins with Given a guest user with an empty cart, but the page already has a cart item left over from another test, the execution layer may see that the desired final state is easier to reach if it does not first empty the cart. The instruction is treated as context, not as a check.

This creates a specific failure mode: the scenario passes because the final page satisfied the assertion before the intended action was performed. The test did not prove that a guest can add an item and see the correct total. It proved that the page could still show the expected total when something had already been added before the test began.

Three Kinds of State That Leak Between E2E Scenarios

To fix state pollution, it helps to separate the types of state that can survive from one browser scenario to the next.

Playwright's browser context isolation is real. Playwright creates a fresh browser context for each test by default, and contexts do not share cookies, cache, or local storage (Playwright: Browser contexts). That default removes one class of browser-state leaks. But it does not remove backend state. If a test creates a real account in a shared staging environment, that account can affect the next run. If a test uses storageState to restore an authenticated session on purpose, then the shared authentication is intentional, but it must be documented as a pre-condition (Playwright: Auth).

The most dangerous leaks are the ones that are invisible to the browser execution layer. A backend record created by a previous test may still be present after the browser context is destroyed. The next test opens a clean browser, but the application behavior is already polluted.

A Natural-Language Example That Hides State Pollution

This scenario is written for a guest checkout flow:

The scenario says Given a guest customer with an item in the cart. But how did the item get there? If the execution layer reused a browser session from a previous signed-in test, the user may not be a guest at all. If the cart already contained a different item created by a preceding checkout test, the displayed shipping rate may be correct for the wrong item. The final assertion still passes because a shipping rate is visible.

Now consider a variant that would fail only under polluted state:

Playwright Context Isolation Is Not the Whole Answer

Playwright's default model is helpful. In a standard Playwright test run, each test receives its own isolated browser context. That means cookies, localStorage, and sessionStorage do not carry over from one test to another unless the test explicitly uses storageState or browser.newContext() with shared options (Playwright: Browser contexts, Playwright: Test fixtures). This is better than many legacy Selenium implementations that reused a single browser profile.

But the default does not solve everything.

First, many teams intentionally reuse storageState for authenticated tests. The login flow is slow, so they store the session and restore it before each authenticated scenario. That is reasonable, but it means the test assumes the session is valid and that the user's backend state has not been changed by another test. If a password reset test changes the password for the shared account, every subsequent authenticated test fails.

A Release QA Gate for State-Aware Natural-Language E2E

The fix is not to stop using natural-language tests. The fix is to make the starting state part of the release contract. A state-aware release gate treats every scenario as if it has three phases: preconditions, action, and outcome. The preconditions must be verified, not assumed.

Weak natural-language scenarios often leave the starting state invisible:

This says nothing about whether the customer was already signed in, whether another session was active, or whether the account had a previous password.

Why Deterministic Playwright Anchors Still Matter

Natural-language scenarios are excellent for describing a user journey. They are less precise when the release gate needs a deterministic check that state has not leaked. That is why release-critical workflows should pair natural-language scenarios with specific Playwright anchors.

A Playwright anchor is a small, deterministic check that sets a boundary around the scenario. It should verify the starting state, run the user action, and verify the final outcome with explicit assertions. The natural-language scenario then documents the same contract for less technical stakeholders.

Consider a password change flow. The natural-language scenario says:

How CueTest Fits a State-Aware Release Gate

CueTest is an AI-native browser testing platform built for natural-language E2E testing and Playwright maintenance. In CueTest, the natural-language scenario is the source of truth, and the platform translates that scenario into executable browser checks.

That design helps with state pollution only when the scenario author writes the preconditions into the test. CueTest can verify that the browser has no authenticated session, that the cart is empty, or that the account is the seeded test account. But CueTest cannot know that you intended those preconditions if the scenario only says checkout works. The instruction has to say Given the cart is empty or And the browser has no authenticated session.

The practical first step is to audit the release-critical flows your team runs most often. Look at the natural-language scenarios and ask whether each one names a starting state that could be wrong. Usually the highest-risk flows are guest checkout, password reset, signup, refunds, and any authenticated settings change. Those flows rely on sessions, cart data, and backend records that easily cross between tests.

Key takeaways

  • A green natural-language E2E test can be produced by leftover browser sessions, cached data, or server-side records from earlier scenarios.
  • Playwright isolates each browser context by default, but state can still leak through shared storage state, intentionally reused authentication, or persistent backend records.
  • A release QA gate should make starting state explicit, enforceable, and verifiable before the main user action runs.
  • CueTest can execute the preconditions you write, but it cannot infer that a scenario was meant to start from a clean state if the scenario never says so.

Related CueTest resources