Research guide

Your AI Browser Tests Can Pass Because of Leftover State: A Release QA Gate for Scenario Isolation

A natural-language E2E test looks simple: describe the user, describe the action, describe the expected result. But if the scenario does not describe the starting state, the test inherits whatever state already exists in the browser or backend. A checkout test can pass because a previous scenario left the cart populated. A password change test can pass because the browser is still authenticated from an earlier login. The release gate turns green, but the committed evidence is not that the workflow works. The evidence is that the workflow worked under whichever conditions happened to exist before the test began.

By CueTest QA Team · Published 2026-08-25

A natural-language E2E test looks simple: describe the user, describe the action, describe the expected result. But if the scenario does not describe the starting state, the test inherits whatever state already exists in the browser or backend. A checkout test can pass because a previous scenario left the cart populated. A password change test can pass because the browser is still authenticated from an earlier login. The release gate turns green, but the committed evidence is not that the workflow works. The evidence is that the workflow worked under whichever conditions happened to exist before the test began.

State pollution is one of the most persistent false-confidence problems in browser testing. It is especially dangerous in natural-language E2E systems because the scenario language often treats state as an assumption rather than an assertion. The test author writes Given a guest user with an empty cart, but the AI execution layer may not verify that condition. It may simply navigate to the cart, see that the page already contains an item from the previous test run, and continue with the flow. The final result can still look correct on screen while the intended pre-condition was never true.

Why a Clean Scenario Is Not Automatic

Traditional Playwright tests are usually written as scripts. The test author controls the exact steps, including whichever setup or cleanup is required before the main action. A scripted test may still be flawed if setup is forgotten, but the missing setup is visible in code.

Natural-language E2E tests invert that relationship. The author describes an outcome, and the execution layer decides how to reach it. If the scenario begins with Given a guest user with an empty cart, but the page already has a cart item left over from another test, the execution layer may see that the desired final state is easier to reach if it does not first empty the cart. The instruction is treated as context, not as a check.

This creates a specific failure mode: the scenario passes because the final page satisfied the assertion before the intended action was performed. The test did not prove that a guest can add an item and see the correct total. It proved that the page could still show the expected total when something had already been added before the test began.

The same problem appears in reverse. A test can fail because a previous scenario left a modal open, changed a user setting, or expired a cached token. The failure is real, but it does not describe a product regression. It describes test data that leaked.

State-dependent failures are difficult to triage because they are not reliably reproducible in isolation. A test that fails at 10:14 in a shared execution environment may pass when run alone at 10:20. The intermittent nature makes it tempting to dismiss the failure as a flake, but the underlying condition is a false signal in the release gate: either a false green that masks a broken flow or a false red that wastes release engineering time.

Three Kinds of State That Leak Between E2E Scenarios

To fix state pollution, it helps to separate the types of state that can survive from one browser scenario to the next.

Playwright's browser context isolation is real. Playwright creates a fresh browser context for each test by default, and contexts do not share cookies, cache, or local storage (Playwright: Browser contexts). That default removes one class of browser-state leaks. But it does not remove backend state. If a test creates a real account in a shared staging environment, that account can affect the next run. If a test uses storageState to restore an authenticated session on purpose, then the shared authentication is intentional, but it must be documented as a pre-condition (Playwright: Auth).

The most dangerous leaks are the ones that are invisible to the browser execution layer. A backend record created by a previous test may still be present after the browser context is destroyed. The next test opens a clean browser, but the application behavior is already polluted.

A Natural-Language Example That Hides State Pollution

This scenario is written for a guest checkout flow:

Scenario: Guest can see shipping rates after entering an address
Given a guest customer with an item in the cart
When they enter a valid shipping address
Then they see a shipping rate
And the order total includes the shipping rate

The scenario says Given a guest customer with an item in the cart. But how did the item get there? If the execution layer reused a browser session from a previous signed-in test, the user may not be a guest at all. If the cart already contained a different item created by a preceding checkout test, the displayed shipping rate may be correct for the wrong item. The final assertion still passes because a shipping rate is visible.

Now consider a variant that would fail only under polluted state:

Scenario: Guest can submit a payment
Given a guest customer with an empty cart
When the guest does not add any item
Then the guest sees an "Add an item to continue" message

If the cart is empty, this scenario passes. If a previous test left a cart item in shared state, the scenario fails for no product reason. The failure is state leakage, not an application bug.

The problem is not that the natural-language format is weaker than code. The problem is that the scenario language does not distinguish between an assumption and an assertion. The same sentence can be used as context for the execution layer or as a check that must be verified before proceeding. For release QA, that distinction is critical.

Playwright Context Isolation Is Not the Whole Answer

Playwright's default model is helpful. In a standard Playwright test run, each test receives its own isolated browser context. That means cookies, localStorage, and sessionStorage do not carry over from one test to another unless the test explicitly uses storageState or browser.newContext() with shared options (Playwright: Browser contexts, Playwright: Test fixtures). This is better than many legacy Selenium implementations that reused a single browser profile.

But the default does not solve everything.

First, many teams intentionally reuse storageState for authenticated tests. The login flow is slow, so they store the session and restore it before each authenticated scenario. That is reasonable, but it means the test assumes the session is valid and that the user's backend state has not been changed by another test. If a password reset test changes the password for the shared account, every subsequent authenticated test fails.

Second, backend state is not controlled by the browser context. The browser may be clean, but the application still sees the same staging database. A test that creates a customer record, an order, or a saved payment method leaves those records for later tests unless an explicit cleanup step removes them.

Third, parallel execution makes leakage worse. When tests run in parallel against the same backend, they race over shared fixtures. Two tests may both attempt to use qa-user@example.com with different expected account states. The result is nondeterministic: the same test may pass or fail depending on scheduling.

Playwright itself recommends keeping tests isolated and avoiding dependencies between tests. The Playwright test runner is built around the idea that tests should be able to run independently (Playwright: Test isolation). But that guidance only applies to browser state. The test author still has to design backend cleanup and shared fixtures.

A Release QA Gate for State-Aware Natural-Language E2E

The fix is not to stop using natural-language tests. The fix is to make the starting state part of the release contract. A state-aware release gate treats every scenario as if it has three phases: preconditions, action, and outcome. The preconditions must be verified, not assumed.

Stage 1: Make the starting state explicit and observable

Weak natural-language scenarios often leave the starting state invisible:

Scenario: Customer reset password
When the customer resets the password
Then they can sign in with the new password

This says nothing about whether the customer was already signed in, whether another session was active, or whether the account had a previous password.

A state-aware scenario names the exact state that must be true before the main action:

Scenario: Signed-out customer can reset password
Given the browser has no authenticated session
And the customer has not requested a reset in the last 60 minutes
And the customer knows their current email address
When they open the login page and use the visible Forgot password? link
Then they can create a new password
And the old password no longer works when signing in

The stronger scenario gives the execution layer an observable pre-condition: no authenticated session. If the previous test left a session, that condition is false, and the scenario should fail at the precondition rather than silently continuing with the wrong state.

Stage 2: Separate browser state from backend state

Use a reset strategy for each domain. For browser state, rely on Playwright's per-test context or explicitly clear storage:

test.use({
  storageState: { cookies: [], origins: [] }
});

For backend state, create a reset endpoint or fixture that restores the staging environment to a known condition before each release-critical scenario. A simple Playwright setup block can call a test-only reset API:

import { test, expect } from '@playwright/test';

test.beforeEach(async ({ request }) => {
  await request.post('https://staging.example.com/api/test/reset', {
    data: { tenant: 'e2e' }
  });
});

test('guest can add an item and see the correct total', async ({ page }) => {
  await page.goto('/cart');
  await expect(page.getByText('Your cart is empty')).toBeVisible();

  await page.goto('/products/widget');
  await page.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByRole('link', { name: 'View cart' }).click();

  await expect(page.getByText('1 item in cart')).toBeVisible();
  await expect(page.getByText('$42.00')).toBeVisible();
});

The reset endpoint is not a product feature. It is test infrastructure that creates reproducibility. If a reset endpoint is unavailable, use a fixture account that the test creates and deletes in a finally block, or use a database transaction that rolls back after the test. The important property is that the setup becomes deterministic.

Stage 3: Verify the precondition as an assertion

A natural-language scenario should not treat Given as decoration. It should turn the precondition into a check that the execution layer runs before the main action.

For a guest checkout scenario, add an assertion that the cart is empty before the test adds the first item. In Playwright, that can be written directly:

test('guest checkout does not inherit prior cart state', async ({ page }) => {
  await page.goto('/cart');

  // Fail before the main action if cart state leaked from another test.
  await expect(page.getByText('Your cart is empty')).toBeVisible();

  await page.goto('/products/widget');
  await page.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByRole('link', { name: 'Checkout' }).click();

  await expect(page.getByRole('heading', { name: 'Checkout' })).toBeVisible();
});

If the cart already contains an item from a previous run, this test fails at the precondition. That is the right failure. It tells the team that the release evidence was contaminated, not that the product is broken.

Stage 4: Audit shared authentication and persistent server fixtures

Authenticated tests often use a shared storageState file because the team wants to avoid repeating the login flow. That is fine, but only if the shared session is treated as a first-class precondition. Write the scenario so that it explicitly depends on the session:

Scenario: Signed-in customer can update account email
Given the browser is using the seeded authentication session
And the account has not reached any email change limit
When the customer changes their email on the account settings page
Then the confirmation message shows the new email
And the account page shows the new email after reload

This tells the execution layer that the test expects the seeded session. It also makes the dependency visible to release engineering. If someone later changes the seeded account, the scenario will fail loudly instead of passing with an erroneous session.

For backend fixtures, audit shared accounts that are used by multiple tests. A common failure is a single qa-user@example.com that password reset tests and settings update tests both use. If the settings update test runs first and changes the account state, the password reset test may fail for unknown reasons. The fix is to separate test users by workflow or to reset the account state before each test group.

Why Deterministic Playwright Anchors Still Matter

Natural-language scenarios are excellent for describing a user journey. They are less precise when the release gate needs a deterministic check that state has not leaked. That is why release-critical workflows should pair natural-language scenarios with specific Playwright anchors.

A Playwright anchor is a small, deterministic check that sets a boundary around the scenario. It should verify the starting state, run the user action, and verify the final outcome with explicit assertions. The natural-language scenario then documents the same contract for less technical stakeholders.

Consider a password change flow. The natural-language scenario says:

Scenario: Signed-in customer can change password
Given the browser is using the seeded authentication session
When they change their password from the account settings page
Then the old password no longer works
And the new password works on the next sign-in

The Playwright anchor enforces the same conditions with code:

test('password change is isolated from previous password changes', async ({ page, request }) => {
  await request.post('https://staging.example.com/api/test/reset', {
    data: { account: 'qa-settings@example.com' }
  });

  await page.goto('/account/settings');
  await page.getByRole('link', { name: 'Change password' }).click();
  await page.getByRole('textbox', { name: 'New password' }).fill('new-pw-123');
  await page.getByRole('textbox', { name: 'Confirm new password' }).fill('new-pw-123');
  await page.getByRole('button', { name: 'Save password' }).click();

  await expect(page.getByText('Password changed')).toBeVisible();

  const session = await request.post('https://staging.example.com/api/auth/login', {
    data: {
      email: 'qa-settings@example.com',
      password: 'new-pw-123'
    }
  });
  expect(session.ok()).toBeTruthy();
});

This anchor will fail if the account already has a different password left over from a previous test. It will fail if the reset endpoint did not restore the account to the expected pre-change state. It will fail if the browser session belongs to a different account. It turns state pollution into an actionable failure signal.

How CueTest Fits a State-Aware Release Gate

CueTest is an AI-native browser testing platform built for natural-language E2E testing and Playwright maintenance. In CueTest, the natural-language scenario is the source of truth, and the platform translates that scenario into executable browser checks.

That design helps with state pollution only when the scenario author writes the preconditions into the test. CueTest can verify that the browser has no authenticated session, that the cart is empty, or that the account is the seeded test account. But CueTest cannot know that you intended those preconditions if the scenario only says checkout works. The instruction has to say Given the cart is empty or And the browser has no authenticated session.

The practical first step is to audit the release-critical flows your team runs most often. Look at the natural-language scenarios and ask whether each one names a starting state that could be wrong. Usually the highest-risk flows are guest checkout, password reset, signup, refunds, and any authenticated settings change. Those flows rely on sessions, cart data, and backend records that easily cross between tests.

Rewrite one flow so that its preconditions are observable. In CueTest, you can write a scenario that explicitly checks the empty cart before adding an item, then connect the repository at https://cuetest.dev. Over the next two release cycles, compare the failure signal from that scenario against the same flow before it was state-aware. You should expect fewer false greens when a previous test contaminated the environment, and fewer false reds when a test failed because of missing data that should have been reset.

Ready to remove leftover state from your release gate? Pick your most release-critical natural-language flow, add the missing starting-state assertions, and make state isolation a blocking condition before your next merge.

Sources

• Playwright: Browser contexts
• Playwright: Test isolation
• Playwright: Test fixtures
• Playwright: Auth
• MDN Web Docs: Web Storage API
• Martin Fowler: Eradicating Non-Determinism in Tests
• Google Testing Blog: Just Say No to More End-to-End Tests