Research guide

Your Playwright Suite Is Rotting from the Inside: A Maintenance Gate for Browser Tests

A Playwright suite can be green for months while the test code behind it quietly becomes expensive to maintain. A GitHub Actions badge says “passing,” but every component refactor, design-system update, or marketing page change silently tightens the coupling between the tests and the DOM. The suite still runs. It may even still pass. But the cost of keeping it green grows with each release, and the team does not see that cost because the pipeline only reports success or failure, not maintenance debt.

By CueTest QA Team · Published 2026-09-02

A Playwright suite can be green for months while the test code behind it quietly becomes expensive to maintain. A GitHub Actions badge says “passing,” but every component refactor, design-system update, or marketing page change silently tightens the coupling between the tests and the DOM. The suite still runs. It may even still pass. But the cost of keeping it green grows with each release, and the team does not see that cost because the pipeline only reports success or failure, not maintenance debt.

This rot is different from a flaky test. Flaky tests fail intermittently and announce themselves. Maintenance debt is more insidious: the tests remain green because someone updated a selector, because the AI execution layer healed around a broken locator, or because the test happened to depend only on visible text that did not change. The pipeline never alerts anyone that the suite has become fragile.

Selector Drift Is the Debt You Never See in a Green Pipeline

Consider a Playwright test written during the first implementation of a signup flow. At the time, the primary call to action was a button with a CSS class:

await page.locator('.btn-primary').click();

The test passed for three months. Then the design system replaced .btn-primary with .button--primary as part of a broader visual refresh. A teammate updated the test to match the new class. The UI automation stayed green, and the release shipped. Nothing in the pipeline indicated that the test had become more tightly coupled to an implementation detail.

A month later, the component library changed the button from a <button> element to a styled <div role="button">. The test failed, and another selector update was required. Each update added friction, but the product still shipped on time, so the maintenance cost was invisible.

Now compare that with a semantic locator, the kind Playwright explicitly recommends:

await page.getByRole('button', { name: 'Create account' }).click();

This locator continues to work after both changes because it is tied to the user-visible role and accessible name, not the CSS implementation. The Playwright documentation is direct: locators based on user-facing attributes such as roles, labels, and text are more resilient than CSS or XPath chains (Playwright: Locators). When a suite drifts away from those user-facing contracts, it cannot be fixed by making the pipeline greener. It can only be fixed by changing the way the tests locate controls.

Why Green Pipelines Hide Maintenance Debt

Traditional CI does not measure locator health. It measures pass and fail. If an engineer updates a bad selector, the test is green. If an AI execution layer heals around a stale selector, the test is green. If the test happens to pass because it clicks the first element that looks close enough, the test is green.

None of those green results tells you that the test is one UI change away from a maintenance headache. The result is a slow accumulation of selector debt that shows up later as:

• Routine refactors cause multiple unrelated test failures. A single component change should affect a few tests. With DOM-coupled selectors, it affects dozens.
• Test triage dominates release QA. Engineers spend more time deciding whether a failure is a real product regression or a stale locator than they spend reviewing the product.
• AI healing becomes the norm instead of a fallback. If every run requires the AI execution layer to repair the path, the deterministic plan is not doing its job.
• New hires avoid touching old tests. Tests become so tightly coupled to specific DOM structures that no one can safely update them.

The pass/fail badge is the wrong instrument for detecting this class of problem. It is like using a single green light to tell you whether an engine is still healthy while the oil level drops below the minimum. The light stays green until something breaks; the maintenance condition was visible only if someone checked the dipstick.

Natural-Language Tests Can Mask Maintenance Debt

Natural-language browser tests introduce a subtle version of this problem. Suppose a team describes a flow in plain language:

Scenario: Customer creates an account
When they open the signup page and enter valid details
Then they see a welcome heading and an email confirmation message

On first execution, the platform observes the actual UI, identifies the fields and controls, and stores a deterministic plan. That plan uses the selectors and interactions that worked against the live page. If the plan is good, it uses semantic locators and a clear expected result.

Now suppose the component library changes the signup button from a native <button> to a styled <div role="button">. A brittle CSS selector in the deterministic plan will fail. In some AI-native browser testing platforms, that failure may trigger a healing step. The AI layer searches for a control that satisfies the described step, clicks the new <div role="button">, and verifies the heading appears. The run passes.

The product is still functionally correct in this example, so the pass is not a false green in the same way as a regression. But the maintenance debt did not disappear. The deterministic plan still contains the brittle selector. The next run must heal again, unless the platform teaches a new plan. If every run depends on AI healing, the test is no longer deterministic enough to serve as a low-variance release anchor.

Playwright's own guidance draws a useful line: a test that passes only after a retry should be treated as flaky, not as an ordinary first-run pass (Playwright: Retries). The same principle applies to AI healing. When a plan fails and is repaired, that event is a maintenance signal. It tells you the test relied on something that changed. Treating it as a normal pass erases the signal.

A Maintenance Gate for Browser Tests

You do not need a separate maintenance team to detect browser test rot. You need a gate that makes maintenance evidence visible and acts on it before the suite reaches the point where every refactor is a triage emergency.

Stage 1: Track unplanned healing as a maintenance event

When a deterministic plan fails and the AI execution layer has to resolve the step another way, log that as a healing event. The event should include:

• The step that failed
• The locator or plan that failed
• What the AI layer chose instead
• Whether the expected result was independently verified after healing
• Whether the healed interaction was persisted as a new plan

If a step heals once during a one-off infrastructure issue, that is low priority. If the same step heals on every run, the plan is stale and needs attention.

Stage 2: Measure locator strategy coverage

Playwright provides a clear hierarchy of locator quality. User-visible attributes such as role, label, placeholder, and text are preferred; CSS and XPath should be used only when no user-facing contract exists (Playwright: Locators). Review the suite periodically and categorize each locator:

A healthy release-critical suite should favor low-risk locators. There is no universal ratio, but if the proportion of high-risk CSS and XPath selectors is increasing, the suite is accumulating debt faster than it is being modernized.

Stage 3: Require semantic locators for release-critical flows

Login, signup, checkout, password reset, account settings, and any flow that blocks a release should not depend on CSS classes that are likely to change during a design refresh. Make role- or label-based locators the default for these flows.

Natural-language scenarios can support this indirectly. The scenario should name the control in the same way a user would perceive it:

Weak:

When they click the primary button

Stronger:

When they press Tab to reach the Create account button
And they press Enter
Then they see a welcome heading with the text "Welcome"

The stronger version describes the user-visible role and the outcome. It gives the execution layer the semantic intent needed to prefer a resilient locator. It does not guarantee that the stored plan will be ideal, but it makes poor locator choices visible when healing occurs.

Stage 4: Treat selector drift as a release gate condition

Add maintenance signals to the release QA checklist. Before the release gate approves a build, the team should ask:

• Did any deterministic plan fail and require AI healing on the last two runs?
• Did a locator change cascade into multiple tests?
• Did the percentage of CSS/XPath locators in release-critical tests increase?
• Did any test pass after an unplanned selector adjustment without a corresponding product requirement?

If the answer to any of these is yes on a release-critical flow, the gate should flag the flow for maintenance before approving the release. The product code may be working, but the test suite is losing its ability to protect future changes.

The goal is not to create a bureaucratic metric. The goal is to make the invisible cost of locator churn visible enough that the team acts on it before the suite becomes too expensive to maintain.

How CueTest Fits a Maintenance-First Release Gate

CueTest is an AI-native browser testing platform built for natural-language E2E testing and Playwright maintenance. Its builder demonstrates focused user-flow steps in a live browser and captures the successful interactions into a deterministic plan. Each step also includes an independent expected-result check, so passing is not just the absence of an exception.

That architecture gives you two important maintenance controls. First, the deterministic plan is the default execution path. AI is not invoked unless the plan fails. Second, when AI does heal a step, the resolution is bounded and the expected result is verified afterward. If the healed interaction succeeds, CueTest can teach a new deterministic plan from it. That means healing becomes a visible transition, not an invisible workaround.

What CueTest cannot do is evaluate your locator strategy for you. If the stored plan uses a brittle CSS selector and the page changes, CueTest will try to heal and may succeed. That success does not tell you whether your underlying test maintenance is healthy. The value CueTest provides is that the need for healing appears in the run evidence. You can review that evidence, notice that the same step healed twice in a row, and conclude that the plan needs to be regenerated or the scenario needs to be made more explicit. The tool surfaces the signal; your team still has to decide what that signal means.

A Practical Starting Point

Do not try to audit the entire suite in one afternoon. Begin with one release-critical flow that currently has brittle locators or frequent manual selector updates.

1. Pick the flow: login, checkout, password reset, or account settings.
2. Open the current Playwright test or natural-language scenario and inspect how each control is located.
3. Replace at least three CSS or XPath selectors with user-facing alternatives: getByRole, getByLabel, or getByPlaceholder.
4. Run the test on two consecutive runs. If it heals on either run, review the healing event and update the plan.
5. Connect CueTest to your repository at https://cuetest.dev and run the same flow as an ordered set of natural-language steps with independent expected results. Compare the healing events over the next two release cycles.

The outcome you want is not a suite that never fails. You want a suite that fails for the right reasons. When a product regression occurs, the test should fail on the intended user-visible path. When the UI changes in a way that breaks a bad selector, the maintenance gate should catch it before it becomes release-blocking. Both outcomes depend on making selector drift and healing visible rather than assuming that a green pipeline means a healthy test suite.

Ready to stop treating selector drift as a silent cost? Audit one release-critical flow, replace brittle selectors with user-facing locators, and make unplanned healing a release-gate signal before your next merge.

Sources

• Playwright: Locators
• Playwright: Retries
• Playwright: Best Practices
• Testing Library: Guiding Principles
• Martin Fowler: Eradicating Non-Determinism in Tests
• Google Testing Blog: Just Say No to More End-to-End Tests