Frontend tests that pass in a preview environment and fail in CI are usually not random. They are telling you that the test is sensitive to a difference you have not measured yet. The trick is to avoid the common reflex of rerunning immediately. A rerun can hide the signal you actually need.

If you have ever seen frontend tests pass in preview but fail in CI, the problem is often one of five things: build artifact drift, timing differences, browser or driver mismatch, data and service variance, or missing observability around the failure itself. This guide focuses on the exact logs and artifacts worth collecting before the next rerun, so you can turn a vague flaky failure into a narrow hypothesis.

The goal is not just to make the test green. The goal is to understand why the same scenario behaves differently across environments.

Start by defining the two environments precisely

People often say “it works in preview” as if preview were a single thing. In practice, preview environments vary in at least four dimensions:

  • the commit or build artifact being deployed,
  • the runtime configuration injected at deploy time,
  • the browser and device profile used for validation,
  • the network and backend dependencies available to the app.

CI has its own differences, and some are subtle. CI might run in a container with different fonts, different locale defaults, a different Node or browser version, a different CPU budget, or stricter resource limits. That is why “preview versus CI” is not a binary comparison, it is an environment matrix problem.

Before adding more logging, capture the identity of each environment in a machine-readable form. You want to know whether both systems are testing the same artifact, the same commit, and the same runtime settings.

For background on the general practice, see continuous integration, test automation, and software testing.

Minimum environment identity to log

At the top of each test run, record:

  • git commit SHA
  • branch name
  • build or deployment ID
  • environment name, such as preview, staging, or CI
  • test suite version or package lock hash
  • browser name and version
  • test runner version
  • operating system and container image tag
  • Node, Java, Python, or other runtime version
  • feature flags and environment variables that affect the app

If you cannot prove that the preview and CI runs exercised the same build artifact, everything else becomes noisy speculation.

Log the artifact lineage first

One of the most common causes of preview environment drift is simple artifact mismatch. The frontend code in preview may not be the exact same bundle that CI is validating. This happens when preview is built from a different commit, uses a different dependency lockfile state, or reconstructs assets with a different compiler configuration.

The most useful artifact logs are those that answer three questions:

  1. What source code generated this build?
  2. What exact output bundle was deployed?
  3. What runtime environment produced that bundle?

What to capture

  • commit SHA and parent SHA
  • package manager lockfile digest, such as package-lock.json, pnpm-lock.yaml, or yarn.lock
  • build command and build parameters
  • bundler version, for example Vite, Webpack, or Turbopack
  • environment variables used during build time, especially public config
  • output asset hashes or manifest entries
  • deployment timestamp and artifact ID

A small manifest file can help. Generate it during build, store it as an artifact, and attach it to the test run.

{ “commit”: “a1b2c3d4”, “buildId”: “preview-1842”, “node”: “20.11.1”, “browser”: “chromium-127.0”, “lockfileHash”: “9f31e1…”, “publicEnv”: { “API_BASE_URL”: “https://api-preview.example.com”, “FEATURE_X”: “true” } }

This does not solve the bug, but it stops the guessing.

Capture the browser and test runner state

A frontend test often fails because the browser state is not what the test assumes. That can mean a cookie is missing, local storage contains stale data, the viewport differs, or the page loaded a different language locale. CI is less forgiving than a preview click-through because CI usually starts from a clean, deterministic state, which can expose hidden dependencies in the test itself.

Log the test runner and browser state at the moment just before the failure:

  • browser version and channel, such as Chromium stable or Firefox
  • headless or headed mode
  • viewport size and device scale factor
  • timezone and locale
  • storage state, cookies, local storage, session storage
  • authenticated user or test account identity
  • permissions granted, such as geolocation, clipboard, or notifications
  • any network stubbing or route interception enabled

If your test framework supports attachments, save a snapshot of the browser context or session state for the failed run. In Playwright, for example, a saved storage state can be very useful when the issue is related to login, routing, or session renewal.

import { test } from '@playwright/test';

test('checkout flow', async ({ page, context }) => {
  await page.goto('/checkout');
  await context.storageState({ path: 'artifacts/storage-state.json' });
});

The exact storage snapshot is often more useful than a generic “test failed” message, because it tells you whether the failure came from authentication, CSRF expiry, or a missing app state transition.

Log the network path, not just the final error

Preview environments often sit close to the backend, or they hit test doubles with stable behavior. CI may be running behind a different egress path, DNS resolver, proxy, or rate limit. The test may fail only because the page request sequence changes slightly in that environment.

If a test depends on network requests, collect these details:

  • request URLs and HTTP methods
  • response status codes
  • request timing and total latency
  • retry counts
  • cache headers and cache hits
  • redirects
  • request payloads for important API calls
  • response bodies for failed or unexpected responses, if safe to store
  • whether the request was mocked, intercepted, or real

For browser-based E2E runs, consider capturing HAR files or framework-specific network logs. A HAR is especially useful when the failure is “element never appeared” because it can show whether the data request actually returned, returned too slowly, or failed entirely.

If the UI is waiting on data, the network trace is usually more honest than the UI assertion.

If your CI environment allows it, log DNS resolution or upstream service health for the backend endpoints that the frontend depends on. A service that is reachable from preview but intermittently slow from CI can produce timeouts that look like flaky selectors.

Record console output and runtime errors from the browser

A surprising number of CI-only failures are already visible in the browser console, but teams do not persist the logs. That is a missed opportunity, because console warnings and uncaught exceptions often point directly to the difference between environments.

Capture:

  • console.error, console.warn, and unexpected console.log output
  • uncaught exceptions
  • promise rejections
  • CSP violations
  • hydration warnings
  • failed script loads
  • cross-origin access errors
  • source map resolution errors, if they affect stack traces

In Playwright, a lightweight listener can collect this data without much overhead.

page.on('console', msg => {
  if (['error', 'warning'].includes(msg.type())) {
    console.log(`[browser:${msg.type()}] ${msg.text()}`);
  }
});

page.on(‘pageerror’, err => { console.log([pageerror] ${err.message}); });

If the error is intermittent, include timestamps. A console error that appears 150 ms before a selector timeout is a much better clue than a single stack trace at the end of the test.

Log timing and state transitions, not only assertion failures

Most flaky frontend tests fail because the assertion is too early or the UI is not yet in the state the test assumes. In preview, a page may be slightly faster, the browser may have warmed caches, or a backend dependency may respond faster. In CI, the same test can cross a threshold and fail.

Instead of logging only the final assertion result, log intermediate state changes:

  • page navigation start and completion
  • DOM content loaded and load events
  • route changes in SPA navigation
  • API readiness checks
  • loading skeleton visibility
  • spinner disappearance
  • enabled or disabled button states
  • text content changes that signal data arrival

A practical pattern is to add structured “milestone” logs around key waits.

typescript console.log(‘waiting for user profile to render’);

await page.getByTestId('user-name').waitFor({ state: 'visible' });
console.log('user profile visible');

These markers help you determine whether the failure is in the app, the selector, or the waiting strategy.

Distinguish state failure from locator failure

A locator problem means the element is not found or does not match. A state problem means the element exists but is not ready, visible, or interactive. Logging should make that distinction obvious.

For example, capture:

  • whether the element exists in the DOM,
  • whether it is visible,
  • whether it is disabled,
  • whether it is obscured by another layer,
  • whether its text is stable or still changing.

That information can reduce a broad “timeout” to a concrete issue such as animation delay, delayed hydration, or a modal overlay that appears only in CI because of slower loading.

Save screenshots, DOM snapshots, and video at the moment of failure

Visual evidence is especially helpful when the difference between preview and CI is subtle. A missing font, an overflowing container, a clipped modal, or a layout shift can produce an interaction failure that logs alone will not explain.

Collect at least one of the following on failure:

  • screenshot of the current page state
  • full-page screenshot if supported and useful
  • DOM snapshot or HTML dump
  • video of the session, if the test is long-lived or interactive
  • accessibility tree snapshot if the issue involves aria state or focus management

A screenshot is often enough to show that the app rendered a login wall, a cookie banner, or a server error page instead of the expected view. A DOM snapshot is better when the page looks right but a selector fails because the node is absent or replaced.

If the app has hydration or rendering issues, a DOM snapshot taken before and after key waits can expose a mismatch between server-rendered markup and client state.

Keep a focused timeline of events

When a frontend test fails in CI, the most helpful artifact is often a timeline, not a stack trace. A timeline can stitch together build metadata, browser events, network calls, and assertion milestones.

A useful format is a single JSON log stream with timestamped entries.

{ “ts”: “2026-07-10T12:00:01.124Z”, “runId”: “ci-88421”, “event”: “navigation_start”, “url”: “/dashboard” }

A good timeline answers questions like:

  • Did the page load before the API response arrived?
  • Was the feature flag enabled when the page rendered?
  • Did the test click before the element became enabled?
  • Did a redirect or auth refresh happen in CI but not preview?
  • Did a network retry or server delay change the page state?

You do not need to log every DOM mutation. You need enough signal to reconstruct the failure path.

Compare configuration, not just code

Many preview environment drift bugs are configuration bugs in disguise. The frontend source is the same, but runtime values are not.

Common configuration differences include:

  • API endpoints
  • auth issuer URLs
  • cookie domain and secure flags
  • feature flag defaults
  • analytics scripts
  • CSP headers
  • image optimization settings
  • CDN behavior
  • locale and currency settings

A useful debugging step is to diff the resolved runtime config between preview and CI. If your app reads config from environment variables, generated files, or server-side templates, dump the final resolved values into the test artifact.

If you use feature flags, log both the flag name and the resolved variant. A test can fail only in CI because CI is routed to a different experiment bucket, even if everyone assumes the app is “the same.”

Log the test data and seeding path

Frontend tests often depend on seeded users, fixture data, or mock API responses. If preview and CI are not using the same data creation path, the UI can render differently even when the app code is identical.

Capture:

  • seed script version or fixture file hash
  • test account ID
  • data creation timestamp
  • any cleanup status from previous runs
  • mocked payload versions
  • seed service response codes
  • fixture IDs used by the test

The most valuable question is whether the test is asserting against immutable data or against data that may be changed by another process. If a preview environment uses a manual seed and CI uses a fresh database, the “same” test may be behaving correctly in one environment and correctly failing in the other.

Watch for browser-specific differences

A browser test that passes in a preview smoke check and fails in CI may simply be running in a different browser engine or mode. That includes differences in:

  • scrollbar behavior
  • text rendering and line wrapping
  • focus management
  • click target calculations
  • drag and drop semantics
  • file input behavior
  • date and time formatting
  • clipboard APIs

Log the exact browser family and version. If the test only fails in headless mode, log that too. Headless and headed runs can differ in timing, layout, and certain Web API behaviors.

If your team supports multiple browsers, compare failures by engine. A CI failure in WebKit but not Chromium is not the same bug as a universal failure across browsers.

Use targeted screenshots and traces, not blanket verbosity

It is tempting to turn on maximum logging everywhere. That can help once, then create a long-term maintenance problem. Excessive logs slow CI, bury useful information, and increase the size of artifacts you need to inspect.

A better approach is to log aggressively only around:

  • navigation
  • authentication
  • data loading
  • interactive actions
  • failure boundaries

For browser automation, a trace or step-level diagnostic artifact is often more useful than raw verbose output. Playwright’s tracing, for example, can provide snapshots, network data, and DOM state in one artifact, while still keeping the default logs manageable.

await context.tracing.start({ screenshots: true, snapshots: true });
// run test
await context.tracing.stop({ path: 'artifacts/trace.zip' });

The point is to create enough context that a single failure can be interpreted without rerunning ten times.

A practical triage checklist before rerunning

If a frontend test passes in preview but fails in CI, use this order of operations before rerunning:

  1. Confirm the commit SHA and artifact hash match.
  2. Confirm browser, runner, and OS versions match or note the delta.
  3. Compare runtime config and feature flags.
  4. Check whether the same data seed and test account were used.
  5. Review network logs for status, timing, and redirects.
  6. Review browser console errors and page errors.
  7. Inspect screenshot, video, or DOM snapshot from the failed run.
  8. Read the timeline of state transitions, especially before the assertion.
  9. Decide whether the failure looks like test fragility, app defect, or environment drift.

If you rerun before collecting these signals, you may replace a reproducible environment mismatch with a one-off green result and lose the evidence.

When the failure is probably the test, not the environment

Not every CI failure is caused by infrastructure or preview drift. Sometimes the test is simply too dependent on timing, CSS structure, or implementation details. Signs that the test itself is brittle include:

  • selecting elements by fragile CSS position or nth-child paths,
  • waiting for arbitrary timeouts instead of observable state,
  • asserting on transient animation frames,
  • assuming network and rendering timing that is not guaranteed,
  • depending on exact copy that changes during localization or A/B testing.

If the preview environment is stable and CI is not, ask whether the test depends on a hidden assumption about load speed or page order. The right fix may be to wait for a semantic state, not to keep adding logs.

A sample CI log bundle that is actually useful

A helpful failure artifact bundle for browser test observability usually contains:

  • run-metadata.json, commit, artifact, browser, and environment data
  • console.log, browser console output
  • network.har, request and response trace
  • screenshot.png, failure-state image
  • trace.zip, if your runner supports it
  • dom.html, a snapshot of the current page
  • test-timeline.json, timestamped state changes

You do not need all of these every time. But you should know which one answers which category of bug.

Make failure logging part of the test contract

Teams often treat logging as a debugging convenience. For flaky frontend tests, it should be part of the contract. A test that cannot explain its own failure is expensive to operate.

A good rule is this: every end-to-end test should emit enough structured context that someone who did not run it can still answer the following:

  • What was the environment?
  • What data was loaded?
  • What did the browser see?
  • What did the network return?
  • What event failed to happen in time?

That standard is especially important in CI because the person reading the log is rarely the same person who wrote the test.

Closing thought

When frontend tests pass in preview but fail in CI, the issue is usually not a lack of retries. It is a lack of evidence. The best debugging workflow is to collect the environment identity, artifact lineage, browser state, network trace, console errors, and a short timeline of meaningful events before you rerun anything.

Once those signals are in place, preview environment drift becomes much easier to reason about. You can separate true product defects from test brittleness, and you can make CI debugging logs useful instead of noisy. That is the difference between chasing flakes and building a test system you can trust.