The control cohort problem: tracking GrowthBook URL-redirect experiments

You want to test a full page redesign — not a few tweaked strings, the whole thing. New cart, new template, new URL. The clean way to do that in GrowthBook is a URL Redirect experiment: split traffic 50/50 between the old URL and the new one, and let the SDK send each user to their side.

And then, a week in, you look at the results and the new design is “winning” — except the numbers feel thin. Half your traffic is missing. The treatment cohort is all there. The control cohort is a ghost.

This is a trap I hit on a cart-redesign test, and it’s worth writing down because the failure is silent. Nothing errors, nothing is misconfigured — the redirect splits traffic exactly as designed, the variant renders, the metrics tick up. The gap is structural, baked into how this kind of experiment reports rather than anything in the setup. The experiment that’s broken is the one that looks like it’s working.

Where the coinflip actually happens

There are two kinds of GrowthBook experiment, and they enrol users in completely different places.

A feature-flag experiment is evaluated by the SDK in the browser. Your code asks growthBook.getFeatureValue("my-experiment", -1), GrowthBook hashes the user, returns a variation, and your render callback does something with it. You own that code path — enrolment, tracking, DOM changes, all of it run where you can see them.

A URL Redirect is a different animal. It’s an auto-experiment — same family as visual-editor changes — and the SDK applies it automatically. On the plain JS / Script Tag SDK (no edge worker, which is the default and what most of my clients run), that means: the page loads, the SDK fetches its payload, hashes the user locally in the browser, and then calls window.location.replace(newUrl) after a short navigateDelay. The redirect is client-side. The coinflip never leaves the browser.

The catch is that an auto-experiment does not map to a feature key. So when your code asks:

growthBook.getFeatureValue("cart-redesign-redirect", -1)

it gets back -1. The default. There’s no feature by that name to evaluate — the redirect is applied through the SDK’s auto-experiment path, not the feature-evaluation API your render callback reads. As far as your framework is concerned, this experiment doesn’t exist. You can’t enrol users into it, you can’t read which variation they’re in, and — critically — you can’t fire a view event or wire up click tracking from inside it.

(GrowthBook does fire its trackingCallback for auto-experiments — it’s the one browser-side hook you get for any bucketed experiment. But if, like mine, that callback only logs, the signal goes nowhere useful.)

That’s fine for the redirect itself. Its only job is to move people to a different URL, and it does. The problem is what happens to tracking on each side of the split.

The asymmetry

Walk the two cohorts through, and the bug falls out on its own.

Treatment cohort. The SDK redirects them to /cart-new. That’s a different URL, so you point a perfectly ordinary URL-targeted experiment at it — call it cart-text. It enrols normally (the SDK can see a URL-targeted feature experiment), applies the redesign’s copy treatment, fires gb_view_experiment under its own exp_id, and registers click tracking on every cart element. Fully observed.

Control cohort. No redirect. They stay on /cart. And there is… nothing there. No experiment is targeting the old URL, because in your head the redirect “is” the experiment. These users never enter a tracked experiment at all. No view event, no click tracking, no rows in your analytics.

So you end up with treatment data and no control data, which is the one shape of A/B result that’s worse than no data — because it looks like data. You can compute a “conversion rate” for the variant. There’s just nothing valid to compare it against.

The root cause in one sentence: the redirect is applied outside the feature-evaluation path your code reads, so neither cohort is observable for free — and it’s easy to accidentally fix only the half that landed on a new URL.

The fix: a paired A-A tracker

The treatment side got tracked for free because it moved to a new URL that you happened to target. So give the control side the same treatment: point a client-side experiment at the old URL whose entire job is to observe.

This is an A-A test — both variants identical, no DOM changes — targeted at the control path:

{
  "country": "de",
  "pathname": { "$regex": "^/cart/?$" }
}

100% of traffic to a single variant. No DOM mutation, no treatment, no cohort assignment — it does not decide anything. It just notices a confirmed control-cohort visit (the “confirmed” is load-bearing — more on that in a moment) and does three things:

• fires gb_view_experiment with its own exp_id, one event per control visit;
• registers the same click tracking (cart quantity, remove item, change channel, whatever you’re measuring) under that exp_id;
• optionally dispatches a custom DOM event so any third-party listeners get the same signal they’d get from a real experiment.

Now both cohorts emit identically-shaped events under two experiment IDs — one per side of the split. The cohort comparison is then a union of the two tracker IDs, partitioned by which one fired:

SELECT exp_id, ...metrics...
FROM events
WHERE exp_id IN ('cart-text', 'cart-tracker')
GROUP BY exp_id

cart-tracker is your control column, cart-text is your treatment column, and the redirect experiment itself contributes nothing to analytics — it’s pure routing. Every metric you care about (views, clicks, downstream conversion) lines up across the two IDs because you deliberately wired the same tracking into both.

Don’t infer the cohort — ask GrowthBook

Targeting /cart is necessary but not sufficient, and this is the bit that actually decides whether you can safely launch the tracker at all. On a client-side redirect, the control URL is also where treatment users briefly sit — for the navigateDelay window — before the SDK moves them. A tracker that enrolled everyone it found on /cart would scoop up those in-transit treatment users and mis-file them as control. That’s the whole SRM problem from the next section, baked straight into your enrolment logic.

So don’t infer the cohort from the URL. Ask GrowthBook whether it has decided to redirect this user, with a page-condition gate:

condition: () => window._growthbook?.getRedirectUrl?.() === ""

getRedirectUrl() is a three-state oracle:

• a URL string → this user is bucketed into the redirect (treatment); the SDK is about to move them → don’t enrol;
• undefined / falsy → GrowthBook hasn’t finished evaluating yet; the coinflip could still land either way → don’t enrol;
• "" (empty string) → GrowthBook has evaluated and decided no redirect applies → a confirmed control user, staying put → enrol.

The strict === "" is the whole trick. It collapses three states into one safe answer: enrol only once GrowthBook itself has told you the coinflip is done and the result is “stay.” You’re not racing the redirect — you’re waiting for it to not happen. That’s how you know it’s safe to start: a redirect isn’t imminent, because GrowthBook just told you there won’t be one.

Fail-closed matters here. A loose check — getRedirectUrl() !== "/cart-new", or “any URL that isn’t the treatment path” — lets undefined through, so you’d enrol mid-settle and contaminate control. Strict equality against the empty string fails closed: anything other than a settled no-redirect verdict excludes the user.

The trap inside the trap: SRM

Here’s the part that bites after you’ve congratulated yourself on the fix. A client-side redirect is racy. The SDK does its window.location.replace() on a timer (navigateDelay), and your tracker’s enrolment check runs synchronously in the same browser tick — while the SDK may still be settling the redirect. Get the timing wrong and a treatment-bound user is briefly still on the control URL when your tracker looks, so they get counted as control. Multiply across enough traffic and you get sample ratio mismatch: your 50/50 split reports as 53/47, and now the whole experiment is suspect.

This isn’t a bug in your tracker; it’s inherent to redirecting on the client. GrowthBook’s own docs recommend implementing redirect tests at the edge or backend for exactly this reason — bucket before the origin responds, no race. If you’re on the plain JS SDK, you don’t have that luxury, so you mitigate:

• Gate on the decision, not the URL — the getRedirectUrl() === "" check above is the mitigation, not an afterthought. Enrolling on GrowthBook’s settled no-redirect verdict (rather than “I’m on a control-looking URL”) is what keeps in-transit treatment users out of the control bucket. Bias toward under-counting control over mixing treatment into it: a slightly thin control column is recoverable; a contaminated one is not.
• Give the redirect room. Bumping navigateDelay buys the redirect time to complete before anything else reads the URL.
• The real escape hatch: move bucketing to a Cloudflare Worker (or your edge of choice). The worker hashes before the origin sees the request and issues a 302 — no flicker, no race, and a bonus: edge bucketing fires trackingCallback server-side under a single unified exp_id, which collapses this whole paired-tracker pattern into one ordinary experiment. If a client has edge capability, use it and skip everything above.

One more edge case for the test plan

A user who lands directly on the new URL — a bookmark, a shared link, someone typing it in — never passes through the redirect, so GrowthBook never buckets them on the redirect experiment. They’ll still trip the URL-targeted treatment tracker (it fires on the URL, not on the bucket), so they show up as “treatment” in your event union but are absent from the redirect experiment’s own cohort count. In practice it’s noise — almost all traffic reaches the page through in-site navigation that goes through the redirect — but it’s the kind of thing you note up front rather than discover in the analysis.

And gate the tracker behind GrowthBook’s preview/QA targeting until you’ve confirmed both exp_ids are actually firing in your analytics, then drop the gate to go live. An A-A tracker that silently isn’t tracking is the same failure you started with, one level deeper.

---

URL Redirects are the right tool for testing a whole-page redesign. Just remember what kind of experiment you’ve actually created: an auto-experiment the SDK applies in the browser, invisible to the feature-evaluation path your tracking lives on. The treatment cohort gets observed by accident because it changed URL. The control cohort gets observed only if you decide to — and watch the SRM while you do.

Happy testing 🙂