Building the ultimate A/B testing developer experience

Six clients, six GrowthBook tenants, six different production sites. The third-party-tool era (Convert, Optimizely) had already taught me what a bad loop felt like — paste into the platform’s UI, save, refresh, hope. By the time we moved everything to GrowthBook, I had a rudimentary version of this Toolbox running: build locally, then a third-party script-injection extension would shim the dist file into the live site. Better than the UI-paste loop, but each save still meant manual build, browser switch, manual refresh. Across six clients in six markets, that’s a lot of context-switching tax.

So I built the loop I wanted. Three locally-hosted pieces pointing at live production sites: a Vite hub that builds per-client bundles, a Manifest V3 Chrome extension I wrote to replace the third-party injector, and a WebSocket reload loop that keeps every browser tab in sync with the editor.

The pieces

Vite hub. One repo, six client configurations. Each client gets its own bundle target, experiments folder, and tracking layer — but they share the recoveryArea framework underneath. A small CLI on top handles the unglamorous bits. npm run new:experiment walks an interactive prompt — client, experiment type, directory layout, registry updates, Prettier, git add — so I never start a test by copy-pasting boilerplate. npm run remove:experiment archives finished tests and cleans up the import graph. Neither of those sounds exciting; both save the same fifteen minutes of YAML-and-imports fiddling every single time.

RequestLite. A Manifest V3 Chrome extension that intercepts production script requests at the network layer. Two modes: redirect (production loads my localhost build instead of the real script) and inject (production doesn’t have a script at this URL yet, but here’s one from my dev server). I wrote it up properly in a separate post — go there for the network-layer detail and the chrome.declarativeNetRequest quirks.

WebSocket reload. A small server on port 5678 watches for file changes, rebuilds the affected client bundle, and tells RequestLite to refresh every matching tab across every browser instance. Save a file in WebStorm, every relevant tab refreshes. It’s hot module replacement, but for production websites you don’t own.

What a session looks like

npm run watch -- qlf

That spins up Vite in watch mode for QLF, a preview server on localhost, and the WebSocket reloader. I flip the QLF rule on in the RequestLite popup, navigate to the production site, and the page is now loading my local script. Edit the experiment, save, watch the tab refresh. Open a second Chrome profile signed in as a different user — both tabs refresh in lockstep.

The feedback loop went from “minutes per save” to “the tab is already showing the new version by the time I look up”. That’s the whole win, and the rest of the system exists to protect it.

Deployment is the boring part

When an experiment is ready, deployment isn’t a separate ceremony. Each client has a Cloudflare Pages project pointed at its own release branch. The flow is:

git push origin main
git checkout -b release-qlf
git push origin release-qlf

Pages builds and deploys to a per-client subdomain (growthbook-toolbox.qlf.com). No build server to babysit, no upload step to forget, no staging environment that almost-but-doesn’t match production. The same Git workflow used for everything else handles the production CDN.

---

Two principles underneath all of this. First, the feedback loop has to be measured in seconds, not minutes — minutes-per-save quietly kills the small bets and quick rollbacks that good experimentation depends on. Second, the boring stuff (scaffold, archive, deploy) has to be one command each, because the moment it requires thought you start cutting corners. Everything else in the system is downstream of those two requirements.

Happy testing 🙂

Update — 2026-05

Six clients still, but the toolbox itself moved on: semantic-architecture v2.8 with the QA Dev Panel sitting inside it (Force Variation banner, runtime profiler, dark/light tracer), a Dep Guardian weekly run that audits dependencies and flags risky updates, and three Jira MCP + three GrowthBook MCP servers feeding scaffolders straight from a Jira ticket. The three pieces above (Vite hub + RequestLite + WebSocket reload) are still the load-bearing core; everything since has been built on top of that loop, not replaced it.