A green test forgives what a rendered frame keeps#
A test suite and a video renderer run the same browser through the same flow. They part ways on one word: rerun. When a test flakes, the runner retries it, and a pass on the second attempt is still a pass. That safety net is load-bearing at scale. Google, measuring its own corpus, reported that about 1.5% of all test runs report a flaky result, that almost 16% of its tests carry some level of flakiness, and that about 84% of the transitions it sees from pass to fail turn out to be flaky rather than real regressions1. A suite survives those numbers because the retry hides them.
A render has no retry. The flake is not a red check you can run again to green; it is a frame in the file. A click that lands a beat early is a recorded misclick. A wait that resolves while a panel is still sliding is a recorded stutter. A spinner caught at a different angle is a different picture. The bar for a test is "eventually passes." The bar for footage is "produces the same pixels twice," and the second bar is strictly higher: every flaky behavior a suite would have quietly retried away is, on camera, a defect you ship.
That is why treating a demo as a committed spec you can diff and rebuild only pays off if the rebuild is deterministic. A spec that renders a slightly different video each run is not a spec; it is a random seed. The rest of this is the checklist that closes the gap between the two bars, variable by variable: what drifts between runs, what that drift looks like on screen, and how to pin it.
Auto-waiting beats a sleep, and on camera it beats it by more#
The oldest determinism bug is sleep(2000). A fixed pause is a bet that the thing you are waiting for finishes inside the guess, and the guess is wrong in both directions. Too short and the next action fires into a half-loaded page. Too long and you have bought dead frames of a screen doing nothing. In a test the first failure just retries; on video both failures are permanent. The misfire is recorded, and the dead air pads the runtime you then have to trim.
The fix is to wait on the condition, not the clock. Playwright's actionability model waits, before every action, until the target element is visible, stable, receiving events, and enabled, and it defines "stable" precisely: the element has held "the same bounding box for at least two consecutive animation frames"2. That last check is the one that matters for footage, because it refuses to click a button that is still animating into place. Playwright also ships auto-retrying assertions that poll a condition until it holds instead of failing on the first look. The one escape hatch to distrust is force, which disables the non-essential checks; it makes a test pass and a video stutter, because it clicks whether or not the element was ready.
Two conditions cover most demos: the element is stable and hittable, and the network the step depends on has gone quiet. Key the action to those, and the same flow paces itself identically whether the page painted in 80 ms or 800.
Freeze the motion: reduced-motion, pinned animations, a fake clock#
Motion is the richest source of per-run drift, because most of it is driven by the wall clock at the instant the frame is grabbed. A spinner's rotation, a card's fade-in, a skeleton shimmer, a toast that dismisses itself on a timer: each is at a different phase every run. On camera the same scene shows the spinner at a new angle, a modal caught 40% slid-in on one take and 70% on the next, a fade that is complete in one render and still going in another. Nothing is broken; nothing matches.
Three controls, coarse to fine:
- Emulate reduced motion. The
prefers-reduced-motionmedia feature signals that the user wants an interface that "removes, reduces, or replaces motion-based animations"6; a well-built app already tones its animations down under it. Playwright can emulate it directly withreducedMotion: 'reduce', so the app serves its calmer, less time-dependent motion for the capture4. That is free determinism for any motion the app agreed to drop. - Pin the clock. For motion the app keeps, take away its clock. Playwright's clock API, added in v1.45, replaces the page's
Date,setTimeout,setInterval,requestAnimationFrame, andperformancewith a fake one you control, so every timer- and animation-frame-driven effect advances by exactly the ticks you allow3. Install it before you navigate and set it just behind the intended time, the docs note, so page-load timers still fire and nothing hangs. A pinned clock is also what makes a "3 minutes ago" label read the same in every render. - Drive the motion you keep. For the one move you want on purpose, a reveal or a highlight, do not leave its duration to the browser. Compute it yourself over a fixed number of frames, so it is both smooth and identical every time.
Pin the frame geometry: viewport, pixel ratio, and fonts#
Geometry decides what the same DOM actually looks like. Three settings move it, and all three have to be nailed down before the first frame.
Viewport. Width drives layout. Capture the flow at 1280 wide in one run and 1360 in another and a responsive grid drops a column, a nav collapses to a hamburger, a tooltip flips sides: the same demo, a different composition. Set an explicit viewport (Playwright takes viewport: { width, height } on the context) and never let the window size float4.
Device-pixel-ratio. DPR is sharpness. A capture at deviceScaleFactor: 1 upscaled to a 1080p canvas is soft, and its text anti-aliases onto different subpixels than a native 2x render. Pin deviceScaleFactor (2 is the safe default for crisp text) and every run rasterizes to the same grid.
Fonts. A web font that arrives after the first paint gives you the flash of unstyled text: the fallback renders, then the real font swaps in and the line reflows. If the glyph is missing entirely you get a tofu box instead. The wait is one line. document.fonts.ready resolves once "the document has completed loading fonts, layout operations are completed, and no further font loads are needed"7, so block the first frame on it. Availability is a separate problem: a CI runner that lacks the font family paints the box no matter how long you wait, which is its own thing to provision on the runner.
Take the network out of the shot#
Network variance is the drift you cannot pin from inside the browser, so the move is to remove it. Three flavors, all corrosive to a repeatable render:
- Latency jitter. A request that returns in 120 ms one run and 900 ms the next changes when content paints and how long your idle wait sits. Even with condition-based waits, the pacing wobbles.
- Live data. A feed, a counter, a relative timestamp, a "trending now" rail: anything that reflects the real world films whatever the world was doing at capture time, so the numbers on screen change between runs.
- Variants and third parties. Production may serve an A/B branch, a feature flag, a cookie wall, a late-loading chat widget or an ad slot. Occasionally the render simply films the wrong UI.
The remedy is to record against a target you own: localhost, a seeded staging build, or a fixture with a pinned dataset, and to stub or mock the requests so responses come back instant and identical. This is the same reason the agent seeds state before it writes the spec: the demo should depend on data it controls, not on whatever production happened to be serving. Filming a real service and hoping is how a golden render quietly drifts out of true.
The determinism checklist, in one table#
Every variable above, its on-camera symptom, and its pin, in one place. This is the difference between a flow that turns a test green and one that renders the same footage twice.
| Variable | What drifts between runs | On camera | How to pin it |
|---|---|---|---|
| Waits | when a step is "ready" | early misclick, or dead air | actionability plus a network-quiet condition, never sleep(n) |
| CSS/JS animation | phase at capture | fade or slide caught mid-way | reducedMotion: 'reduce'; drive kept motion yourself |
| Timers and clock | Date, setTimeout, rAF |
spinner angle, "3 min ago" text | install a fake clock before navigating |
| Viewport | width to layout | dropped column, hamburger nav | an explicit viewport, never floating |
| Device-pixel-ratio | rasterization grid | soft text, different anti-aliasing | pin deviceScaleFactor (2 for crisp text) |
| Fonts | load timing and availability | reflow on swap, tofu box | await document.fonts.ready; ship the family |
| Network | latency, data, variants | pacing wobble, changed numbers | localhost or fixture plus stubbed responses |
| Scroll | smooth-scroll duration and offset | jumpy or subpixel-different scroll | an eased curve over fixed frames to an integer offset |
Scroll earns the last row on its own. A native scroll-behavior: smooth hands both the duration and the final resting offset to the browser, and both come back slightly different per run, which reads as a jitter no zoom can hide. Compute the scroll instead: an easing curve over a fixed frame count to a pinned integer offset, so the move looks hand-glided and repeats to the pixel.
None of this is exotic. It is a fixed viewport, a locked pixel ratio, waits keyed to conditions rather than a countdown, motion under a frozen clock, and a network you own. An engine built for this bakes the list in: aidemo, which is ours, drives a real Chrome through a fixed action-spec with an injected cursor, pins the viewport and pixel ratio, and waits on conditions, so one storyboard renders the same frames on every run. It works only in the browser, its storyboards come from an agent rather than a drag-and-drop timeline, and it has no click-to-trim editor, which are the same constraints that make the take reproducible. Get the list right and the demo stops being a performance you hope repeats and becomes something a CI job can rebuild on the commit that changed the UI.
Sources#
- Google Testing Blog — Flaky Tests at Google and How We Mitigate Them (1.5% of runs, 16% of tests, 84% of pass-to-fail transitions)
- Playwright — Auto-waiting and actionability checks (visible, stable, receives events, enabled)
- Playwright — Clock API (v1.45; mock Date, timers, requestAnimationFrame)
- Playwright — Emulation (viewport, deviceScaleFactor, reducedMotion, colorScheme)
- Playwright — Videos (recordVideo on newContext, viewport-sized, saved on context close)
- MDN — prefers-reduced-motion media feature
- MDN — FontFaceSet.ready (promise resolves when fonts and layout are complete)
FAQ#
Why does deterministic replay matter more for a video than for a test?#
A flaky test is recoverable: the runner reruns it and a later pass counts, so a suite tolerates the roughly 1.5% of runs that flake. A render has no rerun. Whatever the browser did on that pass is baked into the frames, so a mistimed click or a spinner caught mid-spin is not a retry, it is a defect in the deliverable. The practical consequence is a higher bar, not "eventually passes" but "produces identical footage twice."
Can Playwright record a demo video deterministically on its own?#
Playwright can record the session: the recordVideo option on newContext captures the page as it runs, sized to the viewport and written when the context closes5. But recording is not determinism. That recorder films whatever the run did, flakes included, so you still have to pin the waits, animations, clock, fonts, and network yourself, or two recordings of the same script come out different. The recorder captures determinism, it does not create it.
How do I keep a loading spinner from ruining a screen recording?#
A spinner is a timer-and-animation effect, so it lands at a different phase every capture. Emulate prefers-reduced-motion so the app serves its calmer state, or install a fake clock so the spinner and the request behind it advance by the same ticks each run. Then wait on the loaded condition rather than a fixed sleep, and trim the idle so the finished video never sits on the spinner at all.