"Binary files differ" is all a pull request tells you about a video#
Open a pull request that changes a committed demo video and the diff has nothing to say. Git stores the MP4 as an opaque blob, and where a code change would show red and green lines, a re-encoded video shows one line: Binary files a/demo.mp4 and b/demo.mp4 differ. That message is not a GitHub quirk. It is git's defined behavior for any path whose diff attribute is unset, the default for anything git detects as binary3. The reviewer learns that four bytes changed, or four million did, with no way to tell which and no way to see what.
That is the whole problem in one line. Code review works because a reviewer can read every line of a change, understand it, and rule on whether it improves the codebase; Google's own review guidance is blunt that if a reviewer "can't understand the code, it's very likely that other developers won't either"6. A binary that renders as "files differ" fails that test before it starts. So a media PR gets approved the way media PRs get approved today: by trusting the author, or by leaving the tab to watch the file somewhere else. Neither is review.
The fix is not a smarter diff tool for MP4s. It is to stop asking the pull request to review the pixels at all, and to hand it three things it can read: an image where an image will do, a rendered preview parked beside the diff, and above all the text that produced the video. Treating the demo as a committed spec is what makes that last one possible.
What a pull request will render, and what it won't#
GitHub is not helpless with binaries; it is selective. For images it ships a real visual diff with three modes, and for one common format it does nothing at all. Knowing where the line falls tells you which artifact to put in the diff.
| File in the diff | What the PR shows | Diff modes available | Reviewer verdict |
|---|---|---|---|
Committed .mp4 / .mov |
Binary files differ, no preview |
none | can't review it |
Committed .gif |
renders inline as an animation | 2-up, swipe, onion skin (per frame) | reviewable as a short loop |
Committed .png / .jpg still |
renders inline | 2-up, swipe, onion skin | fully reviewable, one state |
Committed .svg |
renders as an image and as XML text | image modes plus a line diff | reviewable two ways |
Storyboard .json / .tape (spec) |
a line-by-line text diff | text | the primary review surface |
The image modes are worth naming, because they are the reason a still beats a clip for review. GitHub renders PNG, JPG, GIF, PSD, and SVG, and lets you "visually compare images in three different modes: 2-up, swipe, and onion skin"1. 2-up is the default and shows both versions with any dimension change called out; swipe drags a slider across the two so you compare the same pixels; onion skin fades one into the other on an opacity slider, which "comes in handy when elements move around by small, hard to notice amounts." PSD renders but will not diff, and SVG diffs twice over, as an image and as the XML it really is. Video appears nowhere on that list: a committed .mp4 gets no preview, no modes, nothing.
So the review-surface ranking sorts itself. A frozen key state committed as a PNG is fully reviewable, in three modes. A short loop as a GIF is reviewable as motion, though swipe and onion skin only help on a held frame. A committed video is not reviewable in the diff at all. And the storyboard that produced the video diffs as ordinary text, line by line, which is the surface a reviewer can rule on most precisely of the lot.
Put the render where the reviewer already is#
A reviewer should not have to check out the branch and run a render to see what changed. GitHub already hosts video: drop an .mp4, .mov, or .webm into a pull request description or comment and it uploads the file and embeds a player inline, no external host needed2. The catch is a size ceiling that doubles as a length discipline: 10 MB per video on a free plan, 100 MB on a paid one, with H.264 recommended because "video codec compatibility is browser specific." A 60-second 1080p UI capture lands near 40 MB, so the free-tier 10 MB cap is really a nudge toward the short, low-motion clip a reviewer will actually finish.
Attaching the preview by hand is fine for a one-off. The durable version is a CI job that renders the demo on the pull request and posts the result back as a comment, so every push refreshes the clip next to the diff. That is the same render-in-CI pipeline that rebuilds a demo when the product changes, aimed at the PR instead of the release. The productized form is a visual-review service: Chromatic, for one, runs on the PR build, generates "a changeset highlighting the exact visual differences between your feature branch and the main branch," lets you assign reviewers who approve or deny that changeset, and posts a UI Review status check you can require in branch protection so an unreviewed visual change cannot merge4. Hand-dropped clip or bot-driven changeset, the move is identical: bring the rendered result to the reviewer's tab.
Review the spec, not the pixels#
The preview lets a reviewer see the change. The spec lets them understand it, which is the part that counts. When the demo is a storyboard rather than a captured take, the meaningful change is a text diff: a renamed selector, an edited narration line, a reordered scene. That reads in the pull request with no special tooling, and it carries the one thing the pixels never will, which is why the video changed.
The terminal world has run this way for years. vhs writes a terminal demo as a .tape file of human-readable commands (Type, Enter, Sleep, Wait /regex/) and renders the GIF from it; the reviewable artifact is the tape, and for a regression check the project's advice is to emit .txt or .ascii output and commit those text files so a later run's diff surfaces any change5. The GIF is build output; the tape is the source you review. A browser demo is the same shape against a richer target.
Git even sanctions the general trick. A binary path can be handed a textconv program that converts it to text purely for diffing, so a .jpg can diff as its EXIF fields and a PostScript file as its ASCII rendering instead of as a blob3. The lesson outruns any single textconv: the reviewable form of a media file is a text projection of it, and for a demo that projection already exists as the storyboard. Reviewing the spec is not a workaround for the binary diff; it is the diff, moved to the layer a human can read. It is also why the finished render does not belong in the commit at all.
The gate hands the reviewer a verdict, not a video#
Seeing the change and reading the spec still leaves one question a person should not answer by squinting: did the flow itself break? That is a machine's job, and it belongs in a merge gate rather than in the reviewer's eyes. The gate's mechanics, a committed golden, a CI re-render, a diff that blocks the merge, are their own discipline, and the pixel comparison inside it is its own tool choice; the review-relevant point is narrow. A good gate reduces a media change to a verdict the reviewer can trust: the flow still resolves, or it does not.
Our engine, aidemo (ours; the limits are real, in that it records inside a browser only, its storyboards are written by an agent as text, and it ships no drag-and-drop timeline), emits exactly that verdict. A probe --golden run replays the storyboard's actions against the current build with no voice and no encode, then exits non-zero on the first step whose selector stops resolving and prints a field-level line a reviewer reads at a glance. Because that check answers "did it break," the human is freed for the questions only a human can settle: is the frame readable, is the narration still true, is this change intended. The gate and the reviewer split the labor; neither does the other's job.
One rule keeps the split honest, and it is a review rule, not a machine one: whoever changes a demo blesses its new baseline in the very same pull request, so no green check ever certifies an update that nobody looked at.
A checklist for approving a media change#
Google's reviewers look for design, functionality, tests, naming, and comments, and above all for a change they can actually understand6. A media PR earns the same rigor, pointed at different artifacts. Here is what a reviewer actually approves, and where each answer lives.
| What the reviewer decides | Where the answer lives | The red flag |
|---|---|---|
| Was this change intended? | the storyboard text diff | a selector or URL changed with no reason in the PR |
| Did the flow still resolve? | the golden / probe check | a step flips from resolved to not-found |
| Does the frame still read? | the attached preview or a still diff | tiny text, a spinner, or an empty state in shot |
| Is the narration still true? | the narration lines in the diff | the script names a control the UI no longer has |
| Is this the source or the output? | the changed-files list | a committed MP4 where a spec should be |
| Was the baseline blessed on purpose? | the golden update in this same PR | a baseline bumped in a separate or automated commit |
The checklist has a shape worth saying outright. Every row a human should judge points at text or a still, never at the committed video, because the video is the one artifact the pull request cannot show. Make the change land as a spec diff, plus a rendered preview, plus a green flow-check, and approving a demo stops being an act of faith and becomes what code review already is: reading the change, understanding it, and blessing it on purpose.
Sources#
- GitHub Docs — Working with non-code files (image diff modes: 2-up, swipe, onion skin; supported formats)
- GitHub Docs — Attaching files (video .mp4/.mov/.webm rendered inline; 10 MB free / 100 MB paid; H.264)
- Git — gitattributes ("Binary files differ", the -diff attribute, and textconv to diff a binary as text)
- Chromatic — UI Review (changeset, assign reviewers, approve/deny, required UI Review status check)
- Charmbracelet vhs — tape files as source and .txt/.ascii golden output committed to git
- Google Engineering Practices — What to look for in a code review (review every line; you must understand the change)
FAQ#
Can you preview a video in a GitHub pull request?#
Not from the file diff itself: a committed video shows only as "Binary files differ," with no player. But you can drop an .mp4, .mov, or .webm into the pull request description or a comment, and GitHub uploads it and embeds a playable preview inline, capped at 10 MB on a free plan and 100 MB on a paid one. The reliable pattern is a CI job that renders the demo on each push and posts it back as a comment, so the preview always matches the current branch.
Does GitHub show image differences in pull requests?#
Yes for images, though not for video. GitHub renders PNG, JPG, GIF, PSD, and SVG and offers three comparison modes: 2-up (both versions side by side), swipe (a slider you drag across the two), and onion skin (an opacity fade that exposes small shifts). PSD renders but will not diff, and an SVG diffs both as an image and as its XML source. That is why freezing a key screen as a committed PNG makes a demo's state reviewable in a way the MP4 never is.
How do you review a change to a binary file in git?#
You review its text projection, not its bytes. Git can attach a textconv program that converts a binary to text purely for diffing (EXIF for a JPG, ASCII for PostScript), and the same principle drives demos-as-code: review the storyboard that renders the video, since the render is build output. For a genuinely opaque asset with no source, attach a before/after preview to the PR and gate the merge on an automated visual check.
