Engagement Report: Re-Verifying a Silent Diff-Engine Failure

Fourth sample engagement report, on Cyclops, a Playwright visual-regression and AI-polish loop. This one has a different shape than the previous three: there's no automated test suite in this repo to run, and no new bug surfaced tonight. The finding here is the re-verification itself, confirming a previously claimed fix still holds on today's dependency versions, which is a real and separate kind of engagement work from finding something new.

Scope and Objective

Target: src/differ.js, the component that compares a before/after screenshot pair and reports the mismatched pixel count, built on the pixelmatch library.

Objective: a prior pass on this repo documented a real bug: pixelmatch@7 shipped ESM-only, which broke every diff run under plain CommonJS require(), silently, because Node's interop returns a wrapper object instead of throwing. The claimed fix was accessing .default on the required module. Confirm that fix is still correct and still necessary on the dependency versions actually installed right now, not the versions in place when it was first found.

Finding: The Original Failure Mode, Reproduced

Calling require("pixelmatch") directly in the currently installed [email protected] returns a module object whose real function lives on .default, not the object itself. Calling the returned object as a function throws immediately. This confirms the original finding is not stale: the ESM-only export shape that broke diff runs then is the same shape installed today. The fix in the codebase, require("pixelmatch").default, is still the correct and necessary one, and src/differ.js carries it with an inline comment explaining why, so the next person who upgrades this dependency doesn't rediscover the same failure from scratch.

Verification: A Real Diff, Run Tonight

Rather than trust the fix from reading the import line alone, two real 100×100 PNGs were generated tonight, identical except for one deliberately altered 20×20 block, and run through differ.js's actual exported diffScreen() function, not a mock of it:

{
  "name": "home",
  "mismatch": 361,
  "skipped": false
}

361 mismatched pixels against a 400-pixel altered region is the expected result, not a round number: pixelmatch's default anti-aliasing threshold excludes some edge pixels from the count by design, so a smaller-than-exact mismatch count on a hard-edged synthetic test image is correct behavior, not a sign of under-detection. The diff engine ran, read both files, and produced a real, explicable number. The engine works on the versions actually installed.

Secondary Check: The Readline Race

The same prior pass documented a second fix: three CLI entry points had a readline race, the callback-based readline module's interface could receive input before the prompt text had finished writing, corrupting interactive setup flows. Checked tonight: src/onboard.js now imports readline/promises, the promise-based API introduced specifically to remove this class of race by making the prompt a single awaited call instead of a callback racing against stdout. The fix is in place and is the structurally correct one, not a workaround.

Mitigations and Next Steps

1. The real gap here isn't a bug, it's test coverage. Both fixes verified above were checked by hand, tonight, because no automated test exists for either. A single test that imports pixelmatch and asserts the result is callable would have caught the original ESM break in CI the moment the dependency was bumped, instead of silently breaking every diff run until someone noticed by hand.
2. Pin and re-verify on every dependency bump, same lesson as the image_payload_injection report: a fix that is correct today depends on an upstream package's export shape that can change again on the next major version. The inline comment explaining why the .default access exists is good practice and should be the template for the rest of this codebase's dependency-shape assumptions.
3. Add one regression test covering exactly the failure mode found here: require pixelmatch, assert it's callable, so this specific class of silent breakage gets a CI failure instead of a manual re-check.

---

GhostInThePrompt.com // A fix that held once still has to hold on the versions installed today. This one did.