The Untestable Game Won

Two Breakout builds that exposed a test API passed the Breakout driver.

Two others exposed no test API. The driver could not judge them.

Then Konstantin played all four. The strongest practical artifact came from the second group. The runner-up did too.

The instrumented builds won the machine verdict. The uninstrumented grafts won the human ranking.

That is what “untestable” means in this title: unavailable to one named driver in the artifact I tested. It does not mean beyond testing in principle. The missing API could be grafted in later. In this comparison, however, the machine had no mechanical verdict for either overlay, while the human verdict favored both.

The previous Breakout pass had already shown that a complete mechanical verdict could coexist with an incomplete player lifecycle; this pass changed the production route instead of extending that contract.

One model, four routes through the problem

This was not another model comparison. Every artifact came from Gemini Flash. The experiment crossed two decisions in a 2x2.

That produced four cells: scratch design, scratch presentation, overlay design, and overlay presentation. Same model, same game, two ways to build, two kinds of improvement.

The point was not to crown a model. It was to ask which process preserved the strongest human-facing base.

The objective layer

At the generic floor, every artifact ran, rendered, and produced no console or page errors.

Then the paths separated. Both scratch artifacts voluntarily exposed <code>window.gameTestAPI</code>. The Breakout driver could inspect their state, step them deterministically, and exercise the named mechanical contract. Both returned PASS.

The overlay artifacts exposed no such API. The driver did the honest thing and declined to issue a mechanical verdict.

The overlay driver was skipped because the test API was absent.

The ledger contains two different kinds of evidence. API and driver status are objective observations. The human notes are Konstantin's play assessment. Combining them into one leaderboard would erase the most important fact in the experiment: the two verdict systems covered different surfaces.

The scratch cells were mechanically legible to the driver. The overlay cells were not. That gives the scratch cells a stronger mechanical evidence record. It does not tell us that they were better games.

The human layer

The overlay design artifact was the strongest practical build. The overlay presentation artifact was runner-up.

The scratch cells did not fail mechanically. They missed in ways the mechanical contract did not describe.

Scratch design had an interesting idea: exploding bricks could trigger larger clears. But there were too many of them. The mechanic flattened the difficulty instead of sharpening it. The game became too easy.

Scratch presentation was pleasant and coherent enough, but it was not distinctive. Flash returned to a familiar neon arcade prior. Nothing was obviously broken. Nothing made it the artifact to keep.

The overlays had their own visible flaw. Both retained the <code>press Space</code> prompt during active play. The instruction remained on screen after the state it described had passed.

That bug matters. It also makes the ranking more useful. Human review did not select a polished, defect-free winner. It selected the strongest base while seeing a concrete flaw that the generic gate had missed.

The machine gates established run/render health for all four artifacts and mechanical replay consistency for the two scratch cells. They did not establish difficulty balance. They did not notice a stale prompt. They did not decide which artifact had the stronger play feel.

Those were not failed checks. They were absent checks and a human preference.

What the graft preserved

The process difference was simple.

Scratch recreation asked the model to rebuild the improvement and the surrounding game together. The new artifact had to recover the base play feel, the presentation, the interaction rhythm, and the proposed change in one generation.

The overlay route began with a live artifact produced by an open creative prompt. It preserved that seed, then asked for one focused layer: design or presentation.

In this four-artifact slice, the focused graft preserved a stronger human-facing base. The design overlay produced the practical winner. The presentation overlay produced the runner-up. Both scratch artifacts were more instrumented, but neither displaced them in human review.

This is an observed process result, not a causal proof. Four artifacts cannot tell us exactly which feature of the route produced the difference. The seed may have carried useful visual and UX coherence. The narrower instruction may have reduced reconstruction loss. The particular prompts or generation may simply have landed better.

What the evidence supports is smaller: here, open seed -> focused graft produced the stronger practical artifacts than scratch recreation.

An early replication, not a law

Fun Is Not a Spec had already suggested the same working pattern: preserve a lively seed, filter the change for coherence, and graft one focused improvement without erasing what made the seed worth keeping.

The Breakout experiment now provides an early replication of that process result. It is still one model and four artifacts. It does not establish that overlays are universally better. It does not establish mechanical superiority, because the overlay cells were not available to the Breakout driver. It does not establish a stable Flash personality or a model leaderboard.

And the word “won” remains scoped to Konstantin's human assessment of this slice: overlay design was the practical winner; overlay presentation was runner-up. No numerical ranks were recorded for the scratch cells.

The next useful move is not to declare grafting solved. It is to preserve the accepted feel, add the missing test API without redesigning the artifact, fix the stale Space prompt, and see whether the human winner can become mechanically legible without losing what won.

For now, the result is deliberately uncomfortable:

The machine could prove more about the scratch builds. The human still chose the grafts.