Every failure in this one was silent

The working environment learned to make things that move. Getting there took six diagnostic rounds, and every single failure reported success — a video that renders one frame ninety times, a still that captures the moment before the animation starts, a build that passes locally and fails in CI. None of them threw. Here is what each one changed.

A silent failure is worse than a crash by roughly the cost of finding it. A crash names a file and a line. A render that produces ninety identical frames produces a valid mp4, a plausible file size, and a success message — and the only way to know is to watch it. Most of the work described below is not the feature. It is moving each of these failures to the earliest point where something could still be done about it.

First: a fourth way in

The working environment exposes host libraries to agent scripts through three routes, picked by the shape of the library: defineAdapter for anything doing I/O, defineBuilder for stateful builder APIs, and definePureModule for synchronous computation.

All three assume a library. There was no route for a capability — an MCP server, a model, an HTTP API — and the difference turns out to be economic rather than aesthetic.

Consider checking a forty-page PDF for visual defects. As a verb, view_image costs one tool call per page and every answer lands in the context window: forty round trips, and a conversation buried under forty paragraphs about page margins. As a function a script can call, it is a loop:

import { rasterize } from 'env:render';
import { look } from 'env:vision';

const { pages } = await rasterize('/out/report.pdf', '/tmp/pages');
const bad = [];
for (const p of pages) {
  const answer = await look({ path: p.path, prompt: 'Is any text cut off at the page edge?' });
  if (/yes/i.test(answer)) bad.push(p.page);
}
return bad.length ? `clipped text on pages ${bad.join(', ')}` : 'all pages clean';

Forty answers land in a variable. One line comes back. defineTools is the fourth route, and it takes the same ToolFn catalog glove-scratchpad already used — so fnsFromMcp(conn) produces the list directly, and an MCP server becomes an importable module with no adapter written at all.

Writing a file is not delivering it

By the end of a real task, /out holds the report, the superseded draft of the report, and the intermediate workbook that fed it. Only the agent knows which was the answer. Every host was left guessing from filenames and timestamps.

present is the explicit hand-off — one call per finished artifact, with a caption, refused for any path outside /out. It only exists when the host wires onPresent; without that the verb is absent from the tool set entirely, which is the rule the environment already applied to view_image. An agent is never shown a capability that would fail on use.

There is a tail to this that only appeared later. present reports a media type, and the media type is what a host uses to decide between a player and a download prompt. It knew about PDFs, decks, workbooks and images. When the environment learned to render video, an mp4 arrived as application/octet-stream — an opaque blob, delivered successfully.

Time has to be replaced, not measured

The environment could already produce a PDF, a deck, a workbook and a resized image. It could not produce anything that moves. The reason was never the encoder — ffmpeg has been in env:media for a while. It was that nothing could draw a frame.

A browser can draw frames, and that is the whole problem. A browser animation is a function of wall-clock time. Screenshot the same scene twice and you get two different pictures; a renderer that fell behind by 4ms emits a frame from the wrong moment. For video, where frame N must be exactly N, neither is acceptable.

So env:motion does not measure time. It replaces it. Before any scene code runs, requestAnimationFrame becomes a queue nobody drains except the renderer, and performance.now() and Date.now() return a number the renderer sets. One advance is one frame.

Measured: two independent runs of the same 60-frame scene produce byte-identical PNGs for every frame. That is what makes a re-render after an edit a real diff, and it is the property everything else is built on.

Five ways a scene renders frame 1 forever

The goal was React Native Reanimated scenes rendering unchanged — real motion code, not a reimplementation. Five things stood in the way. Each one produces the identical symptom: the first frame renders, nothing moves, no error, nothing to grep for.

That last one cost the most and is worth stating plainly, because it is not in any of the obvious places: in Playwright, addInitScript runs on navigation, not on setContent. The fix is to write the page to disk and reach it with goto(pathToFileURL(...)). It was confirmed by probing directly — the globals are absent after setContent and present after goto — rather than by reasoning about it, which is the only way to settle a question like that.

Deleting the switch that was always set wrong

Scenes come in two shapes. A useFrame() scene is a pure function of the frame number. A Reanimated scene is driven by its own clock. The first design asked the caller which one they had.

Picking wrong produced a valid video of a still image — the sixth silent failure, and this one was self-inflicted. The fix was to stop asking: the renderer advances both signals on every frame, and each signal is inert for the other kind of scene. Any scene animates with no configuration, and the two stay consistent by construction, because frame f is always t = f/fps.

// Both signals, every frame. A clock-driven scene ignores setFrame;
// a frame-driven scene ignores the clock. Neither has to be declared —
// "auto" is the default, and mode survives only as an override.
const drive = async (f: number) => {
  if (options.mode !== "clock") await setFrame(f);
  if (options.mode !== "frame") await advance(f === 0 ? 0 : step);
  await settle();
};

Removing the switch also fixed a bug nobody had reported yet. Stills were implemented as “render frame N”, which for a clock-driven scene meant screenshotting before the animation had moved — so a Reanimated still always captured the initial state. Now a frame-driven scene is jumped straight to the requested frame and a clock-driven one is walked there without intermediate screenshots. Spot-checking frame 90 is cheap either way, and correct either way.

Refuse early, and name the fix

Every frame is a browser screenshot — about 330ms. A ten-second clip at 30fps is 300 frames. The environment's default script budget is 30 seconds, so the first honest render died at the timeout with a generic message, four minutes of work discarded.

Raising the default would be wrong: a 30-second ceiling is right for a script that reads a spreadsheet. Instead the adapter reads the environment's own runTimeoutMs, estimates the render before starting it, and refuses up front with the exact line to add:

env:motion.render: a 300-frame render needs roughly 119s — a browser launch,
then a screenshot per frame — but this environment's script budget
(limits.runTimeoutMs) is 30s, so it would be killed mid-render. Create the
environment with limits: { runTimeoutMs: 180000 } (this package exports
MOTION_LIMITS as a good default), or render fewer frames.

The same principle produced glove-motion-doctor. Whether a host can render at all depends on a browser it may not have, and discovering that by burning a render is the expensive way to find out. One diagnosis is published on three surfaces: a CLI for the developer with a fix command per failing row, capabilities() for the agent at runtime, and an “on this host” section in the generated /std/motion/README.md the agent reads before it writes anything.

$ pnpm exec glove-motion-doctor
✓ browser     /opt/pw-browsers/chromium-1194/chrome-linux/chrome
✓ ffmpeg      bundled with the package (…/@ffmpeg-installer/linux-x64/ffmpeg)
✓ react       bundled with glove-env-motion — no install needed
✓ reanimated  installed with react-native-web and the worklets plugin — React Native motion code renders here

ready — env:motion can render on this host

The check that only passed on Linux

Browser and ffmpeg discovery was written on Linux, for Linux — /usr/bin, a snap path, an ELF binary. It would have failed on every developer machine running macOS.

The fix is ordinary — check /Applications on macOS, Program Files and LOCALAPPDATA on Windows — but the testable version is not. Discovery is parameterized by platform rather than reading process.platform internally, so Linux CI can assert that the macOS candidate list contains the right paths. A cross-platform code path that can only be exercised on the platform it was written for is not covered; it is just untested in a way nobody notices.

One gateway, every surface

Separately: hosts wanted to hand an agent reference material — a corpus, a real source tree — that it can read and grep but never edit.

The implementation question was where to enforce it, and the answer had already been decided by an earlier design choice. Every mutation in the environment goes through one function, assertMutable. Verbs go through it. Scripts using env:fs go through it. Stdlib adapter handles go through it. So readOnlyPaths is one check in one place, and it binds all of them at once — including undo, which would otherwise have been a side door.

const env = await createWorkingEnvironment({
  filesystem: hostDirectory("./project"),
  readOnlyPaths: ["/src"],          // read and grep the source; write only elsewhere
});

await env.mount("./handbook.pdf", "/corpus/handbook.pdf");   // the host door stays open

Two deliberate asymmetries. env.mount() bypasses the check, because seeding content the agent can only read is the entire point of the option — while env.fs, the guarded host handle, obeys exactly the same rules as the model. And the zones are announced in the orientation file the agent reads at startup, so the boundary is learned by reading rather than by being refused. A refusal still carries the fix (cp it to /tmp and work on the copy), but it should be the second way an agent finds out, not the first.

The example is where the wiring gets tested

examples/document-desk is an agent with a real working environment: upload documents, ask for something, watch the code it writes. It now mounts env:motion alongside the six document adapters — which immediately surfaced three things that were invisible while motion lived only in its own test suite.

The media type gap above, found because a rendered mp4 arrived as an opaque blob. The budget, because the desk's 60-second script ceiling was generous for a PDF and far too small for a render — so every render was refused, correctly and uselessly, until the ceiling became MOTION_LIMITS. And the obvious one: a video you cannot watch is not a deliverable. A presented video or image now plays inline in the transcript, and the file explorer previews both instead of offering a download.

There is also a check that needs no model and no API key, because a render is the one capability here that depends on something outside the repo:

pnpm --filter glove-document-desk check:motion
# doctor rows, then a real mp4 on disk — 90 frames at 960x540 in 14.2s

Driven by an actual model, the round trip is more convincing than the numbers. Asked for an animated revenue counter, the agent wrote the scene, rendered it, then wrote a second script to render frame 60 and confirm the count had reached the right figure — checking its own work, unprompted, because the environment's skills tell it to.

A black rectangle with working controls

One more, found while screenshotting that UI. The player showed a black rectangle. The controls worked. The duration was correct. No error anywhere.

The mp4 was fine — ffprobe confirmed a valid H.264 stream. The browser could not decode it: Chromium builds without proprietary codecs, including the one playwright-core install puts on disk, have no H.264 decoder. canPlayType('video/mp4; codecs="avc1.42E01E"') returns an empty string there and "probably" for VP9.

Chrome, Edge, Safari and Firefox all play it, so mp4 remains the right default for anything a person opens — but if you preview a render in a bare Chromium and get a black box, the file is not the problem. Render .webm there. It is now written down in three places, because the next person to hit it will have no reason to suspect the browser.

The failure that belonged to nobody

The last one was not in any package. A site build passed locally and failed on Vercel, on a monorepo install that had been narrowed to the four projects the site actually needs.

The site had never declared @types/node. It had been free-riding on a hoisted copy that some other workspace package pulled in — which is invisible while the whole monorepo installs together, and fatal the moment the install is scoped. The dependency was real, used on every build, and simply not written down.

Worth generalising: an undeclared dependency is not a latent bug, it is a bug that happens to be masked. The masking is provided by an unrelated package's dependency list, which nobody has promised to keep stable.

The pattern

Reading these back, the fixes are all the same fix applied at different distances from the user.

None of that makes the software do more. It moves each diagnosis to the earliest point where it is still actionable, and puts the fix in the message. For a system whose primary user is a model that cannot ask a colleague what went wrong, that is not polish. It is most of the interface.

env:motion · present · the four authoring routes