From Full Reading to Verifiable Film: A Reversible Production Workflow

Why a workflow is necessary

Pairing a few passages with images is easy. The real difficulty appears when the work becomes long: a character state may drift between passages, the narration may enter a new sentence while the image still represents the previous one, a local revision may disturb the entire timeline, and a successful render may conceal black edges in motion, audible joins, or a narrative misreading.

The 5:49 adaptation of Moonlight over the Lotus Pond gave us a complete test case. For the film and its literary interpretation, read the companion essay on the finished work, or watch the public film on Bilibili. This essay addresses a different question: how can long-form text become an audiovisual system that can be built in stages, rolled back locally, and verified repeatedly?

Four design goals

The workflow was derived backwards from four goals:

1. Textual evidence comes first. Every shot must identify the passage it serves. An attractive image cannot cover a gap in interpretation.
2. Real time is the organizing axis. Shots, subtitles, actions, and sounds follow the final narration rather than estimated character counts or a preset rhythm.
3. Changes remain local. A gust of wind, one transition, or a narration join can be compared and withdrawn without forcing the entire film to be regenerated.
4. Validation duties remain separate. Machines check measurable omissions, wrong frames, black edges, and timing; people judge naturalness, rhythm, meaning, and whether an effect deserves to remain.

Together they imply one central choice: store the work as sources, states, timing, and rules, then assemble the film with deterministic tools. The finished film is one output of that system, not the only master that can still be edited.

A seven-step production workflow

1. Begin with a textual responsibility table

The first production artifact is not a storyboard. It is a textual responsibility table. We divide the full text into semantic passages and record the location, time, discourse layer, required objects, character state, continuity with neighboring passages, and prohibited misreadings for each one.

Reality, rhetoric, and cultural memory must be separated here. The Tsinghua night walk belongs to physical space. “Moonlight like flowing water” changes how light is perceived; it does not require a literal river. Jiangnan lotus-picking is a cultural association and cannot quietly become an event that occurred that night. Once these boundaries are registered, image generation receives a stable area of responsibility.

2. Let the final narration establish the timeline

After the main reading is complete, shot lengths are no longer estimated from character counts. We read the actual onsets, pauses, passage boundaries, and total duration from the audio. Subtitles, key states, sound entries, and transitions all attach to that timeline.

Automatic speech recognition (ASR) helps locate when words actually occur. Pulse-code modulation (PCM) waveform analysis helps reveal loudness jumps, clipping, and abnormal silence. Both are locating tools: they can tell us where to listen again, but they cannot judge whether a reading has literary restraint.

3. Express the visual specification as state constraints

Continuity across a long film cannot depend on adding “keep everything consistent” to every prompt. Identity, clothing, period character, weather, light direction, palette, spatial relations, and neighboring states need stable records. Each keyframe then declares what it is allowed to change.

The production unit becomes a state rather than an image. One lotus pond can have thinner moonlight, leaves disturbed by wind, or denser mist. Each revision touches only the variables inside its responsibility. Generated images still make mistakes, but those mistakes become easier to locate and reverse.

4. Divide labor among keyframes, sound, and silence

Keyframes establish locations, relationships, and indispensable visible states. Narration carries sentences and tone. Environmental sound extends space beyond the frame. Silence and held images give viewers time to read.

This division deliberately reduces the number of shots. Laughter beyond the wall does not require a new child on screen. A distant cicada can enlarge a still night scene. A figure of speech does not have to trigger an immediate literal image. One less generated shot often removes several opportunities for identity drift, weather jumps, and purposeless transitions.

5. Assemble a deterministic data timeline

Remotion is a framework that renders video from React and data-driven timelines. Here it combines keyframes, state changes, pans, zooms, dissolves, and subtitles. FFmpeg handles audio assembly, encoding, and media inspection. With unchanged sources, timings, and parameters, the same structure can be produced again.

Determinism does not require every encoding run to produce identical bytes. It requires the more important relationships to remain reproducible: the same sentence reaches the same states, the same sound enters at the same time, and a repair cannot drift because someone dragged a clip by hand.

6. Test local samples before returning them to the full film

High-risk passages are compared first as samples lasting several seconds. A candidate can be retained, replaced, or discarded altogether. It enters the full timeline only after it works with the real narration and its surrounding context.

Local samples form a short feedback loop; the complete film forms a long one. The short loop resolves movement, composition, and sound details. The long loop checks spatial continuity, accumulated fatigue, and whether the return structure still closes. Separating the loops prevents a polished local effect from masquerading as a successful film.

7. Let machines filter first and people judge aesthetics

Machine checks cover file and data contracts, subtitle completeness, the timing of key states, black frames and edges, audio continuity, media specifications, and reproducible builds. Human viewing focuses on whether movement feels natural, whether imagery competes with the prose, whether pauses are sufficient, and whether an effect deserves to exist.

A machine pass only means that predefined faults were not found. A human impression that the film “flows” cannot replace evidence about sources and timing. Keeping both judgments side by side prevents technical closure from being reported as an aesthetic conclusion.

Representative case: state propagation in “a passing breeze”

The sentence about “a passing breeze” requires visible change without looking like a switch. In the first version, the whole field of leaves changed together. The information was clear, but the movement felt mechanical.

The revision divided one gust into twenty-four propagation states and six recovery states. A small tremor begins nearby. Only when the narration reaches “in an instant” does it travel rightward and into the distance. After arriving, it gradually settles. Each state changes a limited region; direction, speed differences, and recovery appear only in continuous playback.

Figure 1 | Six sampled states show local tremor, outward propagation, and gradual recovery. The film uses a denser sequence. AI supplied the base keyframe; propagation and assembly were controlled by the local deterministic workflow.

The reusable lesson is not simply to make more images. Continuous action can be described as constrained state propagation: define its origin, direction, covered region, arrival time, and recovery condition, then select enough intermediate states. Machines check those conditions; people decide whether the result resembles wind.

How three failures became design rules

Narration joins: good segments do not guarantee a continuous reading

Individually clear narration segments can still jump in breathing, timbre, pace, loudness, or noise floor after assembly. The workflow therefore treats every neighboring boundary as an explicit check: ASR and PCM analysis locate candidates, then a person listens through each join. Once the voice is continuous, extra visual transitions are no longer needed to conceal seams.

Black edges during pans: correct still files do not guarantee valid motion

Six panning shots once exposed black canvas at their edges. The source images existed, their dimensions were correct, and the render completed. The error appeared only when image size, displacement, and frame boundaries changed together over time. The repair slightly enlarged the shots, restricted the pan range, and added frame-by-frame edge scanning.

The rule applies to every motion design: validation must include time. Inspecting only input files and final encoding status is insufficient.

Cover redesign: production truth and entry copy need separate layers

The first cover behaved like a film poster. Its atmosphere was coherent, but at thumbnail size it could not quickly communicate “classic school text, full reading, AI production.” The public cover enlarged the title and used direct functional language so an unfamiliar viewer could decide whether to open it. The essay then explains the tools, labor, and boundaries.

Titles and covers serve entry decisions; articles and production records serve accurate explanation. With these duties separated, acquisition copy does not have to absorb every technical fact, and a thumbnail does not have to carry the full production argument.

“Zero cost” in the public title means only that this production did not purchase actors, a studio, or a generative-video service for finished shots. Anyone reproducing the workflow may still incur subscription, computing, and time costs. “AI” likewise identifies only part of the image source; it does not summarize narration, sound, assembly, validation, or human judgment.

Tools enter the system by responsibility

Stage	Tools actually used	Responsibility in the workflow
Textual interpretation and production orchestration	OpenAI/Codex	Organize textual and historical constraints and establish responsibility tables, visual specifications, and validation tasks
Keyframe generation	OpenAI image generation	Generate and locally correct key states under the visual specification
Reading and supporting voices	Volcengine Doubao TTS 2.0 (seed-tts-2.0)	Produce the complete main reading and a small amount of controlled supporting voice
Audiovisual assembly	Remotion, FFmpeg	Drive shots, motion, dissolves, subtitles, encoding, audio assembly, and media inspection from data
Nonverbal sound	Licensed Mixkit effects	Supply off-screen cues such as insects, footsteps, fabric, breeze, water, and frogs
Data contracts	Node.js, TypeScript, YAML, Zod, Vitest	Store the timeline, validate structure, and reject incomplete inputs
Media checks	ASR, PCM analysis, frame-by-frame inspection	Locate missing words, loudness jumps, abnormal silence, black edges, wrong frames, and state-timing faults

The tool list is not the method. What transfers is the division of responsibility: text constrains visuals, real audio establishes time, state data drives assembly, checks decide whether work can advance, and people receive only questions machines cannot answer reliably.

Scope and boundaries

This workflow suits medium- and long-form audiovisual work with stable text, explicit narration, and continuity across passages. It can also support instructional material, historical narrative, and visual documentary essays when the team is willing to register sources and narrative responsibilities first.

It does not solve literary interpretation automatically, nor can it guarantee the historical accuracy of generated images. We did not run the research systems cited below, did not use their experimental results as evidence of this film’s quality, and did not use a generative video model to create complete shots or the full film directly. The formal adaptation of Moonlight over the Lotus Pond was assembled from keyframes, sound, a data timeline, and deterministic tools.

Method references

1. Lin H, Zala A, Cho J, Bansal M. VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning [C]. COLM, 2024. Inspired planning across multiple scenes before visual generation.
2. Shin A, Kaneko K. Generating Visually Consistent Images for Storytelling via Narrative Graph Prompting [C]. ICCV Workshops, 2025. Inspired narrative records for stable subjects, relationships, states, and canonical attributes.
3. Gao S, Mathew S, Mi L, et al. VinaBench: Benchmark for Faithful and Consistent Visual Narratives [C]. CVPR, 2025. Inspired explicit records of characters, locations, time, discourse layers, and prohibited misreadings.
4. Wang Y, He X, Wang K, et al. Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation [C]. CVPR, 2025. Inspired checks of consecutive events and arrival order.
5. Matsuda R, Kudo K, Yoshida H, Shimizu N, Suzuki J. SLVMEval: Synthetic Meta Evaluation Benchmark for Text-to-Long Video Generation [C/OL]. CVPR, 2026. Inspired injecting one controlled fault to confirm that checking rules catch a known failure.
6. Tang Y, Liu T, Lai Y, et al. KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation [EB/OL]. arXiv:2607.14202, 2026. Inspired separate checks for the presence, fidelity, order, timing, duration, and uniqueness of key states.

Production note

The cover and state figure in this essay come from the formal production assets for Moonlight over the Lotus Pond. This site stores only the WebP files needed for reading and their responsive derivatives. Source paths, purposes, transformations, dimensions, and SHA-256 values are recorded in the article asset manifest. The images explain the workflow; they are not historical photographs, documentary records, or outputs from any cited research system.