Portable game scene generation

SceneAgent.

A Multi-Agent Framework for
Portable Game Scene Generation

1 Tencent2 Fudan University

* Equal contribution † Corresponding authors

Construct rich game scenes with coordinated agents.
Reuse the process and experience across engines.

A tour of generated worldsProject video / 2:30
Jump into a world

The idea

Build worlds.
Carry experience forward.

Moving a scene generator to a new engine requires more than exporting its output. Its planning, tools, and stored experience may depend on the original backend.

SceneAgent separates persistent scene state, task-local reasoning, and reusable skills. Coordinated agents construct the scene; backend adapters handle how it is realized in each engine.

Scene collection

One framework. Many scenes.

From city streets to imagined worlds.
Selected frames from the project video.

Built to travel

Shared reasoning.
Different realizations.

The same construction protocol and skill library,
retargeted through backend integration components.

ENGINE AIce Wonderland
ENGINE BIce Wonderland

Reused Agent protocol, SceneContext, core tools, and semantic skills.

Adapted Backend adapter, asset catalog and database, and capability-tool wrappers.

Qualitative examples of retargeting. The engines use different asset catalogs; these are not matched quality comparisons.

Inside SceneAgent

Persistent worlds. Fresh agents.

Separate what the world remembers
from what each agent needs to reason about.

Within-run construction and cross-run experience reuse.
01 / COORDINATE

Plan, then construct

The Orchestrator retrieves skills. The Planner organizes semantic subtasks. Fresh Builders query the persistent scene state and carry out each task.

02 / REALIZE

Isolate the backend

A canonical SceneContext stores the evolving world. Adapters handle coordinate conventions, asset bindings, and export to the target engine.

03 / REUSE

Retain approved experience

Human-approved traces become asset-ID-free Layout skills and semantic Content skills, ready to retrieve and recombine in later generations.

GameSceneBench

Evaluating complete scenes.

69 prompts: 56 outdoor and 13 indoor.
Common assets and evaluation protocol in Engine A.

Overall human preference83.3%

Tie-adjusted preference across
1,860 human judgments.

Win 76.9%Tie 12.7%Loss 10.3%
SceneAgent preference vs. each baseline
ComparatorHuman preference
AutoUE†89.4%
Holodeck85.2%
ImperativeScene78.2%
Reason-3D81.2%
SceneWeaver82.7%

50% indicates parity. Ties count as half a preference.
† AutoUE uses its original scene-layout stage adapted to the common backend.

The clearest ablation gains are in prompt alignment and scene completeness, with additional runtime. Spatial plausibility remains a challenge. See the paper for full comparisons, protocols, and limitations.

Citation

Citation

@article{li2026sceneagent,
    title={SceneAgent: A Multi-Agent Framework for Portable Game Scene Generation},
    author={Li, Xingyi and Yang, Jiahao and Zhu, Yiqin and Ye, Jingwen and Sun, Yang and Zhang, Weidong},
    journal={arXiv preprint arXiv:2611.17821},
    year={2026}
}