Lumine: Building Generalist Agents in 3D Open Worlds

Mondstadt Region: Lumine successfully completes a full five-hour, three-act main storyline playthrough of Mondstadt region. Even though its reasoning ability is trained only on the content of the first act, Lumine autonomously carries out all story progressions.

Liyue Region: In the completely out-of-distribution region of Liyue, Lumine demonstrates impressive generalization, successfully reaching and completing the main storyline in Liyue Harbor, and even locating and visiting the Adeptus dwelling deep within the mountains.

Beyond Genshin: Lumine exhibits exceptional cross-game generalization ability. Without any additional fine-tuning and modification, Lumine directly completes the entire chapter of Herta Space Station in Honkai: Star Rail within 7 hours; similarly, it successfully finishes approximately 100 minutes of main storyline content in Wuthering Waves.

An Efficient and Scalable Recipe for Building General-Purpose Agents

Lumine adopts a human-like interaction paradigm that unifies perception, reasoning, and action in an end-to-end manner, powered by a vision-language model. It processes raw pixels at 5 Hz to produce precise 30 Hz keyboard–mouse actions and adaptively invokes reasoning only when necessary.

Lumine makes full use of the collected raw gameplay data, following a curriculum that builds skills incrementally:

• 1731 hours of human gameplay for pre-training to master action primitives;
• 200 hours of instruction following data to ground control in language;
• 15 hours of reasoning data to enable adaptive thinking.

The resulting model can not only autonomously complete hours-long missions but also follow diverse instructions to accomplish a broad spectrum of tasks.