AI for Games in the Foundation Model Era

Few AI capabilities in games actually transfer across settings; most conclusions hold only within the original game, engine, interface, and player population.

This survey classifies the literature not by technique but by what the AI's output is directly used for, sorting it into six roles and answering the same three questions for each. That framing lets reinforcement learning agents, world models, code agents, and LLM reviewers be compared in one table, and it exposes the evidence strength behind every conclusion.

01Why the game AI map has to be redrawn

The same pretrained model can work three entirely different jobs within a week, and the evidence that holds differs completely from job to job.

The survey's opening example is direct: ARC-AGI-3 places a model in an unfamiliar interactive environment and requires it to work out the rules before acting; Playco connects the same model to engine tools to generate playable game prototypes. Same model, but the task, the contribution, and the validation all differ. Over past decades, game search, procedural content generation, player modeling, and automated playtesting each evolved in separate communities; foundation models connected them, yet nobody has put their respective evidence side by side.

The authors distill the survey into three recurring conclusions, each pointing at evaluation.

02Why current game AI conclusions cannot be reused directly

Capabilities are broadening, but control schemes, rules, engine interfaces, state representations, and player context still come with each setting.

• FINDING 1Pretraining widens the interface; it does not remove game structureLanguage, vision, code, and tool interfaces let models enter more stages of the pipeline, but what a model can express and what a game can execute remain two different things. GameNGen replaces explicit rules with a learned neural simulation, yet still needs a defined action interface and consistency checks.
• FINDING 2Playing a game yields more than a scoreA trajectory can serve as training data for a simulator, as an experiment inside a learning environment, or as diagnostic evidence for software repair. What a trajectory is worth depends on which information the receiving task needs.
• FINDING 3Evaluation strength is highly unevenBounded competitive play and parts of learning environments already have standardized, executable evaluations. Sustained creation, runtime adaptation, and real player experience still lack shared benchmarks.

Taken together, the three points yield one engineering conclusion: handing one role's output to another role does not mean the capability transferred. Sharing a backbone model, or two components communicating successfully, is not evidence of transfer.

03Method breakdown: one diagram, five steps

The survey's method is not to pile up citations: it first defines classification criteria, then attaches the same set of check questions to every class.

Step 1: Assign roles by the direct use of outputs

Play and Act outputs actions, plans, and messages; Model Players and Games outputs states, transitions, and player predictions; Design outputs levels, rules, and story structure; Build and Maintain outputs code, scenes, and engineering changes; Runtime Generation and Adaptation outputs dialogue, quests, and live content; Test and Evaluate outputs trajectories, verdicts, and diagnostics. Classification follows the use of the output, not the model architecture, so the same system can fall under different roles depending on the claim.

Step 2: Ask each role the same three questions

Boundary: what the game or workflow provides and what the AI is assigned. Transfer and Reuse: which capabilities transfer, which artifacts are reusable, and which remain bound to the setting. Evidence: which claim the evaluation actually supports at the point of use. Transfer means the capability survives a change of game, engine, interface, player population, or task; reuse means a representation, trajectory, specification, model, or feedback signal is consumed elsewhere.

Step 3: Infer the primary role from the claim

When a system spans multiple roles, its core empirical claim decides the assignment. An NPC policy belongs to Play and Act when the claim concerns action and cooperation, and to Runtime when the claim concerns its effect on player experience in conversation; MarioGPT proposes level content and belongs to Design; Play2Code implements and repairs software and belongs to Build and Maintain. Predictions and simulations produced for planning or training stay in Modeling even when running online.

Step 4: Compare evidence on five dimensions

Standardization: whether tasks, protocols, and metrics are shared, or every system ships its own demo. Grounding: whether claims are verified with engines, reference environments, or deterministic tests rather than surface quality. Scope and transfer: which games, genres, engines, interfaces, and player populations the evidence spans, and what actually changed during testing. Duration and revision: whether evaluation covers sustained interaction, repeated editing, regression, and handoff. External validation: whether the reference is independent of the evaluated system and represents the target task or players. The survey explicitly refuses to compress these five dimensions into a single maturity score.

Step 5: Audit cross-role connections with three checks

Compatibility: does the output keep the format and meaning the receiver needs. Use: does the receiver actually consume it in the evaluation pipeline. Benefit: against an appropriate baseline, with costs reported, do downstream results improve. The three do not imply each other — compatible does not mean used, and used does not mean beneficial.

04Where the evidence comes from: the evaluation base of the six roles

This is a survey with no experiments of its own; its data are the most representative benchmarks in each role and their published results.

Chapters 3 through 8 cover methods role by role; Chapter 9 puts the evaluations side by side, defining evaluation goals and interpretation limits for each role; Chapter 10 discusses cross-role connections; Chapter 11 lists six open problems. The main data used in Chapter 9 follows.

Each role has different evaluation goals: Play and Act looks at completion and progress; Modeling looks at world-state fidelity and policy transfer; Design looks at validity and designer control; Build looks at launchability, requirement satisfaction, repair, and regression; Runtime looks at latency, consistency, and player outcomes; Test looks at coverage, verified defects, and verdict accuracy. Collapsing them into one score is meaningless.

05Results: four judgments now backed by numbers

Evaluation numbers hold only for the configured system as a whole; swap any component other than the model and the conclusion changes.

1. FINDING 01

   Agents can advance tasks but can barely complete them

   In GameWorld, the best model reaches 41.9% progress with only a 21.2% success rate. On the same tasks, the novice human reference succeeds 55.3% of the time and the expert 77.1%, with progress of 64.1% and 82.6% respectively. All 18 model configurations fall below both human references. Progress and completion are two different quantities: models often stop halfway, even though the game is paused during reasoning. The two human references are one person each and serve only as examples, not estimates of novice and expert populations; semantic action parsing also does not consistently beat direct computer control.

2. FINDING 02

   A benchmark score is a score of model plus harness

   On ARC-AGI-3, GPT-6 Astra at High reasoning effort scores 54.82 with the Standard harness and 99.95 with the Provider Adapter harness; at Max effort, 62.71 versus 98.55. The two harnesses differ only in how they carry reasoning state and manage conversation history. The harness gap exceeds many gaps between different models. On the same leaderboard under the Standard harness, Claude Opus 5 scores 30.16, GPT-5.6 Sol 7.78, and Grok 4.6 2.11. A score describes the configured system as a whole, not model capability alone.

3. FINDING 03

   Generated games can run the rules but cannot carry the content

   GameCraft-Bench evaluates complete game engineering across 14 configurations. Claude Opus 5 (xhigh) scores 68.44 overall, 76.75 on core mechanics, and 63.31 on content depth; in every reported row, the core mechanics score exceeds the content depth score. Overall scores range from 68.44 down to DeepSeek V4 Pro's 2.15, because submissions that fail to launch or lack a parseable interaction trace are scored zero. Models implement recognizable rules more reliably than coherent progression and content.

4. FINDING 04

   The testing bottleneck is verdicts; state injection costs less than coverage

   On 100 generated web games, GameGen-Verifier combines state injection, visual judgment, and program assertions; its best configuration reaches 92.2% agreement with expert-annotated specifications (Acc@5 over five repeated runs), while a coverage-enforcing baseline reaches only 58.8%. The counterexample is pure visual judgment: VideoGameQA-Bench tops out at 45.2% accuracy on visual regression detection; on industrial footage of 41 hours and 19,738 keyframes, a single-prompt vision-language baseline reaches 0.50 precision and 0.72 accuracy. PlaytestArena reports 84.2% raw criterion agreement (32 games). In NCP-Bench, the best narrative model still leaves 42% of environments conflict-free after 20 turns.

06Closer look: where evaluation is most easily misread

For every impressive-looking number, the survey asks what it actually measured.

World models: looking right is not being right

World model fidelity has at least three goals: perceptual quality, mechanical correctness, and persistent state. FVD catches gross visual failures but checks no specific rules and audits no hidden variables; a beautiful animation can coexist with wrong health deductions, and a transition that is correct right now can still be forgotten later. In PlayWorld's comparison of 9 models, SANA-WM has the second-highest valid-trajectory rate while its two evolution scores sit low on the scale; LingBot-World2 beats LingBot-World on visible evolution but does worse off-screen. Genie 3 scores 2.12 overall (on a 1-to-5 scale) with an 87.1% valid-trajectory rate. Programmable World Model reports 94% counting accuracy and 98% death-state accuracy on 50 videos, but its death check only requires a visible dead character in a sampled frame — not the correct one.

Transfer: new map, new mode, and new game test three different things

Changing the map tests robustness under fixed rules; changing the mode tests adaptation to named rule variants; changing the game tests transfer of the evaluated behavior or model function. SIMA evaluates instruction-following behavior on held-out games, SCOPE tests action-conditioned generation across multiple FPS titles; both cross games, but the latter does not show the policy learned new ways to win. GameFactory's new visual scenes show transfer of appearance and control without restoring each game's original mechanics. Zero-shot, retrieving target demonstrations, and fine-tuning on the target game test different forms of transfer; the overlap between public code, walkthroughs, and benchmark repositories and pretraining data is unknown, so it is evidence neither of contamination nor of a clean split.

Reviewers: self-preference and reward hacking

Model reviewers tend to prefer their own outputs, and optimizing against an imperfect verifier rewards shortcuts. GameGen-Verifier's state injection checks local mechanics but bypasses the path taken to reach that state; Play2Code's playthroughs find paths but only where the player actually went. Fixing against only one kind of feedback leaves the other class of failures untouched. Rerunning the same reviewer or swapping a prompt is not independent validation.

Human studies and reproducibility

The same players repeating levels are not independent subjects; a deployment's calendar duration does not equal each player's multi-session exposure. In personalized level generation, the two player groups already differed in initial level-generation policy before receiving personal history, so later differences cannot all be credited to personalization. A reproducible evaluation must state model and engine versions, prompts, observation and action interfaces, tool permissions, random settings, and retry and best-of budgets.

The survey's own limits

Chapter 10 is a qualitative synthesis of representative systems: it does not rank cross-role connections or measure maturity. Table 13 lists 'downstream results awaiting verification,' not results every system has already delivered. The authors' gray-box plus generative rendering workflow (a code agent builds an executable gray box first, a state-conditioned video model handles appearance) remains a research direction in full production engines; lightweight-engine demos do not demonstrate reliable generation and maintenance for UE or Unity projects.

07Open source and resources

The survey ships a continuously updated repository that organizes 400+ papers into a searchable index by role.

• GitHub repository:Eurekaleo/awesome-ai-for-games
• Project website:eurekaleo.github.io/awesome-ai-for-games
• Appendix A's system index records each system's primary and secondary roles and its testing methods; Appendix B collects game examples and interactive demos.

08What this means for the data and evaluation industry

Games are the cleanest testbed for agent evaluation: clear rules, observable feedback, checkable state — so the evaluation problems they expose will only be worse in other domains.

First, task data must carry the environment, action interface, and state — not just the prompt and the answer. The paper's conclusion is that a trajectory has different value to different receivers: actions need observations, designs need requirements, profiles need context. Every task dataset we deliver should ship a reproducible environment version and action semantics alongside it; miss either and downstream users will misuse it.

Second, verifiers must execute, not just score. FINDING 04 shows state injection plus program assertions reaches 92.2% agreement, while pure visual judgment reaches only 45.2% on visual regression. Whatever rules can be written as assertions should go to engine checks; reviewer models handle only what assertions cannot express, with references from an independent source — the generator and the reviewer must not share the same blind spots.

Third, evaluation reports must treat the harness as a first-class variable. The 54.82-to-99.95 gap shows that leaderboards reporting only model names carry no information. Every benchmark score should come with its harness, tool permissions, retry budget, and reasoning effort, plus a separate harness ablation.

Fourth, separate progress from completion, typical attempts from best-of results, and single sessions from sustained use. GameWorld's 41.9% versus 21.2% is exactly the value of that distinction. Our evaluation datasets provide layered metrics by default, never a single total score.

Fifth, the real gap is persistence: persistent state, repeated revision, sustained adaptation, and real-player coverage have no mature benchmarks today. These are exactly where task data and verifiers are thinnest, and where we build first.

09Cite

@article{luo2026aiforgames,
  title   = {AI for Games in the Foundation Model Era},
  author  = {Luo, Meng and Li, Yanlin and Li, Hao and Lin, Hongzhan and Zhou, Pengfei and Ju, Tianjie and Zhang, Ran and Jin, Yeying and Lee, Mong-Li and Hsu, Wynne},
  journal = {arXiv preprint arXiv:2609.16679},
  year    = {2026}
}