LLMs Inside Your Game
When it works, and when it does not
Verified against a working setup on 2026-08-10
This is the other half of "AI in Unity", and it is a completely different problem from MCP. There, a model helps you build the game. Here, a model runs inside the shipped game, and every constraint changes: latency is player-facing, cost scales with players, and failures happen in front of an audience.
The shape of it
Three pieces, and the middle one is not optional:
- 1 Your game
Sends a request over HTTPS. Holds no API key.
- 2 Your proxy server
Holds the key, adds it, forwards the request, enforces rate limits, and logs cost. This is the part people skip and regret.
- 3 The model API
Answers. Your proxy relays the answer back to the game.
What it is genuinely good at
| Use case | Verdict | Why |
|---|---|---|
Ambient NPC chatter | Strong | Latency is invisible; failure is harmless. |
Conversational NPCs | Strong | Players accept a short pause when talking. |
Player-authored content | Strong | Naming, describing, summarising a run. |
Dynamic quest text | Good | Generate ahead of time, not on demand. |
Combat decisions | Poor | Hundreds of milliseconds is far too slow. |
Anything in Update() | Wrong | A network call cannot run per frame. |
Rules the game depends on | Wrong | Output varies. Game logic must not. |
The four things that sink projects
1. Latency is not optional to design around
A response takes hundreds of milliseconds to several seconds. That is fine for a conversation and fatal for anything reactive. Design the interaction so waiting reads as natural — the NPC pauses, thinks, looks up — rather than bolting a spinner onto a system that assumed instant.
Streaming helps enormously here, because words start appearing almost immediately even though the full reply takes seconds. That is the subject of Streaming NPC Dialogue.
2. Cost scales with players, not with you
It costs nothing while you are testing alone, and it is a real bill at ten thousand players. Model the arithmetic before you design around it: requests per session, tokens per request, sessions per player, players per month.
Two levers matter most. Pick a cheaper, faster model for chatter and reserve the expensive one for moments that matter, and cache the parts of your prompt that never change — both covered in Cost, Latency & Caching.
3. The network will fail
Players play on trains, on hotel wifi, and offline. Every call needs a timeout and a fallback line, and the fallback needs to be good enough to ship on its own.
The practical test: turn off your wifi and play. If the game is broken rather than merely less interesting, the model is load-bearing and should not be.
4. Players will try to break it
If there is a text box, someone will type an instruction into it to see what happens. Treat player text as untrusted input, keep your instructions in the system prompt rather than concatenated into player text, and never let model output directly execute game actions without validating it against a fixed list of allowed moves.
Generate ahead of time where you can
The most reliable pattern is often not runtime at all. Generate a few hundred lines of NPC dialogue during development, review them, and ship them as data. You get the variety without the latency, the cost, the network dependency, or the moderation problem.
What to take away
- Three pieces: game, your proxy, the API. Never two.
- Good for conversation and flavour; wrong for game rules.
- Design the interaction around latency instead of hiding it.
- Model the cost at your target player count before building.
- Every call needs a timeout and a shippable fallback.
- If it does not need to be per-player and live, pre-generate it.