# Game AI: perception and movement playground

The example game's **AI Perception & Brains** scene demonstrates noise investigation, visual pursuit, last-seen search, and return to patrol. An in-game panel owns the instructions, noise controls, and guard feedback in both editor Play and standalone builds. Noise emission uses the same `hi.ai.Stimulus.emit()` API that gameplay can call.

The current AI kernel includes [reusable Brain assets](/docs/guides/brain-assets.md), graph authoring, decision tracing, and an allocation-free flat state/task executor with ground `move_to`, `wait`, completion/failure transitions, an interrupt entry, bounded flat-arena routing, vision admission/occlusion queries, multi-target observation memory, and a World-scoped perception scheduler. Hierarchical graphs, utility selection, navmesh navigation, and navigation/decision job scheduling are not supported.

## Choose a focused demo

| Scene | Purpose | Starts with |
|---|---|---|
| Tutorials → AI Perception & Brains (`ai_hearing`) | Shared Brain, independent parameters, hearing, sight, memory, route failure, decision history | Two guards, one undetectable target |
| Tutorials → AI Gameplay Tasks (`ai_tasks`) | Typed task success/failure, progress, cancellation and lifecycle history | One guard and a closed door |
| Tutorials → AI Swarm Population (`ai_population`) | Compact per-member execution, shared programs, bounded work and member debugging | 100 moving members |
| Diagnostics → AI Sensing Scale (`ai_sensing`) | Multi-target selection, observer admission, sensing budgets and latency | 100 stationary observers, three targets |

Each scene has its own instructions, controls and overview camera. Scale fixtures
are isolated from the behavior and task encounters. All support editor Play and
standalone; the same SDK APIs remain usable directly from Zig.

## Try it

Build the editor through Kaji, without launching it from the build command:

```sh
bin/kaji/kaji editor --workspace=<repo> --project=../hikari-games/example --type=dynamic --config=debug
```

1. Open the example project and **Tutorials → AI Perception & Brains** (`scenes/ai_hearing.json`).
2. Press **Play**. The in-game **AI Perception** panel appears and the cursor stays visible. No editor tool or Escape is needed.
3. Leave interaction on **Noise** and leave **Targets detectable** off to isolate hearing. Left-click the floor near a guard. A yellow ring previews the noise radius; the heard noise interrupts patrol and sends the guard to that point.
4. Choose **4 / 8 / 16 m** and **Footstep / Impact** directly in the panel. Select **Blue / Red** to inspect a guard’s state, hearing result and distance. Enable **Decision history** for its recent transitions.
5. Set radius to **16 m** and click across the centre wall within a guard's hearing range. A cyan route shows its detour. Click the floor inside the closed pen on the right: a guard that hears it reports **unreachable / no_route** and returns to patrol.
6. Wait for investigation to finish. The guard looks around and returns to patrol. Click another point to interrupt it again.
7. **Settings / scenes** opens the normal settings sheet. Close it to resume floor interaction. Escape toggles Settings; it is not a prerequisite for clicking.
8. Check **Targets detectable**, choose **Move player**, and click the floor to move the green capsule. Move it into a yellow guard cone: the cone turns red and the guard pursues it. Move around the centre wall to break line of sight: the guard visits its last-seen marker, searches, then returns to patrol.
9. Switch back to **Noise** during pursuit/search and click nearby. The panel reports that the noise was heard but the visual target takes priority. Re-enter the cone while it is searching to trigger immediate pursuit.
10. Pause permits preview but suppresses noise and movement commands. Stop/Play resets the probe and guard state.

The same controls work in a standalone game after selecting AI Perception & Brains from the scene browser. This scene uses a fixed overview camera and a free pointer. Ordinary movement scenes still request capture in standalone builds; Settings and modal dialogs temporarily release it. In editor Play, keep the game camera possessed for game interaction; editor camera detachment disables the in-game world probe.

The generic Session card is hidden in this scene so its guide and controls do not compete with the hearing panel. Panel backgrounds, controls, Settings, modal dialogs, and editor chrome do not emit noise. Rays use the camera behind the HUD and viewport-local UI points, so editor dock offsets and display/UI scaling do not shift the hit.

The AI demos' [cursor theme](/docs/guides/ui-cursors.md) shows cyan sonar rings for Footstep and an amber burst for Impact over a valid world hit, with a hand over buttons. The centre marks the click location. The theme is installed on entering an AI demo and cleared when leaving it; other scenes use native defaults. Changing cursor appearance does not change capture or click ownership.

The blue guard hears up to **6 m**; the red guard up to **11 m**. Blue accepts **Footstep (category bit 0)** and **Impact (category bit 1)** with mask `3`; red accepts only Impact with mask `2`. Use Impact to compare range, then Footstep to test filtering. Noise radius limits propagation independently of the listener's hearing range.

Hearing uses 3D distance between the stimulus and the guard's actor position, with an inclusive boundary:

```text
heard = accepted category AND distance <= min(noise radius, guard hearing range)
```

The floor ring visualizes horizontal propagation from the source. The guard's sphere visualizes its hearing range. Their visible intersection with the floor differs slightly because a standing guard's position is above the floor. Walls do not attenuate this first hearing model. Viewport placement uses the existing physics raycast; mesh/hull geometry may be approximate when the physics snapshot is used.

With target detection disabled, the guards deliberately share the same patrol offsets and speed, so their initial motion matches. Their different hearing ranges are the comparison: a noise can interrupt one while the other continues patrolling.

## Author the behavior

Add the exported `GuardAI` component from `hikari-games/example/src/entities/ai_guard_entity.zig` to an actor with Physics and a character controller. The sample `ai_guard` archetype also supplies rendering. The component is attachable; the AI kernel has no dependency on that archetype.

Waypoints are offsets from the guard's initial position, so duplicating a guard moves its patrol with it. The three offsets are `waypoint_a`, `waypoint_b`, and `waypoint_c`. Movement uses the existing `CharacterMotor` following cached routes from `hi.ai.navigation`. The planner operates in XZ and keeps clearance from scene-authored boxes; elevation remains the motor's responsibility.

The guards share `assets/brains/guard.brain.json`, assigned through the component's **Brain** asset picker. Its states, completion/failure transitions, ordered fact conditions, and search-duration default are edited in the Brain asset editor and cooked by Shinra. Per-actor fields supply patrol offsets, speed, hearing range, and an optional `search_seconds` override (−1 inherits the asset). See [Brain assets](/docs/guides/brain-assets.md) for authoring and the two-guard reuse check.

The shared program follows patrol → investigate noise → look around → return, or visual pursuit → last seen → search → return. Vision takes priority over noise through authored conditions. Movement failures take the state's failure destination; old failure reports cannot change a newer task activation. Invalid assets or missing guard bindings stop the brain and appear in `brain_status`.

Configuration is authored in Edit; Play inspection is read-only. `enabled` provides a component-level simulation switch. Actor disable cancels execution and zeros movement; re-enable restarts the Brain entry state. `active_state_name`, `transition_name`, and bounded `decision_history` explain the current behavior. Saved Brain changes apply to a new Play World.

## Navigation and failure feedback

The example copies geometry when a move activation begins. Exactly one active floor carries `ai_navigation_floor`; up to **16** tagged `ai_navigation_obstacle` actors supply blockers. Both tags must be declared in `configs/gameplay_tags.json`; World discards undeclared scene tags. Both use their existing Physics box dimensions and world scale, so editing the collider changes the next planned route. The adapter explicitly requires unparented, unrotated static boxes. Invalid geometry and excess capacity fail closed instead of silently omitting blockers. Obstacles represent full-height barriers in this flat model.

The SDK planner borrows a floor rectangle and obstacle rectangles, expands obstacles and shrinks floor bounds by clearance, and searches visible corners with Dijkstra. The sample clearance is the scaled capsule radius plus **0.15 m**, with an additional **0.08 m** corner margin. Storage is bounded at **66 graph points**; neither planning nor following allocates. Search work is bounded by the small obstacle cap, and direct routes take a visibility fast path. This is a small-arena implementation, not a population scheduler or navmesh service; large-population performance has not been measured.

Each guard retains only its route values and activation serial. An accepted noise, visual interrupt, or disable cancels that route. Static destinations plan once per move activation. Pursuit may refresh its route at most once every 0.35 seconds when the sampled goal moves over 0.5 m, rather than planning every tick. Corners have a tighter arrival tolerance than the final task target. Two seconds without at least 5 cm of progress toward the current corner reports `stalled`; the timer uses simulation time and freezes with Pause. Failed movement zeros the motor and takes the authored failure transition. Unreachable patrol points are skipped, with at most one failure transition per tick.

The panel and Play inspector expose **moving**, **arrived**, and **unreachable**, the last completed movement result, and the last failure reason. The failure reason remains visible after recovery. `route_points` and `route_plans` expose route size and replanning count. Cyan segments show the remaining route during movement. A return may begin on the next tick after failure, so use the retained failure reason when inspecting recovery.

Edit obstacle geometry before Play. Cached routes do not rebuild for moving walls; unexpected runtime blockage reports `stalled`. Untagged colliders still physically collide but do not participate in route planning. There is no dynamic crowd avoidance, slope traversal, sound occlusion, or arbitrary mesh navigation in this slice. Clicking a wall top projects the target into its blocked XZ footprint and reports unreachable.

## Vision, memory, and game priorities

`hi.ai.perception.Vision` supplies a 3D range/cone test and a filtered physics segment query. `geometry()` returns geometric admission only; `query()` additionally tests occlusion. The nearest non-sensor collider blocks sight unless it belongs to the target. The observer is excluded, and `occlusion_mask` selects collision categories. A clear segment does not require a target collider hit. Physics uses the existing snapshot-first query API, so results inherit its published-pose timing and approximation limits. Render transparency does not imply visibility through a collider.

`hi.ai.perception.Memory` stores a weak actor handle, copied last observed position, age, and known/visible flags. `remember()` is called only for a positive observation. `miss()` clears visibility while retaining the previous position. `advance()` consumes simulation time; no pointer escapes and Pause does not age memory. The guard keeps its selected target in component transient state, while the World-scoped service retains the bounded set of observations used to select that target. Neither stores a process-global actor registry.

The shared perception service discovers candidates spatially from active `ai_target` actors for each `ai_observer`, aiming for a **10 Hz** observation cycle within its configured work budget. It applies range, category, and cone admission before spending a raycast. There is no hardcoded player lookup in guard sensing. The eye and target sample are 0.5 m above their actor centres. Vision defaults to **12 m**, with a **55° half-angle**. This is one sample point, not full-body visibility. The yellow horizontal cone section turns red while a target is visible; the pink point is the last observed actor position. `show_vision` controls the game-owned overlay. Live inspection exposes `sight_result`, `target_visible`, `target_known`, `last_seen_position`, `last_seen_age`, and `sight_samples`.

The shared guard Brain authors this priority: visible target → remembered visual investigation/search → noise → patrol. Seeing a target interrupts lower-priority tasks. Loss of sight immediately redirects to the remembered point, never the unseen target's live position. If that observation becomes older than `memory_seconds` (default 8), the guard searches from its current location instead of continuing the route. It rotates for the resolved `search_seconds` duration, clears its local pursuit memory, and returns to patrol; shared observations remain available until their retention expires. Reacquisition interrupts either last-seen movement or search. Pursuit/path failure also enters search, with a one-second retry delay to bound retries while an unreachable target remains visible. There is no attack/damage behavior in this test.

The green `ai_target` actor reuses the same motor and route planner at 4 m/s. **Move player** commands travel through the ordinary World event dispatcher. A new click replaces the route; blocked destinations report Unreachable. Movement is silent so hearing and vision can be tested independently. **Targets detectable** controls candidate admission for the player and both scouts. Removing a target from the authoritative candidate snapshot prunes its shared memory; the next delivered observation clears or changes the guard's focus. Cached visibility also expires after 0.25 seconds without a fresh check. For isolated hearing checks, disable it and wait for search/return, or restart Play and then disable it. The `player` tag identifies the directly controlled actor; `ai_target` identifies all detectable actors. Both tags are declared in the gameplay catalog. Scout A and Scout B automatically move between authored offsets at 1.5 m/s without emitting noise.

## Shared scheduling and load tests

`hi.ai.scheduler.Service(observer_capacity, target_capacity, memory_slots)` is the reusable engine service. The example instantiates it with **1,024 observers, 64 targets, and eight observations per observer**. Its game adapter in `ai_service.zig` creates separate storage keyed by `WorldScope.id`, destroys it on detach, and clears the active World's state before scene unload. The host calls this adapter after gameplay joins; no runtime code imports the editor. All inputs and callbacks are borrowed for one tick, and the service retains copied values and generation-safe handles only.

Each tick receives authoritative observer and target snapshots. Stable handle ordering preserves scheduling progress; removed/replaced handles and changed observer activation IDs discard old observations and selection state. Stable input identity ordering reuses the sorted index mapping and updates observer memory in place; cohort changes merge retained rows by handle. Remembered subjects use indexed lookup, and candidate continuation selects the next handle without sorting candidate lists. Targets enter a shared hashed XZ grid with 16 m cells; very large ranges use a bounded linear fallback. Candidate admission also checks 3D distance and game-authored category masks (`guard_ai.target_mask` against `ai_target.category`). Snapshot validation rejects duplicates, invalid input, and excess capacity explicitly rather than silently truncating work.

The scheduler rotates through observers and through their nearby candidates. One visit evaluates one candidate, with at most one visit per observer per frame. A full candidate cycle waits until its nominal 0.1-second period has elapsed; overload continues on subsequent frames instead of catching up in a burst. **Steps** bound candidate evaluations/publications and **rays** independently bound occlusion queries. Neither sorting, spatial lookup, memory maintenance, nor callbacks allocate during the tick. Snapshot synchronization and game-side actor-field gathering still have population-dependent cost; the budgets are work-count caps, not a hard wall-clock deadline. Fairness assumes nonzero budgets and a stable active cohort; continuously changing candidate sets can change revisit intervals.

The game supplies target selection: retain a recently visible focus, otherwise select the nearest recently visible target, then the most recently seen remembered target. This policy lives in `ai_guard_policy.zig`, outside the engine scheduler. Known positions change only after a positive observation. Visibility expires after 0.25 seconds without a positive check; remembered knowledge expires at the guard's `memory_seconds`. Full memory slots evict the oldest checked observation, counted by `Metrics.evictions`.

The adapter sends the selected observation through the ordinary World event dispatcher. A four-argument game protocol carries position, ages, and packed sight/activation flags; the weak subject is `Message.from`. Restart/disable advances the guard's perception epoch so delayed messages cannot revive an old activation. Delivery occurs at the next dispatcher opportunity, adding up to a normal gameplay tick to the service's sensing delay. Dispatcher saturation is best-effort; guards expire stale visibility even if a result is dropped. This service budgets perception; guard movement, navigation planning, rendering, and physics remain outside that budget.

In **Diagnostics → AI Sensing Scale → Play**, use **Observer count → 0 / 10 / 100 / 1000**. The dedicated scene starts with 100 stationary observers, two moving scouts and one controllable target. All observers use sensor bodies and disable cone drawing; there are no full-behavior guards or swarm members. Creation/removal is limited to 16 actors per simulation frame. Reducing the count retires fixture-owned actors. Stop or scene unload removes them with their scene layer. These controls do not appear in the other AI demos.

**Perception budget → Low / Normal / High** sets steps/rays to **16/4**, **128/32**, or **1024/1024** per frame. The panel reports observer/target counts, checks, rays, deferred ready observers, oldest cached check, and adapter CPU milliseconds (including actor gathering, service work, and event submission, excluding later delivery, movement and physics). `Metrics.max_wait` tracks the largest time since an observer was serviced. The guard inspector adds observation age and its selected target handle; the panel shows the target's name. Under severe pressure a stale visible target becomes remembered rather than remaining visible indefinitely.

Reproduce the headless benchmark from `src/hikari`:

```sh
zig build ai-perception-bench -Dgame-src=../../../hikari-games/example/src -Doptimize=fast
```

It runs 10, 100, and 1,000 observer snapshots against eight moving targets for 600 simulated 60 Hz frames, at Normal and High budgets. It fails on exceeded budgets or a starved observer and reports mean/p95 service CPU time, worst first detection, maximum service gap, oldest check, ray counts, minimum deliveries, and fixed storage size. Occlusion uses an analytic box; these measurements exclude Tenkai, actor gathering, event dispatch, rendering, and navigation. See [measured results](/docs/reports/ai-perception-benchmark.md). Navmesh navigation, multi-threaded AI jobs, and general decision scheduling are not supported.

## Emit a stimulus from game code

```zig
const hi = @import("hikari_game");

const noise: hi.ai.Stimulus = .{
    .position = .{ 4, 0, -2 },
    .radius = 8,
    .category = 0,
};
_ = noise.emit();
```

Call on the World owner thread. `emit` validates the payload and broadcasts `hikari.ai.stimulus` through the existing World event dispatcher. Its boolean reports payload validity, not guaranteed delivery under dispatcher saturation. Listener components decode with `Stimulus.fromMessage`, filter with `heardBy`, then interrupt their machine. No actor/context pointers are retained. The sample shows brief diagnostics for stimuli; production games choose their own diagnostic presentation.

`hi.ai.Machine` borrows `Program.states` and target slices only during calls. It stores a state index, timer, activation serial, enabled flag, and transition reason. There are no allocations, jobs, asynchronous receipts, or service ownership in this initial kernel. The host/game module's existing component execution and lifecycle rules still apply.

For concrete Zig examples, see [Run an asset from Zig](/docs/guides/brain-assets.md#run-an-asset-from-zig) and [Author a Brain entirely in Zig](/docs/guides/brain-assets.md#author-a-brain-entirely-in-zig). Both paths use the same task executor; asset-backed behavior also supports editor authoring.

## Check your integration

Run `python3 ../hikari-games/example/tools/test_ai_scene.py` from the engine root to validate the example scene's tags, observer/target setup, navigation membership, and box geometry. Reopen the project after changing its tag catalog so newly created Worlds read the updated declarations.

Use **AI Sensing Scale → Observer count** and perception quality controls to see the query budget in action. More observers can increase deferred work and observation age without increasing the per-frame query ceiling. Reducing the population lets that work drain. Pause stops simulation and sensing; Stop followed by Play starts fresh observations.

Build and headless-test results do not establish visual or gameplay correctness. Test your own geometry, tag catalog, and desired pursuit behavior in Play as well.

## Swarm populations

The same Brain executor also runs in swarm rows. See [Brain execution for swarm populations](/docs/guides/swarm-ai.md) for compact state, shared asset binding, mixed actor/swarm sensing, bounded decisions, and the moving 1,000-member acceptance scene.

## Gameplay interactions

[Gameplay AI tasks](/docs/guides/ai-tasks.md) adds registered actions with typed parameters, capabilities, cancellation and safe delayed completion. **Tutorials → AI Gameplay Tasks** isolates one guard and a door for success, locked-door failure and sight interruption.