Game AI: perception and movement playground

The example game's AI Perception & Brains scene demonstrates noise investigation, visual pursuit, last-seen search, and return to patrol. An in-game panel owns the instructions, noise controls, and guard feedback in both editor Play and standalone builds. Noise emission uses the same hi.ai.Stimulus.emit() API that gameplay can call.

The current AI kernel includes reusable Brain assets, graph authoring, decision tracing, and an allocation-free flat state/task executor with ground move_to, wait, completion/failure transitions, an interrupt entry, bounded flat-arena routing, vision admission/occlusion queries, multi-target observation memory, and a World-scoped perception scheduler. Hierarchical graphs, utility selection, navmesh navigation, and navigation/decision job scheduling are not supported.

Choose a focused demo

Each scene has its own instructions, controls and overview camera. Scale fixtures are isolated from the behavior and task encounters. All support editor Play and standalone; the same SDK APIs remain usable directly from Zig.

Try it

Build the editor through Kaji, without launching it from the build command:

bin/kaji/kaji editor --workspace=<repo> --project=../hikari-games/example --type=dynamic --config=debug

1. Open the example project and Tutorials → AI Perception & Brains (scenes/ai_hearing.json).
2. Press Play. The in-game AI Perception panel appears and the cursor stays visible. No editor tool or Escape is needed.
3. Leave interaction on Noise and leave Targets detectable off to isolate hearing. Left-click the floor near a guard. A yellow ring previews the noise radius; the heard noise interrupts patrol and sends the guard to that point.
4. Choose 4 / 8 / 16 m and Footstep / Impact directly in the panel. Select Blue / Red to inspect a guard’s state, hearing result and distance. Enable Decision history for its recent transitions.
5. Set radius to 16 m and click across the centre wall within a guard's hearing range. A cyan route shows its detour. Click the floor inside the closed pen on the right: a guard that hears it reports unreachable / no_route and returns to patrol.
6. Wait for investigation to finish. The guard looks around and returns to patrol. Click another point to interrupt it again.
7. Settings / scenes opens the normal settings sheet. Close it to resume floor interaction. Escape toggles Settings; it is not a prerequisite for clicking.
8. Check Targets detectable, choose Move player, and click the floor to move the green capsule. Move it into a yellow guard cone: the cone turns red and the guard pursues it. Move around the centre wall to break line of sight: the guard visits its last-seen marker, searches, then returns to patrol.
9. Switch back to Noise during pursuit/search and click nearby. The panel reports that the noise was heard but the visual target takes priority. Re-enter the cone while it is searching to trigger immediate pursuit.
10. Pause permits preview but suppresses noise and movement commands. Stop/Play resets the probe and guard state.

The same controls work in a standalone game after selecting AI Perception & Brains from the scene browser. This scene uses a fixed overview camera and a free pointer. Ordinary movement scenes still request capture in standalone builds; Settings and modal dialogs temporarily release it. In editor Play, keep the game camera possessed for game interaction; editor camera detachment disables the in-game world probe.

The generic Session card is hidden in this scene so its guide and controls do not compete with the hearing panel. Panel backgrounds, controls, Settings, modal dialogs, and editor chrome do not emit noise. Rays use the camera behind the HUD and viewport-local UI points, so editor dock offsets and display/UI scaling do not shift the hit.

The AI demos' cursor theme shows cyan sonar rings for Footstep and an amber burst for Impact over a valid world hit, with a hand over buttons. The centre marks the click location. The theme is installed on entering an AI demo and cleared when leaving it; other scenes use native defaults. Changing cursor appearance does not change capture or click ownership.

The blue guard hears up to 6 m; the red guard up to 11 m. Blue accepts Footstep (category bit 0) and Impact (category bit 1) with mask 3; red accepts only Impact with mask 2. Use Impact to compare range, then Footstep to test filtering. Noise radius limits propagation independently of the listener's hearing range.

Hearing uses 3D distance between the stimulus and the guard's actor position, with an inclusive boundary:

heard = accepted category AND distance <= min(noise radius, guard hearing range)

The floor ring visualizes horizontal propagation from the source. The guard's sphere visualizes its hearing range. Their visible intersection with the floor differs slightly because a standing guard's position is above the floor. Walls do not attenuate this first hearing model. Viewport placement uses the existing physics raycast; mesh/hull geometry may be approximate when the physics snapshot is used.

With target detection disabled, the guards deliberately share the same patrol offsets and speed, so their initial motion matches. Their different hearing ranges are the comparison: a noise can interrupt one while the other continues patrolling.

Author the behavior

Add the exported GuardAI component from hikari-games/example/src/entities/ai_guard_entity.zig to an actor with Physics and a character controller. The sample ai_guard archetype also supplies rendering. The component is attachable; the AI kernel has no dependency on that archetype.

Waypoints are offsets from the guard's initial position, so duplicating a guard moves its patrol with it. The three offsets are waypoint_a, waypoint_b, and waypoint_c. Movement uses the existing CharacterMotor following cached routes from hi.ai.navigation. The planner operates in XZ and keeps clearance from scene-authored boxes; elevation remains the motor's responsibility.

The guards share assets/brains/guard.brain.json, assigned through the component's Brain asset picker. Its states, completion/failure transitions, ordered fact conditions, and search-duration default are edited in the Brain asset editor and cooked by Shinra. Per-actor fields supply patrol offsets, speed, hearing range, and an optional search_seconds override (−1 inherits the asset). See Brain assets for authoring and the two-guard reuse check.

The shared program follows patrol → investigate noise → look around → return, or visual pursuit → last seen → search → return. Vision takes priority over noise through authored conditions. Movement failures take the state's failure destination; old failure reports cannot change a newer task activation. Invalid assets or missing guard bindings stop the brain and appear in brain_status.

Configuration is authored in Edit; Play inspection is read-only. enabled provides a component-level simulation switch. Actor disable cancels execution and zeros movement; re-enable restarts the Brain entry state. active_state_name, transition_name, and bounded decision_history explain the current behavior. Saved Brain changes apply to a new Play World.

Navigation and failure feedback

The example copies geometry when a move activation begins. Exactly one active floor carries ai_navigation_floor; up to 16 tagged ai_navigation_obstacle actors supply blockers. Both tags must be declared in configs/gameplay_tags.json; World discards undeclared scene tags. Both use their existing Physics box dimensions and world scale, so editing the collider changes the next planned route. The adapter explicitly requires unparented, unrotated static boxes. Invalid geometry and excess capacity fail closed instead of silently omitting blockers. Obstacles represent full-height barriers in this flat model.

The SDK planner borrows a floor rectangle and obstacle rectangles, expands obstacles and shrinks floor bounds by clearance, and searches visible corners with Dijkstra. The sample clearance is the scaled capsule radius plus 0.15 m, with an additional 0.08 m corner margin. Storage is bounded at 66 graph points; neither planning nor following allocates. Search work is bounded by the small obstacle cap, and direct routes take a visibility fast path. This is a small-arena implementation, not a population scheduler or navmesh service; large-population performance has not been measured.

Each guard retains only its route values and activation serial. An accepted noise, visual interrupt, or disable cancels that route. Static destinations plan once per move activation. Pursuit may refresh its route at most once every 0.35 seconds when the sampled goal moves over 0.5 m, rather than planning every tick. Corners have a tighter arrival tolerance than the final task target. Two seconds without at least 5 cm of progress toward the current corner reports stalled; the timer uses simulation time and freezes with Pause. Failed movement zeros the motor and takes the authored failure transition. Unreachable patrol points are skipped, with at most one failure transition per tick.

The panel and Play inspector expose moving, arrived, and unreachable, the last completed movement result, and the last failure reason. The failure reason remains visible after recovery. route_points and route_plans expose route size and replanning count. Cyan segments show the remaining route during movement. A return may begin on the next tick after failure, so use the retained failure reason when inspecting recovery.

Edit obstacle geometry before Play. Cached routes do not rebuild for moving walls; unexpected runtime blockage reports stalled. Untagged colliders still physically collide but do not participate in route planning. There is no dynamic crowd avoidance, slope traversal, sound occlusion, or arbitrary mesh navigation in this slice. Clicking a wall top projects the target into its blocked XZ footprint and reports unreachable.

Vision, memory, and game priorities

hi.ai.perception.Vision supplies a 3D range/cone test and a filtered physics segment query. geometry() returns geometric admission only; query() additionally tests occlusion. The nearest non-sensor collider blocks sight unless it belongs to the target. The observer is excluded, and occlusion_mask selects collision categories. A clear segment does not require a target collider hit. Physics uses the existing snapshot-first query API, so results inherit its published-pose timing and approximation limits. Render transparency does not imply visibility through a collider.

hi.ai.perception.Memory stores a weak actor handle, copied last observed position, age, and known/visible flags. remember() is called only for a positive observation. miss() clears visibility while retaining the previous position. advance() consumes simulation time; no pointer escapes and Pause does not age memory. The guard keeps its selected target in component transient state, while the World-scoped service retains the bounded set of observations used to select that target. Neither stores a process-global actor registry.

The shared perception service discovers candidates spatially from active ai_target actors for each ai_observer, aiming for a 10 Hz observation cycle within its configured work budget. It applies range, category, and cone admission before spending a raycast. There is no hardcoded player lookup in guard sensing. The eye and target sample are 0.5 m above their actor centres. Vision defaults to 12 m, with a 55° half-angle. This is one sample point, not full-body visibility. The yellow horizontal cone section turns red while a target is visible; the pink point is the last observed actor position. show_vision controls the game-owned overlay. Live inspection exposes sight_result, target_visible, target_known, last_seen_position, last_seen_age, and sight_samples.

The shared guard Brain authors this priority: visible target → remembered visual investigation/search → noise → patrol. Seeing a target interrupts lower-priority tasks. Loss of sight immediately redirects to the remembered point, never the unseen target's live position. If that observation becomes older than memory_seconds (default 8), the guard searches from its current location instead of continuing the route. It rotates for the resolved search_seconds duration, clears its local pursuit memory, and returns to patrol; shared observations remain available until their retention expires. Reacquisition interrupts either last-seen movement or search. Pursuit/path failure also enters search, with a one-second retry delay to bound retries while an unreachable target remains visible. There is no attack/damage behavior in this test.

The green ai_target actor reuses the same motor and route planner at 4 m/s. Move player commands travel through the ordinary World event dispatcher. A new click replaces the route; blocked destinations report Unreachable. Movement is silent so hearing and vision can be tested independently. Targets detectable controls candidate admission for the player and both scouts. Removing a target from the authoritative candidate snapshot prunes its shared memory; the next delivered observation clears or changes the guard's focus. Cached visibility also expires after 0.25 seconds without a fresh check. For isolated hearing checks, disable it and wait for search/return, or restart Play and then disable it. The player tag identifies the directly controlled actor; ai_target identifies all detectable actors. Both tags are declared in the gameplay catalog. Scout A and Scout B automatically move between authored offsets at 1.5 m/s without emitting noise.

Shared scheduling and load tests

hi.ai.scheduler.Service(observer_capacity, target_capacity, memory_slots) is the reusable engine service. The example instantiates it with 1,024 observers, 64 targets, and eight observations per observer. Its game adapter in ai_service.zig creates separate storage keyed by WorldScope.id, destroys it on detach, and clears the active World's state before scene unload. The host calls this adapter after gameplay joins; no runtime code imports the editor. All inputs and callbacks are borrowed for one tick, and the service retains copied values and generation-safe handles only.

Each tick receives authoritative observer and target snapshots. Stable handle ordering preserves scheduling progress; removed/replaced handles and changed observer activation IDs discard old observations and selection state. Stable input identity ordering reuses the sorted index mapping and updates observer memory in place; cohort changes merge retained rows by handle. Remembered subjects use indexed lookup, and candidate continuation selects the next handle without sorting candidate lists. Targets enter a shared hashed XZ grid with 16 m cells; very large ranges use a bounded linear fallback. Candidate admission also checks 3D distance and game-authored category masks (guard_ai.target_mask against ai_target.category). Snapshot validation rejects duplicates, invalid input, and excess capacity explicitly rather than silently truncating work.

The scheduler rotates through observers and through their nearby candidates. One visit evaluates one candidate, with at most one visit per observer per frame. A full candidate cycle waits until its nominal 0.1-second period has elapsed; overload continues on subsequent frames instead of catching up in a burst. Steps bound candidate evaluations/publications and rays independently bound occlusion queries. Neither sorting, spatial lookup, memory maintenance, nor callbacks allocate during the tick. Snapshot synchronization and game-side actor-field gathering still have population-dependent cost; the budgets are work-count caps, not a hard wall-clock deadline. Fairness assumes nonzero budgets and a stable active cohort; continuously changing candidate sets can change revisit intervals.

The game supplies target selection: retain a recently visible focus, otherwise select the nearest recently visible target, then the most recently seen remembered target. This policy lives in ai_guard_policy.zig, outside the engine scheduler. Known positions change only after a positive observation. Visibility expires after 0.25 seconds without a positive check; remembered knowledge expires at the guard's memory_seconds. Full memory slots evict the oldest checked observation, counted by Metrics.evictions.

The adapter sends the selected observation through the ordinary World event dispatcher. A four-argument game protocol carries position, ages, and packed sight/activation flags; the weak subject is Message.from. Restart/disable advances the guard's perception epoch so delayed messages cannot revive an old activation. Delivery occurs at the next dispatcher opportunity, adding up to a normal gameplay tick to the service's sensing delay. Dispatcher saturation is best-effort; guards expire stale visibility even if a result is dropped. This service budgets perception; guard movement, navigation planning, rendering, and physics remain outside that budget.

In Diagnostics → AI Sensing Scale → Play, use Observer count → 0 / 10 / 100 / 1000. The dedicated scene starts with 100 stationary observers, two moving scouts and one controllable target. All observers use sensor bodies and disable cone drawing; there are no full-behavior guards or swarm members. Creation/removal is limited to 16 actors per simulation frame. Reducing the count retires fixture-owned actors. Stop or scene unload removes them with their scene layer. These controls do not appear in the other AI demos.

Perception budget → Low / Normal / High sets steps/rays to 16/4, 128/32, or 1024/1024 per frame. The panel reports observer/target counts, checks, rays, deferred ready observers, oldest cached check, and adapter CPU milliseconds (including actor gathering, service work, and event submission, excluding later delivery, movement and physics). Metrics.max_wait tracks the largest time since an observer was serviced. The guard inspector adds observation age and its selected target handle; the panel shows the target's name. Under severe pressure a stale visible target becomes remembered rather than remaining visible indefinitely.

Reproduce the headless benchmark from src/hikari:

zig build ai-perception-bench -Dgame-src=../../../hikari-games/example/src -Doptimize=fast

It runs 10, 100, and 1,000 observer snapshots against eight moving targets for 600 simulated 60 Hz frames, at Normal and High budgets. It fails on exceeded budgets or a starved observer and reports mean/p95 service CPU time, worst first detection, maximum service gap, oldest check, ray counts, minimum deliveries, and fixed storage size. Occlusion uses an analytic box; these measurements exclude Tenkai, actor gathering, event dispatch, rendering, and navigation. See measured results. Navmesh navigation, multi-threaded AI jobs, and general decision scheduling are not supported.

Emit a stimulus from game code

const hi = @import("hikari_game");

const noise: hi.ai.Stimulus = .{
    .position = .{ 4, 0, -2 },
    .radius = 8,
    .category = 0,
};
_ = noise.emit();

Call on the World owner thread. emit validates the payload and broadcasts hikari.ai.stimulus through the existing World event dispatcher. Its boolean reports payload validity, not guaranteed delivery under dispatcher saturation. Listener components decode with Stimulus.fromMessage, filter with heardBy, then interrupt their machine. No actor/context pointers are retained. The sample shows brief diagnostics for stimuli; production games choose their own diagnostic presentation.

hi.ai.Machine borrows Program.states and target slices only during calls. It stores a state index, timer, activation serial, enabled flag, and transition reason. There are no allocations, jobs, asynchronous receipts, or service ownership in this initial kernel. The host/game module's existing component execution and lifecycle rules still apply.

For concrete Zig examples, see Run an asset from Zig and Author a Brain entirely in Zig. Both paths use the same task executor; asset-backed behavior also supports editor authoring.

Check your integration

Run python3 ../hikari-games/example/tools/test_ai_scene.py from the engine root to validate the example scene's tags, observer/target setup, navigation membership, and box geometry. Reopen the project after changing its tag catalog so newly created Worlds read the updated declarations.

Use AI Sensing Scale → Observer count and perception quality controls to see the query budget in action. More observers can increase deferred work and observation age without increasing the per-frame query ceiling. Reducing the population lets that work drain. Pause stops simulation and sensing; Stop followed by Play starts fresh observations.

Build and headless-test results do not establish visual or gameplay correctness. Test your own geometry, tag catalog, and desired pursuit behavior in Play as well.

Swarm populations

The same Brain executor also runs in swarm rows. See Brain execution for swarm populations for compact state, shared asset binding, mixed actor/swarm sensing, bounded decisions, and the moving 1,000-member acceptance scene.

Gameplay interactions

Gameplay AI tasks adds registered actions with typed parameters, capabilities, cancellation and safe delayed completion. Tutorials → AI Gameplay Tasks isolates one guard and a door for success, locked-door failure and sight interruption.