← Back

AI Navigation: NavMesh vs Custom System (2026)

SOLO 6 WEEKS Unity

Final Demos

Final demonstrations of the custom A* and Unity NavMesh implementations.

Custom A* Navigation

Final demonstration of the custom A* AI navigation system.

Unity NavMesh

Final demonstration of the Unity NavMesh AI navigation system.

NavMesh vs Custom A*

How much control do you gain by building an AI navigation system yourself?

This project explores that question by comparing Unity's NavMesh-based AI navigation with a custom-built A* alternative. Both systems were designed to support a minimum of 1,000 AI-controlled enemies at a stable 60 FPS.

1,000+

AI Agents Tested

60 FPS

Target Performance

2

Navigation Systems

Project Overview

Both implementations share the same underlying enemy state machine, movement, player controller and Level of Detail (LOD) systems. The key difference was the navigation layer: one relied on Unity's NavMesh system, while the other used a custom A* implementation.

The project also explored optimisation techniques for large numbers of AI agents, including squad-based navigation, AI-aware LOD and frustrum culling.

Key Takeaway

NavMesh proved to be the more practical solution for large-scale AI, while building A* demonstrated the trade-off between engine-provided performance and the flexibility of a custom system.

A* Implementation

Custom A* AI navigation system

Grid Generation

The custom navigation system generates a 2D grid across the level and uses physics checks and layer masks to determine which nodes are walkable. The grid supports eight-directional movement while preventing diagonal movement through obstacles.

Pathfinding

A* evaluates potential routes using movement cost and a distance-based heuristic before reconstructing the resulting path into world-space waypoints.

Path Request Architecture

To support large numbers of agents, I implemented a queued path-request system, allowing AI agents to submit navigation requests without directly managing the pathfinding process. This separated grid generation, pathfinding and request management into independent systems, making the navigation architecture easier to manage and optimise.

Scaling to 1,000+ AI Agents

Supporting large numbers of agents required optimisation beyond the navigation algorithms themselves.

Leader behaviour
AI squad system
AI Level of Detail system
Sprite Level of Detail
Frustum culling
Optimisation 01

Squad-based Navigation

Enemies were organised into squads, with the leader responsible for navigation requests while other members followed the leader's movement. This reduced the number of independent pathfinding and navigation checks required across the AI population.

Optimisation 02

AI-aware LOD

Enemy simulation was scaled based on distance from the player. Nearby enemies retained full AI and visual fidelity, while distant enemies transitioned through lower-cost representations. At sufficient distances, AI behaviours were suspended entirely, putting agents into a sleep state while maintaining a visually convincing scene.

Optimisation 03

Frustrum Culling

Enemy models outside the camera's view were disabled while their underlying AI logic continued to run. When they re-entered the camera's view, their visual representation was restored and the appropriate LOD was selected.

Together, these techniques reduced the cost of both AI simulation and rendering, allowing significantly larger numbers of enemies to be presented without requiring every agent to operate at full fidelity.

Outcomes & Performance

The same optimisation techniques were applied to both systems to provide a consistent comparison.

Custom A*

Navigation

1,000

AI Agents

60 FPS

After Optimisation

Before optimisation: 1,000 enemies at approximately 30 FPS.

After optimisation: 1,000 enemies at 60 FPS, with performance falling below the target when increasing the population further.

Unity NavMesh

Navigation

3,000

Agents @ 60 FPS

5,000

Agents @ 30 FPS

Before optimisation: 1,000 enemies at just under 200 FPS.

After optimisation: Over 200 FPS at 1,000 enemies.

Further stress testing supported 3,000 enemies at 60 FPS and 5,000 enemies at 30 FPS.

Performance Summary

Unity's NavMesh demonstrated substantially greater scalability in this implementation, supporting 3× as many agents at 60 FPS compared with the custom A* system.

Conclusion

The project demonstrated that both Unity's NavMesh and a custom A* implementation can provide a scalable foundation for large-scale AI systems. While the custom A* approach offered significantly greater control and customisation over navigation behaviour, it required substantially more development effort and, in this implementation, performed less efficiently than Unity's NavMesh.

For this particular use case, NavMesh was the stronger solution, providing significantly greater scalability with less development overhead. The A* implementation nevertheless provided valuable experience in designing navigation systems from the ground up and could be better suited to future projects requiring specialised navigation behaviour or greater control than an engine-provided solution allows.

Future Applications

Open-world

Strategy

Simulation

Survival

In the future, this system could be expanded for use in large-scale open-world, strategy, simulation or survival games, where large numbers of AI agents need to navigate across complex environments. Further development could explore dynamic navigation, improved squad formations and more advanced AI behaviours.

Development & Technical Demos

Development footage demonstrating the implementation and progression of the AI navigation and optimisation systems.

A* Progress #1

A* Progress #2

NavMesh Progress #1

NavMesh Progress #2

NavMesh Progress #3

NavMesh Progress #4

NavMesh Progress #5