Rough-Terrain Path Tracking with Hybrid Deep Reinforcement Learning

The problem

A path-tracking controller tuned on flat ground gets into trouble the moment the ground stops cooperating. Slopes change the effective steering response, loose surfaces break the tire model, and the vehicle ends up cutting corners or oversteering. Retuning per surface doesn’t scale, and handing the whole job to a learned policy trades one problem for another: you lose the structure that made the classical controller predictable in the first place.

What I built

A hybrid framework: a model-based controller keeps its role as the stable backbone, and a deep reinforcement learning policy learns the corrections the model can’t express. The learned part handles what varies, whether that’s terrain, traction or geometry, while the model-based part keeps behaviour bounded and legible.

Choosing that split wasn’t obvious, so I evaluated policy architectures across several deep reinforcement learning frameworks rather than committing to one and hoping.

Building terrain worth training on

The policy is only as good as the ground it learned on, so the terrain wasn’t an afterthought. I authored the 3D rough-terrain assets in Blender and exported them through OpenUSD into the simulator, which meant I could define repeatable test protocols instead of eyeballing whether two runs were comparable.

The real-world test

The framework was validated on multiple unseen real-world tracks under that same repeatable protocol, reaching 98.5% precision in the real world. Below is one of those runs, with the vehicle tracking its path across an actual grass slope, not a rendered one.

Where it landed

Published at the 2024 IEEE International Conference on Advanced Intelligent Mechatronics (pp. 685–690). The work also became the core of my dissertation, Hybrid Learning for Rough Terrain Navigation of Actively Articulated Wheeled Vehicles.