Isaac Bi3D Freespace Segmentation on HPC and Hardware
Taking a freespace segmentation model from a single workstation to cluster-scale evaluation, and from a rendered off-road scene to an actual skid-steer robot.
The problem
Freespace segmentation tells a robot where it can actually drive, which matters much more off-road than on a paved surface where the answer is usually “the road.” Evaluating it properly means many runs across many scenarios, and that doesn’t fit on one machine.
What I did
Implemented NVIDIA’s Isaac Bi3D freespace segmentation on Clemson’s Palmetto HPC cluster, scaling evaluation across L40 GPUs so the study could cover more scenarios than a single workstation allows. The off-road simulation scenario was authored in Isaac Sim and visualized through the Omniverse streaming client — which is what makes cluster-side rendering usable at all, since the GPUs doing the work are nowhere near your desk.
The physics detail worth knowing
Deploying to a Clearpath Husky with a stereo depth camera surfaced a simulation question that has nothing to do with perception: skid-steer vehicles turn by deliberately violating the no-slip assumption, so how the solver handles contact matters. I quantified the trade-offs between PhysX’s PGS and TGS solvers for skid-steer fidelity — a choice that silently decides whether your simulated robot turns like the real one.
Credit
This was a group effort in the ARM Lab; Ameya Salvi led the work and I contributed the HPC deployment, the off-road scenario and the solver study.
More projects
See all →Current work
Digital Twins and Edge Autonomy at Arrive AI
An autonomy stack combining RL policies with vision-language-action models, the edge inference that runs it on Jetson hardware, and a digital twin to test it before deployment.
ICRA 2025 Coordinated Payload Transport with Biped-Wheeled Robots
One reinforcement learning agent driving two balancing robots to carry a shared payload — trained in Isaac Lab, transferred to hardware with no fine-tuning.
IEEE/ASME AIM 2024 Rough-Terrain Path Tracking with Hybrid Deep Reinforcement Learning
A model-based controller and a learned policy working together, so an Ackermann-steered vehicle tracks a path across terrain neither was tuned for.