Isaac Bi3D Freespace Segmentation on HPC and Hardware
Taking a freespace segmentation model from a single workstation to cluster-scale evaluation, and from a rendered off-road scene to an actual skid-steer robot.
The problem
Freespace segmentation tells a robot where it can actually drive, which matters much more off-road than on a paved surface where the answer is usually “the road.” Evaluating it properly means many runs across many scenarios, and that doesn’t fit on one machine.
What I did
Implemented NVIDIA’s Isaac Bi3D freespace segmentation on Clemson’s Palmetto HPC cluster, scaling evaluation across L40 GPUs so the study could cover more scenarios than a single workstation allows. The off-road simulation scenario was authored in Isaac Sim and visualized through the Omniverse streaming client, which is what makes cluster-side rendering usable at all, since the GPUs doing the work are nowhere near your desk.
The physics detail worth knowing
Deploying to a Clearpath Husky with a stereo depth camera surfaced a simulation question that has nothing to do with perception: skid-steer vehicles turn by deliberately violating the no-slip assumption, so how the solver handles contact matters. I quantified the trade-offs between PhysX’s PGS and TGS solvers for skid-steer fidelity, a choice that silently decides whether your simulated robot turns like the real one.
Credit
This was a group effort in the ARM Lab; Ameya Salvi led the work and I contributed the HPC deployment, the off-road scenario and the solver study.
More projects
See all →Current work
Simulation, Learning and Edge AI at Arrive AI
Digital twin simulation, reinforcement learning, vision-language-action models, and the edge inference stack that has to run modern models fast enough to be useful.
ICRA 2025 Coordinated Payload Transport with Biped-Wheeled Robots
One reinforcement learning agent driving two balancing robots to carry a shared payload, trained in Isaac Lab and transferred to hardware with no fine-tuning.
IEEE AIM 2024 Rough-Terrain Path Tracking with Hybrid Deep Reinforcement Learning
A model-based controller and a learned policy working together, so an Ackermann-steered vehicle tracks a path across terrain neither was tuned for.