NVIDIA collaboration
Real-Time 3D Scene Reconstruction with NVIDIA Isaac ROS Nvblox
Getting a GPU-accelerated reconstruction stack running end to end, in simulation and on the robot, and then writing down how so the next person doesn't have to rediscover it.
The problem
Nvblox computes a live 3D reconstruction and the 2D costmap a navigation stack needs, on the GPU. The capability is excellent; the distance between “the package exists” and “it runs on your robot” is where most of the time goes: drivers, transforms, camera calibration, the mismatch between what works in simulation and what the hardware does.
What I built
A deployment pipeline covering both halves. In simulation, a test arena authored in Isaac Sim so the reconstruction had something structured to build against. On hardware, a modified TurtleBot3 Burger carrying an Intel RealSense depth camera and a Jetson Orin Nano doing the GPU-accelerated inference, with the whole stack driven through ROS 2 in Python.
Working with NVIDIA
This was a cross-organization collaboration with NVIDIA’s robotics team on integration and hardware bring-up. Working that way, where the person who wrote the package and the person deploying it have to agree on what “working” means, taught me as much about communication as about the stack.
The write-up
I published a complete step-by-step guide so the setup is reproducible by anyone with the same hardware. It’s the piece of this project that has been most useful to other people, which I think says something about which parts of research work are actually worth the effort.
More projects
See all →Current work
Simulation, Learning and Edge AI at Arrive AI
Digital twin simulation, reinforcement learning, vision-language-action models, and the edge inference stack that has to run modern models fast enough to be useful.
ICRA 2025 Coordinated Payload Transport with Biped-Wheeled Robots
One reinforcement learning agent driving two balancing robots to carry a shared payload, trained in Isaac Lab and transferred to hardware with no fine-tuning.
IEEE AIM 2024 Rough-Terrain Path Tracking with Hybrid Deep Reinforcement Learning
A model-based controller and a learned policy working together, so an Ackermann-steered vehicle tracks a path across terrain neither was tuned for.