NVIDIA collaboration
Real-Time 3D Scene Reconstruction with NVIDIA Isaac ROS Nvblox
Getting a GPU-accelerated reconstruction stack running end to end — in simulation and on the robot — and then writing down how, so the next person doesn't have to rediscover it.
The problem
Nvblox computes a live 3D reconstruction and the 2D costmap a navigation stack needs, on the GPU. The capability is excellent; the distance between “the package exists” and “it runs on your robot” is where most of the time goes — drivers, transforms, camera calibration, the mismatch between what works in simulation and what the hardware does.
What I built
A deployment pipeline covering both halves. In simulation, a test arena authored in Isaac Sim so the reconstruction had something structured to build against. On hardware, a modified TurtleBot3 Burger carrying an Intel RealSense depth camera and a Jetson Orin Nano doing the GPU-accelerated inference, with the whole stack driven through ROS 2 in Python.
Working with NVIDIA
This was a cross-organization collaboration with NVIDIA’s robotics team on integration and hardware bring-up. Working that way — where the person who wrote the package and the person deploying it have to agree on what “working” means — taught me as much about communication as about the stack.
The write-up
I published a complete step-by-step guide so the setup is reproducible by anyone with the same hardware. It’s the piece of this project that has been most useful to other people, which I think says something about which parts of research work are actually worth the effort.
More projects
See all →Current work
Digital Twins and Edge Autonomy at Arrive AI
An autonomy stack combining RL policies with vision-language-action models, the edge inference that runs it on Jetson hardware, and a digital twin to test it before deployment.
ICRA 2025 Coordinated Payload Transport with Biped-Wheeled Robots
One reinforcement learning agent driving two balancing robots to carry a shared payload — trained in Isaac Lab, transferred to hardware with no fine-tuning.
IEEE/ASME AIM 2024 Rough-Terrain Path Tracking with Hybrid Deep Reinforcement Learning
A model-based controller and a learned policy working together, so an Ackermann-steered vehicle tracks a path across terrain neither was tuned for.