Robot localization is the base for path planning and navigation, and for AR and VR. This project uses a single camera as its only sensor to build a 3D map of an entire room and localize the robot within that map, which can then be used for navigation.
// Overview
Built on OpenVSLAM and ROS, the system extracts ORB features and uses multi-view geometry and graph optimization to map the room and track the robot pose. It tackles three classic visual SLAM problems: dynamic objects (masked out with a fine-tuned ICNet segmentation model so moving people are not used as landmarks), varying lighting (light-invariant feature extraction), and low-texture scenes. Heavy computation runs on a PC over ROS while a differential-drive robot executes the navigation commands.
// Gallery
// Features
Uses one camera as the only sensor to build a 3D map of a room and localize the robot within it, built on OpenVSLAM with ORB features, multi-view geometry and graph optimization.
Fine-tuned an ICNet semantic segmentation model on a custom dataset to mask humans as dynamic obstacles so the SLAM pipeline does not treat them as valid landmarks.
Integrated the ROS Navigation Stack with an occupancy grid map and scaled odometry, with heavy computation on a PC over ROS while the robot executes navigation commands.