About the Project
University of Maryland - Spring 2026
Course: ENPM 673 Perception for Autonomous Robots (final project)
Recovering a camera's trajectory from a single video stream. Tuning keyframe selection and preprocessing cut closed-loop drift 4.8x, to 2.7% (5.4 cm end-point error on a 2.0 m loop). I recorded a four-wheel skid-steer robot driving a 0.5 m closed square with a forward-facing Raspberry Pi camera (1920x1080, 30 fps), then built and compared two pose-recovery pipelines in Python and OpenCV: pairwise visual odometry that chains essential-matrix poses between consecutive keyframes, and a local-map SLAM pipeline that solves absolute pose with PnP-RANSAC against an evolving 3D landmark cloud. The known 2.0 m loop perimeter anchors the metric scale that monocular geometry cannot observe.
Project Highlights
- Camera calibration from 149 chessboard images (0.59 px RMS), with k3 fixed after finding that a free k3 overfit and extrapolated poorly at the image edges
- ORB features with Lowe's ratio test, 5-point essential-matrix RANSAC, and relative pose recovery between keyframes
- Three fixes to naive every-frame integration: rotation-only updates during in-place pivots, holding a keyframe across motion-blur bursts, and weighting translations by the frame gap they span
- Local-map SLAM with landmark triangulation, parallax and reprojection gates, landmark aging, map reset after pivots, and an essential-matrix fallback when PnP has too few inliers
- Controlled 2x2 comparison (pairwise vs. SLAM, every-frame vs. long-baseline keyframes) showing the 4.8x drift reduction came from the keyframe-selection and preprocessing settings, not the pose-recovery architecture
- Three-panel visualization video synchronizing the camera view with ORB keypoints and both estimated trajectories
Results
Closed-loop drift on the 2.0 m loop: pairwise VO went from 12.9% (25.7 cm) with every-frame keyframes to 2.7% (5.4 cm end-point error) with long-baseline keyframe selection and retuned undistortion preprocessing. Local-map SLAM went from 12.4% to 3.6% on the same inputs. PnP fell back to essential-matrix integration on 76% of pairs because ORB matches against the local map did not survive the longer baselines, which points to the descriptor frontend as the next thing to improve.
Technologies Used
- Python 3, NumPy
- OpenCV (calibrateCamera, ORB, findEssentialMat, recoverPose, solvePnPRansac)
- Jupyter
- Raspberry Pi camera on a skid-steer robot