For this project I will be reconstructing the 3D shape of an object from a series of observations.

Keypoint Selection

The image of the selected keypoints overlayed over the first video frame is:

Feature Tracking

Extra Credit: Iterative refinement to your KLT tracker added.

For 20 random keypoints, the 2D path over the sequence of frames using line segments are shown in green lines as below:

The points which have moved out of frame at some point along the sequence are shown in blue as below:

Structure from Motion

The predicted 3D locations of the tracked points for 3 different viewpoints are as below:

The predicted 3D path of the cameras are shown as below:

References

1. Shape and Motion from Image Streams under Orthography: a Factorization Method (Tomasi and Kanade 1992)
2. A Sequential Factorization Method for Recovering Shape and Motion from Image Streams, Morita and Kanade, 1997