CS 143 Introduction to Computer Vision

Fall 2011, MWF 11:00 to 11:50, CIT 368.
Instructor: James Hays

TAs: Evan Wallace (HTA), Sam Birch, Paul Sastrasinh, Libin "Geoffrey" Sun, and Vazheh Moussavi.


Computer Vision, art by kirkh.deviantart.com

Course Description

Course Catalog Entry
How can computers understand the visual world of humans? This course treats vision as a process of inference from noisy and uncertain data and emphasizes probabilistic, statistical, data-driven approaches. Topics include image processing; segmentation, grouping, and boundary detection; recognition and detection; motion estimation and structure from motion. This offering of CS 143 will emphasize the core vision task of recognition in particular. We will train and evaluate classifiers to recognize various visual phenomena.

The course will consist of five programming projects, two written quizzes, and a self-chosen final project. Students can earn graduate credit for the course but will need to meet higher requirements on all projects throughout the semester and need the instructor's permission. This course can satisfy the graduate A.I area requirement.

Prerequisites

This course requires programming experience as well as linear algebra, basic calculus, and basic probability. Previous knowledge of visual computing will be helpful. The following courses (or equivalent courses at other institutions) are helpful prerequisites: Some of the course topics overlap with these related courses, but none of the assignments will.

Assignments

Winning projects

All Results

Hybrid images with Laplacian pyramids Andy Loomis, Emanuel Zgraggen, Dylan Field Project 1 results
pB Lite: boundary detection Paul Sastrasinh, Li Sun, Hang Su Project 2 results
Scene recognition with bag of words Paul Sastrasinh, Chen Xu, Yun Zhang Project 3 results
Face detection with a sliding window Emanuel Zgraggen, Hang Su, Paul Sastrasinh Project 4 results
Tracking and Structure from Motion or ... Hang Su Project 5 results
Your choice for final project Seth Goldenberg, Emanuel Zgraggen Final project results
It is strongly recommended that all projects be completed in Matlab. All starter code will be provided for Matlab. Students may implement projects through other means but it will generally be more difficult.

Textbook

Readings will be assigned in "Computer Vision: Algorithms and Applications" by Richard Szeliski. The book is available for free online or available for purchase.

Grading

Your final grade will be made up from You have three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

Graduate credit is available and each project will specifiy the minimum requirements to earn such credit.

Contact Info and Office Hours:

You can contact the professor or TA staff with any of the following: James' office hours will be held in his office (CIT 445). TA office hours will be held in the Brindy Bowl (CIT 271).

Tentative Syllabus

Class Date Topic Slides Reading Projects
W, Sept 7th Introduction to computer vision .ppt, .pdf Szeliski 1
Image Formation and Filtering
F, Sep 9th Cameras and optics .ppt, .pdf Szeliski 2.1, especially 2.1.5 Project 1 out
M, Sep 12th Light and color .ppt, .pdf Szeliski 2.2 and 2.3
W, Sep 14th Pixels and image filters .ppt, .pdf Szeliski 3.2
F, Sep 16th Thinking in frequency .ppt, .pdf Szeliski 3.4
M, Sep 19th Image pyramids and applications .ppt, .pdf Szeliski 3.5.2 and 8.1.1
Machine Learning Crash Course
W, Sep 21st Machine learning: overview .ppt, .pdf
F, Sep 23rd Machine learning: clustering .ppt, .pdf Szeliski 5.3
M, Sep 26th Machine learning: classification .ppt, .pdf Project 1 due
Grouping and Fitting
W, Sep 28th Edge detection and line fitting w/ Hough transform .ppt, .pdf Szeliski 4.2 Project 2 out
F, Sep 30th Robust fitting (Hough Transform) .ppt, .pdf Szeliski 4.3
M, Oct 3rd Robust fitting (RANSAC and others) .ppt, .pdf Szeliski 4.3
W, Oct 5th Mixture of Gaussians and EM .ppt, .pdf
F, Oct 7th Gestalt cues, MRFs, and graph cuts .ppt, .pdf Szeliski 5.5
M, Oct 10th No classes Project 2 due
Recognition
W, Oct 12th Recognition Overview and History .ppt, .pdf Szeliski 14 Project 3 out
F, Oct 14th Image features and bag of words models .ppt, .pdf Szeliski 4.1.2, 14.4.1, and 14.3.2
M, Oct 17th Interest points: corners .ppt, .pdf Szeliski 4.1.1
W, Oct 19th Quiz 1
F, Oct 21st Interest points and instance recognition .ppt, .pdf Szeliski 14.3
M, Oct 24th Large-scale instance recognition .ppt, .pdf Szeliski 14.3.2 Project 3 due
W, Oct 26th Detection with sliding windows .ppt, .pdf Szeliski 14.1
F, Oct 28th Guest talk: Jim Rehg, Behavior Imaging and the Study of Autism
M, Oct 31st Detection with sliding windows continued .ppt, .pdf Szeliski 14.2 Project 4 out
W, Nov 2nd Context and Spatial Layout .ppt, .pdf Szeliski 14.5
F, Nov 4th Guest talk: Gabriel Taubin, 3d photography
Multiple Views and Motion
M, Nov 7th Feature Tracking .ppt, .pdf Szeliski 4.1.4
W, Nov 9th Optical Flow see above Szeliski 8.4
F, Nov 11th Guest lecture: Deqing Sun, Optical flow Project 4 due
M, Nov 14th Epipolar Geometry .ppt, .pdf Szeliski 11
W, Nov 16th Stereo Correspondence .ppt, .pdf Project 5 out
F, Nov 18th Structure from Motion .ppt, .pdf Szeliski 7 Final Project out
Advanced Topics
M, Nov 21st Activity Recognition .ppt, .pdf
W, Nov 23rd No classes
F, Nov 25th No classes
M, Nov 28th Internet Scale Vision .ppt, .pdf
W, Nov 30th Guest lecture: Pedro Felzenszwalb, Object Detection .pdf
F, Dec 2nd Crowdsourcing .ppt, .pdf
M, Dec 5th Attributes and Course Summary .ppt, .pdf
W, Dec 7th Quiz 2
F, Dec 9th No classes, reading period
M, Dec 12th No classes, reading period Final Project / Project 5 due
T, Dec 13th, 9:00 AM Exam Period - final presentations

Acknowledgements

The materials from this class rely significantly on slides prepared by other instructors, especially Derek Hoiem and Svetlana Lazebnik. Each slide set and assigment contains acknowledgements. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.

Previous Versions of Course

Michael Black's most recent offering of CS 143 can be found here