CSCI 1430: Introduction to Computer Vision

Fall 2017, MWF 13:00 to 13:50, Metcalf Friedman Auditorium

Instructor: James Tompkin
TAs: Aaron Gokaslan (HTA), Spencer Boyum, Harsh Chandra, Joshua Chipman, Qikun (Tim) Guo, Yue Guo, Yunshu Mao, Luke Murray, Eleanor Tursman, Vivek Ramanujan.

Waitlist | Schedule | General Course Policy

Computer Vision, art by HarperCollins / Clare Skeats / James Tompkin

Contact

Office/Piazza Hours

Course Description

This course provides an introduction to computer vision, including fundamentals of image formation, camera imaging geometry, feature detection and matching, stereo, motion estimation and tracking, image classification, scene understanding, and deep learning with neural networks. We will develop basic methods for applications that include finding known models in images, depth recovery from stereo, camera calibration, image stabilization, automated alignment, tracking, boundary detection, and recognition. We will develop the intuitions and mathematics of the methods in class, and then learn about the difference between theory and practice in projects.

This course is based upon James Hays' computer vision course, previously taught at Brown as CS143, and currently taught at Georgia Tech as CS 4476. Significant thanks to him and his staff, across the years, for all their hard work.

Learning Objectives

Upon completion of this course, students will:
  1. Be familiar with both the theoretical and practical aspects of computing with images;
  2. Have described the foundation of image formation, measurement, and analysis;
  3. Have implemented common methods for robust image matching and alignment;
  4. Understand the geometric relationships between 2D images and the 3D world.
  5. Have gained exposure to object and scene recognition and categorization from images;
  6. Grasp the principles of state-of-the-art deep neural networks; and
  7. Developed the practical skills necessary to build computer vision applications.

Prerequisites

No prior experience with computer vision is assumed, although previous knowledge of visual computing or signal processing will be helpful (e.g., CSCI 1230). The following skills are necessary for this class:

Time Commitment

Task Hours
In class 40
Projects: 20 ×6
Office Hours/Piazza:    20
Total: 180

Capstone

This class can be taken as a capstone. You will need to complete 10 points of extra credit in each of the projects.

Textbook

There is no requirement to buy a textbook. The goal of the course is to be self contained, but sections from two textbooks will be suggested for more formalization and information. These two books are available free online. If you find a word or concept that you do not understand, then please also consider the computer vision dictionary listed third.

  1. Computer Vision: Algorithms and Applications by Richard Szeliski.
  2. Deep Learning, by Goodfellow, Bengio, and Courville.
  3. Dictionary of Computer Vision and Image Processing, by Fisher et al. Note: Full text is available in 'Online Resources' section.

Software

Projects

Projects are released every two weeks, with deliverables due each week at Friday at 9pm. Each project has two parts: written, and code. You have one week to complete the written part, and two weeks to complete the code part.

Hand-in for both parts is electronic via Gradescope. Instructions coming soon.

Projects 1, 2, 3 and 5 must be completed in MATLAB. Project 4 must be completed in TBD.

Project 6 will present a team challenge. More details to come.

Projects Instructions Written Qs Gradescope Hand in
0. MATLAB Primer PDF LaTeX 15th Sept. 9pm
1. Image Filtering and Hybrid Images Webpage LaTeX Written: 15th Sept. 9pm; Code: 22nd Sept. 9pm
2. Local Feature Matching
3. Scene Recognition with Bag of Words
4. Convolutional Neural Nets
5. Camera Calibration and Fundamental Matrix Estimation
6. Group Project

Grading

Your final grade will be 100% from 7 projects. All projects are graded. We leave ourselves a little flexibility to make minor adjustments. Say, if one project ends up being a little more difficult, then we can tweak that project to be less significant in your final grade.

Project Percent
0 2%
1–5 ~14.6%
6 ~25%

Late Submissions

You will lose 10% from each project for each day that it is late. However, you have three late days for the whole course: the first 24 hours after the due date and time counts as one late day, up to 48 hours counts as two, and 72 hours counts as three. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible. Late days cover unexpected clustering of due dates, travel commitments, interviews, hackathons, etc. Do not ask for extensions to due dates—we give you a pool of late days to manage yourself.

Tentative Schedule

Date Topic Slides More info Projects
Wed 06 Sep Introduction to Computer Vision PPTX | PDF Szeliski 1 0 out
Image Formation and Filtering
Fri 08 Sep Light and Color PPTX | PDF Szeliski 2.2, 2.3 1 out
Mon 11 Sep Image Filtering PPTX | PDF Szeliski 3.2
Wed 13 Sep Thinking in Frequency PPTX | PDF Szeliski 3.4
Fri 15 Sep Thinking in Frequency, part 2 PPTX | PDF Szeliski 3.5.2, 8.1.1 0 due; 1 written due
Feature Detection and Matching
Mon 18 Sep Thinking in Frequency, part 3 PPTX | PDF Szeliski 3.5.2, 8.1.1
Wed 20 Sep Edge Detection PPTX | PDF Szeliski 4.2
Fri 22 Sep Interest Points and Corners Szeliski 4.1.2 1 code due, 2 out
Mon 25 Sep Local Image Features and Feature Matching Szeliski 4.1.3, 4.3.2
Machine Learning Crash Course
Wed 27 Sep Machine Learning: Unsupervised Learning Szeliski 5.3
Fri 29 Sep Machine Learning: Supervised Learning Szeliski 5.3 2 written due
Recognition
Mon 02 Oct Recognition Overview and Bag of Features Szeliski 14
Wed 04 Oct Large-scale Instance Recognition Szeliski 14.3.2
Fri 06 Oct Large-scale Category Recognition and Advanced Feature Encoding 2 code due; 3 out
Mon 09 Oct No class—Indigenous Peoples' Day
Wed 11 Oct Detection with Sliding Windows: Viola Jones Szeliski 14.1 and 14.2
Fri 13 Oct Detection with Sliding Windows: Dalal Triggs Szeliski 14.1 3 written due
Mon 16 Oct Pascal VOC and Big Data Szeliski 14.5
Wed 18 Oct Big Data 2
Fri 20 Oct Project Explanation +
Time Warp Design Challenge
3 code due
Deep Learning
Mon 23 Oct Neural Networks Goodfellow 6
Wed 25 Oct Neural Networks Part 2 Goodfellow 6
Fri 27 Oct Convolutional Networks for Recognition Goodfellow 9 4 out; form project teams
Mon 30 Oct Neural Network Regularization Goodfellow 7.1-7.5, 7.12
Wed 01 Nov R-CNNs and FCNs
Fri 03 Nov We Have To Go Deeper 4 written due
Mon 06 Nov Exploring Architectures
Wed 08 Nov Social Good and Dataset Bias
Cameras, Multiple Views, and Motion
Fri 10 Nov Model Fitting and RANSAC Szeliski 6.1, 2.1 4 code due; project brief due; 5 out
Mon 13 Nov Cameras and Optics Szeliski 2.1, esp. 2.1.5
Wed 15 Nov Stereo Introduction Szeliski 11
Fri 17 Nov Camera Calibration Szeliski 6.2.1 5 written due; project start
Thanksgiving Week
Mon 20 Nov No class—project work
Wed 22 Nov Thanksgiving
Fri 24 Nov Thanksgiving
Cameras, Multiple Views, and Motion Continued
Mon 27 Nov Epipolar Geometry and Structure from Motion Szeliski 7
Wed 29 Nov Epipolar Geometry and Structure from Motion Continued Szeliski 8.1 and 8.4
Fri 01 Dec Stereo Disparity and Optical Flow 5 code due
'Reading period'
Mon 04 Dec Research paper class
Wed 06 Dec Research paper class
Fri 08 Dec Research paper class
Mon 11 Dec Projects fast forward!
Tue 12 Dec Projects due

General Policy

Welcome!

Our intent is that this course provide a welcoming environment for all students who satisfy the prerequisites. Our TAs have undergone training in diversity and inclusion, and all members of the CS community, including faculty and staff, are expected to treat one another in a professional manner. If you feel you have not been treated in a professional manner by any of the course staff, please contact any of James (the instructor), Ugur Cetintemel (Dept. Chair), Tom Doeppner (Vice Chair) or Laura Dobler (diversity and inclusion staff member). We will take all complaints about unprofessional behavior seriously. Your suggestions are encouraged and appreciated. Please let James know of ways to improve the effectiveness of the course for you personally, or for other students or student groups. To access student support services and resources, and to learn more about diversity and inclusion in CS, please visit http://cs.brown.edu/about/diversity/resources/.

Prof. Krishnamurthi has good notes on this area.

Quiet Hours

This class runs quiet hours from 9pm to 9am every day. Please do not expect a response from us via any channel. Likewise, we won't ask you to do anything between these times, either, like hand in projects.

Academic Integrity, Collaboration, and Citation

Feel free to talk to your friends about the concepts in the projects, and work through the ideas behind problems together, but be sure to always write your own code and perform your own write up. You are expected to implement the core components of each project on your own, but the extra credit opportunties often build on third party data sets or code. Feel free to include results built on other software, as long as you credit correctly in your handin and clearly demark your own work. In general, if you use an idea, text, or code from elsewhere, then cite it.

Brown-wide, academic dishonesty is not tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Brown Academic and Student Conduct Codes.

Accommodations

Brown University is committed to full inclusion of all students. Please inform me if you have a disability or other condition that might require accommodations or modification of any of these course procedures. You may email me, come to office hours, or speak with me after class, and your confidentiality is respected. We will do whatever we can to support accommodations recommended by SEAS. For more information contact Student and Employee Accessibility Services (SEAS) at 401-863-9588 or . Students in need of short-term academic advice or support can contact one of the deans in the Dean of the College office.

Mental Health

Being a student can be very stressful. If you feel you are under too much pressure or there are psychological issues that are keeping you from performing well at Brown, we encourage you to contact Brown's Counseling and Psychological Services. They provide confidential counseling and can provide notes supporting extensions on assignments for health reasons.

Incomplete Policy

We expect everyone to complete the course on time. However, we certainly understand that there may be factors beyond your control, such as health problems and family crises, that prevent you from finishing the course on time. If you feel you cannot complete the course on time, please discuss with James Tompkin the possibility of being given a grade of Incomplete for the course and setting a schedule for completing the course in the upcoming year.

Electronic Etiquette

Laptops are discouraged, please, except for class-relevant activities, e.g., to help answer questions and show items relevant to discussion. No social media, email, etc., because it distracts not just you but other students as well. Read Shirky on this issue ("Why I Just Asked My Students to Put Their Laptops Away"), or Rockmore ("The Case for Banning Laptops in the Classroom").

We will release course lecture material online. In considering laptop use for note taking, please be aware that research has shown note taking on paper to be more efficient than on a laptop keyboard (Mueller and Oppenheimer), as it pushes you to summarize the content instead of transcribe it.


Previous course runs

Acknowledgements

The materials from this class rely significantly on slides prepared by other instructors, especially James Hays, Derek Hoiem, and Svetlana Lazebnik. Each slide set and assignment contains acknowledgements. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.

Thanks to Tom Doeppner and Laura Dobler for the text on accommodation, mental health, and incomplete policy.

Thank you to the previous TAs who helped to teach and improve this class.