Syllabus
All the information you need will be posted here. This page will be updated regularly so check it often.
All data for the course is in /course/cs296-4/JFK/data
Several readings will be from Richard Szeliski's book, which is available on-line in draft form (pdf)
Sept 1: Introduction
Introduction to the class, organization, history of the Zapruder film.
Video: Image of an Assasination: A New Look at the Zapruder Film
If you missed the video presentation, you can borrow a copy to view. To borrow the DVD contact my assistant, Lori Agresti (laa@cs.brown.edu) who is in room 312. You can sign it out for a few hours. Several people will want to see this, so please try to watch it right away and get it back to Lori.
Sept 8: Background on the JFK assasination and conspiracy theories
I'll be out of town. Matt Loper will show a DVD of an ABC News special, Peter Jennings Reportingh, on the Kennedy Assassination: Beyond Conspiracy.
If you are unfamiliar with this history, this provides a good introduction. It also describes a detailed 3D reconstruction of the scene which will be of interest.
Sept 15: Analysis of conspiracy theories
Assignment 1: (10 points) Due Tuesday, Sept 14; (link)
Reading: Have a quick look at:
Secrets of a homocide, Dale Myers 3D reconstruction (website)
The report is in /course/cs296-4/soh-acoustics-report.pdf (or web link)
This is a long report and I DO NOT expect you to read it all. But may provide interesting insights, covers some photogrametric analysis (are there any problems with how it is done?), and is in the Peter Jennings Special.
Background:
Lots of stuff on the Zapruder film
Sept. 22: Deblurring (reading about it)
It all looks so easy on TV: enhance it!
Assignment 2 (10pts)
Everyone will read the intro to blur and sharpening in the Szeliki Book. Then each person will present a paper in 15 minutes. Your goal is to *TEACH* it to the rest of the class. Be clear. Prepare and practice. Use images liberally to illustrate the idea. Explain the math concisely. Be prepared to answer questions. Think max 10 slides. PDF or PPT sent to me by 11am day of class.
You'll be graded on how well you present the work.
Reading:
Everyone:
Assignment 2 (10pts). Individual presentations (now in the order I think we'll present them):Szeliski book chapters 10.1.4, 10.3
(Jason) Removing Camera Shake from a Single Photograph
Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman
http://cs.nyu.edu/~fergus/research/deblur.html(Tom) PSF Estimation using Sharp Edge Prediction
Neel Joshi, Richard Szeliski, David J. Kriegman
http://research.microsoft.com/en-us/um/redmond/groups/ivm/psfsharpedge/(Soumya) A. Levin, Y. Weiss, F. Durand, W. T. Freeman. Understanding and evaluating blind deconvolution algorithms.
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2009.
http://www.wisdom.weizmann.ac.il/~levina/(Drew) Non-uniform Deblurring for Shaken Images
Whyte, Sivic, Zisserman, Ponce, CVPR 2010
http://www.di.ens.fr/willow/research/deblurring/ (website)(Mike) High-quality Motion Deblurring from a Single Image
Qi Shan Jiaya Jia Aseem Agarwala
Siggraph 2008 (web)(Hoy - are you on the mailing list?) Single Image Deblurring Using Motion Density Functions
Ankit Gupta, Neel Joshi, C. Lawrence Zitnick, Michael Cohen, and Brian Curless
ECCV 2010
(spatially varying) (web)
Extra
Image Deblurring and Denoising using Color Priors
Neel Joshi C. Lawrence Zitnick,, Szeliki and Kriegman
http://research.microsoft.com/en-us/um/redmond/groups/ivm/imagedeblurring/
Background on camera (link)
Sept 29: Deblurring (doing it)
Finish deblurring talks
(Sung) Two-Phase Kernel Estimation for Robust Motion Deblurring
Li Xu and Jiaya Jia, ECCV 2010
(web) (talk)(Kefei) Hirsch, M., S. Sra, B. Schölkopf and S. Harmeling: Efficient Filter Flow for Space-Variant Multiframe Blind Deconvolution, CVPR 2010
does multiple blurs from a sequence.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.9207&rep=rep1&type=pdf
(Geoff) Generating Sharp Panoramas from Motion-blurred Videos
Yunpeng Li, Sing Bing Kang, Neel Joshi, Steve M. Seitz, Daniel P. Huttenlocher, CVPR'10, http://www.cs.cornell.edu/~yuli/papers/deblur.pdf
Assignment 3: (10 points) Deblur frame 158 and a high res frame of your choosing. Due 11am Wed Sept 29.
Email me a pdf with a writeup describing what you did, a comprison of different methods you tried, and how you evaluated your method. You should have a simple test case where you know the "ground truth".
Also email me your best deblurred as an uncompressed png file to the email.
If you just try a built-in Matlab function, you will get a bad grade. Start with something simple and then explore this further. There is no "right" answer. Be creative.
If you try existing code from the web, you should analyze where it fails and why. What assumption of the algorithm are violated? Write this up.
If you branch out and try something new or implementing it yourself -- kudos!
Come to class prepared to talk about your result, your approach, problems you encountered, and what you want to try next.
Oct 6:
Part 1: Optical flow with Deqin Sun.
Part 2: Camera calibration and 3d geometry with Matt Loper.
Deqing is a leading expert in optical flow estimation. His method is currently the most accurate on the Middlebury optical flow benchmark. We're going to need this for next week.
Matt worked on forensic video analysis in the Virginia murder case and is an expert on camera calibration.
Readings:
Sun, D., Roth, S., and Black, M. J.,
Secrets of optical flow estimation and their principles,
IEEE Conf. on Computer Vision and Pattern Recog., CVPR, June 2010.
(pdf) (code)Calibration and geometry, from Szeliski:
Calibration (intrinsic) 6.3
Radial distortion 6.3.5
Structure from motion 7
If you want to play with camera tracking and 3D structure estimation you can try this software:
voodoo camera tracker http://www.digilab.uni-hannover.de/docs/manual.html
Background reading for your reference:
Super detailed:
Moons, T., Van Gool, L., and Vergauwen, M. (2010). 3D reconstruction from multiple
images. Foundations and Trends in Computer Graphics and Computer Vision, 4(4).
ftp://ftp.esat.kuleuven.ac.be/psi/visics/konijn/ICVSS08/vangool.pdfTriggs, B., McLauchlan, P. F., Hartley, R. I., and Fitzgibbon, A. W. (1999). Bundle adjustment—
a modern synthesis. In International Workshop on Vision Algorithms, pp. 298–
372, Kerkyra, Greece.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.1928&rep=rep1&type=pdfLourakis, M. I. A. and Argyros, A. A. (2009). SBA: A software package for generic sparse
bundle adjustment. ACM Transactions on Mathematical Software, 36(1).
http://www.ics.forth.gr/~lourakis/sba/sba-toms.pdf
code: http://www.ics.forth.gr/~lourakis/sba/Engels, C., Stewenius, H., and Nister, D. (2006). Bundle adjustment rules. In Photogrammetric
Computer Vision (PCV’06), Bonn, Germany.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.6901&rep=rep1&type=pdf
Oct 13: Motion estimation, segmentation and blur
Our first batch of blur papers did not really use motion explicitly. The next batch (mostly) does.
Assignment 4 (10 points). Present one of the following papers:
Motion, segmentation, flow and blur
(Andrew) Motion from Blur
Shengyang Dai and Ying Wu,CVPR 2008
(Geoffrey) Segmention and blur estimation together: Analyzing Spatially-varying Blur
Ayan Chakrabarti, Todd Zickler, William FreemanCVPR 2010
(pdf)
(Kefei) L. Bar, B. Berkels, M. Rumpf, and G. Sapiro. A variational framework for simultaneous motion estimation and restoration of motion-blurred video. In ICCV, 2007.
(pdf)
(Jason) Robust Dual Motion Deblurring, Jia Chen Lu Yuan Chi-Keung Tang Long Quan, CVPR 2008(pdf)
(Soumya) A. Levin. Blind motion deblurring using image statistics. In NIPS, 2006.
(pdf)
(Tom) MULTIFRAME IMAGE RESTORATION IN THE PRESENCE OF NOISY BLUR KERNEL, Jung, Marquina, Vese, ICIP 2009
(pdf)
Extras/other
W. H. Richardson. Bayesian-based iterative method of image restoration. Journal of the Optical Society of America, 1972.
B. Bascle, A. Blake, and A. Zisserman. Motion deblurring and super-resolution from an image sequence. In ECCV, 1996.
(pdf)
H. Jin, P. Favaro, and R. Cipolla. Visual tracking in the presence of motion blur. In CVPR, 2005
(pdf)
Removing Non-Uniform Motion Blur from Images
Sunghyun Cho, Yasuyuki Matsushita, Seungyong LeeICCV 2007
(pdf)
P. Favaro and S. Soatto. A variational approach to scene reconstruction and image segmentation from motion-blur cues.
In CVPR, 2004.(pdf)
RAV-ACHA, A., AND PELEG, S. 2005. Two motion-blurred images are better than one. Pattern Recognition Letters 26, 311– 317.
Robust deblurring of random blur, ML Hambaba, Applied Optics, vol. 33, issue 14, pp. 2877-2882, 1994.
(pdf?)
(pdf)
YITZHAKY, Y., MOR, I., LANTZMAN, A., AND KOPEIKA, N. S. 1998. Direct method for restoration of motion-blurred images. Journal of Opt. Soc. Am. A. 15, 6, 1512–1519.
(pdf)
IMAGE RESTORATION THROUGH L0 ANALYSIS-BASED SPARSE OPTIMIZATION IN TIGHT FRAMES
Javier PortillaICIP 2009
(pdf) (web and code)
An Improved FoE Model for Image Deblurring
Dahong Xu · Runsheng Wang
Int J Comput Vis (2009) 81: 167–171
(pdf)
BEN-EZRA, M., AND NAYAR, S. K. 2004. Motion-based motion deblurring. IEEE Trans. Pattern Analysis Machine Intelligence 26, 6, 689–698.
(pdf)
W. Chen, N. Nandhakumar, and W. Martin. Image motion estimation from motion smear-a new computational model. IEEE Trans. on PAMI, 18(4):412–425, 1996.
Oct 20: Out of town; no class.
Assignment 5a. (5pts) Project pre-proposal. Send me an email by Oct 20 at 11am with a few paragraphs describing what project you want to puruse. Discuss what data you will use, what code from the web you will use, what you will implement yourself, how you will test your method, what is a success, what's a failure, is there a hypothesis, does this address some open question in the Zapruder data? Equations are desirable -- I want to know you have a concrete idea of what you want to do.
See 11 Seconds in Dallus Redux for some things to focus on.
Oct 27: Project proposals
Assignment 5b (5pts) Present your project proposal in class. Send me ppt or pdf slides by 11am on Oct 27. Presentations should be on the order of 15min each plus questions. You should have some preliminary data and some experiment to show that you have made a stab at this (it can be a failure but you should learn something from it).
Nov 3: Super-resolution
Nov 10: 3D tracking
Out of town. Reschedule?
Nov 17:
Nov 24: Thanksgiving Break
Segmenting all the images. Grab cut. Video grab cut? Co-segmentation of groups of pictures (CVPR'10).
Szeliski book
image stitching 9
tracking/stabilization parameteric motion 8
panorma (link)
grunt work: create foreground/background masks.
Denoising
A High-Quality Video Denoising Algorithm based on Reliable Motion Estimation
Ce Liu, William T. Freeman
http://research.microsoft.com/en-us/um/people/celiu/ECCV2010/
Assignment ?? Try using the voodoo camera tracker to recover the 3D camera motion and structure. Note that the foreground and background move differently. You may need to create masks for the two regions and process them separately. I'm not sure how easy this is with this software.
voodoo camera tracker http://www.digilab.uni-hannover.de/docs/manual.html