(Look at image on right from very close, then from far away.)

Project 1: Hybrid Images
CS 143: Introduction to Computer Vision

Brief

Overview

The goal of this assignment is to create hybrid images using a modification of the approach described in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns. Hybrid images are static images that change in interpretation as a function of the viewing distance. The basic idea is that high frequency tends to dominate perception when it is available, but, at a distance, only the low frequency (smooth) part of the signal can be seen. By blending the high frequency portion of one image with the low-frequency portion of another, you get a hybrid image that leads to different interpretations at different distances.


Details

This project is intended to familiarize you with MATLAB, image filtering, and multiscale frequency representations. If you don't already know MATLAB, you will find this tutorial on MATLAB helpful. There are two sample images in the /course/cs143/asgn/proj1/ directory (of Derek and his former cat Nutmeg) and some starter code that can be used to load two images and align them. The alignment is important because it affects the perceptual grouping (read the paper for details).

First, you'll need to get a few pairs of images that you want to make into hybrid images. You can use the sample images for debugging, but you should use your own images in your results. You will be generating Gaussian and Laplacian pyramids for both images and merging them to produce the hybrid image. The hybrid will be the sum of a low-pass filtered version of the first image and a high-pass filtered version of the second image. The cutoff-frequency of each filter can be set by choosing the first or last N levels of the Laplacian pyramid to include in the sum, where N is a configuration parameter that you can vary per image to produce the best results. This creates a new pyramid from the top of one pyramid and the bottom of the other pyramid, which is then used to construct the hybrid image using the exact reverse of the pyramid construction process. Don't forget to include the last level of the Gaussian pyramid in the sum for the low-pass filter (it contains all lower frequencies after the end of the pyramid). Your image pyramid should look similar to Figure 7 in the Oliva et al. paper:


The images for the Laplacian pyramid were visualized by adding 0.5, so light gray values are positive and dark gray values are negative. Try creating different types of hybrid images (change of expression, morph between different objects, change over time, etc.). The site has several examples that may inspire.


Useful MATLAB functions: imfilter, imresize, imwrite, cell
Forbidden functions: impyramid


Bells & Whistles (Extra Points)

Try using color to enhance the effect. Does it work better to use color for the high-frequency component, the low-frequency component, or both?

Think of your own clever extensions to get more extra credit.


Writeup

For this project, and all other projects, you must do a project report in HTML. In the report you will describe your algorithm and any decisions you made to write your algorithm a particular way. Then you will show and discuss the results of your algorithm. Also discuss any extra credit you did. Feel free to add any other information you feel is relevant.


Rubric


Web-Publishing Results

All the results for each project will be put on the course website so that the students can see each other's results. In class we will have presentations of the projects and the students will vote on who got the best results. If you do not want your results published to the web, you can choose to opt out. If you want to opt out, email cs143tas[at]cs.brown.edu saying so.


Handing in

This is very important as you will lose points if you do not follow instructions. Every time after the first that you do not follow instructions, you will lose 5 points. The folder you hand in must contain the following:

Then run: cs143_handin proj1
If it is not in your path, you can run it directly: /course/cs143/bin/cs143_handin proj1

Credits

Assignment developed by Derek Hoiem