Project 5: Face Morphing

Due Date: 11:59pm on Friday, March 26th, 2010

Brief

• This handout: /course/cs195g/asgn/proj5/handout/
• Stencil code: /course/cs195g/asgn/proj5/stencil/
• Data: /course/cs195g/asgn/proj5/data/
• Handin: cs195g_handin proj5
• Required files: README, code/, html/, html/index.html

Requirements

You are required to implement the following:

• Implement an algorithm that takes two images and interpolates between them
• Calculate and display the mean face

Details

MATLAB stencil code is available in /course/cs195g/asgn/proj5/stencil/. You're free to do this project in whatever language you want, but the TAs are only offering support in MATLAB. The TAs have supplied a few files to get you started.

• getpoints.m: Click on an image and get an Nx2 list of x,y coordinates
• assignpoints.m: Iterates over all images and calls getpoints
• facemorph.m: Given 2 images, their correspondence points, a cross dissolve value and morph ratio, create the new face

Correspondence Points

Before you can start morphing, you need to define correspondence points between the images; e.g. left eye of image A maps to the left eye of image B. This can be tedious, but is necessary to get good morph results. Select points on features that you want to align during the morph: eyes, ears, nose, mouth, etc. The TAs use 22 points and get good results, but you are free to use as few or as many as you want.

Morphing

The actual morphing is in two parts: cross dissolving and image morphing. Cross dissolving is easy: interpolate a value from 0 to 1 and use A*value + B*(1-value) as the frame value. Morphing is a little trickier.

There are different methods of doing morphing. We will be morphing by smoothly blending offsets from one one point to another. First you need to calculate the intermediate points from image A to image B based on a value from 0 to 1. Then calculate the offsets of the points in A to the intermediate points and do the same for B. These offsets map back to each image and gives us the pixel values that will be blended at the intermediate points.

Next you need to calculate the offsets of non-marked points. You know the offsets of certain pixels, but the rest of the image is unknown. Does this problem sound familar? It should, its just like Poisson blending. You can use your Poisson blending code from project 2, but you will need to modify it slightly. Instead of blending between images, we are just going to be doing a single image fill, so you will probably have to modify your code. If you remember from class, the system of equations for doing a poisson fill are much simpler. For an unknown pixel, the value is the average of its 4 connected neighbors whereas a known pixel is simply its value.

Mean Face

From the correspondence points and morpher, you can create the mean face of some data set. You would need to compute the average shape, warp the images to that shape, then average the colors.

Write up

For this project, just like all other projects, you must do a project report in HTML. In the report you will describe your algorithm and any decisions you made to write your algorithm a particular way. Then you will show and discuss the results of your algorithm. Also discuss any extra credit you did. Feel free to add any other information you feel is relevant.

Extra Credit

You are free to do whatever extra credit you can think of, but here are a few examples.

• (up to +5 pts) Use the mean face to exaggerate features and produce several caricatures
• (up to +5 pts) Try your algorithm on non-face input. Compare how well it worked to when you used faces for input.
• (up to +10 pts) Try other image morphing techniques and compare this to it. One such way is to triangulate the points and deforming the resulting mesh, often referred to as a lattice deformer.
• (up to +20 pts) Automatically assign feature correspondence points.

Graduate Credit

You are required to do at least 10 points of extra credit.

Handing in

This is very important as you will lose points if you do not follow instructions. Every time after the first that you do not follow instructions, you will lose 5 points. The folder you hand in must contain the following:

• README - text file containing anything about the project that you want to tell the TAs
• code/ - directory containing all your code for this assignment
• html/ - directory containing all your html report for this assignment (including images)
• html/index.html - home page for your results

Then run: cs195g_handin proj5
If it is not in your path, you can run it directly: /course/cs195g/bin/cs195g_handin proj5

Rubric

• +70 pts: Face morphing algorithm
• +10 pts: Calculating the mean face
• +20 pts: Write up
• +20 pts: Extra credit (up to twenty points)
• -5*n pts: Lose 5 points for every time (after the first) you do not follow the instructions for the hand in format

Final Advice

• Start using small images, since there are many unknowns it will always be slow to solve.
• Have fun with this and be creative. This is a fairly simple project open to many extensions.

Credits

Project derived from Alexei A. Efros' Computational Photography course, with permission.

Good Luck!