CS180 Project 3

Part 1: Defining Corrrespondences

Part 2: Computing the "Mid-way Face"

In this section, I used the Delaunay triangles to define a forward and an inverse transformation between image 1/2 and the halfway image. First, I computed the halfway image's positioning points by taking average of the points from either image. Second, with the target face structure found, I computed a forward and a inverse transformation for each pair of corresponding triangles between the original image and the halfway image. I did this for both image 1 to halfway and image 2 to halfway. Finally, I used the inverse transformations to select color values from the original images, after which I averaged the two halfway images (image1 - halfway and image2 - halfway) and created the morphed halfway face. The results are as follow.

Part 3: The Morph Sequence

In this part, I completed the morphing processing by running the previous part with different warp and dissolve fractions. I set up a arithmetic sequence from 0 to 1 with the 1 / 60 as the step size, with which I repeatedly called the morph function from part 2 with the warp and dissolve fractions incrementally chosen from the sequence. Then, I combined all the results and created a animated GIF. Here is the result.

Part 4: The "Mean face" of a population

In this part, I used the FEI database to compute its average face. To ensure the completeness of the morphed photos, I also manually added points at each corner for all annotated faces I then morphed all faces into the mean face, using the morph sequence described in part 3. Finally, I warped my face into the average geometry and the average face into my geometry, which was achieved by keeping the dissolving fraction to 0 or 1 depending on which target I'm using. Here are the results.

Part 5: Caricatures: Extrapolating from the mean

In this part, I extrapolated my face from the mean face from the previous part to create a caricature of myself. To be honest, my face, when warped into the mean geometry, was funny enough. However, I wanted to deliver a more dramatic caricature. To this end, I used a warp fraction of 1.8 and a dissolve fractionof 0.9 to morph my face into the mean face, thus creating a hilarious caricature of myself.

Bells & Whistles

Gender Change

I managed to change my gender from male to female by morphing my face into an averaged Chinese female face. I downloaded a photo of the average Chinese female face and defined correspondences similar to part 1. Using the morph sequence from part 3, I successfully morphed my face into the average female face while recording the intemediate images at each step. I tried morphing only my appearance, only my shape, and both. After visualizing all intermediate images, I have chosen the one with my face half morphed into the mean face, thus changing my gender from male to female. Here are the results.

In my opinion, the best morphing result was with both shape and appearance as it captured the female face shape while retaining my facial details. The appearance only transformation remotely looked like me, and the shape only transformation looked too much like me.

PCA

To find the principle axes, I first dimension stacked all 400 gray-scaled, flattened faces from FEI to create a (360 * 260) by 400 array. Then, I used `sklearn.decomposition.PCA` to find the principle components of the combined data set. Using the top 100 principle axes (axes top 100 largest singular values), I transformed my face and the mean face from FEI into the an eigen space. I then simply dissolved my face into the mean face. The dissolve fractions are 1 for my face and 0.9 for the mean face. Finally, I used the inverse transformation function to restore the dissolved face in the RGB space. Here are the results.

Compared with the RGB space caricature from part 5, the caricature through the Eigenspace was less dramatic and more blend, as it did not have any color. However, the morphing process is significantly faster than using the RGB space.