Women of house Capulet is a project combine machine learing, AI training, and video editing. My group member Shiyi, Yue and myself selected the iconic Romeo and Juliet (1968) as our base video, we read through the script and selected the part of conversations between female characters. Taking away the love and hate theme between the two family and gave a spotlight on women of house Capulet. We replaced the voices of characters with the overdub voice of all three of us, and replaced their face with the combination of our faces.

While this video turn out to be hilarious, we also notice the vibrant emotions and personalities in Romeo and Juliet as the female characters interact with each other in the context of love, marriage and family hatred.

To create the faces of our characters, the three of us recorded videos of our own face, submitted it onto the open source machine learing platform Runway ML to create combined faces. The image training function allows us to submit pictures of our faces as the dataset, by combining our faces with random face data, we got three different (yet the same) faces for our characters.

We use loose hair and neutral/kind/innocent facial expression to represent Juliet.

Nurse is a kind lady who really cares about Juliet, we choose to use smile as her facial expression.

Lady Capulet  has to keep a strict face all the time. Based on her costume in the movie, we tied our hair up to get closer to her look.

We used Descript to clone our voices for the three characters. Similar with what we did for their faces, we recorded and combined our voices with different pitch and tone.

First, we read through the script of Romeo and Juliet with the tone of Juliet, Nurse and Lady Capulet. Each of them has different personality, so their voice should also represent their characteristics. Juliet’s voice is a combination of our normal voice. Nurse has an optimistic personality so we made her voice sounds hilarious. And Lady Capulet’s voice sounds serious, decent and low. (Below is a recoding of the three of us read through the script with the tone of Lady Capulet).

Video editing is the most complex part in our working process. First we need to cut the scenes we needed from the original movie. Then, we separated each characters faces by masking and editing. After the preparation is finished, we submitted the scenes of each character and our combined faces onto Deep Fake to swap faces.

Original scene

After face swap

After the face swap is finished, we combined all of the video together by masking and collage.