Comic Face

Generative Adversarial Network AR Application

Product Manager, Tencent, Shanghai

Nov 2020

In 2019, Snap's gender-swap filter went viral, and so did the stock price. This was the first application of GAN on the mobile phone across the world.

QQ is the biggest online social platform for young teenagers, who love ACG culture. I foresaw the potential application of GAN to generate comic face and proposed the comic face AR filter.

Effect User Testing

Typical machine learning algorithms like segmentation have a universal truth to evaluate the accuracy. GAN highly relies on subjective user testing on the effect.

To give a scientific evaluation result, I built up a testing dataset, taking the real use case into consideration from a variety of dimentions.

For each iteration, we recruited 10 people to carry out blind user testing on the preferred effect. By summarizing the result, I provided feedback on what dimension requires improvement. Together with designers and developers, we came up with solutions to improve the effect. 

Performance Testing

Since the comic face filter was the first application of GAN on mobile, we encountered great difficulty in performance. When the GAN model run on a mobile phone valued at around 1,000 RMB, it resulted in noticeable latency and generated heat.

The developers compressed the model to different extend and we matched different models with phones with different CPU and GPU to ensure the best performance and effect.

Applications

Across the entire QQ, I have developed more use cases for the comic face filter.

QQ users reported great fun using the comic face filter while having video chats with their friends.

企业微信20221020-203219_2x.png

In QQ sticker, the comic face filter allows users to make their own stickers with prompts like 'goodnight', 'thank you', and 'yes'

漫画表情包.gif
Other GAN Effects

The comic face acted as the first GAN application in a selfie camera and developed the pipeline for later GAN effects.

图片 2.png
IMG_6095.JPG
image.png
Outcomes

 

Within 5 days, the filter received 34 million visits, generating 4.3 million images. It also broke the record with a 57.4% spread rate.

Our team was also honored with a Business-group level award.