Generative Adversarial Network AR Application
Product Manager, Tencent, Shanghai
In 2019, Snap's gender-swap filter went viral, and so did the stock price. This was the first application of GAN on the mobile phone across the world.
QQ is the biggest online social platform for young teenagers, who love ACG culture. I foresaw the potential application of GAN to generate comic face and proposed the comic face AR filter.
Effect User Testing
Typical machine learning algorithms like segmentation have a universal truth to evaluate the accuracy. GAN highly relies on subjective user testing on the effect.
To give a scientific evaluation result, I built up a testing dataset, taking the real use case into consideration from a variety of dimentions.
For each iteration, we recruited 10 people to carry out blind user testing on the preferred effect. By summarizing the result, I provided feedback on what dimension requires improvement. Together with designers and developers, we came up with solutions to improve the effect.
Since the comic face filter was the first application of GAN on mobile, we encountered great difficulty in performance. When the GAN model run on a mobile phone valued at around 1,000 RMB, it resulted in noticeable latency and generated heat.
The developers compressed the model to different extend and we matched different models with phones with different CPU and GPU to ensure the best performance and effect.
Across the entire QQ, I have developed more use cases for the comic face filter.
QQ users reported great fun using the comic face filter while having video chats with their friends.
In QQ sticker, the comic face filter allows users to make their own stickers with prompts like 'goodnight', 'thank you', and 'yes'
Other GAN Effects
The comic face acted as the first GAN application in a selfie camera and developed the pipeline for later GAN effects.
Within 5 days, the filter received 34 million visits, generating 4.3 million images. It also broke the record with a 57.4% spread rate.
Our team was also honored with a Business-group level award.