AR Birthday Cake

3D Content Placement in AR

Product Manager, Tencent, Shanghai

May 2021

The AR Birthday Cake project was initiated by QQ monetization team as a premiere gift a user can purchase and send to his friend. When a user receives the cake notification in the message, he can launch the experience by clicking it.

To make the experience more surprising, I explored and compared existing 3D content placement designs and created the most suitable one, which auto-places the content on a real surface closest to the scene center with no movement required.

Existing Design Research

from left to right: Douyin, Google, IKEA

Our Design

Since instant experience is our first priority to build a surprising AR cake experience, I decided to design on top of Douyin’s framework while solving the not-on-a-surface problem. Cooperating with machine learning engineers, we introduced surface normal estimation into the instant placement process. With around 200ms undetectable waiting, our system determines the real surface area and automatically places the object on it.

surface norm.png

surface normal estimation

Evaluation and Outcome

A blind user test among 10 colleagues suggested that our approach resulted in more realistic placement in the indoor and outdoor environment.

After launching, the data analysis showed that it boosted the income from virtual gifts. In terms of social connection, we also found that message exchange became more frequent with an AR cake compared with a plain card.