With the growing number of users, Augmented Reality (AR) privacy issues have aroused researchers’ concerns. Our work focuses on the protection of privacy-sensitive objects in indoor settings. We proposed a novel privacy protection mechanism MagicCloth for AR live streaming. When the user covers the privacy-sensitive object with a specially designed cloth, MagicCloth can accurately track it in real time, and replace it with a virtual object based on its shape and size seamlessly. By wisely integrating lightweight 0-1 segmentation, pattern detection, and AR plane tracking techniques, our MagicCloth system achieves real-time performance on Android smartphones. We extensively analyze the challenges involved in object replacement and have put forth corresponding solutions. In our future work, we aim to implement and refine these proposed solutions to effectively address the identified challenges and achieve seamless and privacy-preserving object replacement in AR scenarios.

Cite this work: Yuming Hu, Mingyu Zhu, Qiao Jin, Feng Qian, and Bin Li. 2023. MagicCloth: Protect User Privacy in AR Streaming. In Proceedings of the 1st ACM Workshop on Mobile Immersive Computing, Networking, and Systems (ImmerCom ’23). Association for Computing Machinery, New York, NY, USA, 222–228. https://doi.org/10.1145/3615452.3617936