Objective:
Address the limitations of static preservation of edible intangible cultural heritage (ICH), particularly its lack of interactivity, temporal expression, and cultural engagement, by transforming ephemeral artifacts into dynamic, spatially interactive experiences.
Methods:
· Proposed an AI-driven Mixed Reality (MR) pipeline (SugArt 2.0) to capture, analyze, and animate sugar paintings as dynamic cultural artifacts.
· Designed a semantic animation framework that classifies motifs into five culturally grounded categories (Fly, Swim, Walk, Jump, Grow) and maps them to corresponding motion behaviors.
· Integrated computer vision preprocessing, OpenAI vision-based semantic recognition, and MR interaction (Meta Quest 3 + Unity XR Toolkit) for real-time rendering and interaction.
· Conducted user studies (n=8) combining Likert-scale questionnaires and semi-structured interviews to evaluate cultural authenticity, emotional engagement, and user experience.
Results:
· Developed SugArt 2.0, an AI-augmented MR system that transforms static sugar paintings into semantically animated, interactive cultural experiences.
· Achieved high user evaluation scores in AI trustworthiness (M=4.88), cultural authenticity (M=4.50), and artistic engagement (M=4.75).
· Demonstrated that dynamic animation significantly enhances emotional resonance, memory retention, and cultural reflection compared to static documentation.
· The work was accepted at ACM Multimedia Asia 2025 (CCF-B), validating its contribution to digital heritage and XR research.
Contribution:
Led AI-driven animation pipeline design and system implementation, including semantic recognition, animation mapping, and MR interaction integration; contributed to bridging AI, XR, and intangible cultural heritage preservation.
Abstract:
Edible intangible cultural heritage, such as sugar painting, is inherently ephemeral and difficult to preserve using traditional documentation methods. Static representations often fail to capture its temporal, spatial, and experiential qualities, limiting cultural transmission and engagement.
To address these challenges, this study presents SugArt 2.0, an AI-driven Mixed Reality (MR) system that transforms static sugar paintings into dynamic, semantically animated cultural artifacts. The system integrates computer vision preprocessing, AI-based semantic recognition, and a culturally grounded animation framework that maps traditional motifs to motion behaviors.
Using Meta Quest 3, users can capture, interact with, and experience animated sugar paintings within a spatial environment, enabling gesture-based interaction and persistent digital storage.
User studies combining quantitative and qualitative methods demonstrate high levels of cultural authenticity, AI trustworthiness, and artistic engagement. Compared to static preservation, dynamic animation enhances emotional resonance, memory retention, and user reflection on cultural experiences.
This work reframes ephemeral heritage as interactive, reconfigurable digital experiences, offering a novel approach to AI-assisted cultural preservation and extending the role of XR technologies in digital heritage.
Keywords:
Mixed Reality (MR), Digital Heritage, Artificial Intelligence, Sugar Painting, Intangible Cultural Heritage (ICH), Cultural Computing