Embodied AI Agent Integrated Galaxy Apps
Migrated Galaxy native apps design to the XR platform with embodied AI Agent features.
Overview
In the summer of 2024, I continued working as a Product Design intern in MPS IMEX (Immersive Experience) Lab at Samsung Research America (SRA), where I researched and designed on embodied AI agent features for Galaxy Apps in XR platforms to explore the opportunities and user benefits of AI integration.
Disclaimer
Due to the NDA, I cannot share every detail of my work, all images here are NOT the actual designs of my work.
My Contribution
🧐 Research & Conceptualization: Conducted research on Embodied AI Agents for XR scenarios, shaping the team’s approach to human-AI interaction and agent features.
🖼️ User Scenario Creation:Developed comprehensive use cases that integrated AI capabilities into native apps, focusing on key user scenarios within the Galaxy mobile ecosystem.
🥽 Design System Adaptation & Prototyping: Migrated Galaxy OneUI design system to native apps in the XR platform and created key components for PoC prototyping.
Company
Samsung Research America
Collaborator
Duration
July-Sept, 2024 (12 Weeks)
Tools
Figma, Meta Quest 3,
Blender, Bezi
Overview
What are Galaxy Apps?
My Files
Notes
Messages
Call
Samsung TV+
Gallery
Galaxy Apps are Samsung’s native applications, trusted and used by millions of Galaxy users worldwide. These apps deliver essential tools and seamless experiences across Samsung devices, enhancing productivity, communication, and entertainment for a vast, global user base.
Where are we? - Industrial Trend of AI Agents
Tool
Assistant
Collaborator
We are here!
Companion
Digital Agent
Microsoft Clippy
(1995-2007)
Voice Assistant (VUI)
Apple Siri (2011)
Microsoft Cortana (2014)
Amazon Alexa (2014)
Samsung Bixby (2017)
Conversational AI (CUI)
ChatGPT Voice (2023)
AI powered Siri (2024)
Google Gemini (2024)
Microsoft Copilot (2023)
Meta AI Agent (2023)
Embodied AI agent
Speculative film created by Niantic & Keiichi Matsuda (Liquid City)
Missions of the Project
mission 1: Use Case+ Design
As part of emerging galaxy AI era, Samsung XR platform will demonstrate the power of AI via seamless multi-device experience and help billions of Galaxy users.
Mission 2: Embodied AI Agent Definition
Explore what the embodied AI agent user experience should be on Galaxy devices, starting with the XR platform.
Explore opportunities
Contextualization
The upcoming Galaxy AI-enabled XR platform lacks a clear definition and user experience framework for embodied AI agents, limiting its potential to create engaging interactions.
Competitive Research
Scenario Brainstorming
Framework & Scoping
Specialized Research : Embodied AI Agent
User Flow Ideation
Feature evaluation
User Flow
Iteration
Prototyping
Refinement
PoC Handoff
Design
Due to NDA restrictions, detailed work is private.
Recruiters: Password is in my resume.
Interested parties: Feel free to contact me for access.
🔐
Private Content for Detailed Design
Reflection
Feedback & Impact from Samsung HQ
Team Lead/PM:
“Great job! Your work really showcases the depth of our research and aligns perfectly with the direction we’re heading. This research and design work will definitely help drive our next steps clearly on the Embodied AI agent design.”
Senior Designer (HQ):
“The research and conclusions align closely with our team’s design direction at HQ. Your analysis and approach have clarified our vision and reinforced our current trajectory. The way you articulated user needs and embodied AI potential provides a solid foundation for the next steps.”
Principle Designer (HQ):
“Your insights are crucial for integrating these interactions into our system-level experience. This level of alignment and clarity is precisely what we need to push the boundaries of our product design.”
Perfectly align with HQ’s direction
Push the boundaries
Clarify the vision
Drive the next steps
Key for the system-level integration
PoC MVP validates the direction of Embodied AI agent design.
At the early stage of Embodied AI agent design and its integration into the XR platform, feedback from HQ was crucial. During the final PoC MVP presentation, the HQ design team praised my work, validating the effectiveness of my research and design direction. I’m glad my work can significantly contribute to the next steps.
*Quotes from the final presentation to the Samsung Korean HQ, XR UX Team.
Feedback on Teamwork
Senior Staff XR Designer/Developer:
“Yuehui worked really well with the team by providing insightful, thorough, and well-communicated research while being receptive to comments and ideas. Whenever we were in group ideation sessions, she was actively listening, and provided valuable ideas that we were able to develop in our work”
Senior XR Developer:
“Yuehui is passionate designer and produces quality results. She has been involved in researching for some of the novel interactions for a new product and have provided good insights that has helped team to make good design decisions. Whenever there is a need for input for new design ideas she has been always open to get involved to take up on tasks.”
Senior XR Designer:
“Yuehui always comes to the meeting prepared. She has done a great job communicating her ideas and research results to the team and turning abstract feedback into concrete actions. She is a strong team player eager to make meaningful contributions, and I enjoyed working with her.”
Senior XR Designer (Intern Mentor):
“Yuehui made such a strong impact during her internship that we were thrilled to welcome her back! As her mentor, I saw her dive deep into AI trends, especially in data handling and embodied AI, while collaborating closely with senior designers on privacy-focused XR projects. Her insights and initiative were key in driving our user-centered design efforts forward!.”
Made a strong impact
Dive deep into AI trends
Key insights and initiative
Passionate designer
Always open to get involved
Excellent communicator
💬✨
Effectively turn feedbacks to actions
Strong team player
Active listener
Insightful, thorough, and well-communicated work
Extraordinary
Hardworking
Talented designer
dedication and collaborative spirit
🎨❤️🔥
🤝👥
🌟👩💻
💪🎯
*Quotes from SRA teammates' feedback about Yuehui's teamwork.
Takeaways
As a designer in a XR Lab, I often faced uncharted territory, like defining the concept of an Embodied AI agent—a tough challenge with no existing guidelines. I took the lead in building our understanding from the ground up, quickly gathering insights and refining them in the UX process. This experience fueled my love for learning, problem-solving, and embracing new challenges.
To better sync with the PM, developers, or designers, I made sure to be well-prepared for every single meeting, translating research into concise insights to keep discussions focused. I set clear goals, synced design progress with research, and provided references before meetings to ensure alignment. This approach fostered a collaborative environment where everyone could confidently engage with new concepts.
As a young designer, I initially struggled without clear direction but quickly shifted from an executor’s mindset to a project owner’s. I took the initiative to ask the right questions, understand stakeholder needs, and turn research into actionable design proposals rapidly. By owning the project scope and aligning with the team, I helped drive decision-making and guided us toward shared success.
© Yuehui Du 2024