电机螺纹型电磁阀分拣任务
返回任务列表
002-032-17
4件
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
002-032-17
图片待上传
4件
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
002-032-17
图片待上传
4件
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
002-032-17
图片待上传
4件
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
Glass 2 AR Solutions
for
Rokid AR Glass 2 for Midea's Assembly Line
Optimized the AR interface on Rokid Glass 2 for assembly line workflows by integrating physical ring controls for seamless audio-based interactions.
My Contribution
🔍 In-house & On-site User Research: Conducted user research at both Rokid HQ and client Midea's sites, gathering insights to inform AR interface improvements.
🎛️ Hardware Prototype Definition & Design: Defined and implemented hardware (ring) prototypes to align with user needs, bridging design with technical feasibility.
🕶️Interface Enhancements: Improved the current user interface to integrate physical ring control seamlessly with the existing audio-based input system on Rokid Glasses 2.
Outcome
🌟 Client Praise: The client Midea recognized the innovative approach to enhance their workflow, noting its alignment with their employee's needs.
🚀 Boosted Workflow Productivity: Extended task duration and efficiency on the assembly line by reducing worker fatigue.
😊 Enhanced User Satisfaction on the AR solutions: Improved user satisfaction rates, reflecting a more intuitive experience.
Company
Rokid
Collaborator
Duration
Jun-Sept, 2021
Tools
Figma, Blender 3D,
Rokid AR SDK
Project Snapshot in 60 Seconds
Rokid AR Solutions is collaborating with Midea’s Manufacturing System
However,
the current solutions faces issues like...
Mask-induced speaking barriers
Noisy environment challenges
Repetitive commands fatigue
Cognitive load of command recall
Glass 2
Audio--based control AR
designed without controller for hands-free assembly line sorting operation
Audio Control
Ring Control
Physical User Interface (PUI)
Voice User Interface (VUI)
*The black background means transparent in the Rokid AR interface development for the optical see-through.
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
盘底转接线5分拣任务
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
“
Back to home screen
“
“
Finished
“
“
Upload
“
“
Go back
“
“
Confirm
“
OR
The original design of “Task Step List”
⏱️ Extended task duration by 86%,
boosting the overall workflow productivity.
By integrated the physical interaction models that reduces worker fatigue, the solution enabled workers to perform continuous tasks for longer periods—from an average of 21 minutes to 39 minutes. This improvement significantly boosted overall workflow efficiency and productivity.
😊 Increased satisfaction rates to 90%,
creating a more intuitive and comfortable interaction experience.
Targeting to solve the problems that collected from assembly line sorters, the evolved AR interface and defined multi-model interactions made them feel more satisfied, raising satisfaction ratings from 60% to 90%. This was achieved by creating a more intuitive interaction experience.
Research
Rokid AR Solutions Enhance Midea’s Manufacturing with Rokid Glass 2
Rokid’s long-term AR collaboration with Midea has enhanced assembly line efficiency, generating over $30 million in annual revenue while driving smart manufacturing innovation and improving employee experience.
Monocular AR glasses with 40° FoV
Independent computing
Optical see-through
Audio-based input system
Glass 2
Global leader in home appliances
Innovation in smart manufacturing
AR Solutions for smart manufacturing
Target User: Assembly Line Sorter Utilizing Rokid’s Streamlined AR Solutions
1
Review the task list
2
Start with one task
3
Follow the step-by-step
4
Operation on each step
5
Take a picture and upload
6
Confirm the completion
Start working
The streamlined workflow of assembly line sorting
Next step
“Confirm”
“Go Back”
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
盘底转接线5分拣任务
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
002-032-17
轴承 (Ø20mm)
同时检查轴承中转珠的个数以及顺滑的旋转，丢弃不合规的零件。
# 4件
“Take the picture”
“Next Step”
The assembly line sorter follows a streamlined, repetitive workflow involving the identification, sorting, and organization of parts. With current Rokid's solution, they can follow the instructions easily and efficiently. While efficient, their tasks require precision and adherence to protocols for maintaining quality and productivity in high-volume production.
Pain-points in the Current "Audio-only" AR Solutions for Assembly Line Sorting
Noisy Environment Challenges
High noise levels hinder audio-only interactions, reducing the effectiveness of current input methods.
Mask-induced Speaking Barriers
Constant mask-wearing makes the verbal communication inconvenient, decreasing the current audio-based inputs effectiveness.
Repetitive Commands Fatigue
The current user flow requires users to repeat commands frequently, leading to a tedious experience.
“
Next Step
“
“
Confirm
“
“
Go Back
“
“
Take Picture
“
“
Upload
“
Cognitive Load of Command Recall
Users must remember fixed commands, adding unnecessary mental effort that detracts from productivity and focus on physical tasks.
“
Next Step
“
“
Next Step
“
“
Next Step
“
“
Next Step
“
Audio-only Input Control
Based on the assumption that integrating physical (PUI) and voice (VUI) controls will improve user experience, design a new AR interface paradigm and evaluate its fidelity for enhanced interaction efficiency.
🧑💻 Engineers: "We build this ring prototype, but where does it fit?"
🧚🏻 Designers: "Hold my pen — let's make magic happen!"
Based on the engineer’s prototype, we had three undefined buttons: two for navigation and one for confirmation. This rapid prototype was developed with time and resource constraints, while considering the harsh working conditions of the assembly line. The goal was to keep it as simple as possible, yet effective.
“Left-Confirm-Right”
Primary
“Up-Confirm-Down”
Secondary
We determined the initial design of the three-button ring and how it should be worn on the hand based on user studies. The participants included 10 in-house employees (engineers, HR, etc.) and 3 on-site assembly workers. The core criteria is the "comfortable level" on different gestures.
Where?
to wear the ring
participants think the second knuckle is the most comfortable.
2
1
3
“Although I get used to wear ring on the third knuckle, but with gloves, my finger feels uncomfortable for wearing this way a long time.”
46%
“The ring disturb my hands-on works when it on the first knuckle, the second one is a little bit better than the third one to me”
Comfortable level for using 15 min
Comfortable level on moving thumb
Which direction?
feels more comfortable
participants think the left-right moving on the second knuckle is the most comfortable.
2
3
3
2
“It makes my thumb tired when clicking the ring buttons on the third knuckle for a longer time, sliding my thumb on the second one is best to me.”
76%
“Buttons on the second knuckles feel more horizontal to me, and the vertical direction on the third one works better than it on the second one.”
2
3
3
2
1
3
2
Applying Natural Mapping Principle: Integration of PUI and VUI in Digital Interfaces
Physical Interface
“Buttons”
Reactive
Digital Interface
“Screens”
Audio Inputs
“Commands”
Behavioral Similarity
Behavioral Similarity
Spatial Similarity
Metaphorical Similarity
Go back
“Go Back”
“Confirm”
The 🔗Natural Mapping Principle from Nielsen Norman Group promotes intuitive design by aligning user actions with expected responses. I used metaphorical and behavioral similarities for audio inputs and spatial and behavioral similarities for physical interfaces, making interactions seamless and easy to understand.
Design for PUI + VUI
Modify the original design with PUI integration.
Overall Layout: Symmetrical Layout
The original screens didn’t have unified layout, some of them are “side-by-side”.
AS-IS
Since the physical ring has symmetrical behavior mapping, the new screens follow its as symmetrical layouts.
TO-BE
Call-to-action Buttons: Horizontal Layout
The original CTA buttons were vertical layout based on the “up” & “down” mental model.
AS-IS
Modified the CTA buttons to horizontal layout to match with “left” & “right” behavior on the physical ring.
TO-BE
List Navigation: Horizontal Two Dimensions
The original design was grids layout that required four dimensions inputs, and the command “next one” is not clear enough.
AS-IS
The physical ring only includes two dimensions for “right” & “left” mental model.
TO-BE
However...
the vertical “task list” was kept to distinguish between “task list” and the subsequent “task steps list” in a rapid streamlined workflow.
V.S
Based on the on-site user studies, we found it’s essential that users are effortlessly differentiated at first glance of different list of information. If both screens appear too similar, users might easily experience fatigue and confusion, especially when they have repeated the flow many times.
*The black background means transparent in the Rokid AR interface development for the optical see-through.
Home Screen
“
Settings
“
“
Task List
“
“
Account
“
OR
Task List
“
Next one
“
“
Back to home screen
“
“
Last one
“
OR
Task Steps List
“
Next one
“
“
Back to task list
“
“
Last one
“
OR
Operational Step-by-Step Instructions
“
Go back
“
“
Finished
“
OR
Post-Operation Photo Capture
“
Take the photo
“
OR
Confirm Photo Upload for Documentation
“
Confirm
“
“
Retake
“
OR
Reflection
Takeaways
Working with Rokid Glasses...
Best workstation ever!
Furry colleague...
Tackling complex design challenges requires more than just theory—it demands the ability to translate those theories into practical, user-focused solutions. In this project, I drew upon principles from PUI and digital interface design, adapting them to our specific context. This approach involved blending theoretical insights with real-world application to create an intuitive user experience.
When engineers developed the initial ring hardware prototype, they weren’t sure how it fit within the overall design vision. I as one of designers stepped in to bridge this gap, working closely with engineers to refine and adapt the concept. Through collaborative iterations, the prototype evolved into a solution that respected hardware constraints, seamlessly integrated with software interactions, and met user expectations. This partnership underscored the power of combining engineering precision with design innovation
If I had more time…
In the post-design phase, user testing was limited to the duration until sorters first took a break. This left gaps in understanding potential issues during continuous use over 3-4 hour task sessions or full 8-hour workdays. Extended testing could reveal hidden problems like physical interaction fatigue or discomfort that only emerge during prolonged use.
The project did not include iterations of the physical ring design, though feedback suggested valuable improvements. For example, rotating the button panel to a vertical orientation could offer more flexible interactions, and curving the panel to fit finger contours could enhance comfort. These design possibilities highlight untapped potential for more ergonomic and user-friendly solutions.
© Yuehui Du 2024