ALLPCB
Overview
Researchers led by Prof. Chengkuo Lee at the National University of Singapore proposed an augmented tactile-perception and haptic-feedback ring (ATH ring) that integrates multimodal sensing and feedback. By combining multiple sensing modalities and feedback actuators, the system enables cross-space perception in an interactive metaverse platform, providing a face-to-face virtual social experience.
Background: multimodal VR interaction
The metaverse concept refers to a network of 3D virtual environments that strengthen the connection between physical space and cyberspace using virtual reality (VR) technology. A VR-based digital environment can simulate human senses to provide users with immersive interactions, with potential applications in social networking, education, gaming, training, and rehabilitation. Beyond visual stimulation delivered by head-mounted displays, wearable devices that sense body motion and simulate tactile sensations, such as data gloves and VR suits, have recently received attention as means to further bridge the physical and virtual worlds.
Hand sensing and low-power sensing approaches
Hand sensory and motor functions are more complex than those of other body parts, allowing the fingers to perform flexible and intricate interactions in VR systems. Established finger-motion tracking methods include camera-based and inertial measurement unit (IMU)-based rigid solutions, and flexible solutions that use stretchable materials with resistance, capacitive, or optical mechanisms. To reduce device power consumption for long-term portable operation, self-powered sensing mechanisms have unique advantages. In particular, triboelectric nanogenerator (TENG)-based sensors offer diverse material choices, simple fabrication, low cost, and self-generated signals, which enable highly flexible, ultra-low-power finger sensing systems.
Wearable actuation and integration
Wearable actuation systems are also essential for enhanced interaction in virtual environments, and various tactile feedback mechanisms have been developed. Most current wearable solutions for VR sensing and feedback are glove-based, which are mechanically complex and often require large external power supplies. A compact, highly integrated ring-shaped device with self-powered sensors and low-voltage feedback actuators is fully compatible with portable IoT platforms and suitable for long-term VR use.
ATH ring design and components
The ATH ring designed for VR applications integrates the following components:
- A TENG-based tactile sensor for continuous bending sensing.
- A flexible thermoelectric sensor for temperature detection.
- An eccentric rotating mass (ERM) vibrator for vibration tactile feedback.
- A nickel-chromium (NiCr) wire for thermal tactile feedback.
All sensors and tactile actuators are integrated into the ATH ring and wired to a wireless IoT module. Compared with glove-based solutions, the ATH ring offers higher integration and better portability. The combination of self-powered sensing units and low-voltage-driven feedback elements enables low power consumption. A proposed voltage-integration signal processing method allows continuous finger motion detection based on TENG sensor outputs. Signal collection and processing via the IoT module and machine learning analysis can enable advanced human-machine interactions for industrial automation and healthcare, such as robotic collaboration and sign-language translation.
Multimodal feedback for remote perception
By incorporating vibration and thermal actuators, the ATH ring provides tactile feedback that allows a physical object handled by one user to be identified, reconstructed in virtual form, and remotely perceived in real time by another user within the same virtual environment. This supports a face-to-face virtual social experience. Under the metaverse infrastructure, future work aims to fuse VR display technologies with multimodal body sensing to realize whole-body sensing and feedback, enabling intelligent virtual communities for social interaction, education, entertainment, and healthcare.
