ALLPCB
Introduction
Human-machine interfaces (HMI) are central to almost any electronic system, enabling operators to interact with a device to perform functions or access information. This article reviews the success of touchscreens and examines how that success has driven the development of new HMI techniques that can extend and diversify user interaction.
From a principle-of-operation perspective, the basic HMI concept is not new; its primitive forms date back to the Industrial Revolution. Examples include the Jacquard loom and Babbage's difference engine, where control functions were implemented by manually entering descriptive inputs to machines. With the advent of computers, mechanical keyboards became a primary way to issue commands. More recently, touch-based displays—most notably projected capacitive touch—enabled more advanced and intuitive HMIs. Touchscreens played a key role in the popularity of early MP3 players and were later integrated into the first smartphones and tablets, becoming the default interaction method for many consumer devices. The industrial control sector also adopted touch to replace large mechanical switches and gauges. Increasing interest now exists in expanding HMI beyond touch.
Limitations of Touch Interfaces
Despite the advantages of touch-based HMI, there are scenarios where using touch is problematic. Contact-based HMIs present hygiene concerns, especially in clinical environments or public spaces where bacteria can spread. Many users interact with public terminals without having opportunities to clean touch surfaces. In situations where user attention must remain focused on critical tasks—such as driving or operating heavy machinery—touch interaction can be distracting. For these reasons, non-contact HMI alternatives are receiving increased attention.
Voice control has become common in homes thanks to digital assistants, offering a simple command method, but it is not always suitable. Outdoor public spaces, industrial sites, and automotive environments can have high ambient noise levels that hamper voice recognition and require frequent corrections, which wastes time and may raise privacy concerns. Thus, while voice is effective in some contexts, alternative approaches are still needed.
Adoption of Time-of-Flight (ToF) Technology
One technology gaining attention is time-of-flight (ToF) sensing. ToF can enable control via hand motions without diverting operator attention. In simple terms, an infrared (IR) pulse is emitted and reflected by an object; the reflected signal is received by a sensor array, and the time delay between emission and reception is used to calculate the distance from the emitter to the object. Motion and gestures can be detected to identify different commands.
RF Digital's RFD77402 supports fast, accurate gesture recognition at a 10 Hz refresh rate with ±10% accuracy. The 3D ToF sensor module is packaged in a compact 4.8 mm × 2.8 mm × 1.0 mm surface-mount package and integrates a 29° illumination field with an 850 nm VCSEL emitter, associated drive circuitry, a microcontroller unit (MCU), and onboard memory. It includes a 55° field-of-view photodetector and necessary optical components. Captured gesture data can be transferred to adjacent systems via I2C.
Seeed Studio's DepthEye 3D ToF camera module uses Texas Instruments' 1/6-inch OPT8320 sensor with 80×60 pixel resolution and a 1000 fps frame rate. The camera module measures 60 mm × 17 mm × 12 mm and connects to a laptop or tablet via USB for gesture recognition. Its supported operating systems include Windows 7 and later.
STMicroelectronics' VL6180, based on the company's FlightSense technology, is designed for gesture recognition in smartphones, tablets, and household appliances. This three-in-one optical module (4.8 mm × 2.8 mm × 1.0 mm) integrates an 850 nm VCSEL emitter, a proximity sensor, and a 16-bit ambient light sensor to mitigate background illumination. Melexis offers 320×240 pixel MLX75x23 image sensors and companion ICs such as the MLX75123 targeted at automotive HMI and industrial automation. These solutions provide a complete ToF HMI system that can enable drivers to operate controls without diverting their gaze from the road, allowing functions such as phone calls or infotainment access without compromising safety. Automotive-grade components support operating temperatures from -40°C to 105°C and handle harsh ambient light conditions; systems can process incident background light up to 120 lux. Companion ICs also allow selection of regions of interest or configuration of response triggers.
mmWave Potential in HMI
Texas Instruments' IWR1642 mmWave motion sensor can be used for non-contact gesture recognition without relying on optoelectronics. It captures distance, velocity, and angle data for objects, enabling detection of vertical or horizontal swipes and finger rotations. The sensor operates in the 76 GHz to 81 GHz band, includes a 40 MHz transmitter and a low-noise (-14 dB) receiver. An ARM Cortex-R4F core handles front-end configuration and system calibration, while a high-performance C674x DSP performs signal processing. The single-chip solution integrates a phase-locked loop (PLL), analog-to-digital converters (ADCs), and about 1.75 MB of available memory. A key advantage of mmWave is its ability to sense through materials, removing line-of-sight constraints so the sensor can be enclosed behind protective housings. mmWave sensing can also be implemented with low power consumption.
Electric-Field-Based HMI
Microchip's MGC3140 controller IC is based on the company's proprietary GestIC technology and uses quasi-static electric near-field proximity sensing. This emerging technique can detect gestures up to about 10 cm from the HMI surface. The electric field is emitted from the HMI surface; a DC voltage provides a constant field strength and an AC voltage adds a sinusoidal field. Conductive objects, such as parts of the human body, induce field distortion that the controller detects. This sensing method is immune to ambient light and sound, making it suitable for gaming consoles, medical devices, automotive control pillars, and various home appliances. The MGC3140 supports spatial resolution up to 150 dpi and can report positions at rates up to 200 Hz.
Other Emerging Techniques
Several other HMI technologies are under experimentation or are already in use. Some systems project ultrasound to detect gestures. Elliptic Labs' INNER MAGIC uses the company's patented 180° field-of-view touchless sensing to detect gestures for products such as smart speakers. Ultrahaptics (now Ultraleap) developed midair haptics using a 256-element ultrasound transducer array combined with motion-tracking image sensors to create virtual HMIs that provide tactile feedback despite having no physical controls. These approaches can emulate traditional manual controls—buttons, sliders, and so on—without requiring cleaning or maintenance, which is beneficial in surgical, industrial, retail, home automation, digital signage, and automotive contexts.
Recent academic work has also advanced alternative approaches. Researchers at KAIST demonstrated acoustic localization sensing that can create virtual keyboards using standard smartphones; such systems can turn walls, tables, mirrors, and other surfaces into interactive touch surfaces with reduced latency compared with some prior techniques.
Conclusion
Touchscreens remain important and will continue to play a major role in HMI design. At the same time, a range of technologies—infrared ToF, ultrasound, mmWave, electric-field sensing, and acoustic localization—are extending the HMI toolkit. These methods can overcome limitations of touch in specific applications and offer new interaction modalities. Combining non-contact sensing with existing touch-based approaches will broaden the scope of HMI capabilities and enable new interaction experiences.
