ALLPCB
Introduction
When discussing the most important technologies in VR, eye tracking deserves close attention. Oculus founder Palmer Luckey called it "the heart of VR". This article reviews the technology that enables interaction through gaze.
1. What is eye tracking?
Eye tracking is a scientific application that lets users operate devices without touching the screen. When the eyes look in different directions, subtle changes produce extractable features. Systems use image capture or scanning to extract these features, track eye movement in real time, predict the user's state and needs, and respond so the device can be controlled by gaze.
Many people are familiar with examples: Samsung Galaxy S4 used eye control to pause and play video, and there were rumors that the iPhone 7 would correct words in text based on gaze. Manufacturers continue to explore the area, and VR is currently the most active field researching this technology. So what changes does eye tracking bring to VR?
2. What can eye tracking deliver?
Eye tracking can help address motion sickness in VR. The image formed on the eye adapts to the distance to objects, but with VR headsets the eye-to-screen distance is fixed, which can create a mismatch. When the scene switches between far and near objects, the eyes cannot accommodate properly, which can cause discomfort. Eye tracking can determine the true gaze point and thus obtain depth at the viewpoint on virtual objects, making it a potential technical breakthrough for reducing VR headset sickness.
Advanced eye tracking can also make VR images appear more natural and reduce latency, which would improve usability and immersion. However, several technical challenges must be solved first.
3. What breakthroughs are needed?
Although many companies are researching eye tracking, no current solution is fully satisfactory.
For example, pupil shift causes image distortion in VR as the eyes move. Real-time correction based on eye tracking is required to fully fix this. Palmer Luckey has said Oculus is using optical advantages to try to address the issue, but it remains unresolved and cannot be completely solved without eye tracking.
Pei Yun, head of the graphics and imaging algorithms center at SuperD, notes that VR eye tracking could use devices similar to Tobii eye trackers, but size and power consumption must be addressed. The smallest Tobii devices today are roughly the size of a pen, which is still too large for VR headsets. Power is also a concern because eye trackers emit infrared; external devices attached to laptops can become noticeably warm after use. Applying these devices to VR headsets requires further research to reduce power consumption.
Industry experts believe eye tracking itself is feasible for VR—through measures like external power or larger headset designs—but the greater challenge is developing the image-adjustment algorithms that adapt rendered images to eye movement. Two key metrics are required: natural-looking images and low latency. If both are achieved, VR usability and immersion would improve significantly.
