ALLPCB
Overview
A lens, some sensors, and a few other components: how do they transport you into a virtual environment? VR may seem like magic, but each headset is a carefully engineered device that combines advanced optics, sensors, displays, and processing to present a convincing virtual world. The sections below break down the main subsystems and explain how they work together.
Lenses
Lenses are one of the most important elements. They are designed to fool your eyes into perceiving a wide space rather than a small display panel. To do this, the lenses focus light so the display appears to be far away.
Many headsets use specialized Fresnel lenses, which use a thin array of concentric prismatic rings to achieve the same optical effect as large curved lenses. The lenses also magnify the headset's built-in displays so the image fills most of your field of view and hides the display edges.
Displays
High-performance displays are another key factor for convincing VR. They need sufficient pixel density for clear images and high refresh rates so motion appears smooth.
Headsets such as the HTC Vive and Oculus Rift use two 1080 x 1200 displays, one per eye, capable of showing images at 90 frames per second. That provides smooth motion and a roughly 110-degree field of view, covering a large portion of the user's vision.
Many high-end headsets use separate displays for each eye to create a stereoscopic 3D effect. Each display shows a slightly offset image for each eye, and the brain fuses them into a single image, producing a perception of depth similar to the effect used by the Nintendo 3DS.
Samsung Gear VR and similar designs use a smartphone as the display and processor. To keep costs down and remain wireless, these designs trade off field of view and graphical fidelity. In those systems, stereoscopic imaging is achieved by the headset’s two lenses placed in front of the smartphone display.
Focus and IPD Adjustment
Because the distance between a person's pupils, or interpupillary distance (IPD), varies, the lens position in a headset must be adjustable to provide correct stereoscopic alignment. Some headsets use adjustable lens spacing or allow the user to move the headset up and down to find the best focus.
Some designs use hybrid Fresnel lenses with variable focal characteristics so users can fine-tune the perceived focus by shifting the headset to find the optimal position.
Positional Sensors
To render accurate imagery as you look around, a headset must track head movements with millimeter-level precision. This is done with a combination of onboard sensors so the system can provide six degrees of freedom: translation along the x, y, and z axes and rotation around those axes.
A magnetometer measures Earth’s magnetic field so the headset always knows magnetic north, which helps prevent long-term drift. An accelerometer detects gravity and linear acceleration, which tells the system which direction is up and helps estimate motion. A gyroscope tracks small rotational movements, giving precise rotation information.
Many headsets use an inertial measurement unit (IMU) that combines a magnetometer, accelerometer, and gyroscope into a single module. Compared with standard smartphone sensors, VR-grade IMUs are optimized to reduce latency and improve head-tracking performance.
Infrared Tracking
Some headsets augment inertial tracking with optical or infrared tracking systems to improve positional accuracy.
Oculus uses a constellation of infrared cameras positioned around the play area to track infrared LEDs on the headset and controllers. Each camera independently tracks the emitters; the computer aggregates the data to compute precise position and orientation and render the correct images with minimal latency. Using controllers may require additional cameras to avoid occlusion and ensure consistent tracking.
HTC Vive uses Lighthouse base stations placed in the corners of the play area. The base stations sweep the room with infrared lasers, and sensors on the headset and controllers detect the sweeps. By timing the sweeps, the system computes the devices' positions. Functionally this is similar to the Oculus approach, but the roles of emitter and sensor are reversed.
Guardian System
Some headsets include a forward-facing camera or software-based boundary system to detect the edges of the play area. This allows the system to warn users when they approach walls or furniture, helping prevent collisions with real-world objects.
Controllers
Modern VR controllers are wireless motion controllers that enable interaction with objects in 3D space. Like headsets, controllers typically include a magnetometer, accelerometer, gyroscope, and optical or infrared sensors to achieve millimeter-level tracking accuracy.
Audio and Microphones
Many headsets integrate headphones to provide spatialized audio cues so sounds appear to come from specific directions relative to the user. Built-in or hidden microphones allow voice communication and can enable gameplay features that respond to user sounds or speech.
Cables
While some VR devices are wireless, many high-end systems require a cable to connect the headset to a computer to transmit high-bandwidth video and power needed for low-latency, high-frame-rate displays. Cables introduce practical issues when users move around a play area, so cable management and routing are important design considerations.
Processing and Rendering
Most of the heavy lifting happens on the connected computer. Tracking data from sensors and external trackers is sent to the computer, which updates the scene, renders images for each eye, and streams the frames to the headset.
In mobile or phone-based systems, the "computer" is inside the headset: the smartphone provides the display and performs the processing tasks locally.
