ALLPCB
A lens, some sensors, and a few other components—how do they 'teleport' you into a virtual world?
VR can seem like black magic, but each headset is a carefully engineered device that combines advanced technology and design to create immersion. Below is a breakdown of how headsets work and the components that produce a convincing VR experience.
Lenses
Lenses are among the most important elements. They are designed to trick the eye into perceiving a wide, distant space rather than a two-inch flat display. To do this, lenses focus light so the display appears at optical infinity.
Many headsets use specialized Fresnel lenses. By using a thin, circular array of prisms, a Fresnel lens achieves the same optical effect as a large curved lens while saving weight and thickness. Lenses are also used to magnify the headset's built-in displays so the image fills the user's field of view and hides the display edges.
Displays
High-performance displays are another key factor in convincing VR. They need sufficient pixel density for sharp images and a fast refresh rate so motion appears smooth.
HTC Vive and Oculus headsets use two 1080×1200 displays, one per eye, capable of presenting images at 90 frames per second. That provides smooth motion and a roughly 110-degree field of view, covering a large portion of the user's vision.
Some high-end headsets use dual displays to create stereoscopic 3D, similar to the Nintendo 3DS. Each display shows a slightly offset image for each eye, and the brain fuses them into a single image, producing a sense of depth.
Samsung Gear VR uses a smartphone as the display. To keep costs down and enable a wireless experience, it sacrifices some field of view and graphical fidelity. In that design, creating the stereo image is handled by interchangeable lenses.
Focus Adjustment
Because the distance between each person's pupils, or interpupillary distance (IPD), varies, lens position in a headset must be adjustable to provide correct stereoscopic alignment for each user.
Some headsets, such as certain Oculus models, use hybrid Fresnel lenses with variable focus. Users can adjust focus by moving the headset up or down to find the optimal position.
Positional Sensors
To render the correct image as you look around, a headset must track head movement with millimeter-level precision. That tracking is achieved with several onboard sensors. With data from these sensors, a headset can provide true six degrees of freedom, tracking both translations along the x, y, and z axes and rotations about those axes.
Magnetometers measure the Earth's magnetic field, providing a reference direction and helping prevent heading drift.
Accelerometers detect gravity so the headset knows which way is up, and they measure acceleration along one or more axes, providing useful data about motion. Smartphones use accelerometers to auto-rotate the screen.
Gyroscopes track small angular velocity changes, such as slight head tilts or nods, providing precise rotation information.
Samsung Gear VR relies on an inertial measurement unit (IMU), which combines a magnetometer, accelerometer, and gyroscope into a single device. Unlike the IMUs used in most smartphones, this IMU is tuned to reduce latency and improve head-tracking performance.
Infrared Tracking
Both Oculus Rift and HTC Vive use infrared-based systems to track headset movement, but they take different approaches.
Oculus uses Constellation infrared cameras placed around a desk area to track infrared emitters located on the Rift headset. If you use Oculus Touch controllers, additional cameras may be required so the system can distinguish between infrared lights on the headset and those on the controllers. Each sensor is tracked separately, and the computer collects all of the data to render a correct view from any angle. This processing happens almost instantly to keep latency minimal.
HTC Vive uses Lighthouse base stations placed in the corners of the play area. These emit sweeping infrared lasers that scan the room. Photodiodes on the Vive headset and controllers detect the sweeping laser pulses and measure timing to calculate precise positions. Functionally this is similar to the Oculus approach, but the roles are reversed: Lighthouse units are the emitters and the headset acts as a camera.
Boundary System
In addition to Lighthouse tracking, Vive headsets include a front-facing camera and can use a boundary system to detect when the user approaches the edges of the play area. If you are about to collide with a wall or furniture, the headset can display a visual warning to indicate you are near the play-area boundary.
Controllers
Rift and Vive both provide wireless motion controllers that allow users to interact with objects in 3D space, increasing immersion.
Like the headset, each controller includes a magnetometer, accelerometer, gyroscope, and infrared sensors to enable millimeter-precise motion tracking.
Audio
Many headsets include integrated headphones capable of producing three-dimensional audio. Games can place sounds relative to the user's position so audio appears to come from behind, above, or below.
Built-in microphones give developers options for additional immersive features, such as detecting sound levels produced by the user or enabling voice communication in VR.
Cabling
Some VR devices, like Samsung Gear VR, are entirely wireless, but both Vive and Rift typically require cables to connect the headset to a PC for data and power, allowing high-resolution displays to render at 90 frames per second. Cables are not a problem for seated experiences, but they can be cumbersome when the user moves around the room.
Computing
Most of the heavy lifting happens on the computer. All position-tracking data is sent to the PC, fed into the application, which then renders frames and sends the images to the headset.
For standalone headsets such as Samsung Gear VR and other mobile VR systems, the compute unit sits inside the headset. In Gear VR, the smartphone acts as both the display and the processor.
