ALLPCB
VR or 3D audio is commonly referred to as spatialization. Spatial audio makes sounds appear to originate from specific positions in a 3D space. This capability is critical for immersion because sound provides important cues about our position in a real three-dimensional environment. Like localization, spatialization depends on two key factors: direction and distance.
HRTF (Head-Related Transfer Function) and Directional Spatialization
The geometry and structure of our ears change the sound depending on the source direction. The head-related transfer function (HRTF) captures these directional effects and is used to localize sound.
How HRTF Is Recorded
To collect HRTF data, measurements are typically taken in an anechoic chamber. The subject wears headphones while sounds are played from each possible direction and the resulting sounds are recorded. By comparing the recorded signals with the original sounds, the HRTF for each direction can be computed. Since both ears and the available sample set require recordings from many discrete directions, and because physical anatomy varies between individuals, it is impractical to measure every person’s HRTF. Research labs therefore use generic reference sets that work for most users, especially when combined with head tracking.
Applying HRTF
Once an HRTF dataset is prepared, developers who know the desired position or direction of a sound can select the appropriate HRTF and apply it to the audio. This is implemented via time-domain convolution or FFT/IFFT processing. In practice, these systems filter audio signals so the sound appears to come from a specific direction. Although the concept is straightforward, implementation is computationally expensive and challenging to develop. Headphones are typically required because loudspeaker arrays introduce additional complexity.
Head Tracking
Head movement provides critical cues to recognize and locate sounds in space. Without head tracking, our ability to localize sounds in three dimensions is significantly reduced. When a user turns their head, the audio rendering must reflect that motion; otherwise the sound will seem unnatural and immersion will suffer.
Some higher-end headphones can track head orientation. If developers include head orientation data in their audio pipeline, they can render immersive sounds that remain consistent with the user's head movements.
Distance Modeling
HRTF helps determine the direction of a sound but does not provide reliable distance cues. Various cues can be used or simulated in software to estimate or convey source distance:
Loudness
Loudness is likely the simplest and most reliable cue. Developers can lower sound level based on the distance between the listener and the source.
Initial Time Delay
Reproducing early reflections and initial time delays is difficult. Accurate modeling requires calculations based on a given set of geometries and material properties, which is computationally expensive and complex to operate.
Direct Sound and Reverberation
Systems that aim to model late reverberation and reflections accurately tend to be complex and costly to run.
High-Frequency Attenuation
Air absorption causes high-frequency attenuation, which is less pronounced than other distance cues but can be modeled with a simple low-pass filter by adjusting slope and cutoff frequency. High-frequency attenuation is not as critical as some other cues, but it still contributes to perceived distance and should not be ignored.
