How Is Immersive XR Built?

Introduction

Today we extend our focus to XR.

XR introduces new dimensions to human-computer interaction, and immersive experiences that blend virtual and real environments have been an industry-wide objective. What does "immersion" actually mean, and how far are we from truly immersive XR? This article addresses those questions.

Hardware and the XR Ecosystem

Since the emergence of computer technology, humans have formed interactive relationships with hardware. From the perspective of human-computer interaction, hardware forms over the past 50 years progressed roughly from game consoles to personal computers to smartphones, corresponding to dedicated computing hardware, general-purpose computing hardware, and a trend toward miniaturization. XR brings new depth to this next stage of human-computer interaction.

The content-hardware-ecosystem positive feedback loop driven by iterative XR technologies and devices has brought us to the threshold of the metaverse. Three core underlying technologies that support XR industry development are visual and intelligent computing technologies, large-screen display technologies, and high-speed connectivity technologies.

Unisoc has invested in IP integration capabilities, audio-video multimedia IP, and communications fundamentals such as 5G and Wi-Fi. The company has also addressed requirements for immersive large displays, including high dynamic range (HDR), wide color gamut (WCG), and high frame rate (HFR). Beyond product development, Unisoc participates in XR-related industry standards and collaborates with ecosystem partners on XR technology research and industry advancement.

As cross-sector innovation accelerates, XR technologies and their key capabilities are gradually integrating into consumer applications. With a growing market, always-on sensing, information delivery, and experiences that blend physical and virtual worlds are becoming more accessible.

What Is XR?

Extended Reality (XR) includes virtual reality (VR), augmented reality (AR), and mixed reality (MR). It combines computing and AI, audio-video processing, simulation, and communication technologies to construct virtual environments or entirely new virtual spaces, delivering three-dimensional immersion and multi-sensory virtual interaction.

XR creates multidimensional immersion across three aspects: visual immersion, physical immersion, and cognitive immersion. Visual immersion relies on ultra-high-definition display technologies, very high software and hardware compute capabilities, and high-data-rate transmission. Physical immersion aims to blur boundaries between physical and digital space and includes multimodal interaction, spatial computing, 3D reconstruction, localization, and mapping. Cognitive immersion expands XR's semantic and geometric understanding of real scenes and points to future technologies such as brain-computer interfaces, glasses-free 3D, and light-field displays.

Key XR Technologies

Visual quality is central to XR delivering compelling experiences. About 80% of the information humans receive is visual, so approximating human visual perception requires very high pixel density and an appropriate field of view (FOV). Pixel density depends on image resolution and display resolution; higher resolution reduces the "screen-door effect" and brings clarity closer to the real world. FOV size affects both immersion and perceived clarity: larger FOVs require rendering more content and therefore greater compute resources.

Multimodal perception and interaction fuse technologies such as facial tracking, eye tracking, voice recognition, gesture input, and haptic simulation, allowing users to interact in ways suited to their environment and workflows.

For example, voice recognition requirements vary by scenario. In home environments, noise types are diverse and sound sources can be complex, so issues like long pickup distance and low signal-to-noise ratio are primary concerns for improving user experience. In gaming, audio perception needs to be emphasized, and combining AI-driven noise reduction and AI enhancement can increase presence.

From an immersive experience perspective, Unisoc has developed audio-video system solutions that span clear voice, efficient video codec support, and ultra-high-definition intelligent display, targeting full-scene smart terminal applications and aiming to facilitate XR product and application deployment.

For instance, the M6780 smart display platform, Unisoc's first chip platform with an integrated NPU, includes an end-side intelligent voice solution. It integrates multi-microphone array audio capture algorithms, voice wakeup, and on-device keyword recognition so the chip can detect, understand, and execute voice commands.

5G Enables Richer XR Experiences

XR traffic characteristics such as non-integer periodicity, jitter, high data rates, and strict packet latency requirements present major challenges for communication systems in terms of energy efficiency and capacity. Battery limitations in handheld and wearable devices make terminal power optimization especially important. Optimizing energy use and capacity while ensuring XR service quality will accelerate XR application deployment.

5G supports high throughput, low latency, and high reliability. Considering XR traffic characteristics, networks can also introduce energy-saving and capacity optimization mechanisms. These measures not only meet XR communication needs but also expand potential usage scenarios and provide users with greater flexibility. Combined with AI and cloud computing, the efficiency and quality of immersive content delivery improve significantly, enabling users to actively interact with, select, and create experiences in real time.

Since 2018, 3GPP has included XR as an important element in 5G standards, covering XR concepts, key technologies, device types, and performance indicators. Looking toward 5G-A and 6G, beyond supporting basic XR business scenarios, additional metaverse-related requirements will become priorities. Under the trend of converging sensing, computing, and networking, the ultimate vision is pervasive intelligence.

3GPP TR26.928 Different XR types and their applications

Today, immersive interactive experiences are already present in entertainment exhibitions, industrial manufacturing, social interactions, and surgical procedures. Over the long term, XR must still address challenges in interaction fluidity, responsiveness, cognitive immersion, and content production. However, through technological convergence and broader adoption, XR exploration will expand content ecosystems and penetrate a wider range of scenarios, creating new technical and commercial opportunities.

Note

The "screen-door effect" refers to the visibility of display pixel structure due to limited display resolution, making the image appear as if viewed through a screen door.