ALLPCB
Overview
Polarimetric imaging acquires and interprets polarization information to suppress complex environmental interference, improve image quality, and enhance target perception. It offers unique advantages for optical imaging and detection in challenging environments. This article reviews recent advances in deep learning–based polarimetric imaging, explains the advantages of deep learning for polarization imaging tasks in complex environments, and outlines future directions for the field.
Introduction
Human vision is the primary way of acquiring information about the external world, which makes optical imaging important across many domains. Traditional optical imaging relies mainly on intensity and wavelength information, which are strongly affected by the propagation environment and can fail to reveal certain physical properties of targets. Polarization, as an intrinsic physical property of light, provides additional information about target material and surface characteristics, adding a new information dimension to optical imaging. However, in scattering, low-illumination, and other complex environments, polarization parameter images can be severely degraded. Both research and practical experience show that acquiring and interpreting polarization information can effectively mitigate complex-environment interference, improve image quality, and enhance target sensing capability. The degradation mechanisms of polarization images in complex environments are nonlinear, so interpreting polarization information is a nonlinear inverse problem with high complexity. Deep learning, with its strong feature extraction and learning capability, can learn mappings hidden in large datasets to restore polarization information and is particularly suited to multidimensional, interrelated signal-processing problems such as polarimetric imaging.
Research Background
Polarimetric imaging essentially increases the information dimensions of the optical field. By acquiring and fusing multidimensional polarization data, imaging tasks in various complex environments and application scenarios can be addressed. Methods based on physical models and prior knowledge are important for solving polarization imaging in complex environments. For example, researchers have leveraged scattering models and physical mechanisms to interpret polarization data and improve imaging in water, haze, and other scattering environments. The restoration quality of model-based polarimetric methods depends heavily on how well the constructed physical model matches real-world imaging processes. Simplifications and deviations in the physical model can limit performance in extremely complex environments.
With advances in computing and more powerful hardware for processing large-scale data, deep learning has rapidly developed and demonstrated advantages in optical imaging. Deep learning excels at extracting and learning features from large datasets to infer mappings between input signals and desired outputs, which is especially beneficial for multidimensional, interdependent polarization parameter signals. Deep learning–based polarimetric imaging bypasses the need to explicitly construct a physical model to solve a nonlinear inverse problem and has achieved significant improvements on core objective image-quality metrics. This article summarizes the basic paradigm of deep learning polarimetric imaging and systematically reviews representative work addressing two typical complex imaging scenarios: scattering and noise.
Technical Highlights
1. Deep Learning Paradigm for Polarimetric Imaging in Complex Environments
Unlike approaches that rely on explicit physical models and priors, deep learning methods use multidimensional polarization-parameter images captured by imaging systems as input and learn the nonlinear mapping from polarization-parameter images to restored images through network fitting. Figure 1 illustrates the deep learning paradigm for polarimetric imaging in complex environments.
Figure 1. Deep learning paradigm for polarimetric imaging in complex environments.
2. Deep Learning for Polarimetric Imaging in Scattering Environments
Deep learning methods for polarimetric imaging in scattering media aim to interpret polarization signals affected by scattering, using networks' nonlinear fitting and feature-learning capabilities to map degraded images to restored images. Representative approaches fall into two categories: data-driven methods and physics-embedded methods, and the literature shows how the field has evolved along these two lines.
Haofeng Hu's team at Tianjin University first proposed the Polarimetric Residual Dense Network (PDN) for underwater complex imaging. The network structure is shown in Figure 2(a). Polarimetric-Net takes three polarization images at different angles as input, while Intensity-Net takes only a single intensity image. This design verifies that using multidimensional polarization information as input yields de-scattering results far superior to those obtained using only intensity. The restoration comparisons in Figure 2(b) show PDN achieves the best results in both subjective assessment and objective metrics such as Edge-Model Estimation (EME).
After this initial work, researchers adopted unsupervised and self-supervised learning strategies and used simulation to synthesize the necessary training data to address the lack of public datasets and the difficulty of collecting large-scale polarization data for deep-learning de-scattering methods.
Data-driven methods treat the model as a black box that learns the mapping between input and label images. Training data cannot cover all scattering scenarios, so these methods have limitations when faced with new conditions and may produce restorations inconsistent with physical knowledge. Embedding physical models and priors into network structures enhances the network's ability to reason about the polarization-degradation process and improves adaptability to varying scattering environments.
3. Deep Learning for Polarimetric Imaging in Noisy Environments
Under the same noise level, nonlinear algorithms that derive polarimetric parameter images often show more severe quality degradation than linear algorithms that estimate the Stokes vector. The degradation is especially pronounced in polarization-angle images, where target information can become nearly invisible. Thus, denoising polarization-parameter images degraded by noise is critical for applying polarimetric imaging in low-illumination complex environments. Research in this area can be grouped into three stages. The first stage used supervised end-to-end training to achieve denoising of polarization parameter images. The second stage explored unsupervised, self-supervised, and transfer-learning methods to address the high cost and low efficiency of dataset construction in stage one. The third stage focused on improving the interpretability of network training.
In 2020, Hu's team proposed a polarization denoising network based on a dense residual architecture called PDRDN. Figure 3(d) shows high-quality restoration of polarization parameters under low signal-to-noise ratio conditions compared with ground truth. Deep learning–based denoising methods can effectively suppress noise in polarization images, with particularly strong performance for degree-of-polarization and polarization-angle images.
Figure 3. (a) PDRDN network structure schematic; (b) residual dense block structure; (c) restoration results for polarization parameters across test targets with different materials; (d) detail comparison of restored polarization-parameter images with ground truth.
Conclusions and Outlook
Existing research indicates that because polarization information consists of multiple interrelated parameters, this multidimensional, correlated signal-processing problem is well suited to deep learning. Combining deep learning with polarimetric imaging can further improve optical imaging quality and meet complex-environment imaging requirements. Future work should focus on better embedding physical models, improving network interpretability, exploring multimodal fusion strategies, and strengthening model robustness to environmental variations, to increase generalizability across scenarios.
