ALLPCB
Overview
ADAS feature requirements in the automotive industry are increasing, and driver drowsiness detection is a common focus. Face detection is the basis for monitoring driver state. Only by quickly and accurately detecting faces can facial conditions be analyzed. This article describes a face recognition application implemented on the S32V vision platform.
S32V Vision Processing Platform
NXP began releasing the S32V platform in 2015. The second-generation S32V234 is available and the third generation is currently in sample phase. The platform targets ADAS vision processing and provides the performance and functions required for vision system applications.
On the hardware side, S32V includes two CSI inputs and two parallel camera interfaces, a programmable ISP, the APEX parallel processing engine, and a GPU for 3D rendering. The programmable ISP can process camera image data from CSI inputs and supports HDR, color conversion, tone mapping, and other functions. APEX provides parallel image processing capabilities. The platform also includes H.264 hardware codecs, covering the main hardware resources needed for image processing.
On the software side, the VSDK provides drivers for the S32V platform peripherals and related examples. On the API side, common image processing libraries such as OpenCV, FFmpeg, OpenCL, OpenGL, and EGL are available.
Figure 1 S32V internal block diagram
Face Recognition Solution
The demonstration uses a Sony IMX224 camera as the image input, ports a face recognition library, and builds a demo that draws bounding boxes around detected faces in real time.
1. System block diagram
Figure 2 Face recognition system block diagram
2. Demo setup
- Acquiring camera data
First, implement IMX224 camera data acquisition based on the VSDK. Using S32 Design Studio for Vision, create the IMX224 ISP processing flow as shown below.
Figure 3 ISP processing flow
Here, running the debayer_rgb_simple_interleaced kernel on IPUS0 converts the camera Debayer data to RGB. The result is transferred to DDR via FDMA for algorithm use. If special image processing is required, a custom IPUS kernel can be created to process the image data.
- Cross-compiling required libraries
The face recognition algorithm is built on NCNN. Dependencies include OpenCV, NCNN, and SQLite3. These libraries need to be cross-compiled. Prebuilt ARM versions of OpenCV and SQLite3 for S32V are already provided, so recompilation is unnecessary. The compiled NCNN and face recognition libraries are static libraries and do not need to be copied to the target board.
- Face detection demo
The UI is implemented with Qt. Add the VSDK camera acquisition libraries and the ported algorithm libraries to the project file, then obtain image frames using the following API:
int getCImg(cv::Mat &cimg);
Then call the face detection interface:
void detectMaxFace(ncnn::Mat & img_, std::vector<Bbox> & finalBbox);
3. Demo results
The detection results are displayed in the Qt interface as shown below.
Testing shows the face detection timing as follows.
