Stereo Vision ADAS Solution

The Stereo Vision ADAS Chip - S32V234

The S32V234 chip utilizes four ARM Cortex-A53 cores as the core CPUs to achieve higher performance-to-power ratio. It incorporates an ARM Cortex-M4 as an on-chip MCU for real-time control of critical I/Os, such as CAN-FD, and supports the AutoSAR operating system. 

The chip also features a programmable Image Signal Processor (ISP), allowing the paired image sensors to output raw data. This capability reduces material costs and saves space dimensions.

Furthermore, the chip also includes two vision acceleration engines called APEX2CL. Each APEX2CL engine features 64 local computing units (CUs) along with local memory and dedicated DMA. 

These engines accelerate the image recognition process through SIMD/MIMD (Single Instruction Multiple Data/Multiple Instruction Multiple Data) techniques.

It is worth mentioning that considering the stringent requirements for safety and reliability in ADAS systems, the S32V234 chip incorporates various safety mechanisms such as ECC (Error Checking and Correction), FCCU (Fault Collection and Control Unit), M/L BIST (Memory/Logic Built-In Self-Test), and more. 

These mechanisms ensure compliance with ISO 26262 ASIL B~C requirements.

Advantages of Stereo Vision ADAS

Compared to monocular vision, the key advantage of stereo vision is its ability to utilize dual cameras to capture images of the same object from different angles, allowing for the extraction of disparity information and estimation of target distance. 

In the context of visual ADAS applications, using a single camera typically requires extensive data collection and training for machine learning algorithms to recognize objects such as pedestrians and vehicles, and it struggles to identify irregular objects. 

On the other hand, while using mmWave radar and LiDAR for ranging provides high accuracy, it also comes with higher costs and complexities.

The major advantage of stereo vision is that it achieves target recognition and ranging with a certain level of accuracy while maintaining lower development costs, enabling the implementation of ADAS features like Forward Collision Warning (FCW).

The basic principle of stereo vision ranging is as follows: the disparity of the target point P between the two cameras is given by d=EC+DF. 

By applying the principles of triangle similarity, the distance z can be derived as z=(fq)/d, where the focal length f and camera baseline distance q can be considered fixed parameters. 

Therefore, by obtaining the disparity signal d, the distance z can be determined.

Steps for Stereo Vision Ranging

Camera calibration

Image acquisition

Image preprocessing

Feature extraction and stereo matching

3D reconstruction

Among these steps, camera calibration is performed to obtain the camera's intrinsic and extrinsic parameters, as well as distortion coefficients, and it can be done offline. 

However, challenges arise in implementing stereo vision ADAS on an embedded platform due to requirements such as synchronization of image acquisition between the left and right cameras, the quality and consistency of image preprocessing, and the computational demands posed by real-time requirements of stereo matching (disparity extraction) and 3D reconstruction (distance estimation).

Applications of Stereo Vision ADAS

The S32V234 chip is equipped with two MIPI-CSI2 camera interfaces, each capable of providing a maximum transfer rate of 6Gbps, which can be used for video input from the left and right cameras. 

Since the two cameras input through separate MIPI channels, synchronization between them needs to be considered.

With the cooperation of external image sensors, the S23V324 can support different synchronization methods. Image sensors typically have frame synchronization signals (VSYNC) and line synchronization signals (HSYNC) for signal synchronization:

• When the two cameras work in master-slave mode, the master sends synchronization signals to the slave.

• When both cameras work in slave mode, the S32V234 can generate synchronization signals internally using a timer and send them to both cameras simultaneously.

After the S32V234 obtains the image signals from the external cameras, they can be preprocessed by the internal ISP. 

The ISP module contains multiple internal processing units that use on-chip SRAM to cache input signals and intermediate results. 

It also employs dedicated co-processors to manage the timing of ISP processing units, thereby achieving a pipelined ISP processing.

The built-in ISP not only saves costs for dual-camera setups but also provides sufficient computational resources and bandwidth to support real-time processing of dual high-definition image signals, ensuring the quality and consistency of the dual-camera image signals.

Stereo Vision ADAS Solution

In stereo vision ADAS applications, the biggest challenge lies in the enormous computational power required for stereo matching and 3D reconstruction of the two images.

Taking FCW (Forward Collision Warning) applications as an example, it requires accurate extraction of disparity signals to ensure distance measurement precision while maintaining a certain frame rate to ensure the responsiveness of warnings. Therefore, it is necessary for the embedded platform to have sufficient processing capabilities.

The structure of the image acceleration engine APEX2 integrated into S32V234, with its parallel computing structure, local memory, and dedicated DMA design, ensures high processing efficiency for image signals.

After preprocessing by the ISP, the image data is sent to DDR. The APEX2 engine segments the image and uses a dedicated DMA to transfer it to the local memory CMEM corresponding to each computing unit (CU). 

The algorithms for block matching and other processes required for stereo matching can be parallelly processed in different CUs. 

The processed data is then transferred back to DDR via DMA for further processing by the CPU (such as generating warning signals) or sent to the dedicated DCU (Display Control Unit) module for display output.

In summary, the data flow of the stereo vision application based on S32V234 follows the direction of ISP-APEX2-DCU, with the A53 serving as the main CPU for logical control and necessary data processing.

By employing this pipelined processing approach, each computational resource can be fully utilized.

Using the S32V234 development board to build a stereo vision platform, processing of dual 720p@30fps video signals is performed, and the output results are shown in pic. 

The three images from left to right represent targets at distances of 1m, 2m, and 3m from the camera, and the displayed results indicate the target distance through changes in color temperature.

The results demonstrate that the S32V234 can perform real-time processing of stereo vision signals and accurately obtain 3D distance measurement results. 

When combined with the chip's various safety design features, it can meet the requirements of a stereo vision ADAS system.

Summary

NXP's dedicated vision ADAS chip, the S32V234, integrates various specialized computational units such as the Image Signal Processor (ISP), the Graphics Acceleration Engine APEX2, and the 3D GPU. Through a pipelined processing architecture, it fully utilizes the heterogeneous computational resources.

The support for multiple APIs such as OpenCV, OpenCL, and OpenVG by different computing modules enhances algorithm portability. The functional safety design compliant with the ISO 26262 standard ensures that the chip meets the stringent safety requirements of ADAS systems.

The S32V234 supports various vision ADAS and sensor data fusion solutions, including stereo vision, taking a solid step forward on the path towards autonomous driving.