Mobile Image Algorithm Solutions

With the advent of the smartphone era, photography has undergone tremendous changes. It is no longer just about lenses and sensors; a series of image algorithms are now loaded behind them. These algorithms enable faster image processing and achieve better shooting results.

The era when excellent works relied solely on professional photographers is gone. With the introduction of image algorithms into smartphone photography applications, it has become possible for everyone to capture great photos.

The human perception of color is an important ability that has evolved over a long period of time, allowing us to see a vibrant world. But how are colors presented?

When light passes through the eyes and forms an image on the retina, the photoreceptor cells on the retina perceive the intensity of the light signals. These signals are then transmitted by the visual nerves and processed by the brain, resulting in visual perception.

The generation of color is extremely complex, as the human experience of color not only depends on the physical characteristics of light but is also influenced by psychological and other factors.

Firstly, let's understand the physical characteristics of color. The visible light spectrum that the human eye perceives generally ranges from 380 to 780 nanometers. Different wavelengths of visible light appear as different colors. The ability to discern colors is the result of the visual neural response stimulated by the visible electromagnetic radiation received by the human eye.

The light-sensitive cells in the human eye are divided into two types: rods and cones.

Among them, rods are most sensitive to dim light and have a spectral response range that covers the entire visible light spectrum, reaching a peak around 500 nanometers. However, rods cannot differentiate between light of different wavelengths, thus unable to produce color vision.

In very low light levels, this type of vision is called scotopic vision, and it relies solely on the function of rods. Therefore, in the dark of night, people can see objects but cannot distinguish their colors.

In high light levels, cones are the primary cells involved, allowing for color vision. Cones are further divided into three types: L cones (red-sensitive), M cones (green-sensitive), and S cones (blue-sensitive). Different cones have different response capabilities to light of the same wavelength, enabling color discrimination.

When a light source emits electromagnetic waves with a specific spectral distribution, these waves enter the human eye through reflection or transmission by objects. The cones in the eye detect the light, encompassing the physical properties of color.

Colors are also influenced by psychological factors and people's real-life experiences. The signals produced by the visual cells in response to light in a particular scene do not necessarily correspond to people's perception of that scene.

For example, regardless of whether a white paper is placed under red or green light, it appears white to the human eye. This phenomenon is due to the brain's experiential processing and is known as color constancy.

It can be said that the scenes you see are processed and interpreted by the brain, and what meets the eye is not always the reality.

The primary utility of image algorithms lies in their ability to enhance and restore photos, allowing captured images to exhibit realistic color effects or meet users' specific color preferences.

Automatic White Balance

In Unisoc's ISP (Image Signal Processing) system, there are primarily two modules, namely Automatic White Balance (AWB) and Color Correction Matrix (CCM), used to handle color processing.

Let's begin with Automatic White Balance. The ISP system needs to calculate the gains for the R and B channels based on the statistical information of the image. This calculation aims to achieve automatic white balance functionality, ensuring accurate reproduction of the true colors of objects in various complex scenes.

Unlike the human visual system, which possesses color constancy, allowing us to perceive objects regardless of the light source, image sensors do not possess this characteristic. Therefore, images captured under different lighting conditions may exhibit different colors.

For example, an image captured under a clear sky may appear biased towards blue, while an object captured under candlelight may appear biased towards red.

The purpose of automatic white balance is to simulate the color constancy of the human visual system, thus mitigating the adverse effects of the light source on image rendering.

Color Correction

Cameras aim to mimic the color imaging of the human eye when capturing images. However, due to the different spectral responses of the RGB channels of the camera sensor compared to the photoreceptor cells in the human eye, there are discrepancies between the captured images and what the human eye perceives.

Therefore, color correction (Color Correction Matrix, CCM) is necessary.

The color correction matrix is usually obtained through calibration. By comparing the images captured by the image sensor with reference images, the correction matrix (i.e., CCM matrix) is calculated. The CCM matrix is then used to perform color correction on each pixel of the image, aiming to achieve colors that are closest to the true colors of the objects.

In general, color saturation adjustment accompanies the color correction process. Color saturation refers to the purity of colors, where higher purity results in more vivid image representation, while lower purity leads to a duller appearance.

With the advancements in technologies such as machine learning and the increasing demand for smartphone photography capabilities, color reproduction techniques face new opportunities and challenges. For example, there are opportunities to leverage AI technology to enhance color reproduction effects and achieve more consistent color reproduction on smartphones with multiple cameras.

These additional technologies will provide users with a better smartphone photography experience and contribute to enhancing the competitiveness of the company's products.