Visible light cameras are needed for future AV and ADAS applications alongside thermal cameras which are vital for driving safely in dark and harsh weather conditions.
Thermal cameras are awesome, they really are. They enable seeing things that are not visible to a human eye and are extremely useful in many industries and applications. Their pricy value and military origins held the civil market from fully utilizing and adopting the advanced technology for various improvements and new applications. One of the new arising markets that is driving prices down and innovation up is the automotive industry, with the goal of achieving fully autonomous driving. The size of long wave infrared (LWIR) cameras is decreasing, pixel resolution is increasing, and overall quality is growing fiercely. So, if these cameras are so good and can spot a human or a vehicle from hundreds of meters away, why do we still need visible-light cameras for self-driving cars?
Well, to answer this question, we first need to understand what exactly thermal cameras are and how they work. Long wave infrared cameras, also known as thermal cameras, detect the heat emitted by an object. All real-world objects radiate infrared energy. The difference in the amount of energy emitted by an object depends on two main factors: temperature and material properties.
A thermal camera can detect and quantify the amount of energy emitted by the object or surface it sees, thus differentiating between objects by their heat signature or by their difference in material properties (also called emissivity).
In other words, thermal cameras can distinguish between objects if there is a temperature difference or by the difference in material properties (emissivity) even if their temperature is the same).
For example: If we fill a glass halfway with hot water, a thermal camera will show a temperature difference even though the entire exterior surface of the glass is made of the same material.
Likewise, in the picture below, the ring is the same temperature as the skin but appears colder, because of emissivity (i.e. material) differences.
After we established that material properties affect the thermal image as well as surface temperature, what are the implications for a self-driving car?
Well, due to its unique capabilities, a thermal camera can see even in complete darkness and in harsh weather conditions like rain, fog, and snow. This is where the true strength of thermal cameras appears. On the other hand, thermal cameras cannot really see color tones so they are unable to read road signs or traffic lights efficiently.
We live in a colorful world. We perceive the world in colors and our visual communication is largely in color, but this is true only if we have a strong light source and clear weather to see far ahead. So, I believe the answer to the original question is, yes, we will still need visible light cameras in future AV and ADAS applications alongside thermal cameras which are vital for driving safely in dark and harsh weather conditions.