A bio-inspired vision sensor that can detect spectrally distinctive features

The ability to detect objects in settings with unfavorable lighting, for example at night, in shadowed locations or in foggy conditions, could greatly improve the reliability of autonomous vehicles and mobile robotic systems. Most widely employed computer vision methods, however, have been found to perform under poor lighting.

Researchers at the Hong Kong Polytechnic University recently introduced a new bio-inspired vision sensor that can adapt to the spectral features of the environments it captures, thus successfully detecting objects in a wider range of lighting conditions. This newly developed sensor, introduced in a paper published in Nature Electronics, is based on an array of photodiodes arranged back-to-back.

“In a previous paper in Nature Electronics, we presented a simple in-sensor light intensity adaptation approach to improve the recognition accuracy of machine vision systems,” Bangsen Ouyang, co-author of the paper, told Tech Xplore.

“Building on our past research efforts, we have expanded this strategy from the intensity dimension to the spectra dimension of light.”

The primary objective of this recent work by Ouyang and their colleagues was to design a vision sensor that is better than other sensors at recognizing objects in an environment marked by strong light interference and when there is smoke or fog in the air. The sensor they planned to develop would also collect data with minimal time latency, consuming very little power.

“We aimed to achieve this without relying on optical accessories or complex algorithm resources, which can increase system bulkiness, power consumption and time latency,” Ouyang explained. “The vision sensor we designed is based on back-to-back photodiodes, which consist of switchable junctions with different spectral sensitivities.

“Specifically, the shallow junction is composed of TiO₂/Sb₂Se₃, while the deep junction is made up of Sb₂Se₃/Si. The selection of these two junctions can be controlled through an external bias voltage.”

Selecting the shallow junction in the sensor enhances the sensor’s sensitivity to short-wavelength light, while selecting the deep junction boosts its sensitivity to long-wavelength light. The sensor’s unique design thus allows its photodiodes to be adjusted to match either the broadband visible spectrum or a narrowband near-infrared spectrum.

“The process of spectra adaptation takes tens of microseconds, which is comparable to the frame rate (around 100 kHz) in state-of-the-art high-speed cameras,” Ouyang said. “This spectral adaptation effectively increases the Weber contrast of the scene, leading to improved recognition accuracy of features when exposed to intense visible-light glare.”

In initial tests, the in-sensor spectral adaptation approach introduced by Ouyang and his colleagues attained highly promising results. When applied to the vision system of an autonomous vehicle, the approach was found to enable anti-glare capabilities with low time latency and low power consumption, without requiring additional optical accessories or complex computational algorithms.

“Our study introduces an innovative in-sensor spectral adaptation technology,” Ouyang said. “The process of spectra adaptation is very fast, which is comparable to the frame rate (about 100 kHz) in state-of-the-art high-speed cameras. The practical implications of this in-sensor spectral adaptation approach are significant, as it eliminates the need for optical accessories or complex algorithm resources.”

The new sensor developed by this team of researchers could advance the capabilities of robotic systems, specifically improving their ability to recognize objects in different lighting conditions without increasing their power consumption. The sensor could be integrated in a wide range of technologies, including autonomous vehicles, medical devices, industrial or manufacturing robots and surveillance systems.

“Our future research plans in this area include advancing the performance of the vision sensor in terms of responsivity, dynamic range, response speed, and other key factors,” Ouyang added. “We are also keen on exploring the integration of additional sensing functions in the vision sensors, such as polarization and 3D depth.

“Finally, our next studies will concentrate on fabricating a large-scale array of these vision sensors, along with the corresponding peripheral circuits and software, to develop a mature artificial vision chip for practical applications.”

© 2024 Science X Network