Simultaneous broadband image detection and convolutional processing using van der Waals heterostructures

Simultaneous broadband image detection and convolutional processing using van der Waals heterostructures

Credit: Pi et al.

Efficient processing of broadband signals using convolutional neural networks (CNNs) could improve the performance of machine learning tools for a wide variety of real-time applications, including image recognition, remote sensing and environmental monitoring. However, previous studies suggest that performing broadband convolutional processing computations directly in sensors is challenging, especially when using conventional complementary metal oxide-semiconductor (CMOS) technology, which supports the operation of most existing transistors.

Researchers from Huazhong University of Science and Technology and Nanjing University recently explored the possibility of realizing the convolutional processing of broadband signals using an alternative platform, namely van der Waals heterostructures. Their paper, published in Nature Electronicscould ultimately contribute to the development of better performing image recognition algorithms.

“Our paper is inspired by some of our previous research work,” Tianyou Zhai, Xing Zhou and Feng Miao, three of the researchers who conducted the study, told TechXplore. “In studies published in Advanced materials and Advanced functional materials, we realized band alignments of type III and type II in different heterostructures. We also published a paper in scientific progresswhere we realized a reconfigurable neural network vision sensor based on WSe2

Building on their previous research efforts, Zhai, Zhou, Miao and their colleagues decided to fabricate heterostructures that can be used to process broadband signals using convolutional neural networks. They specifically created heterostructures based on PdSe2/MoTe2using a mechanical transfer method.

The heterostructures created by the researchers exhibit gate-tunable positive and negative photoresponses, as well as broadband linear gate-dependent photoresponsiveness. Their beneficial properties allowed the team to use their heterostructures to create photovoltaic sensors, which enabled them to implement different types of broadband convolutional processing.

Simultaneous broadband image detection and convolutional processing using van der Waals heterostructures

Credit: Pi et al.

“Under a fixed gate voltage, the photoresponse modulation under different photon energies determines the wavelength-dependent convolutional characteristics,” explained Zhai, Zhou and Miao. “In addition, the photoresponse depends on the gate voltage, and modulation of the gate voltage can be used to realize the configuration of different convolution kernels to achieve different operations on remote sensing images.”

As a first proof-of-concept, Zhai, Zhou, Miao and their colleagues used a single device based on their heterostructures to receive pixel images individually and then perform broadband convolutional processing on these images. In the future, however, they could also test their proposed system with two or more devices.

The researchers are among the first to perform broadband image recognition directly in-sensor. Their findings are promising, as their solution significantly outperformed conventional convolutional networks capable of single-band processing only.

In particular, the device created by Zhai, Zhou, Miao and their colleagues achieved photo detection and broadband information processing at the same time. Thus, in the future, it could be used to develop more advanced image recognition tools, as well as alternative solutions to monitor remote environments.

“Using our newly designed special ambipolar vdW heterostructure, we have developed a new approach to realize multifunctional optoelectronic devices in the sensor,” added Zhai, Zhou and Miao. “We now plan to realize hardware-level in-sensor memory computing for more complex functional requirements. Meanwhile, we also plan to realize large-scale integration of these devices to implement practical convolutional processing in the sensor.”

LOEN: Optoelectronic neural network-driven machine vision without a lens

More information:
Lejing Pi et al, Broadband convolutional processing using band-alignment-tunable heterostructures, Nature Electronics (2022). DOI: 10.1038/s41928-022-00747-5

Xing Zhou et al, Tunneling Diode based on WSe2/SnS2 heterostructure with high detection and responsiveness, Advanced materials (2018). DOI: 10.1002 / adma.201703286

Bao Jin et al, Excellent exciting photovoltaic effect in 2D CsPbBr 3 /CdS heterostructures, Advanced functional materials (2020). DOI: 10.1002 / adfm.202006166

Chen-Yu Wang et al, Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor, scientific progress (2020). DOI: 10.1126/sciaadv.aba6173

© 2022 Science X Network

Quote: Simultaneous broadband image detection and convolutional processing using van der Waals heterostructures (2022, May 13) Retrieved May 15, 2022 from

This document is copyrighted. Other than fair dealing for personal study or research, nothing may be reproduced without written permission. The content is provided for informational purposes only.

Leave a Comment