Research Topics
Neuromorphic Computing with Light
Recent advances in artificial intelligence (AI) has resulted in a resurgence of interest in developing photonic neural networks for efficient machine-learning computation. AI and photonic computing form a nice synergy since neural-network models, especially large ones, bring the best out of optical processors by stressing their strengths in data transmission and reuse. Based on these physical properties, photonic neural networks represent a class of analog computing systems that have been experimentally shown to approach the physical limit of computation, using even single photons for each operation. Our interest in photonic neural networks is twofold:
- Hardware: We are interested in exploring a hardware platform that enables photonic neural networks to be both scalable and compact.
- Software: We are interested in using photonic neural networks as a testbed to study novel computing phenomena in the analog noisy regime.
Top: a free-space photonic-neural-network apparatus for large-scale matrix-vector mulitplication (~700×700 in matrix size)
Bottom: a generic photonic chip as an integrated platform for optical information processing; image from imec.
- P.L. McMahon. The physics of optical computing. Nature Review Physics (2023).
- T. Wang, et al. An optical neural network using less than 1 photon per multiplication. Nature Communications 13, 123 (2022).
- S. Lloyd. Ultimate physical limits to computation. Nature 406, 1047–1054 (2000) | Wikipedia Limits of Computation
Computational Sensors based on Neuromorphism
Another opportunity offered by photonic neuromorphic computing is its potential to make smarter and more efficient sensors. Unlike computational imaging, whose goal is to produce full-resolution images through computational reconstruction, the primary goal of image sensing is to capture only salient features at a precision and speed not possible with traditional hardware. This performance is achieved by taking a holistic approach towards system design: By breaking down the walls between imaging, sensing, and computing modules in a conventional system and strategically offloading some computation to optics and sensors, we can train the entire system in an end-to-end fashion for a specific machine-vision task.
In general, obtaining results without recording all intermediate data is a neuromorphic design principle, commonly adopted by biological organisms and autonomous systems to perform tasks in real time. We are interested in demonstrating the impact of this computational sensing scheme by using it to solve real-world problems in high-speed robotics and high-throughput biomedical assays.
In general, obtaining results without recording all intermediate data is a neuromorphic design principle, commonly adopted by biological organisms and autonomous systems to perform tasks in real time. We are interested in demonstrating the impact of this computational sensing scheme by using it to solve real-world problems in high-speed robotics and high-throughput biomedical assays.
Top: The principle of image sensing as compared to traditional computer vision.
Bottom: An envisioned application of ultra-fast image sensing for high-throughput biomedical assays.
- G. Wetzstein, A. Ozcan, S. Gigan, et al. Inference in artificial intelligence with deep optics and photonics. Nature 588, 39–47 (2020).
- P. Sterling and S. Laughlin. Principles of Neural Design (MIT Press, 2015).
- T. Wang*, M.M. Sohoni*, L.G. Wright, et al. Image sensing with multilayer, nonlinear optical neural networks. Nature Photonics 17, 408 - 415 (2023).
Understanding Biological Neural Computation with Light
Despite the breakthroughs in AI, the working principle of its source of inspiration — biological neural networks — is still not fully understood. The difficulty is twofolded, it is both challenging to measure and to analyze neural activity data. We are interested in helping with this endeavor in two ways:
- Seeing through scattered light: While optical imaging provides a minimally invasive way to record the activities of biological neurons across a large population, imaging through turbid biological tissue is challenging due to the highly complex and dynamic nature of tissue-induced light scattering. We are interested in exploring new approaches using a mixture of physics and AI methods for fast light de-scattering. When combined with state-of-the-art optical microscopy or spectroscopy, these methods will enable us to substantially push the depth of brain imaging.
- Modeling neuronal activity data: To test whether current large-scale neuronal recording datasets contain the information necessary for decoding brain functions, we propose using state-of-the-art AI models for neuronal data analysis, with a focus on model's predictive power.
Top: The concept of smart miscroscopy that tailors to sample condition instantly in an autonomous fashion.
Bottom: The open question of predicting neuronal activites based on sensory inputs and behavior outputs; image from https://doi.org/10.48550/arXiv.2305.19654.
- A.E. Carpenter, B.A. Cimini, K.W. Eliceiri. Smart microscopes of the future. Nature Methods 20, 962–964 (2023).
- B.A. Richards, T.P. Lillicrap, P.Beaudoin, et al. A deep learning framework for neuroscience. Nature Neuroscience 22, 1761–1770 (2019).
- S. Gigan, O. Katz, H.B. de Aguiar, et al. Roadmap on wavefront shaping and deep imaging in complex media. J. Phys. Photonics 4, 042501 (2022).
- T. Wang, and C. Xu Three-photon neuronal imaging in deep mouse brain. Optica 7, 947-960 (2020).
- T. Wang, ... and C. Xu Three-photon imaging of mouse brain structure and function through the intact skull. Nature Methods 15, 789-792 (2018).
- D.G. Ouzounov*, T. Wang*, ... and C. Xu In vivo three-photon imaging of activity of GCaMP6-labeled neurons deep in intact mouse brain. Nature Methods 14, 388-390 (2017).
Imaging and Sensing with Quantum Light
Quantum and other exotic states of light are enticing for imaging and sensing applications. This is because quantum resources, such as entanglement, allow us to engineer the collective behavior of multiple photons, enabling each photon to carry more information than its classical counterpart. This extra information translates to various forms of quantum sensing/imaging advantages, such as enhanced phase measurement precision, image resolution, or multiphoton excitation. However, many questions still remain:
Can we engineer a state of light that can lead to scalable quantum sensing advantages, in presence of loss and background noise? How can we utilize quantum resources for an advantage in machine-learning tasks? In the context of optical imaging and sensing, we are interested in developing systems and computational methods to demonstrate practical quantum sensing advantages for real-world applications.
Can we engineer a state of light that can lead to scalable quantum sensing advantages, in presence of loss and background noise? How can we utilize quantum resources for an advantage in machine-learning tasks? In the context of optical imaging and sensing, we are interested in developing systems and computational methods to demonstrate practical quantum sensing advantages for real-world applications.
- P.-A. Moreau, et al. Imaging with quantum states of light. Nature Review Physics 1, 367–380 (2019).
- S. Pirandola, et al. Advances in photonic quantum sensing. Nature Photonics 12, 724–733 (2018).
Artificial Intelligence for Physical Sciences
Recent advances in AI, including reinforcement learning, multimodal, few-shot learning, and large language models, will generate long-lasting impacts on how we conduct scientific research, from how experiments are desinged and performed to how the results are analyzed and circulated. In this regard, we are keen on exploring ways state-of-the-art AI models and tools can be used to accelerate scientific research and rethinking the roles human researchers play in science.
- H. Wang, T. Fu, Y. Du, et al. Scientific discovery in the age of artificial intelligence. Nature 620, 47–60 (2023).
- L.G. Wright*, T. Onodera*, M.M. Stein, T. Wang et al. Deep physical neural networks trained with backpropagation. Nature 601, 549-555 (2022).