Homer Babbidge Library

AbstractA unified framework of data-efficient and generalizable computer vision models is presented for scientific and industrial imaging, connecting various domains such as additive manufacturing, operando microscopy, and biomedical analysis. Extracting meaningful information from complex visual data is challenging due to limited annotations, heterogeneous imaging domains, fine-scale ambiguities, and inherently noisy conditions. To tackle these challenges, we introduce the concept of using paired images as a foundational approach for learning, reframing a wide range of imaging problems, including segmentation, defect detection, and dynamic analysis, as unified change detection tasks. This approach enables robust, data-efficient solutions that can be adapted to new challenges across diverse industrial and scientific imaging fields.The first problem focuses on domain adaptation for additive manufacturing vision tasks. In binder-jet 3D printing, traditional defect detection methods rely on rigid camera setups and handcrafted thresholds. We develop a Semi-Siamese neural network that directly compares a reference schematic to a camera image, allowing for pixel-level defect localization that remains effective under varying illumination and viewpoints without the need to predefine defect types.The second problem we address is the automated analysis of dynamic microscopy data. In operando environmental transmission electron microscopy, accurately segmenting nanoscale features is crucial for quantifying reaction kinetics. However, this process is often complicated by limited annotations and visually ambiguous structures. We present a change-detection-based framework that compares paired frames to capture subtle temporal and structural variations while simultaneously segmenting related regions of interest. This method transforms traditional in-situ imaging into spatially resolved operando characterization, allowing for the automated tracking of nanoscale material evolution and creating new opportunities for data-driven studies of reaction mechanisms and catalyst regeneration.The third problem considers label-efficient learning for biomedical imaging. Medical datasets tend to be small and heterogeneous, which limits the performance of conventional deep learning models. We propose a multi-fidelity framework that automatically generates low-fidelity versions of high-resolution images and trains a Semi-Siamese network to learn from the comparisons between different fidelity levels. This approach enhances the delineation of morphological boundaries while reducing reliance on extensive annotations.Together, these contributions create a cohesive framework that integrates change detection, domain adaptation, and multi-fidelity modeling for small-data, cross-domain scientific imaging. The resulting methods advance the objective of achieving autonomous, data-efficient visual understanding in complex experimental systems, paving the way toward self-driving laboratories and intelligent imaging platforms.

Conferences & Speakers