Title: Model Sparsification on Emerging Applications and Technologies Ph.D. Candidate: Shanglin Zhou Major Advisor: Dr. Caiwen Ding Co-Major Advisor: Dr. Krishna Pattipati Associate Advisors: Dr. Cunxi Yu, Dr. Zhijie Shi Date/Time: Tuesday, September 24th, 2024, 3:00pm - 4:00pm Location: Virtual Abstract: Deep neural networks (DNNs) with higher accuracy often lead to larger models, increasing storage and energy demands. Model sparsification can reduce size but risks compromising accuracy. Balancing these factors is challenging, especially as traditional processors struggle with the requirements of low-power, real-time DNNs, highlighting the need for more efficient solutions. In this thesis, we explore model sparsification in emerging applications. We propose an optimization approach using Surrogate Lagrangian Relaxation (SLR) for weight sparsification, streamlining the typical time-consuming three-step pipeline. It enables faster convergence and maintains high accuracy, even during hard-pruning, with rapid recovery in retraining. We explore two primary emerging applications: (1) Energy-harvesting devices that require dynamic power management. We introduce EVE, an AutoML framework using SLR-based sparsification to find optimal multi-models with shared weights, reducing memory use and adapting to changing environments. (2) Diffractive Optical Neural Networks (DONNs) that are fast and energy-efficient but suffer from accuracy degradation due to interpixel interactions. We propose a physics-aware optimization framework for DONNs, incorporating SLR-based sparsification and roughness modeling to smooth phase changes and preserve accuracy. Additionally, we extend our research to multi-task learning (MTL) with DONNs, which traditionally requires manual reconfiguration. We propose LUMEN-PRO, an automated MTL framework that utilizes a flexible DONN backbone. By rotating shared layers instead of storing task-specific ones, LUMEN-PRO reduces memory usage and enhances accuracy, enabling efficient and high-performance DONNs across various tasks.