Link: https://eu02web.zoom-x.de/j/66719911736?pwd=N5n96kFJbauLi2u79eJI0ZD15hgNsi.1
4:30pm Central European time is (usually) 7:30am Pacific time and 11:30pm Beijing time
Transformers, while revolutionary, face challenges due to their demanding computational cost and large data movement. To address this, we propose HyFlexPIM, a novel mixed-signal processing-in-memory (PIM) accelerator for inference that flexibly utilizes both single-level cell (SLC) and multi-level cell (MLC) RRAM technologies to trade-off accuracy and efficiency. HyFlexPIM achieves efficient dual-mode operation by utilizing digital PIM for high-precision and write-intensive operations while analog PIM for high parallel and low-precision computations. The analog PIM further distributes tasks between SLC and MLC PIM operations, where a single analog PIM module can be reconfigured to switch between two operations (SLC/MLC) with minimal overhead (<1% for area & energy). Critical weights are allocated to SLC RRAM for high accuracy, while less critical weights are assigned to MLC RRAM to maximize capacity, power, and latency efficiency. However, despite employing such a hybrid mechanism, brute-force mapping on hardware fails to deliver significant benefits due to the limited proportion of weights accelerated by the MLC and the noticeable degradation in accuracy. To maximize the potential of our hybrid hardware architecture, we propose an algorithm co-optimization technique, called gradient redistribution, which uses Singular Value Decomposition (SVD) to decompose and truncate matrices based on their importance, then fine-tune them to concentrate significance into a small subset of weights. By doing so, only 5-10% of the weights have dominantly large gradients, making it favorable for HyFlexPIM by minimizing the use of expensive SLC RRAM while maximizing the efficient MLC RRAM. Our evaluation shows that HyFlexPIM significantly enhances computational throughput and energy efficiency, achieving maximum 1.86 × and 1.45 × higher than state-of-the-art methods.
Dr. Mingu Kang is an assistant professor of the Electrical and Computer Engineering (ECE) at the UC San Diego Jacobs School of Engineering. He received the Ph.D. degree in Electrical and Computer Engineering from the University of Illinois at Urbana–Champaign, Champaign, IL, USA, in 2017, and the B.S. and M.S. degrees in Electrical and Electronic Engineering from Yonsei University, Seoul, South Korea, in 2007 and 2009, respectively. From 2017 to 2020, He was a research staff member of the IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA, where he designs machine learning accelerator architecture, which was successfully commercialized by being embedded on IBM Z-mainframe server. From 2009 to 2012, he was with the Memory Division, Samsung Electronics, Hwaseong, South Korea, where he was involved in the circuit and architecture design of phase change memory (PRAM). He is a recipient of Intel 2022 Rising Star Award, 2024 Hellman Fellow Award, UIUC CSL best thesis award in 2018, 2019 MICRO TOP Pick Honorable Mention, IEEE International Symposium on Circuits and Systems (ISCAS) “Neural System and Application” Best Paper Awards in 2016 and 2018.
