Title: Enabling On-Device 3D Reconstruction and Rendering via Architecture-Algorithm-System Co-Design
Chaojian Li
Ph.D. Student
School of Computer Science, Colleague of Computing
Georgia Institute of Technology
Date: Wednesday, September 18th, 2024
Time: 1:00 PM - 3:00 PM EDT
Location (in-person): U A Whitaker Biomedical Engr-165 - 1232 Classroom BME
Zoom: https://gatech.zoom.us/j/94875052120?pwd=A2bvfh68ocbEx6CnJnP7keNhl1DmLA.1
Committee:
Dr. Yingyan (Celine) Lin (Advisor), College of Computing, Georgia Institute of Technology
Dr. Josiah Hester, College of Computing, Georgia Institute of Technology
Dr. Ling Liu, College of Computing, Georgia Institute of Technology
Dr. Hyesoon Kim, College of Computing, Georgia Institute of Technology
Dr. Prasanna Balaprakash, Oak Ridge National Laboratory
Abstract:
On-the-fly 3D reconstruction and rendering are essential for numerous intelligent applications, including the metaverse, secure robots, remote education, and self-driving. Among the growing efforts to improve photorealistic reconstruction and rendering quality, techniques that incorporate neural networks with traditional graphics pipelines have stood out. However, these newly introduced operations also pose efficiency challenges in algorithm and architecture design, limiting the potential of 3D reconstruction and rendering on local devices in daily life. My thesis research aims to achieve both instant reconstruction and real-time rendering without sacrificing photorealistic quality. To accomplish this, my thesis research provides pioneering solutions, including a real-time rendering accelerator and an instant reconstruction accelerator through an algorithm-architecture co-design approach, as well as a real-time rendering pipeline for commercial mobile GPUs via an algorithm-system co-design approach. Lastly, I present ongoing efforts to apply these co-design insights to more applications, such as healthcare and scientific exploration.