Location

Virtual

Matthew Chen

Advisor: Dr. May Dongmei Wang

will defend a master’s thesis entitled,

MENDR: Manifold Explainable Neural Data Representations

On

 Tuesday, May 6th at 10:00 a.m.

Virtually on Zoom: https://gatech.zoom.us/j/5087800067?pwd=8t9C8ETeYd7DFiXe90iSC330YNg2C8.1&omn=97121957923 

Meeting ID: 508 780 0067 
Password: 2025

Abstract:

 

Foundation models for electroencephalogram (EEG) signals have recently demonstrated success in learning generalized representations of EEG, such as Large Brain Model, Large Brain Model (LaBraM), Criss-Cross Brain Model, and BERT-Inspired Neural Data Representations, BERT-Inspired Neural Data Representations (BENDR). These models primarily leverage advancements from self-supervised learning in large language models (LLMs), vision transformers, and time series transformers. Although the foundation models have demonstrated state-of-the-art performance on benchmark EEG datasets, they lack visibility into how the models learn during pretraining and verifying that they efficiently compress EEG signal information into embeddings. Suppose EEG foundation models are to be integrated into clinical applications. In that case, there needs to be complete transparency of the pretraining process, downstream fine-tuning, and interpretation of the learned embeddings. Furthermore, current EEG foundation models primarily analyze and understand the EEG signal in the temporal domain, which overlooks the potential of digital signal processing advancements for generating deterministic, backtraceable features of the EEG, such as Fourier spectra or Wavelet series. To resolve these issues, we present a filter-bank foundation model named Manifold Explainable Neural Data Representations (MENDR) (Manifold Explainable Neural Data Representations), which is based on a novel Manifold Attention Transformer (mATT Transformer) proposed by Pan et al. MENDR is pre-trained on a vast EEG corpus of approximately 4,000 hours, decomposed into different levels of wavelet coefficients using a Discrete Wavelet Packet Transform (DWPT). Afterward, MENDR is evaluated on two downstream tasks, which were also assessed
by state-of-the-art EEG Foundation Models Biosignal Transfomer (BIOT), LaBraM, and CBraMod, and achieves similar performances. However, unlike the current state-of-the-art EEG foundation models, MENDR is designed with significantly fewer parameters, with most of them in the reconstruction decoder. We include a reconstruction decoder as part of the model architecture to ensure that the embeddings still have physical interpretations. In other words, it supports the idea that the model is analyzing the same physical reality as neurologists when they view and annotate EEGs, which contain the same core information, just in a different representation. Additionally, by creating a filter bank representation of the EEG signal, one can perform multiresolution analysis on the other frequency bands of the EEG signal to pinpoint which specific frequencies are associated with which neurological disorders, as the correlation between particular frequencies and specific brain functions has been known for almost a century. Not only is MENDR trained to be a powerfully simple EEG foundation model, but its explainable design also allows one to discover new associations between specific brain frequencies and functions through the perspective of manifold optimization. Finally, a new method for explaining embeddings is proposed by visualizing Symmetric Positive Definite (SPD) embeddings as ellipsoids in a lower-dimensional space, inspired by Principal Component Analysis (PCA).

 

Committee:

  • Dr. Dongmei Wang – School of Biomedical Engineering (advisor)
  • Dr. Yunan Luo – School of Computational Science and Engineering
  • Dr. Daniel Drane (Emory University) – School of Neurology