Abstract

In modern neuroscience, functional magnetic resonance imaging (fMRI) has been a crucial and irreplaceable tool that provides a non-invasive window into the dynamics of whole-brain activity. Nevertheless, fMRI is limited by hemodynamic blurring as well as high cost, immobility, and incompatibility with metal implants. Electroencephalography (EEG) is complementary to fMRI and can directly record the cortical electrical activity at high temporal resolution, but has more limited spatial resolution and is unable to recover information about deep subcortical brain structures. The ability to obtain fMRI information from EEG would enable cost-effective, naturalistic imaging across a wider set of brain regions. Further, beyond augmenting the capabilities of EEG, cross-modality models would facilitate the interpretation of fMRI signals. However, as both EEG and fMRI are highdimensional and prone to noise and artifacts, it is currently challenging to model fMRI from EEG. Indeed, although correlations between these two modalities have been widely investigated, few studies have successfully used EEG to directly reconstruct fMRI time series. To address this challenge, we propose a novel architecture that can predict fMRI signals directly from multi-channel EEG without explicit feature engineering. Our model achieves this by implementing a Sinusoidal Representation Network (SIREN) to learn frequency information in brain dynamics from EEG, which serves as the input to a subsequent encoder-decoder to effectively reconstruct the fMRI signal from a specific brain region. We demonstrate that our proposed SIREN-based model achieves state-of-the-art performance on the Eyes Open - Eyes Closed dataset. However, the success of the model does not generalize well to a different dataset, the Resing-State dataset. The work highlights the potential of leveraging periodic activation functions in deep neural networks for certain neuroimaging tasks, as well as the approach’s limitations with respect to generalizability to different, more noisy data.