Dr. Sam Gijsen
Postdoctoral Researcher
Foundation Models & Representation Learning
for Neural Time Series
I research multimodal representation learning and build foundation models for neural and physiological time series. My neuroscience background shapes how I think about learning in neural nets; I'm fascinated by what representations emerge, how they scale, and what they encode about the world.
GitHub LinkedIn Google Scholar
Recent Work
We develop a self-distilled foundation model for brain dynamics that pretrains in 2 hours and eliminates the need for finetuning. We stabilize self-distillation for noisy neural timeseries through learned tokenization, and find log-linear scaling laws for pretraining data on cross-dataset downstream tasks.
First-of-its-kind EEG-language model for downstream clinical tasks. We show that multimodal models integrating natural language learn more useful representations of neural data.
Led a small team that placed highly using a multi-modal fusion architecture designed from scratch, without any pretraining. Code and report to come!
Latest Blog post
Hillsbrad Diffusion: A World Diffusion Model Criminally Undertrained
A qualitative look at a world diffusion model undertrained on two hours of sparse exploration of a large map.
Some Previous Work
Computational modeling of neural signals using information-theoretic measures shows perceptual learning can be described as a process of probabilistic inference.
Compared to reinforcement learning, active inference models can better describe human sequential decision-making using probablistic surprise minimization.