Dr. Sam Gijsen
Machine Learning Researcher
Foundation & Generative Models for Physiological Time Series
I research multimodal representation learning and build both foundation models as well as generative models for neural and physiological time series. Previously, I completed a PhD in Computational Cognitive Neuroscience (Freie Universität Berlin) on probabilistic inference and worked on pharmaco-imaging at King's College London and Maastricht University.
Recent Work
We propose a per-timestep conditioned diffusion transformer for generating realistic neural time series during unseen experimental tasks by injecting both compositional language and optional spatial priors in-context. Its zero-shot generation can facilitate counterfactual neuroscience by supporting in-silico design and evaluation of novel cognitive experiments before empirical validation. A summary with GIFs can be found here.
We develop a self-distilled foundation model for brain dynamics that pretrains in 2 hours and eliminates the need for finetuning. We stabilize self-distillation for noisy time series through learned tokenization, and find log-linear scaling laws for pretraining data on cross-dataset downstream tasks.
First-of-its-kind EEG-language model for downstream clinical tasks. We show a multiple-instance learning extension of the infoNCE loss enables multimodal models which align physiological time series with natural language, yielding more useful representations.
Led a small team that placed highly using a multi-modal fusion architecture designed from scratch, without any pretraining. Code and report to come!
Some Previous Work
Computational modeling of neural signals using information-theoretic measures shows perceptual learning can be described as a process of probabilistic inference.
We develop active inference models using probabilistic surprise minimization and show these to describe human sequential decision-making better compared to reinforcement learning.
Latest Blog post
Hillsbrad Diffusion: A World Diffusion Model Criminally Undertrained
A qualitative look at a world diffusion model undertrained on two hours of sparse exploration of a large map.