Gabrijel Boduljak
I am a doctoral student in the Visual Geometry Group (VGG) at the University of Oxford, advised by Christian Rupprecht, Iro Laina, and Andrea Vedaldi. My primary research interests are generative world modeling and learning intuitive physics from video. Additionally, I am interested in the theory of flow matching and mathematical optimal transport.
Research
-
JtT: Rethinking Point Trajectory Diffusion for Diverse and Rigid Motion from an ImagePreprint 2026We study forecasting rigid-body motion from a single image: generating diverse and physically plausible future trajectories for arbitrary points in a scene, without building rigidity into the architecture or relying on geometric inputs such as depth. Like video generators, recent trajectory generators fail at this task, even in simple synthetic scenes. Our model, JtT, is a simple diffusion transformer that operates directly in coordinate space and decodes arbitrary and dense query patterns. Through a systematic study of the design space, we find that diffusing per-frame velocities is the only parameterization that captures rigidity. We also observe that physical plausibility and diversity do not improve together during training, and adopt a stricter evaluation protocol under which JtT improves over prior methods with only about 50M parameters.
-
VFMF: World Modeling by Forecasting Vision Foundation Model FeaturesICML 2026Many recent methods forecast the world by generating stochastic videos. While these excel at visual realism, pixel prediction is computationally expensive and requires translating RGB into actionable signals for decision-making. An alternative uses vision foundation model (VFM) features as world representations, performing deterministic regression to predict future states. These features directly translate into useful signals like semantic segmentation and depth while remaining efficient. However, deterministic regression averages over multiple plausible futures, failing to capture uncertainty and reducing accuracy. To address this limitation, we introduce a generative forecaster using autoregressive flow matching in VFM feature space.
-
What Happens Next? Anticipating Future Motion by Generating Point TrajectoriesICLR 2026, Best paper award @ World Modeling Workshop (MILA) 2026While recent high-quality video generators are often presented as world models, we show that they struggle to accurately forecast motion even in simple physical scenarios, such as falling blocks or objects interacting mechanically, even after fine- tuning on such data. We hypothesize that this limitation arises from the overhead of generating pixels instead of motion. To address this, we develop a video-like diffusion model that generates (quasi-)dense motion trajectories rather than pixels, achieving significantly more efficient and accurate predictions. We also propose improved metrics to assess motion forecasting in ambiguous settings, and use them to show that our approach compares favorably against both video generators and prior motion forecasters.
-
On Vanishing Variance in Transformer Length Generalization1st Workshop on Scalable Optimization for Efficient and Adaptive Foundation Models (SCOPE), ICLR 2025It is a widely known issue that Transformers, when trained on shorter sequences, fail to generalize robustly to longer ones at test time. In this paper, we offer a vanishing variance perspective on this issue. To the best of our knowledge, we are the first to demonstrate that even for today's frontier models, a longer sequence length results in a decrease in variance in the output of the multi-head attention modules. On the argmax retrieval and dictionary lookup tasks, our experiments show that applying layer normalization after the attention outputs leads to significantly better length generalization. Our analyses attribute this improvement to a reduction, though not a complete elimination of the distribution shift caused by vanishing variance.