Mathematical Foundations of AI

8 June 2026Diffusion models and associative memories

1. 8:30–9:15Welcome fika and registration
2. 9:15–9:30Welcome address
3. 9:30–10:30Luca Ambrogioni · Invited speakerHow Out-of-Equilibrium Phase Transitions can Seed Pattern Formation in Trained Diffusion Models＋

   The talk described pattern formation in trained diffusion models as an out-of-equilibrium phase transition caused by instabilities in denoising dynamics. Its theoretical framework connects symmetries in the data and architectural constraints, including locality and translation equivariance, to collective spatial modes. Structure emerges when low-frequency modes become unstable and spatial correlations grow rapidly.

   The predictions were tested in a patch-based model, convolutional models trained on Fashion-MNIST and large ImageNet models. Across these settings, pattern formation coincided with increased correlation length and weakened low-frequency modes. Guidance applied near this critical stage improved class alignment compared with guidance applied at arbitrary times.

4. 10:30–11:00Chenxiao Ma · Contributed talkThe interplay between data structure and imbalance in the learning dynamics of diffusion models＋

   This work studied how class heterogeneity and sampling imbalance affect the transition from generalisation to memorisation in diffusion models. A high-dimensional random-features model trained on Gaussian mixtures was analysed through its feature-covariance spectrum to obtain class-specific generalisation and memorisation times.

   Class variance was found to determine the learning order most strongly, with higher-variance classes learned first; centroid geometry had a secondary effect. Strong sampling imbalance could reverse this order and delay speciation for minority classes. Experiments with U-Nets on Fashion-MNIST supported the theoretical predictions.

5. 11:00–11:30Break
6. 11:30–12:30Alberto Fachechi · Invited speakerDense L-Directional Hopfield Networks: Merging Modularity and Higher-Order Interactions for Pattern Disentanglement＋

   The talk began from the Hopfield model as a foundational associative-memory system and considered dense variants with higher-order interactions. It combined this dense-interaction setting with modular Hopfield networks whose modules interact anti-imitatively.

   The resulting dense, modular networks were analysed as a method for disentangling spurious nonlinear combinations of stored patterns, with applications to practical pattern-disentanglement problems.

7. 12:30–12:45Discussion
8. 12:45–14:00Lunch
9. 14:00–15:00Matteo Negri · Invited speakerDetecting Creativity Geometrically: Hopfield networks as a Lens on Generative AI＋

   The talk used associative-memory energy landscapes to ask when generative AI generalises rather than memorises. Spurious mixture states, usually treated as defects, can create basins of attraction around unseen examples and therefore provide a geometric mechanism for generalisation.

   This viewpoint was extended to self-attention, interpreted as a pseudo-energy minimisation process with vector spins representing token embeddings, and to discrete diffusion language models. Attractors near test examples were proposed as a diagnostic for distinguishing memorisation from genuine generative behaviour.

10. 15:00–15:30Vincenzo Schimmenti · Contributed talkEquilibria of Modern Hopfield networks＋

    The work characterised equilibria in dense associative memories formulated as modern Hopfield networks. A convex-dual form of the energy maps the system to a phase-separation problem; minima correspond to pattern weights at fixed points of the retrieval dynamics, determined by the Gram matrix of embedded patterns.

    Bifurcations follow a hierarchy related to spectral clustering in the correlation matrix. The method reproduces replica-based results for Gaussian patterns and was tested on hierarchical toy models, gene-expression networks and standard machine-learning datasets.

11. 15:30–16:00Afternoon fika
12. 16:00–17:00Beatrice Achilli · Invited speakerStatistical Physics of Generative Diffusion＋

    The first part developed a general theory of speciation transitions in reverse diffusion. These sharp changes mark the point at which trajectories commit to data classes. A Bayesian description of class structure and free-entropy differences gives transition times for arbitrary target distributions, including classes that differ only in higher-order moments, and allows successive transitions at increasingly fine class resolutions.

    The second part examined memorisation when data lie on latent manifolds. Treating the empirical score as a proxy for the true score, the high-dimensional process was characterised with spin-glass methods. The analysis showed how structure in the data can reduce the usual dimensionality burden.

13. 17:00–17:30Enrico Ventura · Contributed talkEmergence of Distortions in High-Dimensional Guided Diffusion Models＋

    This talk analysed how classifier-free guidance can reduce sample diversity by making its sampling distribution differ from the true conditional distribution. In high-dimensional Gaussian mixtures, dynamic mean-field theory predicts distortions when the number of classes grows exponentially with dimension; they vanish in the sub-exponential regime through a dynamical phase transition. In the infinite-class limit, distortions remain.

    Standard guidance schedules could not prevent variance shrinkage. A schedule with a negative-guidance window improved class separation and diversity in latent diffusion experiments. Related preprint ↗

14. 17:30–17:45Discussion

9 June 2026Geometric deep learning and transformers

1. 9:00–10:00Morning fika
2. 10:00–11:00Daniel Persson · Invited speakerGeometric deep learning: from equivariance to weather predictions＋

   The talk introduced geometric deep learning through invariance and equivariance, starting with convolutional neural networks and extending the discussion to manifolds, homogeneous spaces and gauge-equivariant models.

   Applications included spherical signals, computer vision, weather prediction and topological materials, showing how ideas from mathematics and physics can inform neural-network design.

3. 11:00–11:30Break
4. 11:30–12:30Giovanni Luca Marchetti · Invited speakerAlgebraic Geometry of Deep Learning＋

   Neural networks parameterise spaces of functions, sometimes called neuromanifolds, whose geometry is connected to expressivity, sample complexity and implicit bias. For algebraic models, such as networks with polynomial activations, these spaces are semi-algebraic varieties.

   The talk presented this algebraic-geometric view and then focused on linear self-attention, an algebraic model of degree three. It covered the geometry of linear self-attention layers and parameter symmetries in deep attention networks as part of the emerging subject of neuroalgebraic geometry.

5. 12:30–12:45Discussion
6. 12:45–14:00Lunch
7. 14:00–15:00Freya Behrens · Invited speakerModel Systems for attention, structure and memory in Transformers＋

   The talk isolated interacting components of language models in controlled settings. It examined how dot-product attention can move from positional to semantic behaviour as more data become available, and used counting tasks to compare architecture-dependent routes to parameter and compute efficiency.

   It also studied information storage in multilayer perceptrons and associative memories, including how correlations with unrelated inputs can support class inference. Related papers: 2402.03902, 2407.11542 and 2506.14457.

8. 15:00–15:30Oskar Allerbo · Contributed talkIs supervised learning really that different from unsupervised?＋

   The work decomposed supervised learning into two stages: model parameters are selected without target values, after which the targets are incorporated without changing those parameters. A new model-selection criterion makes this possible without access to the targets.

   For linear ridge regression, the method's asymptotic out-of-sample risk was bounded relative to the optimum. Experiments covered linear and kernel ridge regression, smoothing splines, nearest neighbours, random forests and neural networks, with target-free training performing similarly to standard training. Paper ↗

9. 15:30–16:00Afternoon fika
10. 16:00–17:00Peter Súkeník · Invited speakerOne token to attend them all? Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention＋

    The talk compared attention sinks and diagonal patterns as switching and anti-oversmoothing mechanisms. It derived geometric conditions for representing sinks, including the required alignment between a sink embedding and other embeddings, and specified when dense attention smooths more than sparse attention.

    Sinks were related to a hard attention switch with zero output. Allowing tokens to communicate with themselves produces a softer alternative through diagonal patterns. A quantitative cost comparison explained why pretrained transformers may favour sinks and clarified when attention layers behave like multilayer perceptrons.

11. 17:30–17:45Discussion
12. 19:30Speakers' dinner

10 June 2026Transformers

1. 8:30–9:30Morning fika
2. 9:30–10:30Alessio Giorlandino · Invited speakerFrom Initialisation to Factual Recall: Two Studies on Transformers＋

   The first study connected signal propagation through transformer networks at initialisation with trainability. The second studied factual recall as an associative-memory problem, using a toy task to derive sharp thresholds for a simple neural network to memorise correct associations.

   Related papers: signal propagation and trainability and factual recall.

3. 10:30–11:00Alessandro Zambon · Contributed talkSampling at intermediate temperatures is optimal for training large language models in protein structure prediction＋

   The work introduced pseudo-Langevin dynamics for efficient Boltzmann sampling of feed-forward neural networks on large datasets using controlled mini-batches. This avoids the repeated full-batch gradients required by exact methods such as hybrid Monte Carlo.

   The method was used to sample transformer loss landscapes for protein-sequence models across temperatures. Unlike feed-forward networks, transformers showed no first-order-like transition and instead had a range of intermediate temperatures with favourable learning properties. Layer parameters were strongly conserved there when the embedding dimension was well chosen, while attention matrices predicted protein contact maps better at higher temperatures and larger embedding dimensions.

4. 11:00–11:30Break
5. 11:30–12:30Yedi Zhang · Invited speakerTraining Dynamics of In-Context Learning in Linear Attention＋

   The talk analysed gradient-descent dynamics in multi-head linear self-attention trained for in-context linear regression. The dynamics contain exponentially many fixed points and move from saddle to saddle; this behaviour can be reduced to scalar ordinary differential equations.

   During training, the model performs principal-component regression in context, with the number of active principal components increasing over time. The results give a theoretical account of how in-context learning develops during gradient descent.

6. 12:20–13:00Discussion and conclusion

The source schedule lists the final discussion from 12:20, overlapping the preceding talk by ten minutes.