Odalric-Ambrym Maillard

Principal Investigator (PI)

Participant

• 2026 Neurips Workshop RL4XS, Paris - "Reinforcement Learning for Experimental Sciences"
• 2026 EWRL, Lille - European Workshop on Reinforcement Learning
• 2024 INFORMS Toulouse – Session organizer at RL4SN Informs conference "Challenges and progresses in Statistical Reinforcement Learning"
• 2019 RLSS – Reinforcement Learning Summer School.
• 2018 ANR BADASS Lecture Series (Invited lecturer: Peter Grünwald)
• 2018 EWRL workshop – European Workshop on Reinforcement Learning.
• 2017 Workshop – Sequential Structured Statistical Learning.
• 2014 NIPS Workshop – “From Bad Models to Good Policies” (Sequential Decision Making under Uncertainty)

Principal Investigator (PI)

Participant

As every researcher knows, there is generally a gap between all what we know/master about, all what we are interested in and would like to do, and what finally appears scarcely in some of our published papers. Here I want to list some topics/keywords/questions I would love working on but do not have time to, being too busy with another exciting project. Feel free to be hooked by some. If this gives you inspiration, please go ahead and work on them. From E-learning to Permaculture or Circular economy, this section tries to embrace the potential of Sequential Decision Making for shaping our future societies.

• Computational permafarming: Given a farm, with plants that are strongly interacting and sharing resources, the goal is to decide which action to perform (planting/moving which plant) in order to maximize the resistance of the system to attacks from the weather, insects or diseases while minimizing the external resources added to the system. Handling the strong dependency between the plants is a beautiful challenge. This directly falls under the scope of computational sustainability. Robust hydraulic systems: Say you want to monitor the level of rivers, in order to avoid flooding or that certain pollution reach critical sites. An incoming rain, storm, or pollution is seen as an attack, and the goal is to control the system in a robust way. Such a formalism may also apply to the management of electrical systems, when we further constraint the communications between control nodes.
• E-Learning: The goal is to recommend a series of exercises to incoming students in order maximize their learning level. Since each learner improves at each time steps, the learning progress of a student is a non-stationary reward signal. Handling such a signal for recommender systems is a great challenge. Job recommendations: Here the goal is to match job announcements to people looking for jobs. The task is fairly close to news article recommendation, and involves some natural language processing. Obviously the potential impact on unemployment is exciting.
• Urban recommendations: Consider you display the activity of a set of town councils to a web platform and record a feedback from citizens. Based on this data, you now want to help a council make better decisions, by recommending projects that may work well and alerting about possibly bad projects. Here one challenge is about long term decisions.
• Sustainable Economy: Consider a graph of economic agents exchanging resources. Here you want to price the additional number of resources needed at the source nodes of the graph, in order to increase the production at a certain node, and then recommend to a user which producer to favor when they both output the same resource. Handling the network activity of many agents is especialy challenging, and crucial for the development of closed-loop economy.
• Information reconstruction in resource networks: In this project, we study a large network of agents who produce, transfer and consumate resources. Only transfer of resources can be observed but neither production nor consumptions. Under some assumptions such that a production can only start if the resources needed for production have been received by the agent, and that transfer of resources systematically occur when a production cannot start, the goal is to study to which point it is possible to reconstruct the information of production and consumption, with quantitive bounds, as well as the network of effective dependency of a specific production.
• Stable and self-moving structures on weakly-differentiable manifolds: Motivated by the loss of differentiability occurring in shocks between « particles » we study manifolds that are only weakly-differentiable, with respective extensions of the tangent space, geodesics, curvature, currents, etc. Also, for certain types of dynamical systems governed by a « flattening » dynamics, we study initial conditions that ensure the existence of stable and « self-moving » structures.
• Co-articulation Optimization: Given a dynamical dystem (known dynamics) and a finite set of landmark points as inputs, we want to compute, for each finite sequence of landmark points, an optimal interpolation path passing maximally close to the targeted landmark points in the given order while being maximally distinct from the other landmark points. Then, we want to do the same when the dynamics is unknown. coarticulation This model naturally applies to the computation of co-articulation complexity of words, each landmark point corresponding to the prononciation parameters of one phoneme for a given speech apparatus. Then, based on a corpus of documents in some natural language, we can for instance compute the average co-articulation complexity of a language with respect to a given model of speech aparatus. In case we moreover have access to a grammar generator, we may generate a new natural language that minimizes the co-articulation complexity of most frequent grammatical structures while ensuring that the phoneme distance (geodesic distance in the parametric model of speech apparatus) between two grammatical structures increases with their co-occurrence frequency.
• Associative Memories with massive storage capabilities, Maximal no-hallucination capacity and optimal reconstruction: An associative memory stores a signal into a (hyper-)graph by creating a (hyper-)clique structure, leading to a sparse and robust representation. Optimal reconstruction can be done via the use of random matrices under some conditions. We want to investigate basic properties of associatove memories with hyper graph of a given size, like how to avoid hallucination (either creation or reconstruction of a clique corresponding to no signal), what is the maximal capacity of the memory, and what are the guarantees of optimal reconstruction under the constraint of avoiding hallucinations.