Centre Inria de l'Université de Lille
Team project Scool -- Spring 2025
Keywords: Agroecology, Reinforcement Learning, Simulator, Library.
Investigator: The project is proposed and advised by Odalric-Ambrym Maillard from Inria team-project Scool.
Place: This project will be primarily held at the research center Inria Lille -- Nord Europe, 40 avenue Halley, 59650 Villeneuve d'Ascq, France.
Simulation-based learning has transformed AI research, enabling rapid experimentation and algorithmic development. However, in agriculture, the integration of Reinforcement Learning (RL) remains limited due to a lack of adaptable, research-friendly simulators. Traditional agronomic models like DSSAT and STICS offer scientific rigor but lack modularity for AI experimentation. Meanwhile, simpler models fail to capture critical agroecological challenges. To bridge this gap, we propose a standardized suite of Gymnasium-compatible agroecosystem simulators, balancing biological realism, computational efficiency, and modularity for AI-driven decision-making in agriculture.
Beyond facilitating AI research in agriculture, this initiative has the potential for a transformative impact. By providing a standardized benchmark for the research community—akin to platforms like the Arcade Learning Environment (Atari) or MuJoCo—it will foster innovation, reproducibility, and collaboration in RL-based agroecosystem modeling.
Our primary goal is to extend Gym-DSSAT and integrate new simulators like Gym-STICS. A crucial challenge is ensuring biological realism while maintaining computational efficiency. Designing structured policy spaces and adaptive observation models will allow RL agents to learn under partial observability, mimicking real-world farming constraints.
Moreover, RL adaptation must address high stochasticity, sparse rewards, and long-term dependencies, which are fundamental in agroecological decision-making. Investigating model-based RL can improve sample efficiency, while exploring multi-objective RL can help balance yield maximization with sustainability constraints.
Last, ensuring real-world transferability is key. We aim to validate RL-driven policies through controlled experimental farms, notably via simulated micro-farms and CIRAD collaborations. Integrating human-in-the-loop RL will refine learned policies with expert feedback, enhancing practical applicability.
Our foundations include Gym-DSSAT, the first RL-compatible crop model simulator, and gamified agroecosystem environments like Farm-gym and Wegarden. The Gym-DSSAT in Mali project has already demonstrated real-world applications in collaboration with CIRAD, see Gautron et al. 2024.
We envision an open, standardized suite of agroecosystem simulators serving as a testbed for RL-driven agricultural decision-making. Inspired by platforms like the Arcade Learning Environment, our goal is to establish public leaderboards, modular benchmarks, and shared datasets, fostering global collaboration. By integrating biophysical simulation, AI-based decision-making, and real-world testing, this initiative aims to develop novel, resource-efficient farming strategies, addressing pressing challenges in sustainability, food security, and climate-resilient agriculture.
The Decision Support System for Agrotechnology Transfer (DSSAT) is a comprehensive software application that includes dynamic crop growth simulation models for over 45 crops. It is supported by various utilities and apps for weather, soil, genetic, crop management, and observational experimental data, and includes example data sets for all crop models. The crop simulation models simulate growth, development, and yield as a function of the soil-plant-atmosphere dynamics.
The STICS (Simulateur mulTIdisciplinaire pour les Cultures Standard) crop model has been developed since 1996 at INRA (French National Institute for Agronomic Research) in collaboration with other research and technical institutes. It synthesizes and concretizes a significant part of French agronomic knowledge, providing a perspective on field and cropping systems. The model's formalisations can be considered as references in the framework of crop sciences
Gym-DSSAT is an RL-compatible crop model simulator that interfaces with the Decision Support System for Agrotechnology Transfer (DSSAT). The Gym-DSSAT interface has been developed to facilitate daily interactions between simulated crop environments and RL agents. This integration allows for the optimization of nitrogen management, among other applications, by training RL agents to make informed decisions based on simulated crop data. Gym-DSSAT provides predefined simulations based on real-world maize experiments and is designed to be as user-friendly as any gym environment. Preliminary experimental results suggest that RL can help researchers improve the sustainability of fertilization and irrigation practices.
Farm-gym is an open-source farming environment written in Python that models sequential decision-making in farms using Reinforcement Learning (RL). It conceptualizes a farm as a dynamic system with many interacting entities, allowing for the creation of both simple and complex environments. Farm-gym features intrinsically stochastic games, utilizing stochastic growth models and weather data. Its modular design enables the activation or deactivation of various entities, such as weeds, pests, and pollinators, resulting in non-trivial coupled dynamics. Each game can be customized with .yaml files for rewards, feasible actions, and initial/end-game conditions. This platform aims to present new challenges to the RL community and stimulate collaboration with the agronomy community
The proejct will be hosted at Centre Inria de l'Université de Lille, in the Scool team. Scool (Sequential COntinual and Online Learning) focuses on the study of sequential decision-making under uncertainty.
References are available upon request or via the Scool project page.