Geofysisk institutt

GFI/BCCR Seminar: Autosubmit: a versatile tool for managing Earth System Models on HPC platforms

Domingo Manubens (University of East Anglia):

Autosubmit: a versatile tool for managing Earth System Models on HPC platforms


Climate experiments present a huge complexity not only for the computational cost of simulations but also for the workflow structure that need to be executed, which have dependencies between tasks difficulting the efficient use of resources. Dealing with those complexities implies the need of management systems that can lead to significant energy costs saving and time saving. Workflow management systems are being developed specialising on climate and meteorological models. They can be used to submit and control tasks, but also to create and share experiments. The experiments can be run on diverse supercomputing environments and in any platform in which computing resources are obtained. In this presentation, we will discuss about Autosubmit, a Python tool being developed at BSC Computational Earth Sciences group. It provides a simple workflow definition capacity that allows running weather, air quality and climate experiments. We will provide a comparison with other widely used workflow management systems available for the weather and climate community such as Cylc and ecFlow. We will illustrate the benefit of using Autosubmit with a demonstrator of a multi-model multi-member high resolution ARPEGE5-NEMO and EC-Earth experiment, including post-processing and data formatting tasks. Recently, we developed in Autosubmit a jobs wrapper, that packs multi-member simulations into a single executable which significantly improves the throughput.