Model-Forum for Levante

There are some significant differences between the Mistral cluster and the new Levante
cluster that can explain performance difference if "old" Mistral setups are used directly on Levante.
Levante nodes use AMD CPUs (EPYC ZEN3 architecture) while Mistral nodes were based on Intel CPUs.
Also, a Levante node in the compute partition has 128 phys. cores, while a node in the Mistral compute / compute2
partition had only 24 or 36 phys. cores. Therefore, some adjustments are required for both
the build and run steps to ensure that the models also run performantly on Levante.

Build Step¶

Luckily AWI-ESM is already supported by the ESM-Tools on Levante now. I.e. appropriate settings for the compiler options in the build step (CMake/Autotools) on Levante are considered.

For a complete list of models and versions already supported also see https://github.com/esm-tools/esm_tools/discussions/663

The ESM-Tools project is hosted at GitHub: https://github.com/esm-tools/ where you can find more detailed information.

Run Step¶

If the runscripts are created with the ESM-Tools, appropriate entries for setting up the runtime environment and in particular for the modules to load (MPI environment, netcdf, hdf5, ...) are created.

For a Levante node to be used efficiently (i.e. all 128 phys. cores), a new domain decomposition is usually required first so that the number of MPI processes per model (ECHAM: nproca*nrpocb; FESOM: nproc) is divisible by 128 each.
Otherwise (e.g. with "old" Mistral setups for an experiment), the Levante nodes are typically underutilized. New partitions for FESOM can be generated via fvom_init.F90 if necessary.
This is explained quite well in the FESOM documentation https://fesom2.readthedocs.io/_/downloads/en/latest/pdf/