Höb, M. (2015):

Performance improvement of pre-defined HMR workflows in DRIHM

Hydro-meteorological calculations are one of the most complex computational tasks in high performance computing today. One of the biggest challenges for this science department is to predict flood events and thus to contribute to the protection of the population. DRIHM (Distributed Research Infrastructure for Hydro-Meteorology) is an EU-funded research project to combine Hydro-Metrological Research (HMR) and Information and Communication Technology (ICT) objectives in order to model weather events and its influence on suburban regions.

This thesis gives an understanding of the project, especially the requirements of the hydro-meteorological science and its embedment in a supercomputing infrastructure. Several HMR models are integrated into DRIHM. They can be configured and executed in the DRIHM front end, a web portal, and directly in the back end via command line. However the back end offers no possibility to compose models to a workflow, in contrast to the portal, which allows compositions of up to three models. These workflows lead to performance problems when submitted to grid resources because they are no real workflows but single model computations in the back end. It will be pointed out that this chaining in the portal is very inefficient and therefore very expensive with regard to computing time and transferred data.

To prevent increasing costs, lost time due to unnecessary data transfers and avoidable queueing time a concept of a complete workflow in one submission was developed. Adapted from this concept, three demonstration chains (demoChains) were added to the model repository. The first one is modularized and can handle all possible compositions of HMR models. Therefore configuration and input files were created to guarantee a smooth execution of every given workflow. The two other chains are pre-defined and cover the most important models within DRIHM.

The thesis' concept offers a framework to improve the performance of workflow executions on any back end resource. Integrated into the web portal, it could give inexperienced users a better understanding of the possibilities of chaining models within the DRIHM project.

• PDF Version (3,322.3 KB)