Large Eddy Simulation Using Linear Finite Elements on GPUS

Abstract

This work presents recent computational advances in Alya for the Large Eddy Simulation of complex geometry problems of industrial interest. For such flows, the velocity is discretized explicitly using a 3rd or 4th order Runge Kutta scheme, and the pressure is treated implicitly. A low dissipation scheme based on pure Galerkin discretization and an EMAC (energy, momentum, and angular momentum conserving) scheme for the convective term lead to an efficient and accurate method free from numerical parameters. A non-incremental fractional step method stabilizes the pressure and allows equal order interpolation of velocity and pressure unknowns. Explicit subgrid-scale models such as Vreman or WALE are used. The two main computational kernels are the creation of a right hand and the solution of a linear system for the pressure. For the solution of the linear system or the pressure, we have interfaced Alya with several libraries developed within EoCoE. The best results have been obtained with AGMG and PSCToolkit. This work will present weak scalability results up to 16000 million elements on 24000 CPU cores using the algebraic multigrid preconditioner within PSCToolkit. Thanks to multigrid, Alya can now obtain correct algorithmic scalability. The RHS vector assembly has been optimized for the GPU to obtain an implementation that is more than 60 times faster than Alya’s previous GPU implementation, reaching a floating performance close to 50% of the peak value in an A100 Nvidia GPU. The improvements involved a better use of local memory, specialization, and register lifetime optimization. The specialization includes concentrating exclusively on linear tetrahedral elements for the moment and focusing only on the explicit time discretization. Applications to the Atmospheric Boundary Layer flow over complex terrain will be presented.