Fellow: ESR1

Host institution: BUW

Ph.D. enrolment: UCY/UNIFE

Project Title: Scalable multigrid solvers for exascale architectures

Objectives: To develop and implement multigrid solvers for the inversion of large sparse matrices such as those encountered in LBM or lattice QCD. We will address: i) Scaling and data movement cost of the algorithm over multiple GPUs: This is an inherent issue with multigrid, not restricted to GPUs but more severe for this architecture since computer time is spent in applying a coarse operator. Algorithmic solutions will be investigated such as to optimize applying powers of the operator kernel, needed in several communications avoiding solvers; ii) Load balancing between CPUs and GPUs to allow specifying which levels run on GPU and which on CPU and finetuning the multigrid performance for a range of CPU architectures; iii) Flexibility for adapting to different lattice formulations such as for example the twisted mass operator. In this case algorithms developed in HPC-LEAP have shown that changing action parameters, such as the bare quark mass, at the different multigrid levels can speed-up solution of the finer system. Furthermore, the optimal solver algorithm for building the aggregate vectors varies with the type of operator discretization. We will implement capabilities for changing the action parameters at different multigrid levels and tune the solvers for the case of twisted mass lattice QCD to be used by ESR2. With ESR3, GPU codes for closed circuits will be developed. In addition, ESR1 will interface with ESR10 who will optimize LBM codes for GPUs.

Expected Results: i) Optimized GPU multigrid implementation for the twisted mass and clover operators in QUDA; ii) interface with ESR2 for implementing algorithms for closed quark loops on GPUs and with ESR3 for closed circuits.

Planned secondment(s):

• NVIDIA, M21-M23, 3 PMs, implementation of the multigrid on the latest GPU architecture.
• FZJ, M26-M28, 3 PMs at the JSC exascale labs for optimization of the code on GPUs.

Fellow: ESR2

Host institution: CyI

Ph.D. enrolment: UNIFE/BUW

Project Title: Scalable algorithms for quark loops

Objectives: To develop methods for the computation of quark loops entering in hadron observables addressing both gauge and stochastic noise. Exascale algorithms and codes are needed to achieve the large statistics required resulting in large datasets that will need advanced data analytics to extract useful information. This is a demanding problem that will be addressed jointly by mathematicians (BUW), computer scientists (UNIFE) and lattice QCD experts (CyI). The goal is to compute key observables such as the nucleon σ-terms needed for dark matter searches, the spin content of the nucleon and parton distribution functions (PDFs). ESR12 will extend these techniques to QCD plus QED, while with ESR5 advanced data analysis methods will be employed for analyzing the resulting millions of correlators.

Expected Results: i) Test and optimize methods for evaluating the quark disconnect contributions on large lattices (L>4 fm) using gauge configurations at the physical value of the pion mass leveraging the methodology developed by ESR8; ii) Optimize these methods for highly parallel computing devices, including Intel Xeon, Intel Xeon Phi, Power processors and with ESR1 for GPUs; iii) evaluate the nucleon σ-terms, disconnected contributions to form factors and PDFs; iv) interface with ESR12 for applying the methodology to QCD plus QED; v) interface with ESR5 for employing data analysis techniques to the large number of correlation functions.

Planned secondment(s):

• NVIDIA, M21-M23, 3 PMs, for optimising applications for GPUs at the NVIDIA exascale labs.
• FZJ, M26-M28, 3 PMs at exascale labs, to optimise software for Intel Xeon, Xeon Phi and Power processors.

Fellow: ESR3

Host institution: HUB

Ph.D. enrolment: UTOV/UCY

Project Title: Splitting methods for Partial Differential Algebraic Systems

Objectives: To develop new structure exploiting methods for fast simulation of dynamic networks described by partial differential algebraic equations and to apply the new methods to electronic circuits for simulation of the interaction of novel electro-magnetic devices operating in super high frequency ranges. It is intended to develop an operator splitting for differential-algebraic equations (DAEs) with network structure that allows the exploitation of symmetrized integration methods for high-dimensional systems with a sparse and partially symmetric Jacobian structure (e.g. for networks with linear and non-linear conducting devices described by 3D spatially discretized Maxwell equations). The challenge of the implicit algebraic coupling for DAEs shall be solved by a recently developed dissection concept for network DAEs. M Günther from WUB will train the ESR novel symmetrized integration techniques.

Expected Results: i) Development of an operator splitting for network DAEs; ii) Convergence analysis of numerical schemes for this splitting approach; iii) Explore the application of symmetrized integration methods for partial differential algebraic systems (with ESR4) describing the interaction of electro-magnetic devices; iv) application software on GPUs (with ESR1)

Planned secondment(s):

• MAGWEL, M14-M16, 3 PMs, to be introduced to modeling of electro-magnetic devices.

Fellow: ESR4

Host institution: HUB

Ph.D. enrolment: UTOV/UCY

Project Title: Path integral evaluation by new mathematical quadrature rules

Objectives: To explore mathematical tools for evaluating high dimensional path integrals. The techniques of quasi Monte Carlo and polynomially exact integration rules through symmetrized integrations can be applied to many fields and will be explored here for gauge theories relevant for high energy physics and to the path integral approach to stochastic differential equations such as those addressed by ESR3. The problem of applying these alternative techniques to higher, up to four dimensions will be addressed. These mathematical methods avoid completely the problem of autocorrelation time and also overcome the sign problem of the standard Markov chain Monte Carlo method. They allow to investigate important phenomena such as the influence of a chemical potential and CP violation in QCD. This new approach can open completely new research directions in the future including HMC methods for rare events tackled by ESR7.

Expected Results: i) Application of completely symmetrized integration rules for 2-dimensional U(1)-gauge theory; ii) Evaluation of this model in the presence of a theta-term; iii) Development and application of the method to 3 and 4 dimensions.

Planned secondment(s):

• DESY, M17-M19, 3 PMs, to be introduced to physical models and to the relevant mathematical techniques.
• MAGWEL, M14-M16, 3 PMs, to be introduced to modeling of electro-magnetic devices.

Fellow: ESR5

Host institution: UCY

Ph.D. enrolment: UTOV/BUW

Project Title: Lattice QCD methods for nucleon observables probing BSM physics

Objectives: To develop innovative lattice approaches for the computation of nucleon observables probing BSM physics with a prime target the nEDM using gauge configurations with physical pion mass. The complexity involved in the evaluation of this quantity requires several innovative approaches: i) larger statistics as compared to CP conserving form factors requiring multigrid scalable codes for new computer architectures, ii) a study of multiple definitions for the topological charge; and iii) noise reduction techniques to obtain the accuracy required. These ground-breaking approaches can be adopted in the evaluation of other baryon matrix elements such as for the determination of the charge radius of the proton and the PDFs addressed by ESR13. Accomplishing these objectives optimally requires co-supervision of groups with complementary expertise, namely theoretical physics tools (UCY & UTOV), new approaches and codes for handling large data sets and extracting information from noisy data (BUW).

Expected Results: i) A comparison of the various definitions of the topological charge, so the optimal can be employed, including definitions which rely on quark loops to be developed by ESR2; ii) Automated analysis of correlators including exploring deep learning; iii) Implementation of noise reduction algorithms; iv) the first extraction of nEDM to an accuracy of about 20% at the physical point.

Planned secondment(s):

• IBM, M21-M23, 3 PMs, to develop the data analysis methods and to be introduced to deep learning approaches.

Fellow: ESR6

Host institution: HU

Ph.D. enrolment: RWTH/CyI

Project Title: Management of large scientific data-sets on multi- and many-core architectures with application in NEET proteins

Objectives: The goal is twofold: i) to investigate and design methodologies to analyze noisy scientific data-sets, both deriving from simulations and experiments, using deep-learning approaches, and to implement and optimize them for recent multi- and many-core HPC systems for the applications of STIMULATE. Numerical simulations used in fundamental physics, biology and engineering allow the investigation of several combinations and configurations of input parameters. This leads to a huge-amount of data-sets that need to be analyzed to extract important information. At the same, increasingly complex experimental procedures produce huge amount of real-life data, often hidden behind badly known noise sources; ii) To apply these methods in the special case of the NEET proteins, that are promising targets for combating neurodegenerative diseases and other pathologies including cancer. NEET proteins feature two protomers intertwound to form a two-domain motif, a cap, and a unique redox-active labile 2Fe-2S cluster binding domain. We will use computational approaches coupled to in vitro and in vivo experiments to determine new effective peptide ligands able to stabilize the NEET proteins clusters and thus regulating the oxidative stress of neuronal cells found in the brains affected by neurodegenerative diseases. We will employ metadynamics simulations to identify the poses of the peptide-ligands onto the protein. The peptides ligands designed here will be tested against neurodegenerative diseases in a novel, integrated approach, where simulations will be coupled with experimental studies. Data analysis and visualization methods will be used in the anatomical studies.

Expected Results: The expected results are: i) characterization of data-analysis methods suitable for the applications of STIMULAE; ii) design of data-analysis algorithms; iii) implementation on multi- and many-core architectures; iv) evaluation and optimization of performances; v) Computational protocol for peptide binding against metal containing proteins; and with ESR14: vi) peptides recognition mechanism; vii) peptides binding free-energies to NEET proteins; viii) peptide library against different NEET proteins; ; ix) In vivo and ex-vivo animal studies.

Planned secondment(s):

• IBM, M14-M16, 3 PMs, to be trained in deep-learning and cognitive computing focusing on the analysis of bio-simulation data connecting them to the experiments

Fellow: ESR7

Host institution: RWTH

Ph.D. enrolment: CyI/UTOV

Project Title: Monte Carlo methods and importance sampling of rare events in stochastic partial differential equations and in stochastic single- and multi-particle problems.

Objectives: i) Path integral investigations of low-dimensional hydrodynamics to address extreme and very rare fluctuations in collaboration with ESR4; ii) Development of HMC algorithm with constraints to enhance the occurrence of rare events and machine learning for identifying rare evenys. 3) Implement an efficient, fully parallelized HMC code for shell models of turbulence and for single and multi-particle problems to go beyond the optimal path approximation.

Expected Results: i) Application of HMC methods to study intense fluctuations in shell models for the energy turbulent transfer; ii) Classification of different instanton solutions from the perspective of MC methods and address their importance for both Eulerian and Lagrangian problems; iii) Provide a link between saddle-point methods and conventional analytic and numerical approaches that address the nature of probability distribution of the particles at short/long time scales.

Planned secondment(s):

• NVIDIA, M21-M23, 3 PMs, for the implementation of HMC algorithms on GPUs.

• DESY, M24-M26, 3 PMs, to perform systematic optimisation of HMC performances and explore different functional constraints for instantons.

Fellow: ESR8

Host institution: UNIFE

Ph.D. enrolment: BUW/CyI

Project Title: Improving stochastic approaches for selected inversion

Objectives: Selected inversion of a matrix means the computation of quantities of its inverse without computing the inverse. The two prominent examples to be addressed in this project are the trace of the inverse and the diagonal of the inverse. These quantities are important for models for many physical systems, including electronic structure of proteins and of observables in quantum field theories. It will combine two approaches considered mainly separately so far, namely stochastic estimation of these quantities and hierarchical approximations for the inverse. The project will elaborate on how a partial inversion, obtained through multigrid technology, e.g., and a judicious choice of stochastic test vectors can be used to accelerate the stochastic estimation. For the latter, the use of known and to be developed decay bounds on the entries of the inverse will be particularly important. The developed techniques will allow improvements in quantities of interest to projects of other ESRs, in particular they will be illustrated on the closed quark loops targeted by ESR2.

Expected Results: i) New decay bounds for the inverse of a matrix based on information on the location of its spectrum; ii) Analysis of the use of restricted random variables in stochastic estimators to reduce the variance; iii) Development of a hybrid and recursive method for the trace and the diagonal of the inverse combining multigrid approximation and stochastic estimation; (iv) Parallel implementation and test of the method for different application problems.

Planned secondment(s):

• FZJ, M24-M26, 3 PMs, to implement parallel implementations of the stochastic approaches developed.

Fellow: ESR9

Host institution: CyI

Ph.D. enrolment: UTOV/RWTH

Project Title: Multiscale behaviour of dense suspensions on highly parallel computing devices: from theory to applications

Objectives: To develop macro-scale computational models using also machine learning techniques for addressing rheology of concentrated particle suspensions and to apply to haemolysis. This requires achieving a unified picture connecting the dense and dilute regimes and/or full control of the effects induced by strong confinement, a challenge owing to the complex shape, dynamics and interactions of the particles involved in blood flow. Synergies between “ab-initio” and Eulerian numerical simulations for blood dynamics and rheology will be explored. Namely, LBM where one can model realistically fine-size cells, with realistic flow-structure interactions, and explore the resulting rheology at changing the volume fraction together with the degree of confinement will be interfaced with fully Eulerian schemes. Ultimately, the objective is a protocol to feed the Eulerian schemes with input parameters able to realistically model realistic blood effects. R. Tripiccione will participate in training the student in the specific optimisation for Lattice Boltzmann Code dedicated to the study of blood cell dynamics in complex geometry.

Expected Results: i) Development/Optimization of computational tools based on LBM for description of dense particle suspensions and on finite elements implementation for dense particle suspensions; ii) Predictive computational tools for estimating haemoglobin release (haemolysis) from red blood cells at the scale of biomedical devices; iii) Thorough understanding of the factors in artificial heart design leading to non-physiological levels of haemolysis, and subsequent development of new design guidelines of high biocompatibility.

Planned secondment(s):

• NVIDIA, M21-M23, 3 PMs, to implement on GPUs Eulerian simulations of blood dynamics.

Fellow: ESR10

Host institution: BUW

Ph.D. enrolment: UNIFE/UCY

Project Title: Algorithms for Relativistic Lattice Boltzmann

Objectives: To explore advanced LBM on new computer architectures able to handle flows in the ultra-relativistic to mildly relativistic regime in 2-dimensions and in 3-dimensions, experimenting with different possible options for the equation of state of the simulated fluids. The algorithmic developments go in the direction of making this algorithmic approach an efficient option for the simulation of physically interesting applications, such as the study of quark gluon plasma dynamics, astrophysical context or even exotic states in condensed matter, e.g. graphene. This is aim is that the developed algorithm can be efficiently parallelized and optimized for current and next generation HPC systems and will be carried out together with ESR1.

Expected Results: i) development of advanced Lattice Boltzmann algorithms suitable for the simulations of fluids in the relativistic regime; ii) analysis of physics results using different equations of state; iii) development of GPU-optimized computer codes.

Planned secondment(s):

• FZJ, M24-M26, 3 PMs, to learn optimization of the developed algorithms on large HPC systems.

Fellow: ESR11

Host institution: UTOV

Ph.D. enrolment: BUW/CyI

Project Title: Two way coupling in turbulence flow: modulating turbulence by preferential concentration of particles

Objectives: To evaluate the effects of turbulent modulation by using particles with preferential concentration on different flow structures. The evolution of small (point like) particles in turbulent flows is a complicated out-of-equilibrium problem with fundamental open questions. In the case of highly diluted systems, the reaction of particles on the flow is neglected (one way coupling). Single particles distributions and multi-particle dispersions for the one-way coupling cases and for non-spherical objects will be studied initially. Cases in which particles can react and affect the flow evolution, the so called two-way coupling regime, will also be studied, in which the degree of complexity escalates in both numerical and fundamental aspects. In the long term, the objective is to study the effects on turbulent modulation by studying the feedback on the flow by particles with different shape, inertia either with or without a certain degree of activity

Expected Results: i) Develop and optimize massively parallel pseudo-spectral codes to study one- and two-way coupling of particles advected by, and reacting with, turbulent flows, in absence of boundaries and at different degrees of turbulence; 2) Develop proof of concepts on how to engineer active particles by using machine and reinforcement learning techniques to target specific turbulent topological structures. iii) devise/perform state-of-the-art direct numerical simulation of particle advection/reaction in complex flows in two and three dimensions.

Planned secondment(s):

• IBM, M21-M23, 3 PMs, to be introduced to deep learning algorithms for optimal turbulence control.

Fellow: ESR12

Host institution: UTOV

Ph.D. enrolment: UCY/HUB

Project Title: Calculation of masses of charged hadrons and leptonic decay rates including QED

Objectives: i) Ab-initio numerical calculation of radiative corrections to the masses of charged hadrons and to the leptonic decay rates of light pseudoscalar mesons, needed for the extraction of the first-row elements of the Cabibbo–Kobayashi–Maskawa (CKM) matrix with sub-percent theoretical uncertainties; ii) Detailed theoretical analysis of infrared divergences arising in finite-volume calculations of electromagnetic radiative corrections to the decay rates of charged hadrons. Development of theoretical methods, based on spectral analysis and density-of-states techniques for avoiding these divergences; iii) Implementation of parallel computer programs, highly optimized for architectures such as Intel Xeon or Xeon Phi processors, to perform numerical simulations of QCD plus QED from firstprinciples, on a finite volume using charge conjugation (C*) boundary conditions;

Expected Results: i) Detailed study of the cost of algorithms for QED plus QCD simulations by using domain-decomposition, chargepreconditioning and mass-preconditioning techniques; ii) Generation of the required SU(3)xU(1) gauge-field configurations using C* boundary conditions with light quark masses, corresponding to pion masses ranging from ~300MeV down to the physical point, and large volumes, ranging from ~4fm at fine lattice spacings to ~8fm at coarser resolutions; iii) Ab-initio numerical calculation of the charged neutral mass splitting of the light pseudoscalar mesons, employing methods for quark loops developed by ESR2 when these contribute; iv) Development of theoretical methods, based on spectral analysis and density-of-states techniques, to compute electromagnetic radiative corrections to decay rates; v) Ab-initio numeric al calculation of the leptonic decay rates π to lv(γ) and Κ to lv(γ) and extraction of the CKM matrix elements V_ud and V_us with sub-percent theoretical uncertainties

Planned secondment(s):

• MAGWEL, M21-M23, 3 PMs, to learn to apply lattice QCD ideas to electromagnetic modeling.

Fellow: ESR13

Host institution: UCY

Ph.D. enrolment: HUB/UTOV

Project Title: Direct calculation of Parton Distribution Functions (PDFs) on the lattice

Objectives: To calculate in lattice QCD PDFs measured in deep inelastic scattering. i) Develop innovative methods for obtaining quasi-PDFs at large momenta, in order to reliably extrapolate to the infinite momentum frame; ii) carry out the renormalisation to connect the quasi-PDFs to the experimentally measured PDFs, including the removal of all divergences and taking into account mixings with unwanted operators; iii) extend the methodology to generalized distribution functions

Expected Results: 1) Results on the unpolarized, polarized and transversity PDFs using simulations with a physical value of the pion mass; 2) Application of data/noise reduction techniques developed with ESR5; iii) Exploratory study of generalized parton distributions.

Planned secondment(s):

• DESY, M26-M28, 3 PMs, for matching lattice quasi-distributions to the ones extracted from experiment.

Fellow: ESR15

Host institution: RWTH

Ph.D. enrolment: CyI/HU

Project Title: HPC and machine learning studies of biomolecules involved in neurodegeneration

Objectives:To predict the conformation of the human α-synuclein protein, associated with the pathogenesis of Parkinson’s. α-synuclein is indeed a naturally unfolded monomer likely to be involved in the structural transitions of the protein to amyloid fibrils. Both the wild-type and disease-linked mutants will be investigated by HPC-based simulation. The calculations will provide the molecular basis of the effect of the mutations on the structural determinants of the protein. The project will furthermore include the use data science approaches to identify ligands interfering with the RNA encoding for the protein Huntington in Huntington’s desease. This work builds on our recent investigation by HPC-based molecular simulation, which has allowed us to identify a ligand - furamidine- affecting huntingtin expression in living cells (Matthes et al, ACS Chem Neurosci. 2018, 9,1399). The ligands will be then tested in Prof. Schulz’s lab at RWTH.

Expected Results: i) Rational design of new ligands using HPC-bases computational protocols; ii) Testing of the ligands in vitro and in vivo.

Planned secondment(s):

• CING, M24-M26, 3 PMs, to analyse simulation data and connect them to the experiments.
• IBM, M21-M23, 3 PMs. To be trained in deep learning algorithms.

Fellow: ESR16

Host institution: HU

Ph.D. enrolment: RWTH/UNIFE

Project Title:Drug Design for the NEET proteins involved in Progressive Neurological Diseases

Objectives: The NEET proteins are 2Fe-2S proteins involved in neurodegeneration, probably due to its involvement in Fe/Fe-S/Redox/Ca+2 homeostasis. MitoNEET (mNT) and NAF-1 have homodimeric structure with the unique “NEET fold” and labile [2Fe-2S] clusters coordinated by 3Cys:1His (His87/114). Molecular dynamics along with quantum chemical calculations showed that the protonation of the histidine leads to the cluster loss, which is associated with a dramatic decrease mNT/NAF-1 structure, which disrupts the protein function. A homozygous mutation in the NAF-1 encoding gene, cisd2, causes the Neurodegenerative progressive disease Wolfram Syndrome Type-2. The single missense mutation at nucleotide 109 where G is substituted with C (G109C), leads to exon skipping, frame shift and premature stop codon that results in the absence of the NAF-1 protein. This mutation is highly abundant (1:40) in regional Palestinian populations. WFS-T2 patients suffer, among other pathologies, from optical nerve atrophy, sensorineural hearing loss, psychiatric episodes and b-cell dysfunction. To avoid these pathophysiological situations, we tested treatments of an iron chelator with/without antioxidant agent on skin fibroblasts obtained from four WFS-T2 patients. The results show that the combined therapy (iron chelation & anti-oxidant), induced a correction of some of the pathophysiological disorders in the WFS-T2 patient. Yet, novel, more powerful drugs are dramatically needed to fully manage the WFS-T2 disease.

Expected Results: i) The PhD student will use advanced, High Performance Computing (HPC)-based simulation and machine learning tools to design drug leads. The studies will use mitoNEET and NAF-1 proteins as a showcase. The effect of drug leads will be tested in vitro.

Planned secondment(s):

• CING, M21-M24, 4 PMs, to learn advanced analysis methods for neuronal networks related data