Seminar Details

An important class of HPC applications are naturally expressed as ensembles, where a set of many, distinct computational tasks execute in a coordinated manner to produce a collective outcome. It is also often the case that scientific problems that have traditionally been expressed as a single computational task can also be reformulated using ensembles. The ensemble computational model is highly relevant with the end of Dennard scaling and has the ability to overcome limitations of single task applications to achieve significant performance gains on large-scale parallel machines. Not surprisingly, the concept of running ensembles on large-scale HPC systems is thus gaining in importance.

Using biomolecular simulations as a case study, we trace developments in ensemble-based applications over the past 15 years. In doing so, we discuss the abstractions (pilot-systems) and software systems (RADICAL-Cybertools) we have developed. We will discuss scientific advances across multiple domains engendered by RADICAL-Cybertools, and how RADICAL-Cybertools along with advances in statistical adaptive algorithms have enabled ensemble-based applications to overcome traditional limitations of single task applications. However, in spite of several orders of magnitude improvement in efficiency, much greater improvements are needed. We will close with recent investigations of novel application architectures motivated by the RADICAL, if not subversive idea of further improving and providing greater power to the many!