Years of scientific domain knowledge and code development have been invested in scientific applications which are now considerably more sophisticated than they were when the community adapted to the transition from vector to parallel supercomputing two decades ago. In terms of sheer size, some of the community codes are an order of magnitude larger than they were in the early 1990s.
Also, many of the significant architectural changes expected in the exascale era will be manifest in the next generations of computers, such as the Cori system at NERSC and in the future systems at other DOE national labs.
To help move the DOE research community along the path to exascale computing, Berkeley Lab has led and/or participated in projects that will help both supercomputing centers and users make the transition to next generation systems.
NESAP, the NERSC Exascale Scientific Applications Program
In 2014, NERSC established the NERSC Exascale Scientific Applications Program (NESAP), a collaborative effort designed to give code teams and library and tool developers a unique opportunity to prepare for Cori’s manycore architecture. NERSC selected 20 projects to collaborate with NERSC, Cray and Intel and access early hardware, special training and preparation sessions with Intel and Cray staff. In addition, another 24 projects, as well as library and tool developers, are participating in NESAP via NERSC training sessions and early access to prototype and production hardware. NERSC subsequently selected six science application teams to participate in the NERSC Exascale Science Applications Program for Data (NESAP for Data) program.
Since the NESAP program was unveiled in 2014, NERSC has been partnering with code teams and library and tool developers to prepare and optimize their codes for the Cori manycore architecture. Like NESAP, the NESAP for Data program joins application teams with resources at NERSC, Cray and Intel; however, while the initial NESAP projects involve mostly simulation codes, NESAP for Data targets science applications that process and analyze massive datasets acquired from U.S. Department of Energy-supported experimental and observational sources, such as telescopes, microscopes, genome sequencers, light sources and particle physics detectors.
Like the initial NESAP program, NESAP for Data also includes a post-doc program in which early career scientists are hired to work closely with applications teams and gain valuable experience in the process.
The DesignForward program initiated partnerships between DOE centers and vendors to accelerate the research and development of critical technologies needed for exascale computing. The principal research areas of interest in the DesignForward program were in the general areas of system integration and interconnect technology.
The FastForward program complemented the DesignForward program and focused on co-design efforts between DOE centers and vendors with the goal of improving processor, memory, storage and I/O technologies. Furthermore, these improvements should be aimed at maximizing energy efficiency and concurrency while increasing performance, productivity and reliability.
Berkeley Lab was a key member of the Combustion Co-Design Center (ExaCT). The work represented a collaboration between applied mathematicians and computational scientists who have developed the Low Mach number combustion code (LMC), and computer scientists focused on performance optimization through auto-tuning and DSLs, performance modeling, and architectural simulation.
The Computer Architecture Laboratory (CAL) was a joint NNSA/SC activity involving Sandia National Laboratories (CAL-Sandia) and Lawrence Berkeley National Laboratory (CAL-Berkeley). CAL teams explored energy efficient and effective processor and memory architecture in preparation for exascale and provided design space exploration for industry collaborations FastForward and DesignForward vendor partner programs described above.
Networking for Exascale Science
The Department of Energy’s Energy Sciences Network (ESnet) participated in developing the final findings and recommendations for DOE’s exascale requirements synthesis report as part of the exascale requirements reviews held with DOE program offices.
To support the increased flow and size of research data sets, ESnet and NERSC built a 400 gigabit-per-second (Gbps) super-channel, the first-ever 400G production link to be deployed by a national research and education network. DOE research is increasingly data-driven as scientists generate massive datasets at experimental facilities and through modeling and simulation at supercomputing centers like NERSC. Having higher-speed networks in place ahead of the growth in data sets will allow researchers to be able to move and analyze data on next generation supercomputers.