New ECP Co-Design Center to Focus on Exascale Machine Learning

Photo by Roy Kaltschmidt, Berkeley Lab.

The Exascale Computing Project (ECP) today announced ExaLearn, a new ECP co-design center focusing on exascale machine-learning technologies. The goal of the project is to provide exascale machine-learning software for use by ECP applications projects, other ECP co-design centers and U.S. Department of Energy (DOE) experimental facilities and computing facilities.

Eight DOE national laboratories are participating in ExaLearn, which is the ECP’s sixth co-design center; each center takes a collaborative approach to the R&D process, bringing together developers of the software, the hardware and the computational science applications. The ExaLearn project will be led by Francis Alexander, Deputy Director of the Computational Science Initiative at Brookhaven National Laboratory. In addition to Brookhaven, the ExaLearn collaboration comprises Argonne, Lawrence Berkeley (Berkeley Lab), Lawrence Livermore, Los Alamos, Oak Ridge, Pacific Northwest and Sandia national laboratories.

The practical end product of the ExaLearn project will be a scalable and sustainable machine learning software framework that allows application scientists and the applied mathematics and computer science communities to engage in co-design for learning, ECP noted in a news release announcing ExaLearn. The ExaLearn co-design center will also collaborate with ECP PathForward vendors on the development of exascale machine-learning software.

“This is an exciting new project, with the goal of bringing machine learning to the forefront of the ECP effort, looking at how we can use it to advance the goals of the ECP applications and tackling exascale-sized problems in machine learning,” said Peter Nugent, the Department Head for Computational Science at Berkeley Lab and the Lab’s principal investigator for ExaLearn. “As data rates grow and researchers want to tackle even larger analysis problems, we can imagine where we would be tuning thousands of hyperparameters simultaneously across an exascale machine or even deploying a single deep neural net across the whole machine.”

Initially, the Berkeley Lab team—which, in addition to Nugent, includes Prabhat, Wahid Bhimji and Quincey Koziol, all from Berkeley Lab Computing Sciences—will focus on the application areas of cosmology and climate and will work on developing benchmarks for current machine-learning methods running on DOE high performance computing facilities. “One of our primary goals is to understand, and then optimize, the I/O issues facing large machine-learning problems at these facilities,” Nugent said.

Berkeley Lab is at the forefront of developing scalable machine learning tools and methods for science applications. Kathy Yelick, Associate Laboratory Director for Computing Sciences at Berkeley Lab, recently testified before the U.S. House of Representatives’ Committee on Science, Space and Technology as part of an expert panel discussing big-data challenges and advanced computing solutions. Yelick emphasized that as scientific data sets continue to grow by leaps and bounds—and will do so even more in the era of exascale—machine learning represents an increasingly important approach for data analytics in science, complementing but not replacing modeling and simulation.

“The Exascale Computing Initiative is addressing one of the three requirements to make machine learning successful in the DOE: availability of extreme computing capabilities,” she stated. “The Exascale Computing Project is addressing some of the underlying computational challenges of data analytics, with applications in cancer research, microbiome analysis and light source imaging, all involving some form of machine learning along with other simulation and analytics methods.”

In addition to ExaLearn, Berkeley Lab currently leads the AMReX: Block-Structured Adaptive Mesh Refinement ECP co-design center, is a partner in the ExaGraph: Combinatorial Methods for Enabling Exascale Application co-design center and is involved in the ExaSky: Computing the Sky at Extreme Scales project.

The ECP is a collaborative effort of the DOE Office of Science and the National Nuclear Security Administration. As part of the National Strategic Computing initiative, ECP was established to accelerate delivery of a capable exascale ecosystem, encompassing applications, system software, hardware technologies and architectures and workforce development to meet the scientific and national security mission needs of DOE in the early 2020s time frame.