The scientific demand for more accurate modeling of the climate system calls for more computing power to support higher resolutions, inclusion of more component models, more complicated physics schemes, and larger ensembles. As the recent improvements in computing power mostly come from the increasing number of nodes in a system and the integration of heterogeneous accelerators, how to scale the computing problems onto more nodes and various kinds of accelerators has become a challenge for the model development. This paper describes our efforts on developing a highly scalable framework for performing global atmospheric modeling on heterogeneous supercomputers equipped with various accelerators, such as GPU (Graphic Processing Unit), MIC (Many Integrated Core), and FPGA (Field Programmable Gate Arrays) cards. We propose a generalized partition scheme of the problem domain, so as to keep a balanced utilization of both CPU resources and accelerator resources. With optimizations on both computing and memory access patterns, we manage to achieve around 8 to 20 times speedup when comparing one hybrid GPU or MIC node with one CPU node with 12 cores. Using a customized FPGA-based data-flow engines, we see the potential to gain another 5 to 8 times improvement on performance. On heterogeneous supercomputers, such as Tianhe-1A and Tianhe-2, our framework is capable of achieving ideally linear scaling efficiency, and sustained double-precision performances of 581 Tflops on Tianhe-1A (using 3750 nodes) and 3.74 Pflops on Tianhe-2 (using 8644 nodes). Our study also provides an evaluation on the programming paradigm of various accelerator architectures (GPU, MIC, FPGA) for performing global atmospheric simulation, to form a picture about both the potential performance benefits and the programming efforts involved.
Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China;Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;Department of Computer Science and Technology, Tsinghua University, Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China;Institute of Software, Chinese Academy of Science, Beijing, China;Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;Department of Computer Science and Technology, Tsinghua University, Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;College of Global Change and Earth System Science, Beijing Normal University, Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China;Department of Computer Science and Technology, Tsinghua University, Beijing, China;Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China;Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China;Joint Center for Global Change Studies (JCGCS), Beijing, China;Department of Computer Science and Technology, Tsinghua University, Beijing, China;National Supercomputing Center in Wuxi, Wuxi, China
Recommended Citation:
Haohuan Fu,Lin Gan,Chao Yang,et al. Solving global shallow water equations on heterogeneous supercomputers[J]. PLOS ONE,2017-01-01,12(3)