Agate is a Hewlett Packard Enterprise (HPE) cluster. Delivery is expected to begin in August 2021, with the cluster available to Minnesota researchers in the fall. The new cluster’s name, Agate, was selected by MSI staff from a pool of names submitted by 133 people. The Lake Superior agate is Minnesota’s State Gemstone.
The Agate cluster will join existing systems in MSI's data center at the University of Minnesota's Walter Library: the HPE/AMD Mangi cluster (2019) and HPE/Intel Mesabi cluster (2015).
System Specifications
The theoretical performance of the new HPE-built cluster is about 7 quadrillion floating point operations per second (7 petaflops), of which 2 petaflops are provided by the cluster's general-purpose processors, and 5 petaflops by the GPU subsystem. "We worked closely with HPE to develop a highly efficient and flexible system …" said Graham Allan, Associate Director for Operations at MSI. "The Agate cluster will provide over 7 times the performance of the existing Mesabi cluster, while consuming the same amount of power and occupying less floor space."
The Agate cluster will contain a total of 416 nodes, including HPE Apollo 2000 XL225 Gen10+ CPU-only systems, and Apollo 6500 Gen10+ GPU servers, all using Direct Liquid Cooling, coupled with a set of large-memory HPE DL385 Gen10+ nodes. In total the cluster will contain 770 AMD Milan 7763 64-core processors, and 264 NVIDIA A100 Tensor Core GPUs, all connected with Mellanox HDR-100 InfiniBand network.
One unique feature of the new cluster is provision of an additional 80 Nvidia A40 single-precision GPUs, to further support interactive applications with server-side GPU rendering, and interactive machine-learning workloads via command line and JupyterHub.
"The new system is designed to position our faculty at the leading-edge of high performance computing capabilities and to minimize obstacles to accelerating data intensive research," said Jim Wilgenbusch, Director of Research Computing
Between now and delivery of the new cluster, MSI will also be upgrading the capacity and performance of its cluster storage systems, starting with updating the existing Panasas parallel storage system to next-generation Panasas Ultra hardware. This will deliver approximately 10 PB of new capacity, representing a net increase of 5 PB, in order to better support a variety of data-intensive research initiatives. Alongside the Panasas storage upgrades, the capacity of MSI's ceph-based tier 2 storage, built using HPE hardware, will also be increased by 5 PB.
Timeline
We anticipate delivery of the Agate system to begin in August 2021, with initial production capability in late fall.