4 Things Data Center Managers Can Learn from HPC

1. Supercomputers are Purpose-Built for Consistency

Unlike most Cloud-computing platforms, like Amazon Web Services or Microsoft Azure which are built to power a variety of applications that can utilize shared resources and infrastructures, most supercomputers are purpose-built for specific needs. The most recent update of the TOP500 list of the world’s fastest supercomputers (publicly-known and declassified), notes not only the locations and speeds of installations, but the primary field of application.

Eleven of the top dozen machines are dedicated for energy research, nuclear testing and defense applications. The only outlier is Frontera, a new NSF-funded petascale computing system at the Texas Advanced Computing Center at the University of Texas, which provides academic resources for science and engineering research partners. Of the next 20 supercomputers on the TOP500 list, almost all are dedicated for Government defense and intelligence applications. Machines between numbers 30-50 on the list are largely dedicated for weather predictions. The last 50 of the top 100 are a mix of corporate computing (NVIDIA, Facebook, et.al), midrange weather predictions, space programs, oil and gas exploration, academic and specific government uses.

These machines aren’t a one-size-fits-all box. They’re custom developed with manufacturers like Intel, Cray, HP, Toshiba and IBM to perform specific types of calculations on very specific datasets ― either in real-time or asynchronous computations.

They have defined acceptable latency thresholds:

• Preset computing resources leveraging millions of processing cores
• Deliver clock rates between 18,000 and 200,000 teraFLOPS.

Their storage capacities are measured in exabytes ― far beyond the petabytes in modern data warehouses.

Systems like Frontera don’t just have to sprint in a peak compute load, but instead have to consistently read vast amounts of data to arrive at a result. A spike in compute performance could actually cause errors in results, thus the emphasis is on consistency.

Today’s data center manager needs to first ask, “What are we doing with the system?,” in order to architect, manage resources and build in predictable fail-safes. Managing a data center that runs a bunch of virtual desktops is a lot different than a 911 call center, or air-traffic control systems. They have different needs, demands, service-level agreements and budgets - and need to be designed accordingly.

Likewise, there needs to be a consideration about how to achieve consistent performance without requiring custom builds. Companies like Amazon, Google and Microsoft have the budgets to engineer custom storage or computing infrastructures, but the majority of service providers have to be more selective with off-the-shelf hardware.

Thus, more data center managers need to set strict criteria for performance benchmarks that address QoS and ensure the greatest emphasis is not only on compute speed and latency, but also consistency.

3. Latency Thresholds, NAND Flash and Tuning DRAM

Most of the latency thresholds are set because of the application demands. In trading scenarios, seconds mean millions, if not billions of dollars. For weather predictions and hurricane tracking, it could mean deciding between evacuating New Orleans or Houston.

Supercomputers operate with the a priori burden of service level - be it latency, computing resources, storage or bandwidth. Most employ fail-aware computing, whereby the system can reroute data streams for optimal latency conditions (based on 𝛱+Δmax clocking), shifting to asynchronous computing models, or prioritizing compute resources to deliver sufficient processing power or bandwidth for jobs.

Whether you’re working with high-end workstations, iron servers, or HPC and scientific workloads, big computers and Big Data require huge DRAM loadouts. Supercomputers like the Tianhe-2, use huge RAM loadouts combined with specialized accelerator cards. The ways in which supercomputing fine-tunes the hardware and controller framework is unique to the application design. Often specific computational tasks, where disk access creates a huge bottleneck with RAM requirements, make DRAM impractical but are small enough to fit into NAND flash. The FPGA clusters are also further tuned for each specific workload to ensure large data sets take huge performance hits if they have to use traditional media to retrieve data.

The teams collaborating between the University of Utah, Lawrence Berkeley Lab, the University of Southern California, and Argonne National Lab have shown new models for Automatic Performance Tuning (or Auto-tuning) as an effective means of providing performance portability between architectures. Rather than depending on a compiler that can deliver optimal performance on more novel multicore architectures, auto-tuned kernels and applications can auto-tune on the target CPU, network, and programming model.