Four things data centre managers can learn from the fastest supercomputers

1. Supercomputers are purpose-built for consistency

Unlike most cloud-computing platforms, such as Amazon Web Services or Microsoft Azure, which are built to power a variety of applications that can utilise shared resources and infrastructures, most supercomputers are purpose-built for specific needs. The most recent update of the TOP500 list of the world’s fastest supercomputers (publicly-known and declassified), notes not only the locations and speeds of installations, but the primary field of application.

Eleven of the top dozen machines are dedicated to energy research, nuclear testing and defence applications. The only outlier is Frontera, a new NSF-funded petascale computing system at the Texas Advanced Computing Centre at the University of Texas, which provides academic resources for science and engineering research partners. Of the next 20 supercomputers on the TOP500 list, almost all are dedicated to Government defence and intelligence applications. Machines between positions 30-50 on the list are largely dedicated to weather predictions. The last 50 of the top 100 are a mix of corporate computing (NVIDIA, Facebook, et.al), midrange weather predictions, space programmes, oil and gas exploration, academic and specific government uses.

These machines aren’t a one-size-fits-all box. They’re custom developed together with manufacturers like Intel, Cray, HP, Toshiba and IBM to perform specific types of calculations on very specific datasets – either in real-time or asynchronous computations.

They have defined acceptable latency thresholds:

• Preset computing resources leveraging millions of processing cores
• Deliver clock rates between 18,000 and 200,000 teraFLOPS.

Their storage capacities are measured in exabytes – far beyond the petabytes in modern data warehouses.

Systems like Frontera don’t just have to sprint in a peak compute load, but instead have to consistently read vast amounts of data to arrive at a result. A spike in compute performance could actually cause errors in results, thus the emphasis is on consistency.

Today’s data centre manager must first ask, “What are we doing with the system?” in order to architect, manage resources and build in predictable fail-safes. Managing a data centre that runs a number of virtual desktops is a lot different than a 999 call centre or air-traffic control system. They have different needs, demands, service-level agreements and budgets – and need to be designed accordingly.

Likewise, there needs to be a consideration about how to achieve consistent performance without requiring custom builds. Companies like Amazon, Google and Microsoft have the budgets to engineer custom storage or computing infrastructures, but the majority of service providers have to be more selective with off-the-shelf hardware.

Thus, more data centre managers need to set strict criteria for performance benchmarks that address QoS and ensure the greatest emphasis is not only on compute speed and latency, but also on consistency.

3. Latency thresholds, NAND Flash and tuning DRAM

Most of the latency thresholds are set because of the application demands. In trading scenarios, seconds mean millions, if not billions of dollars. For weather predictions and hurricane tracking, it could mean deciding between evacuating New Orleans or Houston.

Supercomputers operate with the a priori burden of service level – whether latency, computing resources, storage or bandwidth. Most employ fail-aware computing, whereby the system can reroute data streams for optimal latency conditions (based on 𝛱+Δmax clocking) by shifting to asynchronous computing models or prioritising compute resources to deliver sufficient processing power or bandwidth for jobs.

Whether you’re working with high-end workstations, iron servers or HPC and scientific workloads, big computers and Big Data require huge DRAM loadouts. Supercomputers like the Tianhe-2 use huge RAM loadouts combined with specialised accelerator cards. The ways in which supercomputing fine-tunes the hardware and controller framework is unique to the application design. Often, specific computational tasks with disk accesses that create a huge bottleneck with RAM requirements make DRAM impractical but are small enough to fit into NAND flash. The FPGA clusters are also further tuned for each specific workload to ensure large data sets take huge performance hits if they have to use traditional media to retrieve data.

The teams collaborating between the University of Utah, Lawrence Berkeley Lab, the University of Southern California and Argonne National Lab have shown new models for automatic performance tuning (or auto-tuning) as an effective means of providing performance portability between architectures. Rather than depending on a compiler that can deliver optimal performance on more novel multicore architectures, auto-tuned kernels and applications can auto-tune on the target CPU, network and programming model.