The Supercomputer Race, Revisited

Nine months ago, I wrote a post called “The Supercomputer Race” about China’s then top-ranked supercomputer, the Tianhe-1A and what it meant for the U.S. The Tianhe-1A achieved 2.57 petafolps (10¹⁵ floating point operations per second) on LINPACK, a benchmark based on a series of complex linear equations. In comparison, the world’s next fastest system at the time was Oak Ridge National Laboratory’s Jaguar, clocking in at 1.76 petaflops. (Based on TOP500’s semi-annual ranking of the world’s five hundred fastest supercomputers.)

Today TOP500 released their latest rankings, which put Japan’s K Computer in the number one spot with 8.162 petaflops (PFLOPS), a jump of more than three times the performance of the now number two Tianhe-1A. How was such a sharp increase realized and what does it mean for supercomputing in the future?

A little history: TOP 500 has been ranking the world’s supercomputers since 1993. During this time individual and cumulative performance has followed a steady pattern of exponential growth. As with this latest ranking, individual rankings have shown a significant jump in some years (e.g., 1997, 2002), followed by years of more modest improvement. On the other hand, cumulative totals have been very consistent due to the broader data set and are probably a better indicator of where the trend stands overall. (Interestingly, RIKEN’s K Computer represents a jump not seen since Japan’s last number one, the Earth Simulator in 1992.) Not surprisingly, the plot points for the performance growth of the number one spot approximate a classic series of sigmoid growth curves, as technologies reach their limits and are superseded by others.

The substantial leap forward last year by the Tianhe-1A can mostly be attributed to one significant improvement: the implementation of radically faster interconnects. Rather than focusing on the latest step up in CPU technology, the designers of the Tianhe-1A focused on the biggest bottleneck in the system. Interconnects are networking chipsets that coordinate the data continually being moved between processors, in this case thousands of Intel Westmere and Nvidia Fermi processors. China’s homegrown Galaxy interconnects were a huge improvement in performance at double the speed of the Infiniband interconnects used in many other systems.

This latest ranking saw improvements that are due to a related trend: the transition away from monolithic CPU-based systems to heterogeneous platforms. (Heterogeneous platforms utilize a variety of different types of computational units, including CPUs, GPUs, interconnects, etc.) Looking at the trend line, the Tianhe-1A represented a 50% increase over Oak Ridge’s Jaguar. Japan’s K Computer improves on the Tianhe-1A by almost 200%. During this next year, two U.S. systems are slated to become operational with peak performances in the 20 PFLOP range or a further gain of 150%.

So does this point to a long-term increase in the rate of improvement in supercomputing performance? I’d say, probably not. The elimination of bottlenecks and the transition to new approaches will likely be a blip on the trend line. As the industry moves toward the target of exascale supercomputing later this decade, we’re likely to see improvements slow at various points as we deal with some very considerable challenges of scale. It’s been said that while the move from terascale to petascale computing was evolutionary, the leap from petascale to exascale will be revolutionary. The solutions used in the earlier systems simply won’t scale up without significant changes being made.

A common question among the general public is “why do we even need more powerful supercomputers? Can’t we get by with what we have already?” The simple answer is ‘No’. If the U.S. wants to remain a leading technological and economic force in the world, it will be necessary to invest in a future in which supercomputers play a central role. If we’re to see the nascent technologies of the 21st century realized, we’ll need the vast processing power of exascale systems and beyond. Likewise, we’ll need next-generation supercomputers if we’re to overcome many of the challenges the world now faces. Our digital world is generating enormous quantities of data, data that is itself growing exponentially. Bioinformatics, proteomics and brain simulation are but a few of the fields that will require continuing improvements in supercomputing to deal with their immense data sets. For similar reasons, we’ll need these computers for complex analytic systems such as IBM’s DeepQA Project, more commonly known as Watson. The ability to create tremendously detailed climate models will also be essential as we deal with human-caused climate change, whether to predict its consequences or to implement solutions. In short, to abandon advances in supercomputing is to abandon our place in the future.

(The future of information management is explored in my recent article, “Treading in the Sea of Data”, in the July/August 2011 issue of The Futurist. The article is an abridged version of my paper which will be published this summer in the WorldFuture 2011 conference volume, “Moving From Vision to Action,” editor, Cynthia G. Wagner.)