The Green500 List - November 2012
Heterogeneous Systems Re-Claim Green500 List Dominance
SALT LAKE CITY, UT - November 14, 2012 - The latest Green500 List was released today (http://www.green500.org/lists/green201211) and the top spots on the list have been taken over by machines that combine commodity processors with coprocessors or graphics processing units (GPUs) to form heterogeneous high-performance computing systems.
With all eyes on the new TOP500 number one system, Oak Ridge National Labs' Titan, it was a system belonging to a neighbor at the University of Tennessee that debuted at the top of the November Green500 List. The National Institute for Computational Sciences' Beacon system has set the new energy efficiency bar at nearly two-and-a-half billion floating-point operations per second (gigaflops) per watt. Employing Intel's Sandy Bridge series of Xeon central processing units (CPUs) and four of Intel's Xeon Phi coprocessors per node Beacon achieved a peak 112,200 gigaflops of performance running the LINPACK benchmark while consuming only 44.89 kW of power.
The Intel Xeon Phi--"Knights Corner"-- is a 22nm multicore coprocessor featuring the world's first 3D Tri-Gate transistors. Like its GPU counterparts, the Intel Xeon Phi resides on a PCI Express board that plugs into a machine's expansion slots.
Rounding out the top five systems are three other machines using GPU accelerators combined with traditional AMD or Intel CPUs. "Metaphorically, think of CPU-GPU systems operating like the human brain, where the CPU could be viewed as the left brain and the GPU as the right brain," says Dr. Wu Feng, founder of the Green500 List. "Each side of the brain is suited to process different types of tasks." In second place is the SANAM system from the King Abdulaziz City for Science and Technology, an Intel CPU system that uses AMD's new FirePro S10000 GPU accelerators, and in third and fourth are the Titan and Todi systems by Cray that employ AMD Opteron CPUs and nVIDIA Tesla K20 GPU accelerators. In number five is the previous Green500 List number one system, IBM's Blue Gene/Q that uses PowerPC BQC CPUs. All of these systems are over two gigaflops per watt.
Overall, the performance of machines in the Green500 List has increased at a higher rate compared to power consumption. "That's why the machines' efficiencies are going up," says Feng. "We are more performance for the same amount of power." For machines based on commodity components--machines built with off-the-shelf components-- coprocessors and GPUs are attributing a great deal to the efficiency gains. So much so that they are keeping pace and in the latest list even outpacing purpose built systems like IBM's Blue Gene/Q. "Power consumption is still going up," says Wu, and that is still a concern.
The Green500 List has provided a ranking of the most energy-efficient supercomputers in the world since November 2007. For decades, the notion of "performance" has been synonymous with "speed" as measured in FLOPS. This particular focus has led to the emergence of supercomputers that consume egregious amounts of electrical power and produce so much heat that extravagant cooling facilities must be constructed to ensure proper operation. In addition, the emphasis on speed as the ultimate metric has caused other metrics such as reliability, availability, and usability to be largely ignored. As a result, there has been an extraordinary increase in the total cost of ownership (TCO) of a supercomputer.
The Green500 List
Listed below are the November 2012 The Green500's energy-efficient supercomputers ranked from 101 to 200.
Green500 Rank | MFLOPS/W | Site* | Computer* | Total Power (kW) |
101 | 751.10 | Atomic Weapons Establishment | WillowA - Bullx B510, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 182.00 |
102 | 749.45 | Atomic Weapons Establishment | WillowB - Bullx B510, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 182.00 |
103 | 741.67 | Purdue University | Carter - Cluster Platform 3000 SL6500, Xeon E5 (Sandy Bridge - EP) 8C 2.60GHz, FDR Infiniband | 252.00 |
104 | 741.06 | National Supercomputing Center in Jinan | Sunway Blue Light - Sunway BlueLight MPP, ShenWei processor SW1600 975.00 MHz, Infiniband QDR | 1,074.00 |
105 | 738.73 | Norwegian University of Science and Technology | SGI Altix X, Xeon E5-2670 8C 2.600GHz, Infiniband FDR | 537.00 |
106 | 732.53 | Super Computer Center in Guangzhou | Tianhe-1A Guangzhou Solution - NUDT YH MPP, Xeon X56xx (Westmere-EP) 2.93 GHz, Proprietary | 289.00 |
107 | 731.92 | Sandia National Laboratories | Chama - Xtreme-X GreenBlade GB512X, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 453.60 |
108 | 722.25 | SGI | SGI Altix X, Xeon E5-2690 8C 2.900GHz, Infiniband FDR | 129.00 |
109 | 718.12 | Universitaet Frankfurt | LOEWE-CSC - Supermicro Cluster, QC Opteron 2.1 GHz, ATI Radeon GPU, Infiniband | 416.78 |
110 | 715.14 | Poznan Supercomputing and Networking Center, Institute of Bioorganic Chemistry | Rackable C1103-G15, Opteron 6234 12C 2.40 GHz, Infiniband QDR | 125.50 |
111 | 712.37 | Atomic Weapons Establishment | Blackthorn - Bullx B510, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 446.40 |
112 | 689.56 | Sandia National Laboratories | Dark Sand - Appro Xtreme-X Supercomputer, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 388.80 |
113 | 688.10 | PETROBRAS | Grifo04 - Itautec Cluster, Xeon X5670 6C 2.930GHz, Infiniband QDR, NVIDIA 2050 | 365.50 |
114 | 682.74 | Commissariat a l'Energie Atomique (CEA)/CCRT | airain - Bullx B510, Xeon E5-2680 8C 2.700GHz, Infiniband QDR | 260.00 |
115 | 678.14 | Virginia Tech | BlueRidge - Appro Xtreme-X Supercomputer, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 127.20 |
116 | 669.12 | Petroleum Company | x3650M4 Cluster, Xeon E5-2670 8C 2.600GHz, Infiniband FDR | 285.38 |
117 | 668.10 | National Super Computer Center in Hunan | Tianhe-1A Hunan Solution - NUDT YH MPP, Xeon X5670 6C 2.93 GHz, Proprietary, NVIDIA 2050 | 1,155.07 |
118 | 663.53 | NOAA/Earth Science Research Laboratory/GSD | sJet - Appro Xtreme-X Supercomputer, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 136.00 |
119 | 654.05 | Amazon Web Services | Amazon EC2 Cluster Compute Instances - Amazon EC2 Cluster, Xeon 8C 2.60GHz, 10G Ethernet | 367.08 |
120 | 643.69 | Kyoto University | Laurel - Xtreme-X , Xeon E5-2670 8C 2.600GHz, Infiniband FDR | 210.35 |
121 | 635.15 | National Supercomputing Center in Tianjin | Tianhe-1A - NUDT YH MPP, Xeon X5670 6C 2.93 GHz, NVIDIA 2050 | 4,040.00 |
122 | 634.01 | Vikram Sarabhai Space Centre, Indian Space Research Organisation | SAGA - Z24XX/SL390s Cluster, Xeon E5530/E5645 6C 2.40GHz, Infiniband QDR, NVIDIA 2090/2070 | 297.63 |
123 | 631.33 | DOE/SC/Argonne National Laboratory | Blues - Appro Xtreme-X Supercomputer, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 129.60 |
124 | 628.13 | Lawrence Livermore National Laboratory | Edge - Appro GreenBlade Cluster, Xeon X5660 6C 2.80 GHz, Infiniband QDR, NVIDIA 2050 | 160.00 |
125 | 627.82 | Cray Inc. | Crystal - Cray XC30, Xeon E5-2670 8C 2.600GHz, Aries interconnect | 234.67 |
126 | 627.82 | CSC (Center for Scientific Computing) | Cray Cascade, Xeon E5-2670 8C 2.600GHz, Aries interconnect | 337.33 |
127 | 627.82 | Cray Inc. | Ruby - Cray XC30, Xeon E5-2670 8C 2.600GHz, Aries interconnect | 344.67 |
128 | 627.82 | Swiss Scientific Computing Center (CSCS) | Piz Daint - Cray XC30, Xeon E5-2670 8C 2.600GHz, Aries interconnect | 344.67 |
129 | 603.73 | CEA/TGCC-GENCI | Curie thin nodes - Bullx B510, Xeon E5-2680 8C 2.700GHz, Infiniband QDR | 2,251.00 |
130 | 590.23 | Research Institute for Information Technology, Kyushu University | PRIMEHPC FX10, SPARC64 IXfx 16C 1.848GHz, Tofu interconnect | 282.43 |
131 | 582.00 | NOAA/Oak Ridge National Laboratory | Gaea C2 - Cray XE6, Opteron 6276 16C 2.30GHz, Cray Gemini interconnect | 972.00 |
132 | 579.51 | University of Oklahoma | Boomer - Dell R620/R720, Xeon E5-2650 8C 2.000GHz, Infiniband QDR, NVIDIA 2070 | 158.10 |
133 | 562.27 | International Fusion Energy Research Centre (IFERC), EU(F4E) - Japan Broader Approach collaboration | Helios - Bullx B510, Xeon E5-2680 8C 2.700GHz, Infiniband QDR | 2,200.00 |
134 | 554.65 | Louisiana State University | SuperMike-II - Dell PowerEdge C6220, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 200.00 |
135 | 521.96 | HPC2N - Umea University | Abisko - Supermicro H8QG6, Opteron 6238 12C 2.600GHz, Infiniband QDR | 252.70 |
136 | 506.77 | United Kingdom Meteorological Office | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 803.09 |
137 | 506.77 | United Kingdom Meteorological Office | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 223.08 |
138 | 506.77 | United Kingdom Meteorological Office | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 669.24 |
139 | 499.42 | University of Tokyo/Human Genome Center, IMS | HA8000-tc/HT225, Opteron 6276 16C 2.300GHz, Infiniband QDR | 201.43 |
140 | 492.64 | National Supercomputing Centre in Shenzhen (NSCS) | Nebulae - Dawning TC3600 Blade System, Xeon X5650 6C 2.66GHz, Infiniband QDR, NVIDIA 2050 | 2,580.00 |
141 | 483.12 | Universitaet Mainz | MOGON - Saxonid 6100, Opteron 6272 16C 2.100GHz, Infiniband QDR | 467.00 |
142 | 469.45 | Texas Advanced Computing Center/Univ. of Texas | Stampede - PowerEdge C8220, Xeon E5-2680 8C 2.700GHz, Infiniband FDR, Intel Xeon Phi | 5,666.77 |
143 | 467.72 | University of Bergen | Hexagon - Cray XE6m-200, Opteron 6276 16C 2.300GHz, Cray Gemini interconnect | 342.30 |
144 | 458.33 | DOE/NNSA/LANL | Cerrillos - BladeCenter QS22/LS21 Cluster, PowerXCell 8i 3.2 Ghz / Opteron DC 1.8 GHz, Infiniband | 276.00 |
145 | 455.39 | Government | Sunway 4000H Cluster, Xeon X56xx (Westmere-EP) 3.06 GHz, Infiniband QDR | 199.50 |
146 | 450.95 | Leeds N8 | N8 - Raackable Cluster, Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 180.00 |
147 | 449.41 | HWW/Universitaet Stuttgart | HERMIT - Cray XE6, Opteron 6276 16C 2.30 GHz, Cray Gemini interconnect | 1,850.00 |
148 | 444.35 | DOE/NNSA/LANL | Roadrunner - BladeCenter QS22/LS21 Cluster, PowerXCell 8i 3.2 Ghz / Opteron DC 1.8 GHz, Voltaire Infiniband | 2,345.00 |
149 | 443.08 | Calcul Canada/Calcul Qu�bec/Universit� de Sherbrooke | Rackable C2112-4G3 Cluster, Opteron 12 Core 2.10 GHz, Infiniband QDR | 542.34 |
150 | 441.47 | Slovak Academy of Sciences (SAV) | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 173.36 |
151 | 441.43 | Environment Canada | Power 775, POWER7 8C 3.84 GHz, Custom | 462.30 |
152 | 441.43 | Environment Canada | Power 775, POWER7 8C 3.84 GHz, Custom | 462.30 |
153 | 441.43 | ECMWF | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 1,386.91 |
154 | 441.43 | ECMWF | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 1,386.91 |
155 | 441.43 | IBM Poughkeepsie Benchmarking Center | Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 390.07 |
156 | 438.43 | Institute for Materials Research, Tohoku University (IMR) | Hitachi SR16000 Model M1, POWER7 8C 3.836GHz, Custom | 556.30 |
157 | 429.31 | Government | Sunway 4000H Cluster, Xeon X56xx (Westmere-EP) 2.93 GHz, Infiniband QDR | 339.15 |
158 | 426.72 | Los Alamos National Laboratory | Mustang - Xtreme-X 1320H-LANL, Opteron 12 Core 2.30 GHz, Infiniband QDR | 540.40 |
159 | 423.70 | IBM Development Engineering | DARPA Trial Subset - Power 775, POWER7 8C 3.836GHz, Custom Interconnect | 3,575.63 |
160 | 421.74 | Geoscience | iDataPlex DX360M3, Xeon X5660 6C 2.80 GHz, Infiniband QDR | 212.62 |
161 | 414.92 | University of Southampton | xSeries iDataPlex, Xeon E5645 6C 2.400GHz, Infiniband | 228.22 |
162 | 405.38 | Swiss Scientific Computing Center (CSCS) | Monte Rosa - Cray XE6, Opteron 6272 16C 2.10 GHz, Cray Gemini interconnect | 780.00 |
163 | 404.45 | CLUMEQ - McGill University | Guillimin - iDataPlex DX360M3, Xeon 2.66, Infiniband | 337.00 |
164 | 400.68 | Taiwan National Center for High-performance Computing | ALPS - Acer AR585 F1 Cluster, Opteron 12C 2.2GHz, QDR infiniband | 442.00 |
165 | 390.85 | Institute for Molecular Science | PRIMERGY RX300 S7, Xeon E5-2690 8C 2.900GHz, Infiniband QDR | 299.60 |
166 | 378.77 | EDF R&D; | Frontier2 BG/L - Blue Gene/P Solution | 252.00 |
167 | 378.77 | King Abdullah University of Science and Technology | Shaheen - Blue Gene/P Solution | 504.00 |
168 | 378.76 | IDRIS | Blue Gene/P Solution | 315.00 |
169 | 368.62 | Automotive | BladeCenter HS22 Cluster, WM Xeon 6-core 2.93Ghz, Infiniband | 240.41 |
170 | 368.62 | Automotive | BladeCenter HS22 Cluster, WM Xeon 6-core 2.93Ghz, Infiniband | 240.41 |
171 | 366.58 | DOE/NNSA/LLNL | Dawn - Blue Gene/P Solution | 1,134.00 |
172 | 363.98 | DOE/SC/Argonne National Laboratory | Intrepid - Blue Gene/P Solution | 1,260.00 |
173 | 362.20 | DOE/SC/LBNL/NERSC | Hopper - Cray XE6, Opteron 6172 12C 2.10GHz, Custom | 2,910.00 |
174 | 361.79 | University of Edinburgh | HECToR - Cray XE6, Opteron 6276 16C 2.30 GHz, Cray Gemini interconnect | 1,824.93 |
175 | 359.65 | Energy Company (A) | HP ProLiant SL390s G7 Xeon 6C X5650, Infiniband | 364.80 |
176 | 357.24 | National Institute for Fusion Science (NIFS) | Plasma Simulator - Hitachi SR16000 Model M1, POWER7 8C 3.836GHz, Custom Interconnect | 708.20 |
177 | 355.58 | Vienna Scientific Cluster | VSC-2 - Megware Saxonid 6100, Opteron 8C 2.2 GHz, Infiniband QDR | 430.00 |
178 | 354.32 | Lockheed Martin | BladeCenter HS22 Cluster (WM), Xeon X5650 6C 2.660GHz, Infiniband QDR | 260.62 |
179 | 352.53 | Ohio Supercomputer Center | Oakley - Cluster Platform SL390s G7, Xeon X5650 6C 2.66GHz, Infiniband QDR, NVIDIA 2070 | 225.00 |
180 | 349.78 | South Ural State University | SKIF Aurora - SKIF Aurora Platform - Intel Xeon X5680, Infiniband QDR | 287.04 |
181 | 347.89 | E-Security - Selex Elsag | Aurora - Eurotech Aurora, Xeon X5690 6C 3.470GHz, Infiniband QDR, NVIDIA 2090 | 317.71 |
182 | 342.92 | Information Initiative Center, Hokkaido University | Hitachi SR16000 Model M1, POWER7 8C 3.836GHz, Custom | 354.60 |
183 | 330.98 | Vestas Wind Systems A/S | iDataPlex DX360M3, Xeon X5670 6C 2.93 GHz, Infiniband QDR | 489.75 |
184 | 330.98 | EDF R&D; | Ivanhoe - iDataPlex, Xeon X56xx 6C 2.93 GHz, Infiniband | 510.00 |
185 | 330.98 | IBM Development Engineering | iDataPlex DX360M3, Xeon X5670 6C 2.93 GHz, Infiniband QDR | 316.50 |
186 | 322.11 | Moscow State University - Research Computing Center | Lomonosov - T-Platforms T-Blade2/1.1, Xeon X5570/X5670/E5630 2.93/2.53 GHz, Nvidia 2070 GPU, PowerXCell 8i Infiniband QDR | 2,800.00 |
187 | 314.31 | Volvo | Cluster Platform 3000 BL460c/SL250/ML350 Xeon E5-2670 8C 2.600GHz, Infiniband QDR | 392.40 |
188 | 311.77 | NASA/Ames Research Center/NAS | Pleiades - SGI ICE X/8200EX/8400EX, Xeon 54xx 3.0/5570/5670/E5-2670 2.93/2.6/3.06/3.0 Ghz, Infiniband QDR/FDR | 3,986.96 |
189 | 305.33 | Electronics | xSeries x3550M3 Cluster, Xeon X5650 6C 2.66 GHz, Infiniband QDR | 251.10 |
190 | 305.33 | Electronics | xSeries x3550M3 Cluster, Xeon X5650 6C 2.66 GHz, Infiniband QDR | 251.10 |
191 | 305.33 | Government | xSeries x3650M3 Cluster, Xeon X5650 6C 2.660GHz, Infiniband | 302.81 |
192 | 305.33 | Defence | xSeries x3650M3 Cluster, Xeon X5650 6C 2.660GHz, Infiniband | 334.06 |
193 | 297.44 | National Institute for Computational Sciences/University of Tennessee | Kraken XT5 - Cray XT5-HE Opteron Six Core 2.6 GHz | 3,090.00 |
194 | 297.26 | National Computational Infrastructure National Facility (NCI-NF) | Vayu - Sun Blade x6048, Xeon X5570 2.93 Ghz, Infiniband QDR | 425.22 |
195 | 278.89 | DOE/NNSA/LANL/SNL | Cielo - Cray XE6, Opteron 6136 8C 2.40GHz, Custom | 3,980.00 |
196 | 273.77 | Financial Securities | iDataPlex DX360M4, Xeon E5-2670 8C 2.600GHz, 10G Ethernet | 339.87 |
197 | 271.29 | Financial Services (H) | iDataPlex DX360M3, Xeon E5-2670 8C 2.600GHz, 10G Ethernet | 405.13 |
198 | 271.29 | Financial Services (H) | iDataPlex DX360M3, Xeon E5-2670 8C 2.600GHz, 10G Ethernet | 405.13 |
199 | 271.29 | Financial Services (H) | iDataPlex DX360M3, Xeon E5-2670 8C 2.600GHz, 10G Ethernet | 405.13 |
200 | 271.29 | Financial Services (H) | iDataPlex DX360M3, Xeon E5-2670 8C 2.600GHz, 10G Ethernet | 405.13 |
* Performance data obtained from publicly available sources including TOP500