Preparing users for Titan
In a sense, the morphing of ORNL's flagship supercomputer from Jaguar to Titan resembles earlier installations. Being at the forefront of scientific supercomputing, the Oak Ridge Leadership Computing Facility regularly upgrades its capabilities to give researchers access to the most powerful resources possible.
But Titan is different. Although Jaguar has gone through six upgrades over the past seven years, the challenge was always one of scale. The system got newer processors, and there were more of them, but the type of processor remained the same. Titan, on the other hand, will combine two very different types of processors—CPUs (traditional computer chips) and GPUs (chips originally developed to speed video performance)—and ask users to adapt their applications accordingly.
A Cray technician prepares to upgrade nodes on the Jaguar supercomputer. Photo: Jason Richards
"GPUs are fast as lightning," explains ORNL's Bronson Messer, "but you can't ask them to make a decision. All they can do is take a big chunk of data and do the same thing on it." Because CPUs and GPUs are good at different things, the new configuration asks researchers to rethink their problems. To help, the OLCF created the Center for Accelerated Application Readiness, or CAAR, a collaboration among application developers, Titan manufacturer Cray, GPU manufacturer NVIDIA, and the OLCF's scientific computing experts.
CAAR has been working for nearly two years to establish best practices for code writers. The center is divided into six teams working with six of the OLCF's most advanced and representative applications.
"Applications make use of supercomputers by finding effective ways to spread out the work over the available resources," Messer explains. "On Jaguar, for instance, the first level was to spread it across the nodes. Each node had two six-core AMD processors, so the work at hand would be spread across those.
"With Titan, each node will have not only a 16-core AMD processor but also an NVIDIA GPU capable of tackling hundreds of tasks simultaneously. Therefore, an application using Titan to the utmost must also find a way to keep the GPU busy, remembering all the while that the GPU is fast, but less flexible than the CPU."
The process must also allow researchers to move freely between supercomputing centers, notes OLCF Director Buddy Bland. The solution, he says, is in the creative use of programs known as compilers.
Application developers don't typically write code in languages that can be understood by Titan—or any other computer, for that matter. Rather, they write in languages such as FORTRAN, C, and C++ that must be translated into instructions that a computer can understand. The translator is a program known as a compiler.
"We've been working with the compiler community and the vendor community for a couple of years to help develop accelerator directives," Bland explains. "These are directives that you can put into your FORTRAN or C++ code that describe the parallelism in the code, but you let the compiler figure out how to express that parallelism in the machine language of whatever machine it's running on.
"In our case, it would figure out how to translate those instructions to run on the GPU. But they can also figure out how to run them most efficiently on the myriad of other processors that are out there." —Leo Williams