Titan

Titan Project Timeline

Phase 0: Pre-Upgrade

Phase 0 reflects the current configuration of the Jaguar XT5 machine.

Phase 0: System Configuration

Name:Jaguar
Architecture:XT5
Processor:6-Core AMD
Cabinets:200
Nodes:18,688
Cores/node:12
Total cores:224,256
Memory/node:16GB
Memory/core:1.3GB
Interconnect:SeaStar2+
GPUs:0
Speed:

Titan Project Timeline

Phase 1: October 10, 2011

During Phase One, 56 of the 200 cabinets that comprise the current Jaguar XT5 system will be taken out of service and upgraded from the XT5 architecture to the XK6 architecture. During this phase of the upgrade, users will continue to have access to the remaining 144 cabinets in Jaguar. During this phase, the number of available cores will decrease from 224,256 to 162,240.

Phase 1: System Configuration

Name:Jaguar
Architecture:XT5
Processor:6-Core AMD
Cabinets:144
Nodes:13,824
Cores/node:12
Total cores:162,240
Memory/node:16GB
Memory/core:1.3GB
Interconnect:SeaStar2+
GPUs:0
Speed:

Titan Project Timeline

Phase 2: October 17, 2011

During Phase Two, 40 additional cabinets that comprise the current Jaguar XT5 system will be taken out of service and upgraded from the XT5 architecture to the XK6 architecture. During this phase of the upgrade, users will continue to have access to the remaining 104 cabinets in Jaguar and will continue to run on the XT5 architecture. During this phase, the number of available cores will decrease from 162,240 to 117,120.

Phase 2: System Configuration

Name:Jaguar
Architecture:XT5
Processor:6-Core AMD
Cabinets:104
Nodes:9,984
Cores/node:12
Total cores:117,120
Memory/node:16GB
Memory/core:1.3GB
Interconnect:SeaStar2+
GPUs:0
Speed:

Titan Project Timeline

Phase 3: December 5, 2011

During Phase Three, users will transition over to using the 96 cabinets that were upgraded to the XK6 architecture in Phases 1 and 2. As part of the architecture upgrade, the processor changes from dual 6 core AMD CPUs to a single 16 core AMD CPU and the Seastar interconnect is replaced with Cray's latest Gemini network, offering increased bandwidth, lower latency, and advanced features such as one-sided communication and atomic memory operations. During this phase, the number of available cores will be 142,848. The remaining 104 cabinets will be taken out of service and upgraded to the XK6 architecture.

Phase 3: System Configuration

Name:Jaguar
Architecture:XK6
Processor:16-Core AMD
Cabinets:96
Nodes:9,216
Cores/node:16
Total cores:142,848
Memory/node:32GB
Memory/core:2GB
Interconnect:Gemini
GPUs:0
Speed:

Titan Project Timeline

Phase 4: January 16, 2012

During Phase Four, the machine will be unavailable to users as the entire machine goes through stability and acceptance testing. This period could last anywhere from 2-4 weeks. During this phase, the number of available cores will be 0.

Phase 4: System Configuration

Name:Jaguar
Architecture:XK6
Processor:16-Core AMD
Cabinets:0
Nodes:0
Cores/node:0
Total cores:0
Memory/node:0
Memory/core:0
Interconnect:0
GPUs:0
Speed:0

Titan Project Timeline

Phase 5: Current

During Phase Five, the entire machine will be returned to users. The machine will consist of 200 cabinets running on the XK6 architecture. During this phase, the number of available cores will be 299,008. In addition, there will also be 10 cabinets that contain 960 GPUs.

Phase 5: System Configuration

Name:Jaguar
Architecture:XK6
Processor:16-Core AMD
Cabinets:200
Nodes:18,688
Cores/node:16
Total cores:299,008
Memory/node:32GB
Memory/core:2GB
Interconnect:Gemini
GPUs:960
Speed:

Titan Project Timeline

Phase 6: June 19, 2012

In preparation for the final upgrade phase, additional work will need to be done to ready the cabinets for the accelerator parts. The cabinet work is scheduled to begin June 19, 2012 and last through September. As a result of this effort, 16 cabinets will be removed from the XK6 (Jaguar) system each week, upgraded, and then returned to service the following week. This process will be repeated until all cabinets are upgraded. This rolling upgrade will result in approximately 8% of the system being unavailable during this time period.

Phase 6: System Configuration

Name:Jaguar
Architecture:XK6
Processor:16-Core AMD
Cabinets: 184
Nodes: 274,848
Cores/node:0
Total cores:0
Memory/node:0
Memory/core:0
Interconnect:0
GPUs:0
Speed:0

Titan Project Timeline

Phase 7: Fall 2012

During Phase Seven, there will be both periods of reduced computing capability and/or periods where the entire system will be unavailable while the accelerators are installed. The OLCF will do everything possible to minimize the time when Jaguar is unavailable during this upgrade. This period will last 3-5 months.

Phase 7: System Configuration

Name:Jaguar
Architecture:XK6
Processor:16-Core AMD
Cabinets:0
Nodes:0
Cores/node:0
Total cores:0
Memory/node:0
Memory/core:0
Interconnect:0
GPUs:0
Speed:0

Titan Project Timeline

Phase 8: TBD

Upon conclusion of Phase Six, the machine will be returned to users with the additional GPUs available and the official name will change from Jaguar to Titan.

Phase 8: System Configuration

Name:Titan
Architecture:XK6
Processor:16-Core AMD
Cabinets:200
Nodes:18,688
Cores/node:16
Total cores:299,008
Memory/node:32GB
Memory/core:2GB
Interconnect:Gemini
GPUs:TBD
Speed:

What’s Changing?

Name

The name of the machine will change from Jaguar to Titan after the completion of the final phase of the upgrade which is tentatively scheduled for late 2012.

Architecture

The machine will be upgraded from the Cray XT5 architecture to the Cray XK6 architecture beginning in October 2011. The Cray XK6 supercomputer takes the proven Cray XT5 infrastructure and incorporates two innovative new technologies: AMD’s powerful multi-core processors and the Gemini interconnect. For more information on the new architecture, please visit Cray’s website at http://www.cray.com/Products/XK6/XK6.aspx.

Processor

As part of the architecture upgrade, the processor will change from dual 6-core AMD CPUs to a single 16-core AMD CPU.

Cabinets

The number of cabinets will not change after the upgrade is complete. Some cabinets will not be available during fall 2011 to users while the hardware is being upgraded.

Nodes

The number of nodes will remain 18,688.

Cores Per Node

The number of cores on each node will increase from 12 to 16.

Total Cores

The upgrade will increase the number of CPU cores by 33 percent, from 224,256 to 299,008.

Memory/Node

The memory per node will double from 16GB to 32GB.

Memory/Core

The memory per core will increase from 1.3GB to 2GB.

GPUs

As part of the upgrade from the XT5 architecture to the XK6 architecture, 960 NVIDIA Tesla 20-series GPUs will be installed. During the last phase of the project, additional GPUs will be installed. The number of GPUs installed in the final phase will depend upon the budget but the final system should be in the range of 10—20 PF.