Oak Ridge Leadership Computing Facility

Center-Wide File System

• Spider
  The Spider center-wide file system is the operational work file system for most OLCF systems. An extremely high-performance system, Spider has over 26,000 clients, providing 10.7 petabytes of disk space and can move data at more than 240 gigabytes a second. It is both the fastest and largest-scale Lustre file system in the world.For more information about Spider, see https://www.olcf.ornl.gov/kb_articles/spider-changes-2012/

• ADIOS (module load adios)
  http://olcf.ornl.gov/center-projects/adios/
  The Adaptable I/O System (ADIOS) library, developed at OLCF, is a flexible method for applications to transparently manage data I/O. Using ADIOS, applications can change their I/O patterns and methods (such as MPI-IO or POSIX I/O) by simply editing a configuration file. This allows applications to optimize their I/O workload for different computing systems utilizing different I/O subsystems and filesystems.
• HDF5 (module load hdf5)
  http://www.hdfgroup.org/HDF5
  The Hierarchical Data Format library provides a standard, portable method of describing and interacting with structured datasets. HDF5 includes support for manipulating files in parallel using parallel applications.
• NetCDF (module load netcdf)
  http://www.unidata.ucar.edu/software/netcdf
  The Network Common Data Form library provides a standard, portable method of manipulating array-structured data. Used by many weather and climate modeling applications, netCDF can be configured to use HDF5 as a backend for parallel applications. The netCDF libraries at OLCF support file format versions 3 and 4.
• MPI-I/O (available by default)
  The MPI-2 standard (www.mpi-forum.org) includes MPI-I/O, a set of routines for manipulating files using large numbers of processes. MPI-I/O is available by default on all OLCF systems and is automatically included using the compiler wrapper programs (cc and ftn).
• LibLUT (module load liblut)
  http://olcf.ornl.gov/center-projects/liblut/
  The Lustre User Toolkit, developed at OLCF, is an API for interacting with Lustre filesystems. LUT can also provide I/O timing information, allowing application authors to optimize their use of Lustre filesystems. OLCF has also made available lustre-optimized user utilities developed using LUT. The spdcp (module load spdcp) program is a fast, parallel tool to copy large datasets on and across Lustre filesystems.

• HPSS (High Performance Storage System)
  The mass storage facility at ORNL currently consists of tape and disk storage components, Linux servers, and High Performance Storage System (HPSS) software. As of June 2010, the computing center has over 12 PB stored in over 19 million files.

  Incoming data is written to disk and later migrated to tape for long term archival. Robotic tape libraries provide tape storage.

  The center has 4 SL8500 tape libraries, each holding up to 10,000 cartridges. The libraries house a total of twenty-four T10K-A tape drives (500 GB cartridges, uncompressed) and sixty-four T10K-B tape drives (1 TB cartridges, uncompressed). Each drive has a bandwidth of 120 MB/s.

  As storage, network, and computing technologies continue to change, ORNL’s storage system evolves to take advantage of new equipment that is both more capable and more cost-effective. In 2011, the center expects to see the addition of another SL8500, which will hold 10 petabytes of data with the current generation of tape drives. With the new tape library, ORNL will have room to store and use up to 50,000 tape cartridges.