Mellanox Technologies: Cluster Case Studies

Cluster Case Studies

Performance Case Studies

Mellanox proven performance and scalability abilities are the driving forces of the most powerful compute clusters in the world, as published in the Top500 list.

Galactic Computing - Maui HPCC - NASA - Sandia National Laboratories - Texas Advanced Computing Center/Univ. of Texas - Tokyo Institute - University of Nebraska - Virginia Tech

Galactic Computing

3.413TFlops , 562 Intel Xeon EM64T CPUs
Mellanox InfiniBand MemFree
820MB/s storage throughput
84.35% Linpack efficiency WORLD’S HIGHEST REPORTED RATING on x86!

Galactic Supercomputer

Having installed successfully in Nov. 2004 a GT1000 Supercomputing Blade System (1 Teraflops) at Beijing Biomedical Research Institute, Galactic Computing just completed another GT4000 Supercomputing Blade System (281 blades, 562 Intel® Xeon™ processors) which has attained the world rank of 100 according to the 2005 June 22nd release of Top500 supercomputers list. The peak performance of the entire system is 4.046 Teraflops and the HPLinpack performance is 3.413 Teraflops which corresponds to the efficiency of 84.35%.

GT4000 result demonstrates the highest HPLinpack efficiency in the world among teraflop s scale supercomputers based on x86 processors which have been announced to date.

Dr. Steve Chen, CEO and Founder of Galactic Computing and Chief System Architect of Supercomputing Blade System, said, “While we will crank up system performance gradually, we will continue to focus in creative system architecture and design to achieve highly parallel system efficiency, reliability, usability, and most importantly the actual deliverable performance based on real customers applications solution time and total cost of ownership for a wide range of scientific, engineering and commercial applications in fast growing China and global markets”.

“Incorporating Intel-based platforms and industry standard technologies such as InfiniBand, Gala ctic’s system delivers incredible teraflop s performance in a very efficient manner, enabling faster design and deployment opportunities,” said Pat Gelsinger, Senior Vice President and General Manager, Digital Enterprise Group of Intel Corporation, “This is where high-performance computing is headed. Intel, working with companies like Gala ctic and Mellanox, is committed to continue delivering these super-fast computers that help speed scientific research, exploration and many other important areas to all of humankind.”

Also, this is the one and only supercomputer in China with completely embedded end-to-end InfiniBand 10Gb from blade-to-blade and blade-to-storage with 4 microseconds latency (instead of 50 milliseconds as most 1Gb Ethernet networked server clusters) and can achieve storage data throughput rate up to 820MB/s (two to three times faster and yet cheaper than Fiber Channel SAN).

(Top)

Maui High Performance Computing Center (MHPCC)

41.4TFlops
1300 Dell PowerEdge 1955 servers with 5200 Intel Xeon 5100 CPU cores

The Maui High Performance Computing Center (MHPCC) is an Air Force Research Laboratory Center managed by the University of Hawaii. MHPCC is an Allocated Distributed Center of the Department of Defense (DoD) High Performance Computing Modernization Program, providing computational resources to the research and development, test and evaluation, and warfighter communities.

(Top)

NASA

52TFlops, 10K Itanium-2 CPUs
85% efficiency rating on LINPACK benchmark

The cluster enabled scientists to simulate multiple decades of ocean changes in only days (compared to five years of simulation every year on previous systems) and engineers to compute thousands of flight characteristics of aircraft designs in a single day (versus years on previous systems).

"Columbia will have a huge impact on NASA's ability to solve truly complex problems in engineering design, weather modeling, astronomy, and many other mission critical applications, " said Bill Thipgen, Columbia project manager at Ames Research Center, Mountain View, Calif. "The high performance InfiniBand interconnect is a key component of the system, facilitating highly efficient communication for the second most powerful supercomputer in the world.

(Top)

Sandia National Laboratoires

53TFlops, 84.66% Linpack efficiency
4500 Dell PowerEdge 1850 1U Servers
9000 Dual Intel Xeon EM64T 3.6GHz processors
Lustre storage ~120 TB capacity and 6.0 GB/s BW

"A key metric to HPC application scalability is the bandwidth balance between the system processor, memory, PCI Express, and the cluster network," said Matt Leininger, computational scientist at Sandia National Laboratories, and technical representative for the DoE OpenIB PathForward. "The recent introduction of dual-core processors places additional data contention on the servers I/O system. The introduction of DDR solutions ensures that InfiniBand will remove the network bottleneck in these next generation servers."

“Sandia has been a leader in putting InfiniBand on the high-performance computing map,” said Ken Washington, CIO and director of Sandia’s Information Systems and Services Program. “It is only natural that we be the place where such a large InfiniBand cluster is first realized for meeting an institutional computing requirement.”(See Related Documents)

(Top)

Texas Advanced Computing Center/Univ. of Texas

41.4TFlops
1300 Dell PowerEdge 1955 servers with 5200 Intel Xeon 5100 CPU cores

The TACC Lonestar cluster is one of the largest academic computational resources in the nation. It serves as a computational resource in the NSF TeraGrid partnership, the Texas-wide Computational Grid (HiPCAT), the UT campus grid and the UT research community.

(Top)

Tokyo Institute of Technology

47.3TFlops
Sun Fire x64 servers with 10,480 Opteron processor cores
1300 Mellanox InfiniBand DDR MemFree HCAs
Native InfiniBand storage with 1 petabyte of hard disk storage

The Tokyo Tech TSUBAME is the fastest supercomputer outside of the United States as measured by sustained Linpack performance as of May 2006. TSUBAME is also the largest supercomputer outside the United States and includes more than 21 terabytes of memory and 1.1 petabyte of hard disk storage.

"This is truly an unprecedented achievement, given that TSUBAME was installed in just three weeks, and it has only been a month since the it went into production," said Satoshi Matsuoka, professor in charge of Research Infrastructure at Global Scientific Information and Computing Center, Tokyo Institute of Technology. "And we have only begun to understand the true capability of TSUBAME. We expect further advances in performance with utilization of acceleration, as well as to exploit the other superior aspects of the system, such as very large memory, fat node shared memory architecture with fast interconnect, ultra fast and reliable storage, and most importantly, x86 compatibility with desktop environments."

Partial list of applications:

ABAQUS (Standard and Explicit)
ABAQUS/CAE
MSC/Nastran, MSC/Patran
Gaussian 03 (+Linda), GaussView
Molpro
MOPAC2002
AMBER 8
Materials Explorer, Materials Studio, Discovery Studio Modeling
AVS/Express
Ensight
SAS
MATLAB
Mathematica

(Top)

University of Nebraska

View Case Study: Building One of the World's Largest Windows-Based HPC Clusters

When the University of Nebraska at Omaha’s Holland Computing Center teamed up with Dell to build an InfiniBand-based high-performance computing (HPC) cluster, the result was a flexible dual-boot system of Dell™ PowerEdge™ servers with dual-core AMD Opteron™ processors and Mellanox InfiniBand interconnect —and one of the world’s largest and most powerful Microsoft® Windows® OS–based clusters.

(Top)

Virginia Tech

View Article: How Virginia Tech built a supercomputer with a shoestring budget

View Case Study: Virginia Tech Taps 40Gb/s InfiniBand for Research Cluster

12.2TFlops
1100 Dual 2.3 GHz Apple XServe servers

System X is an important academic tool, available for 'big science' research projects.
A partial list of community codes known to run on System X:

AMBER - molecular dynamics
ARPREC - high-precision numerical methods
ARPS - weather modeling
CHARMM - molecular dynamics
FASTEST- fluid dynamics
GAMESS - quantum chemistry
Global Arrays - shared memory programming interface
LAMMPS - molecular dynamics
METIS/ParMETIS - sparse matrix suite
NWChem - molecular dynamics
PETSc - partial differential equation suite
ScaLAPACK - dense and band matrix software
WRF - weather modeling

(Top)