Mellanox OFED for Linux Version OFED-1.3.1 June 2008 =============================================================================== Table of Contents =============================================================================== 1. Overview 2. Contents of Mellanox OFED for Linux Rev OFED-1.3.1 3. Changes and Bug Fixes Since OFED-1.3-mlnx1 4. Supported Platforms and Operating Systems 5. Supported HCA Adapter Cards and Firmware Versions 6. Tested Switch Platforms 7. Third Party Test Packages 8. Known Issues =============================================================================== 1. Overview =============================================================================== These are the release notes of Mellanox OFED for Linux, version 1.3.1. This software package is a subset of the Open Fabrics Enterprise Distribution (OFED) version 1.3.1 (see http://www.openfabrics.org). Mellanox OFED for Linux is composed of several software modules and is intended for use on a computer cluster constructed as an InfiniBand subnet. Mellanox OFED for Linux is presented as ISO images, one for each supported operating system. See Section 3 below. Note: If you plan to upgrade the OFED package on your cluster, please upgrade all of its nodes to this new version. =============================================================================== 2. Contents of Mellanox OFED for Linux =============================================================================== Mellanox OFED for Linux software contains the following components: - OpenFabrics core and ULPs: - IB HCA drivers (mthca, mlx4) - core - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, RDS - OpenFabrics utilities: - OpenSM (OSM): InfiniBand Subnet Manager - Diagnostic tools - Performance tests - MPI: - OSU MPI (MVAPICH 1.0) stack supporting the InfiniBand interface - Open MPI stack supporting the InfiniBand interface - MPI benchmark tests (OSU benchmarks, Intel MPI benchmarks, Presta) - Extra packages: - ib-bonding: Bonding driver for IPoIB interface - Sources of all software modules (under conditions mentioned in the modules' LICENSE files) - Documentation Notes: 1. All Mellanox OFED components are of production quality. 2. See release notes for each package in the docs directory. =============================================================================== 3. Changes and Bug Fixes Since OFED-1.3-mlnx1 =============================================================================== Note: For details regarding the various fixed bugs, please see the release notes for each component in the docs directory. * New OS support: - SLES 10 SP2 - RHEL 5 up2 * Changes in install scripts: - There is no default configuration for IPoIB interfaces. You need to manually specify the full IP configuration or use the ofed_net.conf file. - A new option was added to the ofed_uninstall.sh script: --unload-modules. This option executes 'openibd stop' before removing the OFED RPMs. * Updated Libraries: - librdmacm 1.0.7 * Kernel changes: - IPoIB bug fixes - SRP bug fixes - RDS bug fixes - Bonding fixes and support for SM failover - Core: SA module fix * Updated utilities: - opensm version 3.1.11 - mstflint - ibutils * Updated low level drivers: - mlx4 (Mellanox) * Updated MPI packages: - MVAPICH 1.0.1 - Open MPI 1.2.6 =============================================================================== 4. Supported Platforms and Operating Systems =============================================================================== 1. RHEL4 (MLNX_OFED_LINUX-1.3.1-rhel4.iso) i686: 2.6.9-42.ELsmp (RHEL4 U4) 2.6.9-55.ELsmp (RHEL4 U5) 2.6.9-55.0.12.Elsmp (RHEL4 U5) 2.6.9-67.ELsmp (RHEL4 U6) x86_64: 2.6.9-42.ELsmp (RHEL4 U4) 2.6.9-55.ELsmp (RHEL4 U5) 2.6.9-55.0.12.Elsmp (RHEL4 U5) 2.6.9-67.ELsmp (RHEL4 U6) ppc64: 2.6.9-55.EL (RHEL4 U5) 2.6.9-67.0.1.EL (RHEL4 U6) 2. RHEL5 (MLNX_OFED_LINUX-1.3.1-rhel5.iso): i686: 2.6.18-8.el5 (RHEL5.0) 2.6.18-53.el5 (RHEL5.1) 2.6.18-92.el5 (RHEL5.2) 2.6.18-92.el5PAE (RHEL5.2) ppc64: 2.6.18-53.el5 (RHEL5.1) x86_64: 2.6.18-8.el5 (RHEL5.0) 2.6.18-53.el5 (RHEL5.1) 2.6.18-92.el5 (RHEL5.2) ia64: 2.6.18-53.el5 (RHEL5.1) 3. SLES10 (MLNX_OFED_LINUX-1.3.1-sles10.iso): i686: 2.6.16.21-0.8_smp (SLES10) x86_64: 2.6.16.21-0.8-smp (SLES10) ia64: 2.6.16.21-0.8-smp (SLES10) 4. SLES10 SP1 & SP2 (MLNX_OFED_LINUX-1.3.1-sles10_sp1_sp2.iso): i686: 2.6.16.46-0.12-smp (SLES10SP1) 2.6.16.60-0.21-smp (SLES10SP2) 2.6.16.60-0.21-bigsmp (SLES10SP2) ppc64: 2.6.16.53-0.16-ppc64 (SLES10SP1) x86_64: 2.6.16.46-0.12-smp (SLES10SP1) 2.6.16.60-0.21-smp (SLES10SP2) NOTE: If you wish install OFED on a different kernel, you need to open the the OFED tarball (under src/) and build the package as explained in the file OFED_Installation_Guide.txt (under the docs/ folder of the opened OFED tarball). Also check the OFED_release_notes.txt for the list of supported operating systems and kernels. =============================================================================== 5. HCAs Supported =============================================================================== Mellanox supports the following IB HCAs with OFED 1.3.1: o Mellanox Technologies HCAs (SDR and DDR Modes are Supported): - InfiniHost (fw-23108 Rev 3.5.000) - InfiniHost III Ex (MemFree: fw-25218 Rev 5.3.000 with memory: fw-25208 Rev 4.8.200) - InfiniHost III Lx (fw-25204 Rev 1.2.000) - ConnectX IB (fw-25408 Rev 2.3.000) For official firmware versions please see: http://www.mellanox.com/support/firmware_table.php =============================================================================== 6. Switches Supported =============================================================================== This release was tested with switches and gateways provided by the following companies: - Cisco - Flextronics - Qlogic - Sun - Voltaire =============================================================================== 7. Third Party MPI Packages =============================================================================== The following third party package has been tested with this release of Mellanox OFED: HP-MPI 2.2.5.1 (without XRC support) =============================================================================== 8. Known Issues =============================================================================== The following is a list of general limitations and known issues of the various components of this Mellanox OFED for Linux release. 1. When upgrading from an earlier Mellanox OFED version, the installation script does not stop the earlier version prior to uninstalling it. Workaround: Stop the old OFED stack (/etc/init.d/openibd stop) before upgrading to this new version. 2. Fork support from kernel 2.6.12 and above is available provided that applications do not use threads. The fork() is supported as long as the parent process does not run before the child exits or calls exec(). The former can be achieved by calling wait(childpid), and the latter can be achieved by application specific means. The Posix system() call is supported. 3. mstdump tool: Running mstdump on a ConnectX HCA may cause the HCA to hang. Workaround: Run the mstregdump tool instead. Both tools have the same output. The mstregdump tool requires the PCI Device ID of the HCA in its command line. You can obtain the PCI Device ID by using the lspci command. Example: > lspci | grep Mellanox 04:00.0 InfiniBand: Mellanox Technologies Unknown device 634a (rev a0) > mstregdump 04:00.0 > /tmp/mstregdump.txt Note: See the release notes of each component for additional issues.