Re: [ewg] http://git.openfabrics.org
Andy Grover wrote: > Not working for me today. > > Hmm, nor http://openfabrics.org ? I can still ssh to sofa... Fixed! Thanks! -- Andy ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] http://git.openfabrics.org
Not working for me today. Hmm, nor http://openfabrics.org ? I can still ssh to sofa... Thanks -- Andy ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] OFED-RDMAoE-1.5 GA is available
Hi, I am pleased to announce that OFED-RDMAoE-1.5 GA release is done Notes: The tarball is available on: http://www.openfabrics.org/downloads/OFED/ofed-rdmaoe-1.5/OFED-RDMAoE-1.5.tgz To get BUILD_ID run ofed_info Please report any issues in bugzilla https://bugs.openfabrics.org/ for OFED-RDMAoE 1.5 Eli Release information: Linux Operating Systems: - RedHat EL4 up72.6.9-78.ELsmp - RedHat EL4 up82.6.9-89.ELsmp - RedHat EL5 up32.6.18-128.el5 - RedHat EL5 up42.6.18-164.el5 - SLES10 SP22.6.16.60-0.21-smp - SLES10 SP32.6.16.60-0.54-smp - SLES112.6.27.19-5-default - OEL 4 up7 2.6.9-78.ELsmp - OEL 4 up8 2.6.9-89.ELsmp - CentOS5.3 2.6.18-128.el5 - CentOS5.4 2.6.18-164.el5 - Fedora Core12 2.6.31.5-127.fc12* - OpenSuSE 11.2 2.6.31.5-0.1-default * - kernel.org2.6.29, 2.6.30, 2.6.31 and 2.6.32* * Minimal QA for these versions Systems: * x86_64 * x86 * ia64 * ppc64 Main Changes from OFED-1.5 Added RDMAoE support. See attached readme file for usage information. === OFED-1.5 with RDMAoE Support README for OFED-RDMAOE-1.5 January 2010 === Contents: = 1. Overview 2. Software Dependencies 3. User Guidelines 4. Ported Applications 5. Firmware Requirements 6. Known Issues 1. Overview === RDMAoE allows InfiniBand (IB) transport over Ethernet networks. It encapsulates IB transport and GRH headers in Ethernet packets bearing a dedicated ether type. While the use of GRH is optional within IB subnets, it is mandatory when using RDMAoE. Verbs applications written over IB verbs should work seamlessly, but they require provisioning of GRH information when creating address vectors. The library and driver are modified to provide for mapping from GID to MAC addresses required by the hardware. 2. Software Dependencies In order to use RDMAoE over Mellanox ConnectX(R) hardware, the mlx4_en driver must be loaded. Please refer to MLNX_EN_README.txt for further details. 3. User Guidelines == Since RDMAoE encapsulates InfiniBand traffic in Ethernet frames, the corresponding net device must be up and running. In case of Mellanox hardware, mlx4_en must be loaded and the corresponding interface configured. - Make sure mlx4_en.ko is loaded - Make sure an IP address has been configured to this interface - Run "ibv_devinfo". There is a new field named "link_layer" which can be either "Ethernet" or "IB". If the value is IB, then you need to use connectx_port_config to change the ConnectX ports designation to eth (see mlx4_release_notes.txt for details) - Configure the IP address of the interface so that the link will become active - All IB verbs applications which run over IB verbs should work on RDMAoE links as long as they use GRH headers (that is, as long as they specify use of GRH in their address vector) 4. Ported Applications == - ibv_*_pingpong examples have been ported too. The user must specify the GID of the remote peer using the new '-g' option. The GID has the same format as that in /sys/class/infiniband/mlx4_0/ports/1/gids/0 - Note: Care should be taken when using ibv_ud_pingpong. The default message size is 2K, which is likely to exceed the MTU of the RDMAoE link. Use ibv_devinfo to inspect the link MTU and specify an appropriate message size - All rdma_cm applications should work seamlessly without any change - libsdp works without any change - Performance tests have been ported 5. Firmware Requirements RDMAoE requires ConnectX firmware version 2.7.000 or newer. Some features require newer, not yet released firmware versions. For example, loopback support is available in firmware versions 2.7.624 and later. 6. Known Issues === - PowerPC and ia64 architectures are not supported. x32 architectures were not tested. - SRP is not supported. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
Superb, that worked fine. Many thanks everyone, time to go play with Infiniband :) On Tue, Jan 12, 2010 at 3:33 PM, Sasha Khapyorsky wrote: > On 15:12 Tue 12 Jan , Ross Smith wrote: >> Sorry, I'm very much a linux newbie. Could you explain what I need to >> do to add that to ld.so.conf. It has just one line at the moment, do >> I just need to add another that says "include /usr/local/lib"? >> And then do I just run ldconfig? > > Assuming that you are with Ubuntu-9.10: > > sudo sh -c "echo /usr/local/lib >> /etc/ld.so.conf.d/local.conf" > sudo ldconfig > >> Will I need to recompile the >> programs, or should opensm just run then? > > No need to recompile, just run it. > > Sasha > ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
On 15:12 Tue 12 Jan , Ross Smith wrote: > Sorry, I'm very much a linux newbie. Could you explain what I need to > do to add that to ld.so.conf. It has just one line at the moment, do > I just need to add another that says "include /usr/local/lib"? > And then do I just run ldconfig? Assuming that you are with Ubuntu-9.10: sudo sh -c "echo /usr/local/lib >> /etc/ld.so.conf.d/local.conf" sudo ldconfig > Will I need to recompile the > programs, or should opensm just run then? No need to recompile, just run it. Sasha ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
Sorry, I'm very much a linux newbie. Could you explain what I need to do to add that to ld.so.conf. It has just one line at the moment, do I just need to add another that says "include /usr/local/lib"? And then do I just run ldconfig? Will I need to recompile the programs, or should opensm just run then? thanks, Ross On Tue, Jan 12, 2010 at 3:11 PM, Sasha Khapyorsky wrote: > On 14:48 Tue 12 Jan , Ross Smith wrote: >> >> However, it looks like I don't need OFED, and I've today found the >> opensm packages on the openfabrics.org website, and have downloaded: >> libibumad-1.3.4.tar.gz >> libibmad-1.3.4.tar.gz >> opensm-3.3.5.tar.gz >> infiniband-diags-1.5.5.tar.gz >> >> I followed the instructions in the README file to install them in >> order. However while everything has compiled, and appears to be >> installed ok, I still cannot run opensm: >> >> # opensm >> opensm: error while loading shared libraries: libosmvendor.so.3: >> cannot open shared object file: No such file or directory >> >> Does anybody have any ideas as to what I might have done wrong? > > By default the libraries are installed in /usr/local/lib. Assuming so > you likely need to add this to your ld.so.conf paths and rerun ldconfig. > > Sasha > ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
On 14:48 Tue 12 Jan , Ross Smith wrote: > > However, it looks like I don't need OFED, and I've today found the > opensm packages on the openfabrics.org website, and have downloaded: > libibumad-1.3.4.tar.gz > libibmad-1.3.4.tar.gz > opensm-3.3.5.tar.gz > infiniband-diags-1.5.5.tar.gz > > I followed the instructions in the README file to install them in > order. However while everything has compiled, and appears to be > installed ok, I still cannot run opensm: > > # opensm > opensm: error while loading shared libraries: libosmvendor.so.3: > cannot open shared object file: No such file or directory > > Does anybody have any ideas as to what I might have done wrong? By default the libraries are installed in /usr/local/lib. Assuming so you likely need to add this to your ld.so.conf paths and rerun ldconfig. Sasha ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
Hi Tziporet, Yes, I'd tried with --without-depcheck option but to no avail. However, it looks like I don't need OFED, and I've today found the opensm packages on the openfabrics.org website, and have downloaded: libibumad-1.3.4.tar.gz libibmad-1.3.4.tar.gz opensm-3.3.5.tar.gz infiniband-diags-1.5.5.tar.gz I followed the instructions in the README file to install them in order. However while everything has compiled, and appears to be installed ok, I still cannot run opensm: # opensm opensm: error while loading shared libraries: libosmvendor.so.3: cannot open shared object file: No such file or directory Does anybody have any ideas as to what I might have done wrong? thanks, Ross On Tue, Jan 12, 2010 at 11:55 AM, Tziporet Koren wrote: > On 1/11/2010 8:21 PM, Ross Smith wrote: >> >> Actually, I was just trying to run opensm on it. I couldn't find any >> downloads for that, but I knew it was included in OFED. I already >> have IPoIB working fine in Ubuntu. >> >> >> On Mon, Jan 11, 2010 at 5:04 PM, Roland Dreier wrote: >> >>> >>> > I'm trying to install OFED on Ubuntu 9.10 without much success. OFED >>> > 1.5 is failing with an error: >>> >>> > > From OFED RN: > 5. When installing OFED on OpenSuse or Ubuntu one should use the > --without-depcheck option of the install.pl script > > Tziporet > > > ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] EWG/OFED meeting minutes for Jan 11, 2010
EWG/OFED the meeting minutes for Dec 23, 2009 Meeting summary: 1. Started to work on OFED 1.5.1. 2. OFED 1.5.1 GA is expected on March 3, 2010 (RC1 on Feb 2) 3. OFED 1.6 features - will be closed at Sonoma 4. Sonoma workshop - everyone with an idea for a session or feedback on the suggested agenda is welcome to approach Bill Boas Meeting details: 1. OFED 1.5.1 status: These are the changes that were done to the source since OFED 1.5 release: Kernel : - rdma/cm: Fixup IPv6 support and IPv4 routing corner cases for RDMA CM - RDS/TCP: Wait to wake thread when write space available - RDS: use IB_CQ_VECTOR_LEAST_ATTACHED for cq's - RDS: update copy_to_user state in tcp transport - sdp: use IB_CQ_VECTOR_LEAST_ATTACHED for cq's - sdp: make statistics per cpu and added statistics instead of prints - Add support for RoCEE - as submitted to the kernel last week. Userspace: - libmlx4, libibverbs - Add RoCEE support Other changes that should be done: - Open MPI update to rev. 1.4.1 - MVAPICH release update to 1.2 - Fix NFS-RDMA bugs and bring it to GA - Bug fixes - More OSes support - be prepared for RHEL 5.5 with back-ports - Ask Doug Ledford if kernel is stable enough to start. - Updated uDAPL package - Support SDP on PPC64 - iSER support for distro Still open: - uMMU notifier kernel module - If it will be added to OFED then Open MPI can add the code to support it. Need to check resources. - SW implementation of RoCEE - need to check with SFW 2. OFED 1.5.1 Schedule: I assume only 3 RCs: - RC1 - Feb 2, 2010 - RC2 - Feb 16, 2010 - RC3 - Feb 24, 2010 - GA - Mar 3, 2010 Reasonable, maybe RHEL 5.5 kernel will take more work. 3. Update on OFED-RDMAoE branch: The code is aligned with OFED 1.5 and the GA is expected tomorrow 4. OFED 1.6: Open discussion on features, base kernel, main features, ... Suggestion from Bill: - First datacenter focused release - Woody suggested to get a written list and have a session in Sonoma on this. - Resolve Scalability issues Other features (from SC09 OFA BOF): - SRIOV - Mellanox vnic - uMMU module (or any other kernel implementation) - Virtualization 5. Sonoma schedule: Everybody should send input and session suggestions for Sonoma to Bill Boas. Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Is this the right place for questions on OFED?
On 1/11/2010 8:21 PM, Ross Smith wrote: > Actually, I was just trying to run opensm on it. I couldn't find any > downloads for that, but I knew it was included in OFED. I already > have IPoIB working fine in Ubuntu. > > > On Mon, Jan 11, 2010 at 5:04 PM, Roland Dreier wrote: > >> > I'm trying to install OFED on Ubuntu 9.10 without much success. OFED >> > 1.5 is failing with an error: >> >> From OFED RN: 5. When installing OFED on OpenSuse or Ubuntu one should use the --without-depcheck option of the install.pl script Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] ofa_1_5_kernel 20100112-0200 daily build status
This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_5/linux-2.6.git git_branch: ofed_kernel_1_5 Common build parameters: Passed: Failed: Build failed on i686 with linux-2.6.21.1 Build failed on i686 with linux-2.6.18 Build failed on i686 with linux-2.6.19 Build failed on i686 with linux-2.6.26 Build failed on i686 with linux-2.6.22 Build failed on i686 with linux-2.6.24 Build failed on i686 with linux-2.6.27 Build failed on x86_64 with linux-2.6.16.60-0.21-smp Log: /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/include/rdma/ib_addr.h:200: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/include/rdma/ib_addr.h:201: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/include/rdma/ib_addr.h:198: warning: unused variable 'cmd' make[4]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.21-smp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.16.60-0.21-smp' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.16.60-0.54.5-smp Log: /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/include/rdma/ib_addr.h:200: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/include/rdma/ib_addr.h:201: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/include/rdma/ib_addr.h:198: warning: unused variable 'cmd' make[4]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.16.60-0.54.5-smp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.16.60-0.54.5-smp' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/include/rdma/ib_addr.h:200: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/include/rdma/ib_addr.h:201: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/include/rdma/ib_addr.h:198: warning: unused variable 'cmd' make[4]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.18-128.el5 Log: /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/include/rdma/ib_addr.h:200: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/include/rdma/ib_addr.h:201: error: dereferencing pointer to incomplete type /home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/include/rdma/ib_addr.h:198: warning: unused variable 'cmd' make[4]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_5_kernel-20100112-0200_linux-2.6.18-128.el5_x86_64_check/drivers/infiniband] Error 2 make[1
Re: [ewg] EWG/OFED meeting agenda for today (Jan 11, 2010)
Brian J. Murrell wrote: > On Mon, 2010-01-11 at 18:26 +0200, Tziporet Koren wrote: >> This is the agenda for the meeting today: >> >> 1. OFED 1.5.1 status: > ... >> Other changes that should be done: > ... >> - Other - let discuss in the meeting today > > How about the ISER/iSCSI on kernels < 2.6.30 situation? We'll probably release a backport for RHEL5.4 and SLES11 in OFED-1.5.1 If RHEL5.5 is out by than, it will be probably supported. I'll send patches soon > > I'm not sure I can make the call, but regardless, it would be nice to > have it discussed and a status included in the meeting minutes. > > b. > > > > > > ___ > ewg mailing list > ewg@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] EWG/OFED meeting agenda for today (Jan 11, 2010)
Brian J. Murrell wrote: > How about the ISER/iSCSI on kernels < 2.6.30 situation? Moni (CC-ed here) will send update on this later today Or. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
