Re: [Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04?
Hi, We are going to provide a Cinder patch to enable iSER in the next few days, it will allow users 5x faster bandwidth compared to using iSCSI TCP. The coming OFED 2.0 version (in couple of months) will allow Eth emulation over IPoIB (called eIPoIB). If you want early access to it you can contact me or openst...@mellanox.com. It will also include SR-IOV support allowing RDMA access and low-latency access from guest VMs. We are planning to publish a Mellanox Quantum plug-in soon that will allow simple integration with our InfiniBand and Ethernet hardware (and will have SR-IOV and NIC based IO Virtualization as part of it) I hope this helps. In case you need more details, please drop me an email. BR, Irena -Original Message- From: openstack-bounces+irenab=mellanox@lists.launchpad.net [mailto:openstack-bounces+irenab=mellanox@lists.launchpad.net] On Behalf Of Narayan Desai Sent: Tuesday, December 25, 2012 4:09 PM To: changliwei07 Cc: openstack Subject: Re: [Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04? We're using IB (QDR connectX and connectX2) on our system. It turns out that the drivers included in version 3.2 of the linux kernel are fine. I've built a ppa for updated management tools though; all of those bits are ancient in precise. The ppa is here: http://launchpad.net/~narayan-desai/+archive/infiniband One word of warning; you'll only be able to use IPOIB for connectivity between the hypervisor nodes and the openstack service nodes; getting to VMs requires full ethernet compatibility, which IPOIB doesn't provide. You will be able to use the IPOIB for fast image transfers and really fast volume storage if you have it. Turning on iSER helps a lot with volume bandwidth, but requires a patch to nova compute. -nld On Tue, Dec 25, 2012 at 7:03 AM, changliwe...@gmail.com wrote: Hi, Merry Christmas ! Have you even tried to use infinitband network in OpenStack base on Ubuntu 12.04? In recent two days, I want to use IPoIB in Ubuntu, but the big problem is the driver of IB, The OFED src is special for enterprise distribution only, such as (RHEL and SLES ), even the so called source code archive is a rpmbuild-tree-like structure. Then the question is how to solve the driver problem in ubuntu 12.04. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04?
Hi Irena, Will you provide drivers for 12.04 as well? And does it support SR-IOV? Thanks, -Vladimir Zadara Storage On Dec 29, 2012, at 10:41 PM, Irena Berezovsky ire...@mellanox.com wrote: Hi, We are going to provide a Cinder patch to enable iSER in the next few days, it will allow users 5x faster bandwidth compared to using iSCSI TCP. The coming OFED 2.0 version (in couple of months) will allow Eth emulation over IPoIB (called eIPoIB). If you want early access to it you can contact me or openst...@mellanox.com. It will also include SR-IOV support allowing RDMA access and low-latency access from guest VMs. We are planning to publish a Mellanox Quantum plug-in soon that will allow simple integration with our InfiniBand and Ethernet hardware (and will have SR-IOV and NIC based IO Virtualization as part of it) I hope this helps. In case you need more details, please drop me an email. BR, Irena -Original Message- From: openstack-bounces+irenab=mellanox@lists.launchpad.net [mailto:openstack-bounces+irenab=mellanox@lists.launchpad.net] On Behalf Of Narayan Desai Sent: Tuesday, December 25, 2012 4:09 PM To: changliwei07 Cc: openstack Subject: Re: [Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04? We're using IB (QDR connectX and connectX2) on our system. It turns out that the drivers included in version 3.2 of the linux kernel are fine. I've built a ppa for updated management tools though; all of those bits are ancient in precise. The ppa is here: http://launchpad.net/~narayan-desai/+archive/infiniband One word of warning; you'll only be able to use IPOIB for connectivity between the hypervisor nodes and the openstack service nodes; getting to VMs requires full ethernet compatibility, which IPOIB doesn't provide. You will be able to use the IPOIB for fast image transfers and really fast volume storage if you have it. Turning on iSER helps a lot with volume bandwidth, but requires a patch to nova compute. -nld On Tue, Dec 25, 2012 at 7:03 AM, changliwe...@gmail.com wrote: Hi, Merry Christmas ! Have you even tried to use infinitband network in OpenStack base on Ubuntu 12.04? In recent two days, I want to use IPoIB in Ubuntu, but the big problem is the driver of IB, The OFED src is special for enterprise distribution only, such as (RHEL and SLES ), even the so called source code archive is a rpmbuild-tree-like structure. Then the question is how to solve the driver problem in ubuntu 12.04. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04?
Hi, Merry Christmas ! Have you even tried to use infinitband network in OpenStack base on Ubuntu 12.04? In recent two days, I want to use IPoIB in Ubuntu, but the big problem is the driver of IB, The OFED src is special for enterprise distribution only, such as (RHEL and SLES ), even the so called source code archive is a rpmbuild-tree-like structure. Then the question is how to solve the driver problem in ubuntu 12.04.___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to Solve InfinitBand Adaptor Driver under Ubuntu 12.04?
We're using IB (QDR connectX and connectX2) on our system. It turns out that the drivers included in version 3.2 of the linux kernel are fine. I've built a ppa for updated management tools though; all of those bits are ancient in precise. The ppa is here: http://launchpad.net/~narayan-desai/+archive/infiniband One word of warning; you'll only be able to use IPOIB for connectivity between the hypervisor nodes and the openstack service nodes; getting to VMs requires full ethernet compatibility, which IPOIB doesn't provide. You will be able to use the IPOIB for fast image transfers and really fast volume storage if you have it. Turning on iSER helps a lot with volume bandwidth, but requires a patch to nova compute. -nld On Tue, Dec 25, 2012 at 7:03 AM, changliwe...@gmail.com wrote: Hi, Merry Christmas ! Have you even tried to use infinitband network in OpenStack base on Ubuntu 12.04? In recent two days, I want to use IPoIB in Ubuntu, but the big problem is the driver of IB, The OFED src is special for enterprise distribution only, such as (RHEL and SLES ), even the so called source code archive is a rpmbuild-tree-like structure. Then the question is how to solve the driver problem in ubuntu 12.04. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
