> Von: [email protected] [[email protected]]" im Auftrag von > "René Koch [[email protected]] > Gesendet: Freitag, 30. Mai 2014 21:08 > An: ml ml > Cc: [email protected] > Betreff: Re: [ovirt-users] oVirt Infiniband Support - Etherrnet vs Infiniband > > On 05/30/2014 04:55 PM, ml ml wrote: > > Hello List, > > > > i am very new to ovirt. I am setting up a 2 Node Cluster with GlusterFS > > Sync Replication. > > > > I was about to buy a 2-Port Intel Gbit nic for each host which is about > > 150Eur each. > > > > However, for a little extra of 180Eur you get a 10GBit Infiniband - > > MCX311A-XCAT. > > > > I would like to have quick opinions on this. > > > > Should i pay a little more and then get 10MBit instead of 2GBit bonded? > > Infiniband is good for GlusterFS / storage connection but never never > never never ever use it as a vm network - believe me it's a nightmare. > > So you would direct connect your 2 nodes as otherwise you would have to > buy an Infiniband switch as well (btw I'm not sure if direct connect is > possible, but I guess it is). Keep in mind that Infiniband cables aren't > cheap, too. > Why don't you go for 10Gbit/s Ethernet?
Welcome to Ovirt. We are on infiniband IPoIB for about 3 years now (started with VMWare). So we jumped on the 10GBit path long before 10GB ethernet had reasonable prices. We are fine with it but you should consider the following aspects. - Go for ConnectX-2 cards or higher. They allow for IPoIB TCP offloading. Do not believe the other information you will find on the internet. A helful statment here: http://permalink.gmane.org/gmane.linux.drivers.rdma/18032 - With IPoIB infiniband you should always be happy to achieve more than 1GBit throughput. A hear a lot of people complaining that they do not get wire speed. The stack is quite complex and from our experience we can reach 5-7GBit speeds easily on a DDR backbone (16 Gbps payload). Better CPUs will increase the throughput. A jump from Xeon 5400 to 5600 gave a nice boost. - Everything around NFS over RDMA is currently a mess. - Cable prices are a real pain (at least in Europe). - Older CX4 Switches are very cheap and rock solid. We have a CISCO 7000D. Though limited to 2044MTU they are fine. - Infiniband requires a subnet manager on the network. Cards will only get a link if that process is available. Either you need a switch with an builtin SM or in case of direct connection it must run on one of the hosts. - always remember that GBit bonding will not improve the throughput of one connection. So you can easily migrate a single VM or disks over infiniband with much more than 1GBit while an ethernet bond will be limited to 1Gbit. Maybe you should have a look at the 10GBit mellanox ethernet adapters. They are floating around for cheap prices ~$60 have a SFP+ port and are build from the same silicone as the infiniband cards. For direct connections they should be fine but switching hardware is expensive. Markus
**************************************************************************** Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. Ãber das Internet versandte E-Mails können unter fremden Namen erstellt oder manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine rechtsverbindliche Willenserklärung. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln Vorstand: Kadir Akin Dr. Michael Höhnerbach Vorsitzender des Aufsichtsrates: Hans Kristian Langva Registergericht: Amtsgericht Köln Registernummer: HRB 52 497 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. e-mails sent over the internet may have been written under a wrong name or been manipulated. That is why this message sent as an e-mail is not a legally binding declaration of intention. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln executive board: Kadir Akin Dr. Michael Höhnerbach President of the supervisory board: Hans Kristian Langva Registry office: district court Cologne Register number: HRB 52 497 ****************************************************************************
_______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
