[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Interesting. I have a lot of stability issues with IPoIB in EL7. In Windows it’s works flawlessly, but on Linux is just unstable. So I never had the courage to put an hypervisor Storage network over IPoIB. Are you using Mellanox OFED, or the upstream one? The iSCSI is only through software? No RDMA involved? Aren’t the CPU being wasted in this scenario since Connect-X3 does not offload TCP/IP connections? Finally the IPoIB mode is connected or dataram? Have you tunned the MTU settings? Thanks, Sent from my iPhone > On 1 Mar 2019, at 21:36, Mike Blanchard wrote: > > Hi Vinicius, i'm sorry, I thought I had replied already. > > I'm mostly using IPoIB, but gluster is using some over RDMA. I'm using it a > few different ways, > 5 IB nodes, all have dual port ConnectX-3 cards @ 40GB > > 2 nodes are running windows, one port on each is used for a sync channel for > starwind Vsan to mirror ~20tb of date. The second ports are using for iSCS > sharing SMB fiile shares for other IB nodes. > 3 nides are running CentOS 7 and ovirt. One of the ports on each are used > for gluster synchronzaton, gluster is set to use both IPoIB & RDMA. Then the > other ports are using for migratons, iSCSI sharing, and a backup admin > channel. > >> On Fri, Mar 1, 2019 at 1:36 PM Vinícius Ferrão >> wrote: >> Darkytoo have you received my message. >> >> I’m really curious about your setup using Infiniband. >> >> Thanks, >> >> > On 14 Feb 2019, at 07:42, Vinícius Ferrão >> > wrote: >> > >> > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or >> > Ethernet over IB? >> > >> > If using NFS on top of it, RDMA is working >> > with NFSoRDMA? >> > >> > I’m really curious about your setup. >> > >> > Thanks, >> > >> > Sent from my iPhone >> > >> >> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote: >> >> >> >> You were just like me, I also thought it would pick up settings from >> >> cockpit, but once I realized you had to configure the network again when >> >> adding to the host, everything sort of fell into line. I would make sure >> >> and monitor the networks so you can see the traffic going through the >> >> infiniband and not the ethernet. >> >> ___ >> >> Users mailing list -- users@ovirt.org >> >> To unsubscribe send an email to users-le...@ovirt.org >> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> >> oVirt Code of Conduct: >> >> https://www.ovirt.org/community/about/community-guidelines/ >> >> List Archives: >> >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/P35FN76GEAOXKZO4VFPUZZ75MD2FMBRL/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Hi Vinicius, i'm sorry, I thought I had replied already. I'm mostly using IPoIB, but gluster is using some over RDMA. I'm using it a few different ways, 5 IB nodes, all have dual port ConnectX-3 cards @ 40GB 2 nodes are running windows, one port on each is used for a sync channel for starwind Vsan to mirror ~20tb of date. The second ports are using for iSCS sharing SMB fiile shares for other IB nodes. 3 nides are running CentOS 7 and ovirt. One of the ports on each are used for gluster synchronzaton, gluster is set to use both IPoIB & RDMA. Then the other ports are using for migratons, iSCSI sharing, and a backup admin channel. On Fri, Mar 1, 2019 at 1:36 PM Vinícius Ferrão wrote: > Darkytoo have you received my message. > > I’m really curious about your setup using Infiniband. > > Thanks, > > > On 14 Feb 2019, at 07:42, Vinícius Ferrão > wrote: > > > > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? > Or Ethernet over IB? > > > > If using NFS on top of it, RDMA is working > > with NFSoRDMA? > > > > I’m really curious about your setup. > > > > Thanks, > > > > Sent from my iPhone > > > >> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote: > >> > >> You were just like me, I also thought it would pick up settings from > cockpit, but once I realized you had to configure the network again when > adding to the host, everything sort of fell into line. I would make sure > and monitor the networks so you can see the traffic going through the > infiniband and not the ethernet. > >> ___ > >> Users mailing list -- users@ovirt.org > >> To unsubscribe send an email to users-le...@ovirt.org > >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >> oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > >> List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/ > > ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DEWEMDDSG5DIP6KOCFJ3MAJBUDIIRC2H/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Darkytoo have you received my message. I’m really curious about your setup using Infiniband. Thanks, > On 14 Feb 2019, at 07:42, Vinícius Ferrão wrote: > > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or > Ethernet over IB? > > If using NFS on top of it, RDMA is working > with NFSoRDMA? > > I’m really curious about your setup. > > Thanks, > > Sent from my iPhone > >> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote: >> >> You were just like me, I also thought it would pick up settings from >> cockpit, but once I realized you had to configure the network again when >> adding to the host, everything sort of fell into line. I would make sure >> and monitor the networks so you can see the traffic going through the >> infiniband and not the ethernet. >> ___ >> Users mailing list -- users@ovirt.org >> To unsubscribe send an email to users-le...@ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CZ3LJIHVRP3BBZMPGMOYIRMO5O5DERSI/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or Ethernet over IB? If using NFS on top of it, RDMA is working with NFSoRDMA? I’m really curious about your setup. Thanks, Sent from my iPhone > On 10 Feb 2019, at 23:55, darky...@gmail.com wrote: > > You were just like me, I also thought it would pick up settings from cockpit, > but once I realized you had to configure the network again when adding to the > host, everything sort of fell into line. I would make sure and monitor the > networks so you can see the traffic going through the infiniband and not the > ethernet. > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DVIIC5ZMV24SFHTUINHTP4P7UNCJRVKX/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
You were just like me, I also thought it would pick up settings from cockpit, but once I realized you had to configure the network again when adding to the host, everything sort of fell into line. I would make sure and monitor the networks so you can see the traffic going through the infiniband and not the ethernet. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Hello, now everything is working. For some reason I had to reconfigure the NIC on my engine machine. Roman ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OEOKATQMSMRPP7M6DTZLOOWAQCW2CWW3/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Hello, thanks for the answer! I was missing that, I thought it would pick up the previously configured IP I set in cockpit before I added the host! But I also found out the main issue: it was set as a VM network. But I don't know if that's meant to be like this or this is just a bug. Now it's added to the host but I can't send traffic through it, not even a ping! Have you some idea why it's still not working? Roman ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PHBT64KGJSTYXVXJ6VSSUJEQZM2E32A3/
[ovirt-users] Re: Cannot assign Network to Mellanox Interface
Hello fellow Infiniband-er! I have a 3 node cluster with Connectx-3 cards under CentosOS and it seems to be working well. One thing I didn't realhotize is once I assigned it to the host, I had to edit it and assign the network manually, otherwise it left it has DHCP, it never got an address (for obvious reasons) and it stopped working. When I originally set the servers up, I installed the infiniband support packages and I see them in cockpit before I Actually assigned them. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Q3SDTA3VHHDLWOABLLZIZM2F3B7OXULL/
