[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-03-02 Thread Vinícius Ferrão
Interesting.

I have a lot of stability issues with IPoIB in EL7. In Windows it’s works 
flawlessly, but on Linux is just unstable. So I never had the courage to put an 
hypervisor Storage network over IPoIB.

Are you using Mellanox OFED, or the upstream one?

The iSCSI is only through software? No RDMA involved? Aren’t the CPU being 
wasted in this scenario since Connect-X3 does not offload TCP/IP connections?

Finally the IPoIB mode is connected or dataram? Have you tunned the MTU 
settings?

Thanks,


Sent from my iPhone

> On 1 Mar 2019, at 21:36, Mike Blanchard  wrote:
> 
> Hi Vinicius, i'm sorry, I thought I had replied already.  
> 
> I'm mostly using IPoIB, but gluster is using some over RDMA.  I'm using it a 
> few different ways,
> 5 IB nodes, all have dual port ConnectX-3 cards @ 40GB
> 
> 2 nodes are running windows, one port on each is used for a sync channel for 
> starwind Vsan to mirror ~20tb of date.  The second ports are using for iSCS 
> sharing SMB fiile shares for other IB nodes.
> 3 nides are running CentOS 7 and ovirt.  One of the ports on each are used 
> for gluster synchronzaton, gluster is set to use both IPoIB & RDMA.  Then the 
> other ports are using for migratons, iSCSI sharing, and a backup admin 
> channel.
> 
>> On Fri, Mar 1, 2019 at 1:36 PM Vinícius Ferrão  
>> wrote:
>> Darkytoo have you received my message.
>> 
>> I’m really curious about your setup using Infiniband.
>> 
>> Thanks,
>> 
>> > On 14 Feb 2019, at 07:42, Vinícius Ferrão  
>> > wrote:
>> > 
>> > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or 
>> > Ethernet over IB?
>> > 
>> > If using NFS on top of it, RDMA is working
>> > with NFSoRDMA?
>> > 
>> > I’m really curious about your setup.
>> > 
>> > Thanks,
>> > 
>> > Sent from my iPhone
>> > 
>> >> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote:
>> >> 
>> >> You were just like me, I also thought it would pick up settings from 
>> >> cockpit, but once I realized you had to configure the network again when 
>> >> adding to the host, everything sort of fell into line.  I would make sure 
>> >> and monitor the networks so you can see the traffic going through the 
>> >> infiniband and not the ethernet.
>> >> ___
>> >> Users mailing list -- users@ovirt.org
>> >> To unsubscribe send an email to users-le...@ovirt.org
>> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> >> oVirt Code of Conduct: 
>> >> https://www.ovirt.org/community/about/community-guidelines/
>> >> List Archives: 
>> >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/P35FN76GEAOXKZO4VFPUZZ75MD2FMBRL/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-03-01 Thread Mike Blanchard
Hi Vinicius, i'm sorry, I thought I had replied already.

I'm mostly using IPoIB, but gluster is using some over RDMA.  I'm using it
a few different ways,
5 IB nodes, all have dual port ConnectX-3 cards @ 40GB

2 nodes are running windows, one port on each is used for a sync channel
for starwind Vsan to mirror ~20tb of date.  The second ports are using for
iSCS sharing SMB fiile shares for other IB nodes.
3 nides are running CentOS 7 and ovirt.  One of the ports on each are used
for gluster synchronzaton, gluster is set to use both IPoIB & RDMA.  Then
the other ports are using for migratons, iSCSI sharing, and a backup admin
channel.

On Fri, Mar 1, 2019 at 1:36 PM Vinícius Ferrão 
wrote:

> Darkytoo have you received my message.
>
> I’m really curious about your setup using Infiniband.
>
> Thanks,
>
> > On 14 Feb 2019, at 07:42, Vinícius Ferrão 
> wrote:
> >
> > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode?
> Or Ethernet over IB?
> >
> > If using NFS on top of it, RDMA is working
> > with NFSoRDMA?
> >
> > I’m really curious about your setup.
> >
> > Thanks,
> >
> > Sent from my iPhone
> >
> >> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote:
> >>
> >> You were just like me, I also thought it would pick up settings from
> cockpit, but once I realized you had to configure the network again when
> adding to the host, everything sort of fell into line.  I would make sure
> and monitor the networks so you can see the traffic going through the
> infiniband and not the ethernet.
> >> ___
> >> Users mailing list -- users@ovirt.org
> >> To unsubscribe send an email to users-le...@ovirt.org
> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> >> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> >> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DEWEMDDSG5DIP6KOCFJ3MAJBUDIIRC2H/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-03-01 Thread Vinícius Ferrão
Darkytoo have you received my message.

I’m really curious about your setup using Infiniband.

Thanks,

> On 14 Feb 2019, at 07:42, Vinícius Ferrão  wrote:
> 
> Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or 
> Ethernet over IB?
> 
> If using NFS on top of it, RDMA is working
> with NFSoRDMA?
> 
> I’m really curious about your setup.
> 
> Thanks,
> 
> Sent from my iPhone
> 
>> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote:
>> 
>> You were just like me, I also thought it would pick up settings from 
>> cockpit, but once I realized you had to configure the network again when 
>> adding to the host, everything sort of fell into line.  I would make sure 
>> and monitor the networks so you can see the traffic going through the 
>> infiniband and not the ethernet.
>> ___
>> Users mailing list -- users@ovirt.org
>> To unsubscribe send an email to users-le...@ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct: 
>> https://www.ovirt.org/community/about/community-guidelines/
>> List Archives: 
>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/CZ3LJIHVRP3BBZMPGMOYIRMO5O5DERSI/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-02-14 Thread Vinícius Ferrão
Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or 
Ethernet over IB?

If using NFS on top of it, RDMA is working
with NFSoRDMA?

I’m really curious about your setup.

Thanks,

Sent from my iPhone

> On 10 Feb 2019, at 23:55, darky...@gmail.com wrote:
> 
> You were just like me, I also thought it would pick up settings from cockpit, 
> but once I realized you had to configure the network again when adding to the 
> host, everything sort of fell into line.  I would make sure and monitor the 
> networks so you can see the traffic going through the infiniband and not the 
> ethernet.
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: 
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives: 
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DVIIC5ZMV24SFHTUINHTP4P7UNCJRVKX/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-02-10 Thread darkytoo
You were just like me, I also thought it would pick up settings from cockpit, 
but once I realized you had to configure the network again when adding to the 
host, everything sort of fell into line.  I would make sure and monitor the 
networks so you can see the traffic going through the infiniband and not the 
ethernet.
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDENXVCTXW6F24742YKDE6ZHK/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-02-10 Thread cbop . mail
Hello,
now everything is working.

For some reason I had to reconfigure the NIC on my engine machine.

Roman
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/OEOKATQMSMRPP7M6DTZLOOWAQCW2CWW3/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-02-10 Thread cbop . mail
Hello, thanks for the answer!

I was missing that, I thought it would pick up the previously configured IP I 
set in cockpit before I added the host!

But I also found out the main issue: it was set as a VM network. But I don't 
know if that's meant to be like this or this is just a bug.

Now it's added to the host but I can't send traffic through it, not even a ping!

Have you some idea why it's still not working?

Roman
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PHBT64KGJSTYXVXJ6VSSUJEQZM2E32A3/


[ovirt-users] Re: Cannot assign Network to Mellanox Interface

2019-02-10 Thread darkytoo
Hello fellow Infiniband-er!  I have a 3 node cluster with Connectx-3 cards 
under CentosOS and it seems to be working well.  One thing I didn't realhotize 
is once I assigned it to the host, I had to edit it and assign the network 
manually, otherwise it left it has DHCP, it never got an address (for obvious 
reasons) and it stopped working.  When I originally set the servers up, I 
installed the infiniband support packages and I see them in cockpit before I 
Actually assigned them.  
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/Q3SDTA3VHHDLWOABLLZIZM2F3B7OXULL/