Hi Gianluca,
My answers below,
cordialement, regards,
Nelson LAMEIRAS
Ingénieur Systèmes et Réseaux / Systems and Networks engineer
Tel: +33 5 32 09 09 70
[email protected]
www.lyra-network.com | www.payzen.eu
Lyra Network, 109 rue de l'innovation, 31670 Labège, FRANCE
From: "Gianluca Cecchi" <[email protected]>
To: "Nelson Lameiras" <[email protected]>
Cc: "ovirt users" <[email protected]>
Sent: Thursday, June 8, 2017 9:36:33 AM
Subject: Re: [ovirt-users] ISCSI storage with multiple nics on same subnet
disabled on host activation
On Wed, Jun 7, 2017 at 2:59 PM, Nelson Lameiras <
[email protected] > wrote:
BQ_BEGIN
Hello,
In our oVirt hosts, we are using DELL equallogic SAN with each server
connecting to SAN via 2 physical interfaces. Since both interfaces share the
same network (Equalogic limitation) we must patch the linux kernel to to allow
iSCSI multipath with multiple NICs in the same subnet with sysctl :
--------------------------------------------------------------------------------------------
net.ipv4.conf.p2p1.arp_ignore=1
net.ipv4.conf.p2p1.arp_announce=2
net.ipv4.conf.p2p1.rp_filter=2
net.ipv4.conf.p2p2.arp_ignore=1
net.ipv4.conf.p2p2.arp_announce=2
net.ipv4.conf.p2p2.rp_filter=2
BQ_END
BQ_BEGIN
Hello Nelson,
nice to see another environment based on EQL iSCSI SAN.
The systctl.conf customized lines above are not what you mean with "we must
patch the linux kernel", is it?
What do you mean exactly?
BQ_END
Sorry, by "patch" I meant "sysctl.conf configuration" indeed.
This configuration is what we use to be able to configure 2 network interfaces
in the same subnet.
BQ_BEGIN
--------------------------------------------------------------------------------------------
BQ_END
BQ_BEGIN
This works great in most setups, but for a strange reason, on some of our
setups, the sysctl configuration is updated by VDSM when activating a host and
the second interface stops working immeadiatly :
--------------------------------------------------------------------------------------------
vdsm.log
2017-06-07 11:51:51,063+0200 INFO (jsonrpc/5) [storage.ISCSI] Setting strict
mode rp_filter for device 'p2p2'. (iscsi:602)
2017-06-07 11:51:51,064+0200 ERROR (jsonrpc/5) [storage.HSM] Could not connect
to storageServer (hsm:2392)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/hsm.py", line 2389, in connectStorageServer
conObj.connect()
File "/usr/share/vdsm/storage/storageServer.py", line 433, in connect
iscsi.addIscsiNode(self._iface, self._target, self._cred)
File "/usr/lib/python2.7/site-packages/vdsm/storage/iscsi.py", line 232, in
addIscsiNode
iscsiadm.node_login( iface.name , target.address, target.iqn)
File "/usr/lib/python2.7/site-packages/vdsm/storage/iscsiadm.py", line 337,
in node_login
raise IscsiNodeError(rc, out, err)
In my environment I don't see these kinds of problems.
I'm using plain CentOS 7.3 hosts as hypervisors; is this your case too or are
you using ovirt-ng nodes?
BQ_END
Plain CentOS 7.3 hosts also. Built from a Minimal centos and adding oVirt
repositories.
BQ_BEGIN
--------------------------------------------------------------------------------------------
"strict mode" is enforced for second interface, and it no longuer works...
Which means - at least - that there is no redundancy in case of hardware
faillure and this is not acceptable for our production needs.
What is really strange is that we have another "twin" site on another
geographic region with simillar hardware configuration and same oVirt
installation, and this problem does not happen.
What can be the root cause of this behaviour? How can I correct it?
BQ_END
BQ_BEGIN
Actually it is very strange to me to understand how you don't have problems on
the other side.
My workaround to get multipath with EQL in oVirt was the one explained here
with screenshots:
http://lists.ovirt.org/pipermail/users/2017-March/080992.html
It has been working now for about 3 months without problems. We have several
Oracle RDBMS server VMs on it, from 2virtual threads and 8Gb of ram to 16
threads (2-4-2) and 64Gb of ram
How did you configure multipath instead? As far as I know oVirt will not let
you do it.
The only way would be to use bonding but that is not certified/supported by
Dell.
HIH sharing experiences,
Gianluca
BQ_END
I've read the link you gave and it seems to me that you are using the same
physical interface for SAN and vm network.
In my setup :
- SAN network is using two disctinct physical interfaces (p2pX) on same subnet
(therefore the need for sysctl patch)
- VM network (p1pX) is using 2 other distinct physical interfaces on "trunk"
mode
- Administration network (emX) is using 2 distinct bonded network interfaces
Our setup, using a total of 6 network interfaces is "simpler" than yours since
each interface has a "disctinct" role.
To make multipath work with oVirt on this setup, we had to configure SAN
network on system before installing oVirt (using sysctl.conf above),
- next declare 2 SAN networks on oVirt that are "dragued" to each SAN interface
in host network configuration,
- and finally, in datacenter listing, chosing a datacenter and selecting "iSCSI
Multipathing" tab below I can add a multipath configuration concerning both SAN
oVirt networks and relevant iqn storage targets.
Hope it's clear, if not, please ask me.
It's very strange to me also since this configuration did work great on our
labs. But half of production sites have the problem decribed above : second SAN
interface stops working uppon host activation because vdsm forces sysctl
net.ipv4.conf.p2p2.rp_filter to 1 (strict mode).
Still searching for clues as why this behaviour...
_______________________________________________
Users mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/users
