Re: [ovirt-users] Logical network error
- Original Message -
> From: "Dan Kenigsberg"
> To: "Maurice James" , from...@redhat.com
> Cc: "Moti Asayag" , "users" , "Antoni
> Segura Puimedon"
> Sent: Monday, July 21, 2014 6:49:56 PM
> Subject: Re: [ovirt-users] Logical network error
>
> On Mon, Jul 21, 2014 at 07:35:00AM -0400, Maurice James wrote:
> >
> > Here are the supervdsm logs
>
> Hmm, it's a trove of errors and tracebacks; there's a lot to debug, for
> example, here Vdsm was asked to create a 'migration' network on top of
> bond0 that was already used by ovirtmgmt. Engine should have blocked
> that. Moti?
Obviously should have been blocked from engine side, however there are couple
of issues with that migration network
which might contempt the engine caps:
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,677::configNetwork::250::root::(addNetwork) validating network...
MainProcess|Thread-43::INFO::2014-04-11
08:56:24,677::configNetwork::272::root::(addNetwork) Adding network migration
with vlan=5, bonding=bond0, nics=['em1', 'em2'], bondingOptions=mode=4
miimon=100, mtu=None, bridged=False,
defaultRoute=False,options={'implicitBonding': True}
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,679::ifcfg::447::root::(_persistentBackup) backing up ifcfg-bond0.5: #
original file did not exist
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,681::ifcfg::313::root::(writeBackupFile) Persistently backed up
/var/lib/vdsm/netconfback/ifcfg-bond0.5 (until next 'set safe config')
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,681::ifcfg::537::root::(writeConfFile) Writing to file
/etc/sysconfig/network-scripts/ifcfg-bond0.5 configuration:
# Generated by VDSM version 4.14.6-7.git41501b3.el6
DEVICE=bond0.5
ONBOOT=yes
VLAN=yes
DEFROUTE=no
NM_CONTROLLED=no
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,684::utils::642::root::(execCmd) '/sbin/ip route show to 0.0.0.0/0
table all' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,690::utils::662::root::(execCmd) SUCCESS: = ''; = 0
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,728::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.2'
(cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,735::utils::662::root::(execCmd) SUCCESS: = ''; = 0
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,741::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.4'
(cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,747::utils::662::root::(execCmd) SUCCESS: = ''; = 0
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,753::utils::642::root::(execCmd) '/sbin/ip -d link show dev bond0.3'
(cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,759::utils::662::root::(execCmd) SUCCESS: = ''; = 0
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,764::__init__::122::root::(_addSourceRoute) Adding source route
bond0.5, None, None, None
MainProcess|Thread-43::ERROR::2014-04-11
08:56:24,764::sourceRoute::68::root::(configure) ipaddr, mask or gateway not
received
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,764::utils::642::root::(execCmd) '/sbin/ifup bond0.5' (cwd None)
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,914::utils::662::root::(execCmd) SUCCESS: = ''; = 0
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,917::libvirtconnection::124::root::(wrapper) Unknown libvirterror:
ecode: 43 edom: 19 level: 2 message: Network not found: no network with
matching name 'vdsm-migration'
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,918::ifcfg::324::root::(_atomicNetworkBackup) Backed up migration
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,919::libvirtconnection::124::root::(wrapper) Unknown libvirterror:
ecode: 43 edom: 19 level: 2 message: Network not found: no network with
matching name 'vdsm-migration'
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,919::ifcfg::333::root::(_persistentNetworkBackup) backing up network
migration: # original file did not exist
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:24,921::ifcfg::313::root::(writeBackupFile) Persistently backed up
/var/lib/vdsm/netconfback/logicalnetworks/migration (until next 'set safe
config')
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:25,031::configNetwork::651::setupNetworks::(setupNetworks) Checking
connectivity...
MainProcess|Thread-43::DEBUG::2014-04-11
08:56:26,035::supervdsmServer::103::SuperVdsm.ServerCallback::(wrapper) return
setupNetworks with None
MainProcess|Thread-43::DEBUG::201
I wasn't able to find the output of getCapabilities for the host in the logs.
Without 'getCaps' it is hard to know which network configuration was reported
to the engine by vdsm.
But if 'vdsm-migration'
Re: [ovirt-users] Logical network error
On Mon, Jul 21, 2014 at 07:35:00AM -0400, Maurice James wrote:
>
> Here are the supervdsm logs
Hmm, it's a trove of errors and tracebacks; there's a lot to debug, for
example, here Vdsm was asked to create a 'migration' network on top of
bond0 that was already used by ovirtmgmt. Engine should have blocked
that. Moti?
MainProcess|Thread-47826::DEBUG::2014-04-11
10:00:26,335::configNetwork::589::setupNetworks::(setupNetworks) Setting up
network according to configuration: networks:{'migration': {'bonding': 'bond0',
'bridged': 'false'}}, bondings:{}, options:{'connectivityCheck': 'true',
'connectivityTimeout': 120}
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 98, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 202, in setupNetworks
return configNetwork.setupNetworks(networks, bondings, **options)
File "/usr/share/vdsm/configNetwork.py", line 648, in setupNetworks
implicitBonding=True, **d)
File "/usr/share/vdsm/configNetwork.py", line 186, in wrapped
return func(*args, **kwargs)
File "/usr/share/vdsm/configNetwork.py", line 256, in addNetwork
bridged)
File "/usr/share/vdsm/configNetwork.py", line 170, in
_validateInterNetworkCompatibility
_validateNoDirectNet(ifaces_bridgeless)
File "/usr/share/vdsm/configNetwork.py", line 154, in _validateNoDirectNet
(iface, iface_net))
ConfigNetworkError: (21, "interface 'bond0' already member of network
'ovirtmgmt'")
There are also repeated failed attempts to create a payload disk: did you
notice when they happen? Could it be related to insufficient disk space?
MainProcess|clientIFinit::ERROR::2014-03-25
22:13:02,529::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) Error
in mkFloppyFs
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 98, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 325, in mkFloppyFs
return mkimage.mkFloppyFs(vmId, files, volId)
File "/usr/share/vdsm/mkimage.py", line 104, in mkFloppyFs
"code %s, out %s\nerr %s" % (rc, out, err))
OSError: [Errno 5] could not create floppy file: code 1, out mkfs.msdos 3.0.9
(31 Jan 2010)
err mkfs.msdos: unable to create
/var/run/vdsm/payload/94632c2e-28a0-4304-9261-c302e0027604.ecac527e731a2a0057dc6a3ae3df9ba3.img
In any case, if you manage to reproduce these issues, please open a detailed
bug entry.
Regards,
Dan.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logical network error
On Fri, Jul 18, 2014 at 03:20:34PM -0400, Maurice James wrote: > Logs attached...more to follow That's an impressive set of logs - but I could not find supervdsm.log among them, the one with the Unexpected Exception. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logical network error
On Thu, Jul 17, 2014 at 10:54:34AM -0400, Maurice James wrote: > > I put the entire cluster into maintenance mode and reboot. I was then able to > make changes to the network Still, if you can trace back the bit of supervdsm.log with the traceback, we may be able to nail down a bug that may harm others. Thanks! ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logical network error
I put the entire cluster into maintenance mode and reboot. I was then able to make changes to the network - Original Message - From: "Moti Asayag" To: "Maurice James" Cc: "users" , "Antoni Segura Puimedon" , "Dan Kenigsberg" Sent: Thursday, July 17, 2014 5:20:34 AM Subject: Re: [ovirt-users] Logical network error - Original Message - > From: "Maurice James" > To: "users" > Sent: Wednesday, July 16, 2014 5:06:23 PM > Subject: [ovirt-users] Logical network error > > While attempting to remove a logical network from one of my hosts, Im getting > the following error. > Error while executing action Setup Networks: Unexpected exception > > > Im seeing the following error in the vdsm.log. > Could you attach the entire vdsm.log and supervdsm.log ? > > Thread-72::ERROR::2014-07-16 > 10:03:46,773::BindingXMLRPC::1086::vds::(wrapper) unexpected error > Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 1070, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 494, in setupNetworks > return api.setupNetworks(networks, bondings, options) > File "/usr/share/vdsm/API.py", line 1297, in setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, options) > File "/usr/share/vdsm/supervdsm.py", line 50, in __call__ > return callMethod() > File "/usr/share/vdsm/supervdsm.py", line 48, in > **kwargs) > File "", line 2, in setupNetworks > File "/usr/lib64/python2.6/multiprocessing/managers.py", line 725, in > _callmethod > conn.send((self._id, methodname, args, kwds)) > IOError: [Errno 32] Broken pipe > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logical network error
- Original Message - > From: "Maurice James" > To: "users" > Sent: Wednesday, July 16, 2014 5:06:23 PM > Subject: [ovirt-users] Logical network error > > While attempting to remove a logical network from one of my hosts, Im getting > the following error. > Error while executing action Setup Networks: Unexpected exception > > > Im seeing the following error in the vdsm.log. > Could you attach the entire vdsm.log and supervdsm.log ? > > Thread-72::ERROR::2014-07-16 > 10:03:46,773::BindingXMLRPC::1086::vds::(wrapper) unexpected error > Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 1070, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 494, in setupNetworks > return api.setupNetworks(networks, bondings, options) > File "/usr/share/vdsm/API.py", line 1297, in setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, options) > File "/usr/share/vdsm/supervdsm.py", line 50, in __call__ > return callMethod() > File "/usr/share/vdsm/supervdsm.py", line 48, in > **kwargs) > File "", line 2, in setupNetworks > File "/usr/lib64/python2.6/multiprocessing/managers.py", line 725, in > _callmethod > conn.send((self._id, methodname, args, kwds)) > IOError: [Errno 32] Broken pipe > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] Logical network error
While attempting to remove a logical network from one of my hosts, Im getting the following error. Error while executing action Setup Networks: Unexpected exception Im seeing the following error in the vdsm.log. Thread-72::ERROR::2014-07-16 10:03:46,773::BindingXMLRPC::1086::vds::(wrapper) unexpected error Traceback (most recent call last): File "/usr/share/vdsm/BindingXMLRPC.py", line 1070, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/BindingXMLRPC.py", line 494, in setupNetworks return api.setupNetworks(networks, bondings, options) File "/usr/share/vdsm/API.py", line 1297, in setupNetworks supervdsm.getProxy().setupNetworks(networks, bondings, options) File "/usr/share/vdsm/supervdsm.py", line 50, in __call__ return callMethod() File "/usr/share/vdsm/supervdsm.py", line 48, in **kwargs) File "", line 2, in setupNetworks File "/usr/lib64/python2.6/multiprocessing/managers.py", line 725, in _callmethod conn.send((self._id, methodname, args, kwds)) IOError: [Errno 32] Broken pipe ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
