Re: [ovirt-users] AIO 3.4 on fedora 19 initial errors before coming up

2014-05-12 Thread Sandro Bonazzola
Il 11/05/2014 09:55, Gianluca Cecchi ha scritto:
 
 Hello,
 I have an environment with All-In-One on Fedora 19.
 Packages version is aligned with oVirt repos up to 3.4.0-1.fc19
 
 I start and shutdown the system every day as it is a personal computer.
 Storage is composed by two local storage domains.
 
 Every time system boots it takes about 7 minutes to have datacenter up.
 The system has 16Gb of ram and internal disk is an ssd one (one of the 
 storage domains is on a sata disk).
 I see some errors for some minutes before the DC and storage domains come up.
 I would like to know if this is expected or if I have some correctable 
 problems as the system comes form several updates up to the current 3.4.0 
 version.
 Here you can find relevant logs with latest lines during yesterday shutdown 
 and today lines up to DC  has been activated.
 
 Thanks in advance,
 Gianluca
 
 engine.log
 https://drive.google.com/file/d/0BwoPbcrMv8mvQWYwT0wxYXpXeFk/edit?usp=sharing
 
 vdsm.log
 https://drive.google.com/file/d/0BwoPbcrMv8mvN3BSNjRpbjFSTjQ/edit?usp=sharing

This one shows errors that seems to be related to  missing IPv6 address on one 
NIC and unfetched / non existent storage domains.
Federico, Antoni, can you take a look?


 
 supervdsm.log
 https://drive.google.com/file/d/0BwoPbcrMv8mvR01Sby1Pd1ZHNEE/edit?usp=sharing
 
 
 ___
 Users mailing list
 Users@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/users
 


-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] AIO 3.4 on fedora 19 initial errors before coming up

2014-05-12 Thread David Sommerseth
On 12/05/14 12:28, Dan Kenigsberg wrote:
 On Mon, May 12, 2014 at 02:33:52AM -0400, Francesco Romani wrote:
 Hi,

 - Original Message -
 From: Gianluca Cecchi gianluca.cec...@gmail.com
 To: Roy Golan rgo...@redhat.com
 Cc: users users@ovirt.org
 Sent: Sunday, May 11, 2014 11:49:06 PM
 Subject: Re: [ovirt-users] getVdsCapabilites unexpected exception [was: Re: 
 AIO 3.4 on fedora 19 initial errors
 before coming up]
 [...]
 it seems the error in vdsm.log when I run the command above is of this type:

 Thread-25::ERROR::2014-05-11
 20:18:02,202::BindingXMLRPC::1086::vds::(wrapper) unexpected error
 Traceback (most recent call last):
 File /usr/share/vdsm/BindingXMLRPC.py, line 1070, in wrapper
 res = f(*args, **kwargs)
 File /usr/share/vdsm/BindingXMLRPC.py, line 393, in getCapabilities
 ret = api.getCapabilities()
 File /usr/share/vdsm/API.py, line 1185, in getCapabilities
 c = caps.get()
 File /usr/share/vdsm/caps.py, line 369, in get
 caps.update(netinfo.get())
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 557, in get
 netAttr.get('qosOutbound'))
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 487, in
 _getNetInfo
 ipv4addr, ipv4netmask, ipv6addrs = getIpInfo(iface)
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 317, in
 getIpInfo
 ipv6addrs = devInfo.get_ipv6_addresses()
 SystemError: error return without exception set


 Based on above errors, I think that for some reason these two python related
 packages that were updated yesterday are causing some problems with vdsm.
 Can you confirm that you can run ok the 3.4 vdsm with those?

 vdsm-4.14.6-0.fc19.x86_64


 May 10 21:24:23 Updated: python-ethtool-0.9-2.fc19.x86_64
 May 10 21:24:23 Updated: python-lxml-3.3.5-1.fc19.x86_64

 I can also try to rollback and see...


 I was right.
 Against what to bugzilla?
 This is a show stopper for fedora 19 ovirt users...

 Unfortunately, you are been hit by 
 https://bugzilla.redhat.com/show_bug.cgi?id=1078312

 It is fixed on gerrit, but you'll need VDSM = 4.14.8.1
 
 Too bad that we did not manage to add a Conflicts: vdsm = 4.18.6 to
 that release of python-ethtool-0.9-2.fc19.x86_64. But now that it is
 out, there is not much that one can do but to upgrade to a new Vdsm or
 roll back python-ethtool.
 
 The propblem was due to Vdsm-proper using libnl1 while that version of
 python-ethtool starting to use linbnl3 and solved by
 http://gerrit.ovirt.org/26514. Backporting this to the now-unsupported
 ovirt-3.3 is not really viable, I a m afraid.

Sorry about that!  I honestly forgot about this when I had a discussion
with Antoni Segura Puimedon last week.  And we discovered some other
ugly issues too, where we need a dependency on a not yet released libnl3
version to fully fix some libnl connection conflicts with vdsm.

I tried to remove this update in the last minute.  But it obviously
slipped through anyway :(  Not sure if I'm able to revert to an older
version now in Koji.


--
kind regards,

David Sommerseth







___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] AIO 3.4 on fedora 19 initial errors before coming up

2014-05-12 Thread David Sommerseth
On 12/05/14 19:27, Dan Kenigsberg wrote:
 On Mon, May 12, 2014 at 05:34:34PM +0200, David Sommerseth wrote:
 On 12/05/14 12:28, Dan Kenigsberg wrote:
 On Mon, May 12, 2014 at 02:33:52AM -0400, Francesco Romani wrote:
 Hi,

 - Original Message -
 From: Gianluca Cecchi gianluca.cec...@gmail.com
 To: Roy Golan rgo...@redhat.com
 Cc: users users@ovirt.org
 Sent: Sunday, May 11, 2014 11:49:06 PM
 Subject: Re: [ovirt-users] getVdsCapabilites unexpected exception [was: 
 Re: AIO 3.4 on fedora 19 initial errors
 before coming up]
 [...]
 it seems the error in vdsm.log when I run the command above is of this 
 type:

 Thread-25::ERROR::2014-05-11
 20:18:02,202::BindingXMLRPC::1086::vds::(wrapper) unexpected error
 Traceback (most recent call last):
 File /usr/share/vdsm/BindingXMLRPC.py, line 1070, in wrapper
 res = f(*args, **kwargs)
 File /usr/share/vdsm/BindingXMLRPC.py, line 393, in getCapabilities
 ret = api.getCapabilities()
 File /usr/share/vdsm/API.py, line 1185, in getCapabilities
 c = caps.get()
 File /usr/share/vdsm/caps.py, line 369, in get
 caps.update(netinfo.get())
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 557, in 
 get
 netAttr.get('qosOutbound'))
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 487, in
 _getNetInfo
 ipv4addr, ipv4netmask, ipv6addrs = getIpInfo(iface)
 File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 317, in
 getIpInfo
 ipv6addrs = devInfo.get_ipv6_addresses()
 SystemError: error return without exception set


 Based on above errors, I think that for some reason these two python 
 related
 packages that were updated yesterday are causing some problems with vdsm.
 Can you confirm that you can run ok the 3.4 vdsm with those?

 vdsm-4.14.6-0.fc19.x86_64


 May 10 21:24:23 Updated: python-ethtool-0.9-2.fc19.x86_64
 May 10 21:24:23 Updated: python-lxml-3.3.5-1.fc19.x86_64

 I can also try to rollback and see...


 I was right.
 Against what to bugzilla?
 This is a show stopper for fedora 19 ovirt users...

 Unfortunately, you are been hit by 
 https://bugzilla.redhat.com/show_bug.cgi?id=1078312

 It is fixed on gerrit, but you'll need VDSM = 4.14.8.1

 Too bad that we did not manage to add a Conflicts: vdsm = 4.18.6 to
 that release of python-ethtool-0.9-2.fc19.x86_64. But now that it is
 out, there is not much that one can do but to upgrade to a new Vdsm or
 roll back python-ethtool.

 The propblem was due to Vdsm-proper using libnl1 while that version of
 python-ethtool starting to use linbnl3 and solved by
 http://gerrit.ovirt.org/26514. Backporting this to the now-unsupported
 ovirt-3.3 is not really viable, I a m afraid.

 Sorry about that!  I honestly forgot about this when I had a discussion
 with Antoni Segura Puimedon last week.  And we discovered some other
 ugly issues too, where we need a dependency on a not yet released libnl3
 version to fully fix some libnl connection conflicts with vdsm.
 
 Could you (or Toni) add more information regrading the last sentence? Is
 there a known issue with ovirt-3.4.1's vdsm?

AFAIU, it's the same old issue, which caused some regression from the
tests we've done earlier, due to vdsm using libnl-1.x while
python-ethtool uses libnl3.  When vdsm in addition uses py-ethtool, the
socket py-ethtool module gets closed by itself, which it uses to query
the kernel's netlink interface.  The result is that py-ethtool gets in a
useless state is not able to re-establish a new socket and cannot query
the system for any useful information.  Thomas Haller (libnl developer)
have provided patches to libnl3 which fixes the connection handling -
which should help avoiding this issue.

I missed the detail that the libnl3 fixes hadn't been pushed out in a
release.  So when py-ethool got updated, it broke vdsm due to this
libnl1/libnl3 issue.

With an updated libnl3 + updated py-ethtool, I believe things should be
better also for the older vdsm versions.  The change in py-ethtool is
primarily improving error handling if libnl3 isn't able to establish a
netlink socket to the kernel.


--
kind regards,

David Sommerseth

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users