[vdsm] Bugzilla bugs moved to MODIFIED
Hi! Yesterday I made some changes to the gerrit hooks to avoid having bugs moved to MODIFIED status when the patch that was merged was in a master branch. That should not be happening again, if you see it happening, ping me with the bug and the patch id and I'll look into it and solve it. Thanks! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization RD Tel.: +420 532 294 605 Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 pgpdhj8ZHleCu.pgp Description: PGP signature ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/ broken
On Wed 05 Feb 2014 01:15:40 PM CET, Eyal Edri wrote: not sure sandro meant the patch was broken, it seems job was hang and stuck for a while. dcaro - can you check what could cause this? e. - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: VDSM Project Development vdsm-devel@lists.fedorahosted.org, infra in...@ovirt.org Sent: Wednesday, February 5, 2014 2:09:51 PM Subject: Re: http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/ broken On Wed, Feb 05, 2014 at 10:24:47AM +0100, Sandro Bonazzola wrote: Hi, It seems that http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/ is broken, never ending also if all dependent jobs ended successfully. This keep the slaves busy avoiding other jobs to run. Please disable that job or fix ASAP. Sorry, I do not understand what is broken and what should be fixed. A recent patch of mine for an ack by this job http://gerrit.ovirt.org/#/c/24103/ but for some reason its link http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/6962/ is broken, as if it was partially moved to http://jenkins.ovirt.org/view/vdsm/job/vdsm_master_pep8_gerrit/6962/ ___ Infra mailing list in...@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra Getting this a little late sorry, the job Sandro referenced does no longer exist and the one dan references seems to be working ok. Still relevant? -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization RD Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] VDSM Storage tests @ Jenkins
El lun 10 feb 2014 17:08:33 CET, Dan Kenigsberg escribió: On Mon, Feb 10, 2014 at 03:10:56PM +0100, Vinzenz Feenstra wrote: Hi, Could we please disable the storage tests until they are really working? These random failures are really creating more noise than being helpful. As much as I think that they are valuable to have, I think Vinzenz is correct - they are not yet dependable. ___ Infra mailing list in...@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra Ok, I've disabled the notifications (Silent mode on) of the tests until they are fixed. @vered: are you working on them? -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization RD Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] New gerrit flags behavior
Hi everyone! With the latest gerrit upgrade it has become easier to add the propagation of the Code Review and Verified flags when doing a trivial rebase or when no code was changed, so I've enabled those features for all the projects! The change should become effective right away, so let me know if you have any issues. Enjoy! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization RD Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Vdsm network functional tests taking slave offline
Hi! We have been seeing the slave that is used for the network tests getting offline from time to time, it was dues to vdsm_functional_tests not cleaning up properly, as removing all the *dummy* interface configurations from /etc/sysconfig/network-scripts and restarting the network brings it back again. Can you take a look on why it would not have been cleaning up properly? the last job that made the host go offline is: http://jenkins.ovirt.org/job/vdsm_network_functional_tests/1173/console The number of the dummy interface that was left over was 6 (not sure it's reelevant, but just in case). Thanks! -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization RD Email: dc...@redhat.com Web: www.redhat.com RHT Global #: 82-62605 signature.asc Description: OpenPGP digital signature ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] stale gerrit patches
On Mon 23 Sep 2013 12:36:58 PM CEST, Itamar Heim wrote: we have some very old gerrit patches. I'm for abandoning patches which were not touched over 60 days (to begin with, I think the number should actually be lower). they can always be re-opened by any interested party post their closure. i.e., looking at gerrit, the patch list should actually get attention, and not be a few worth looking at, with a lot of old patches thoughts? Thanks, Itamar ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel It might helpful to have a cron-like script that checks the age of the posts and first notifies the sender, the reviewers and the maintainer, and if the patch is not updated in a certain period just abandons it. -- David Caro Red Hat Czech s.r.o. Continuous Integration Engineer - EMEA ENG Virtualization RD Tel.: +420 532 294 605 Email: dc...@redhat.com Web: www.cz.redhat.com Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic RHT Global #: 82-62605 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Test failure not free loop devices
On Mon 12 Aug 2013 04:13:26 AM CEST, Zhou Zheng Sheng wrote: Hi David, on 2013-08-09 21:45, David Caro Estevez wrote: Sometimes we get this error when running the vdsm tests: == ERROR: testLoopMount (mountTests.MountTests) -- Traceback (most recent call last): File /ephemeral0/vdsm_unit_tests_gerrit_el/tests/mountTests.py, line 69, in testLoopMount m.mount(mntOpts=loop) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 222, in mount return self._runcmd(cmd, timeout) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 238, in _runcmd raise MountError(rc, ;.join((out, err))) MountError: (2, ';mount: could not find any free loop device\n') begin captured logging Storage.Misc.excCmd: DEBUG: '/sbin/mkfs.ext2 -F /tmp/tmpq95svr' (cwd None) Storage.Misc.excCmd: DEBUG: SUCCESS: err = 'mke2fs 1.41.12 (17-May-2010)\n'; rc = 0 Storage.Misc.excCmd: DEBUG: '/usr/bin/sudo -n /bin/mount -o loop /tmp/tmpq95svr /tmp/tmpcS29EU' (cwd None) The problem is that it seems that the loop devices that are not being released (maybe when the test fails?) and the system runs out of devices eventually. Can you take a look to see where the cleanup fails and fix it? I think this may be related to a known bug [1] of gvfs. To workaround the bug, 1. Reboot Linux 2. See if ps aux | grep gvfs produces any results, then kill all gvfs related processes. 3. cd tests, and run the tests ./run_tests_local.sh mkimageTests.py mountTests.py for 10 times, and losetup -a should give empty result after each run. When gvfs is running, losetup -a would show 2 new loop devices are occupied after each run. gvfs is not installed on the machines, so I doubt this is the same error :( I suggest to add some cleanup code at the end of the tests to free all the used loop devices (if you free only the ones you use, it will not fail when running parallel tests). Adding more loop devices or cleaning them all manually is only a temporary solution. Thanks! -- David Caro Red Hat Czech s.r.o. Continuous Integration Engineer - EMEA ENG Virtualization RD Tel.: +420 532 294 605 Email: dc...@redhat.com Web: www.cz.redhat.com Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic RHT Global #: 82-62605 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Test failure not free loop devices
Sometimes we get this error when running the vdsm tests: == ERROR: testLoopMount (mountTests.MountTests) -- Traceback (most recent call last): File /ephemeral0/vdsm_unit_tests_gerrit_el/tests/mountTests.py, line 69, in testLoopMount m.mount(mntOpts=loop) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 222, in mount return self._runcmd(cmd, timeout) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 238, in _runcmd raise MountError(rc, ;.join((out, err))) MountError: (2, ';mount: could not find any free loop device\n') begin captured logging Storage.Misc.excCmd: DEBUG: '/sbin/mkfs.ext2 -F /tmp/tmpq95svr' (cwd None) Storage.Misc.excCmd: DEBUG: SUCCESS: err = 'mke2fs 1.41.12 (17-May-2010)\n'; rc = 0 Storage.Misc.excCmd: DEBUG: '/usr/bin/sudo -n /bin/mount -o loop /tmp/tmpq95svr /tmp/tmpcS29EU' (cwd None) The problem is that it seems that the loop devices that are not being released (maybe when the test fails?) and the system runs out of devices eventually. Can you take a look to see where the cleanup fails and fix it? Thanks! pd. To free a loop device you have to umount it and then losetup -d loopdevice, that way the device gets liberated. Also it might be a good solution to create a specific device per test, that way it will never run out of them no matter how many tests run in parallel, but that requires a little more modifications. David Caro Red Hat Czech s.r.o. Continuous Integration Engineer - EMEA ENG Virtualization RD Tel.: +420 532 294 605 Email: dc...@redhat.com Web: www.cz.redhat.com Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic RHT Global #: 82-62605 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] environment encoding, LC_ALL and vdsm tests
- Original Message -
From: Dan Kenigsberg dan...@redhat.com
To: Martin Sivak msi...@redhat.com, dc...@redhat.com
Cc: vdsm-devel@lists.fedorahosted.org
Sent: Thursday, June 20, 2013 3:08:29 PM
Subject: Re: environment encoding, LC_ALL and vdsm tests
On Thu, Jun 20, 2013 at 05:50:16AM -0400, Martin Sivak wrote:
Hi,
recently I discovered an issue with our Jenkins test environment. It was
failing in testHooks.py because my Gerrit name contains diacritics and our
code tried to decode it as ascii.
Traceback (most recent call last):
File /usr/lib64/python2.6/unittest.py, line 278, in run
testMethod()
File /ephemeral0/vdsm_unit_tests_gerrit_el/tests/hooksTests.py, line
125, in test_deviceCustomProperties
params={'customProperty': ' rocks!'})
File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/hooks.py, line 70, in
_runHooksDir
scriptenv[k] = unicode(v).encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 12:
ordinal not in range(128)
The relevant code is here:
hooks.py:
60 scriptenv = os.environ.copy()
...
69 for k, v in scriptenv.iteritems():
70 scriptenv[k] = unicode(v).encode('utf-8')
My first instinct was to decode it using the proper encoding:
source_encoding = sys.stdin.encoding or locale.getpreferredencoding()
for k, v in scriptenv.iteritems():
scriptenv[k] = v.decode(source_encoding).encode('utf-8')
But it still did not work. So I tried to print out the environment and
encodings that are used when make check is being run and got this:
sys.stdin.encoding == None
locale.getpreferredencoding() - ANSI_X3.4-1968
os.environ['LC_ALL'] == 'C'
os.environ['LANG'] == 'en_US.UTF-8'
Please notice the encoding part, my system and terminal are using utf-8,
but vdsm reads the environment values using ANSI. That is obviously wrong
and can't work.
So i tried to investigate it further and found out we force LC_ALL to C in
vdsmd.init, run_tests.sh.in and run_tests_local.sh.in.
I also found the commit that introduced this -
107644dbad9af250c00e7f25fc51a92c6250d442 - and finally understood where
the issue was.
Although I understand the reasons for the patch, I do not agree with
it. If we are executing other tools and parse their output, we should
be preparing and passing the updated locale _only_ to those tools. We
should not be setting the locale we need for parsing stuff to the
whole vdsm daemon.
Since vdsm is not intended for direct human control, I actually like the
idea of turning off all locale noise by a global LC_ALL=C. The
alternative, of setting it to C before each application with parsed
output seems tedious and easily forgotten.
Our current practice of setting LC_ALL to C no matter on what terminal
or system we are starting vdsmd is causing us the above mentioned
issue, because the environment can (and does) contain data in the
system encoding. This essentially prevents anybody with utf-8 chars in
their names to submit anything to vdsm.
No doubt that we have to fix it. The easiest hack is to ask our Jenkins
job to clear the Jenkins env vars before calling `make check`. I'm sure
David (CCed) can do it quite easily.
Yes, that should be easy, if you decide to do that, it can be done in 30min
(smallest fraction of time for a task).
So I would like to start a discussion about this that will lead to the
necessary fixes and change in our current practice :)
Unfortunately, I have no idea beyond exterminating non-7-bit chars from
the environment and setting LC_ALL=C in n+1 places.
The first approach may not be so horrible as it seems: I'm not sure we
should pass every vdsm env variable to the hook scripts. Passing only
ascii ones may be good enough.
Obviously, unicode custom properties should continue to be explicitly
added, with utf-8 encoding, to the script environment, as this is a
documented vdsm API.
Dan.
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
