Re: [vdsm] [Engine-devel] Things to be done to support Ubuntu hosts
on 2013/11/14 17:41, Yaniv Bronheim wrote:
Hey,
Most of the issues you mentioned in slides 11-13
(http://www.ovirt.org/images/5/57/Shanghai-VDSM-on-Ubuntu.pdf) are already
solved (if not all of them),
If there are more issues to solve can you please open RFE bz on them to close
those gaps (like making the services' names configurable)?
Hi, the slide 11-13 are about VDSM on Ubuntu compatibility. I believe
almost all the problems are fixed now. The remaining part is
ovirt-host-deploy. Indeed standalone VDSM installs and runs on Ubuntu
well, but we need otopi and ovirt-host-deploy to make a Ubuntu host
under the management of Engine. As I mentioned, the things left are
adding apt-get support in otopi and finishing reviewing the upsteam
configNetwork.py patches in VDSM. Do you mean I should open RFE bz of
otopi and ovirt-host-deploy?
Thanks
Yaniv Bronhaim.
- Original Message -
From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com
To: engine-devel engine-de...@ovirt.org
Sent: Thursday, November 14, 2013 8:57:19 AM
Subject: [Engine-devel] Things to be done to support Ubuntu hosts
Hi,
Recently Ubuntu support is added to VDSM, and .deb binray packages can
be downloaded from launchpad.net PPA [1]. Most of the key features such
as storage management and VM lifecycle work on Ubuntu. The cross
distribution network management patches are upstream as well. One big
piece left is making ovirt-host-deploy support Ubuntu, so that we can
manage Ubuntu hosts from Engine, thus close the gap on host side.
In May 2013 I made some hacks to ovirt-host-deploy and otopi. I made it
skipped parts not supported on Ubuntu and configure the environment
manually, and successfully added Ubuntu host to Engine [2].
Unfortunately I have no plans to continue on it, but I'd like to make a
summary of the things I hacked to help anyone who wants to submit
patches in future. I was to add a WIKI page but I found there were not
many items, so a mail would be enough.
1. Package management operations
Both otopi and ovirt-host-deploy query for dependency packages and
install them on demand. The otopi package management just supports yum,
we need to add apt-get support. Package names are different in Ubuntu,
so I made a list mapping the names. The list in in VDSM source
directory, debian/dependencyMap.txt .
2. Network configuration operations
ovirt-host-deploy asks VDSM's configNetwork.py to create bridge network.
Cross distribution support patches for configNetwork.py are under
review. ovirt-host-deploy supports Ubuntu bridge configuration as long
as they are merged.
[1] https://launchpad.net/~zhshzhou/+archive/vdsm-ubuntu
[2] http://www.ovirt.org/images/5/57/Shanghai-VDSM-on-Ubuntu.pdf
Here goes the detailed hack patch. The hack was base on v1.0.1, I
rebased it to the latest master. The rebased patch are not tested. If
you find this email useless, it's actually a good news, which means
there is not a lot of work and problems ahead ;-)
Hack patch for ovirt-host-deploy.
From 120493a242046d19794ef3da83b32486d372aa39 Mon Sep 17 00:00:00 2001
From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com
Date: Wed, 13 Nov 2013 18:02:26 +0800
Subject: [PATCH] Ubuntu Hacks
Change-Id: Ifb4ebc829101c92d06475619b1b5986e87b83d57
Signed-off-by: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com
---
src/plugins/ovirt-host-deploy/gluster/packages.py | 17 +
src/plugins/ovirt-host-deploy/tune/tuned.py | 3 +-
src/plugins/ovirt-host-deploy/vdsm/bridge.py | 45
---
src/plugins/ovirt-host-deploy/vdsm/packages.py| 9 +++--
src/plugins/ovirt-host-deploy/vdsm/pki.py | 3 +-
src/plugins/ovirt-host-deploy/vdsm/software.py| 2 +
src/plugins/ovirt-host-deploy/vdsm/vdsmid.py | 3 +-
7 files changed, 45 insertions(+), 37 deletions(-)
diff --git a/src/plugins/ovirt-host-deploy/gluster/packages.py
b/src/plugins/ovirt-host-deploy/gluster/packages.py
index 1fecfda..eb37744 100644
--- a/src/plugins/ovirt-host-deploy/gluster/packages.py
+++ b/src/plugins/ovirt-host-deploy/gluster/packages.py
@@ -60,13 +60,13 @@ class Plugin(plugin.PluginBase):
),
)
def _validation(self):
-if not self.packager.queryPackages(patterns=('vdsm-gluster',)):
-raise RuntimeError(
-_(
-'Cannot locate gluster packages, '
-'possible cause is incorrect channels'
-)
-)
+# if not self.packager.queryPackages(patterns=('vdsm-gluster',)):
+# raise RuntimeError(
+# _(
+# 'Cannot locate gluster packages, '
+# 'possible cause is incorrect channels'
+# )
+# )
self._enabled = True
@plugin.event(
@@ -74,7 +74,8 @@ class Plugin(plugin.PluginBase):
condition=lambda self: self._enabled,
)
def _packages
[vdsm] Keeping VDSM Compatible with Ubuntu
Hi all, Recently we merged some patches to make VDSM run on Ubuntu. We also have some packaging scripts in the debian/ sub-dir. You can either build .deb packages manually or find binary packages from VDSM PPA [1] on launchpad.net. Once you add that PPA, you can use apt-get to install VDSM and its dependencies. I'll setup a Jenkins on my laptop to test the master branch automatically by build and install VDSM on Ubuntu, then running unit and functional tests. I suggest when adding a new change, it's better to make sure it is covered by unit or functional tests. If you change the packaging code, for example adding new options to configure.ac, editing vdsm.spec.in, and changing VDSM daemon startup behavior, I am happy to be invited to review your patch. I'd also like to listen to your suggestions on this topic, thanks! [1] https://launchpad.net/~zhshzhou/+archive/vdsm-ubuntu -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] stale gerrit patches
on 2013/09/24 05:21, Ayal Baron wrote: - Original Message - - Original Message - From: Itamar Heim ih...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: David Caro dcaro...@redhat.com, engine-devel engine-de...@ovirt.org, vdsm-devel@lists.fedorahosted.org Sent: Monday, September 23, 2013 1:54:39 PM Subject: Re: [vdsm] stale gerrit patches On 09/23/2013 01:52 PM, Alon Bar-Lev wrote: - Original Message - From: Itamar Heim ih...@redhat.com To: Alon Bar-Lev alo...@redhat.com Cc: David Caro dcaro...@redhat.com, engine-devel engine-de...@ovirt.org, vdsm-devel@lists.fedorahosted.org Sent: Monday, September 23, 2013 1:50:35 PM Subject: Re: [vdsm] stale gerrit patches On 09/23/2013 01:49 PM, Alon Bar-Lev wrote: - Original Message - From: Itamar Heim ih...@redhat.com To: David Caro dcaro...@redhat.com Cc: engine-devel engine-de...@ovirt.org, vdsm-devel@lists.fedorahosted.org Sent: Monday, September 23, 2013 1:47:47 PM Subject: Re: [vdsm] stale gerrit patches On 09/23/2013 01:46 PM, David Caro wrote: On Mon 23 Sep 2013 12:36:58 PM CEST, Itamar Heim wrote: we have some very old gerrit patches. I'm for abandoning patches which were not touched over 60 days (to begin with, I think the number should actually be lower). they can always be re-opened by any interested party post their closure. i.e., looking at gerrit, the patch list should actually get attention, and not be a few worth looking at, with a lot of old patches thoughts? Thanks, Itamar ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel It might helpful to have a cron-like script that checks the age of the posts and first notifies the sender, the reviewers and the maintainer, and if the patch is not updated in a certain period just abandons it. yep - warn after X days via email to just owner (or all subscribed to the patch), and close if no activity for X+14 days or something like that. This will be annoying. And there are patches that pending with good reason. pending for 60 days with zero activity on them (no comment, no rebase, nothing)? http://gerrit.ovirt.org/#/q/status:open+project:ovirt-engine+branch:master+topic:independent_deployments,n,z so how does it help us to have these patches, some without any comment from any reviewer. lets get them reviewed and decide one way or the other, rather than let them get old and stay forever Again... maintainer can close these if he likes. Owner can close these if he likes. right, but why? a patch without activity being abandoned might actually spur someone into motion (rebasing and resubmitting, prodding maintainers etc). I'm +1 for automatically abandoning old patches. At least we all agree on that old patches should be abandoned. I think we can do this in a semi-automatic way. A cron job checks the patch's freshness, and sends an email to warn the author and reviewers of an old patch. If the someone has a good reason to keep the patch, he can leave a comment on the gerrit web page saying I want to #keep the patch# because Then the system skips the patches whose last comment contains #keep the patch#. If no one cares it, the patch is abandoned after some time. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Test failure not free loop devices
Hi David, on 2013-08-12 14:59, David Caro wrote: On Mon 12 Aug 2013 04:13:26 AM CEST, Zhou Zheng Sheng wrote: Hi David, on 2013-08-09 21:45, David Caro Estevez wrote: Sometimes we get this error when running the vdsm tests: == ERROR: testLoopMount (mountTests.MountTests) -- Traceback (most recent call last): File /ephemeral0/vdsm_unit_tests_gerrit_el/tests/mountTests.py, line 69, in testLoopMount m.mount(mntOpts=loop) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 222, in mount return self._runcmd(cmd, timeout) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 238, in _runcmd raise MountError(rc, ;.join((out, err))) MountError: (2, ';mount: could not find any free loop device\n') begin captured logging Storage.Misc.excCmd: DEBUG: '/sbin/mkfs.ext2 -F /tmp/tmpq95svr' (cwd None) Storage.Misc.excCmd: DEBUG: SUCCESS: err = 'mke2fs 1.41.12 (17-May-2010)\n'; rc = 0 Storage.Misc.excCmd: DEBUG: '/usr/bin/sudo -n /bin/mount -o loop /tmp/tmpq95svr /tmp/tmpcS29EU' (cwd None) The problem is that it seems that the loop devices that are not being released (maybe when the test fails?) and the system runs out of devices eventually. Can you take a look to see where the cleanup fails and fix it? I think this may be related to a known bug [1] of gvfs. To workaround the bug, 1. Reboot Linux 2. See if ps aux | grep gvfs produces any results, then kill all gvfs related processes. 3. cd tests, and run the tests ./run_tests_local.sh mkimageTests.py mountTests.py for 10 times, and losetup -a should give empty result after each run. When gvfs is running, losetup -a would show 2 new loop devices are occupied after each run. gvfs is not installed on the machines, so I doubt this is the same error :( I suggest to add some cleanup code at the end of the tests to free all the used loop devices (if you free only the ones you use, it will not fail when running parallel tests). Adding more loop devices or cleaning them all manually is only a temporary solution. Thanks! From what I can there is 3 places would consume a loop device. In tests/parted_utils_tests.py it calls losetup directly and tears down the loop device at the end of each test. In tests/mountTests.py and tests/mkimageTests.py, it consumes loop device via loop mount, and umount the device so the loop device is freed automatically. All three places free loop device at the end of the test, unless there is exception or test fails. In this case we can catch that exception and free the device. Another reason could be other process occupying the mount point or loop device, in this case trying to free it again will fail and can not solve this problem. So I think we should investigate why it is not released and locate the bug. Could you lsof the mount point and the device to see if any process is occupying them? And could you try to run parted_utils_tests.py, mountTests.py and mkimageTests.py separately and see losetup -a's report, to see which of the test module consumes but not free the device? -- David Caro Red Hat Czech s.r.o. Continuous Integration Engineer - EMEA ENG Virtualization RD Tel.: +420 532 294 605 Email: dc...@redhat.com Web: www.cz.redhat.com Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic RHT Global #: 82-62605 -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Test failure not free loop devices
Hi David, on 2013-08-09 21:45, David Caro Estevez wrote: Sometimes we get this error when running the vdsm tests: == ERROR: testLoopMount (mountTests.MountTests) -- Traceback (most recent call last): File /ephemeral0/vdsm_unit_tests_gerrit_el/tests/mountTests.py, line 69, in testLoopMount m.mount(mntOpts=loop) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 222, in mount return self._runcmd(cmd, timeout) File /ephemeral0/vdsm_unit_tests_gerrit_el/vdsm/storage/mount.py, line 238, in _runcmd raise MountError(rc, ;.join((out, err))) MountError: (2, ';mount: could not find any free loop device\n') begin captured logging Storage.Misc.excCmd: DEBUG: '/sbin/mkfs.ext2 -F /tmp/tmpq95svr' (cwd None) Storage.Misc.excCmd: DEBUG: SUCCESS: err = 'mke2fs 1.41.12 (17-May-2010)\n'; rc = 0 Storage.Misc.excCmd: DEBUG: '/usr/bin/sudo -n /bin/mount -o loop /tmp/tmpq95svr /tmp/tmpcS29EU' (cwd None) The problem is that it seems that the loop devices that are not being released (maybe when the test fails?) and the system runs out of devices eventually. Can you take a look to see where the cleanup fails and fix it? I think this may be related to a known bug [1] of gvfs. To workaround the bug, 1. Reboot Linux 2. See if ps aux | grep gvfs produces any results, then kill all gvfs related processes. 3. cd tests, and run the tests ./run_tests_local.sh mkimageTests.py mountTests.py for 10 times, and losetup -a should give empty result after each run. When gvfs is running, losetup -a would show 2 new loop devices are occupied after each run. [1] https://bugzilla.redhat.com/show_bug.cgi?id=853674#c5 Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] About vdsmd init script
Hi, on 05/28/2013 17:26, Yaniv Bronheim wrote: Hey, I think that libvirt_configure part can be an external module (maybe python module) that can be initiated by vdsm-tool, It should work with template or conf.default as you mentioned, and we should call it before starting the service, I think it should be a module as it also should include all the part of libvirtd_sysv2upstart, libvirtd_reconfigure, libvirtd_configure, test_conflicting_conf scripts. Also, keep in mind that we plan to split vdsm to 2 services - one for vdsmd and one for supervdsmd, both should be initiated at startup and should be depended on eachother (http://gerrit.ovirt.org/#/c/11051/). Yes. After supervdsm starts as service, we can add dependency declarations easily. It's not conflicting with refactoring vdsm init script. I can help to review the supervdsm patch to make it done faster. The other parts that you want to take out of vdsmd script are: shutdown-conflict-srv - could be also as part of the tool nwfliter, dummybr - both python scripts that we run, why not part of the tool as well? start_needed_srv, load-needed-modules - only sysv and debian need it if I understand correctly. systemd,upstat,openrc can use their init script parameters. so why take them out? in each start function we'll start and load the needed services and modules. systemd,upstat,openrc don't need custom start function anyway. The Debian ships with /lib/lsb/init-functions, and Red Hat family (such as CentOS, RHEL6) ship with /etc/init.d/functions. To print the error message and daemonize the service process, we call different utility functions in different system thought they are all SysV. The service script boilerplate in Debian is different from Red Hat family as well. So we want provide dedicated init script for respective systems. To re-use start_needed_srv and load-needed-modules in different SysV init scripts, I move them out. gencerts, syslog_available, tune_system, test_space_and_lo, prepare_dirs - can be scripts that we run before start as you did. Regards, Yaniv Bronhaim. I agree some of the initialize operations can be moved to vdsm-tool. I think we can do this in future patch after we port VDSM init script to Ubuntu. I'd prefer start small, not to do all the things in one batch. Once we have VDSM run on Ubuntu, we can improve it step by step. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Potential Bug in cpopen
Hi,
Recently Jenkins unit test sometimes fails on PidStatTests.test, for
example http://gerrit.ovirt.org/#/c/14670/
After it execCmd a sleep command with sync=False, in
/proc/[xxxpid]/stat we should see the name is sleep, but in this case
we get python, which means there is a possible race condition. The
most possible situation is that execCmd returns before the child process
execvp the sleep, then the parent process reads the stat and sees the
process name is still python.
cpopen is designed to avoid this kind of race. It uses a pipe to
synchronize the child and parent. It sets the FD_CLOEXEC on the child
side of the pipe, so that once execvp succeeds, the child pipe is
closed. If execvp fails, child writes error code to the pipe. The parent
reads the other end of the pipe like follow.
if (read(errnofd[0], childErrno, sizeof(int)) == sizeof(int)) {
PyErr_SetString(PyExc_OSError, strerror(childErrno));
goto fail;
}
It assumes that when read returns, the return value is either
sizeof(int), which indicates an error in the child side, or 0, which
indicates the child side of pipe is closed and execvp succeeds. However
this assumption may not always be true. If the parent process gets a
signal, the read invocation would be interrupt, and the code treats this
interruption the same as the case of execvp succeeds. If the system is
very busy like our Jenkins slave concurrently executing jobs, it's
possible the parent gets interrupted before the child execvp succeeds,
so cpopen returns to execCmd and cause the race.
To produce this problem, we can add a sleep(10); before the exec
invocation in cpopen.c, it simulates a busy/slow system. Then in a
Python interpreter, register a signal handler and calls execCmd.
from vdsm.utils import execCmd
from vdsm import utils
def handler(signum, frame):
... print 'Signal handler called with signal', signum
...
import signal
signal.signal(signal.SIGALRM, handler)
signal.signal(signal.SIGCHLD, handler)
p = execCmd(['sleep', '3'], sync=False, sudo=False); s =
utils.pidStat(p.pid); print s; p.wait()
Then kill -SIGCHLD or kill -SIGALRM to this Python interpreter process,
we can see the output.
Signal handler called with signal 17
(9541, 'python', 'S', 9509, 9509, 6843, 34817, 9509, 4218944, 66, 0, 0,
0, 0, 0, 0, 0, 20, 0, 1, 0, 663767, 217919488, 1676,
18446744073709551615L, 4194304, 4197004, 140735665131088,
140735665124456, 268226504400, 0, 0, 16781312, 73730,
18446744071579398713L, 0, 0, 17, 3, 0, 0, 0, 0, 0, 6294952, 6297616,
19152896, 140735665132425, 140735665132432, 140735665132432,
140735665135592, 0)
True
I have not found other ways to produce it, just found this method. Not
sure it is a bug. Is it reasonable to check the read() return value and
retry on EAGAIN/EINTR to fix this?
--
Thanks and best regards!
Zhou Zheng Sheng / 周征晟
E-mail: zhshz...@linux.vnet.ibm.com
Telephone: 86-10-82454397
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Porting vdsmd.init.in and vdsm.spec.in to Ubuntu
Hi guys, You may notice recently I work on porting VDSM to Ubuntu. I managed to run VDSM functional tests with some hacks in the code, then I documented the hacks and try to create long term solution patches for them [1]. I'm a bit stuck on the two files, vdsmd.init.in and vdsm.spec.in. I made a lot of small modifications to vdsmd.init.in to make it work in Ubuntu. Firstly I try to do it in the GNU autotools way to add more macros to this file, and make it adapt to Ubuntu. Then I find this file is already somewhat complicated, adding more macros to configure.ac and this file would make it worse. So I created a separate init file for Ubuntu based on vdsmd.init.in, but I find the structure and most part of the new file is the same as the original one. Dan suggests me break the init file into small functions and put the functions in vdsm-tool. I think this job may take long time and delay the process of porting VDSM. You can have a look at my modifications to this file at [2]. I have another idea. I can create a vdsm-contrib sub-directory under the VDSM source directory. Then put a vdsm-contrib/ubuntu/vdsmd.init.in.patch file in it. When the user build VDSM in Ubuntu, he can firstly patch the vdsmd.init.in. In this way we avoid creating more complicated macros and re-use the existing code. We do not have to maintain vdsmd.init.in.patch. There is no promise on this file to be patched cleanly in the long term. For now I can write new upates for vdsm-contrib/ubuntu temporarily. Once Ubuntu accept VDSM, we have them maintain the init file. How does this plan sound like? As regarding to vdsm.spec.in, I find there are some shell scripts to be executed during/after the installation. When I'm on Ubuntu, I grab the scripts out of vdsm.spec and run them to make VDSM work. If I want to create .deb files, these scripts should be re-used. Otherwise the spec file for .deb contains duplicated content from spec file for .rpm. My idea is crab these script out of .spec.in and create standalone scripts like vdsm_postInstall.sh. Then we distribute these script in vdsm package and invoke them after installation. If I could successfully port these two files, it would be a big step closer to my final goal. Could anyone gives me some suggestions? Really thanks. [1] http://www.ovirt.org/VDSM_on_Ubuntu [2] https://github.com/edwardbadboy/vdsm-ubuntu/commit/0b23299f48c5a341b73f2bb2b3dfdb58e82f5459 -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Porting vdsmd.init.in and vdsm.spec.in to Ubuntu
on 03/20/2013 18:21, Alon Bar-Lev wrote: Why did you change the order of the udev rules? Thanks a lot for the suggestions, really helpful. I sent some patches to gerrit http://gerrit.ovirt.org/#/q/project:vdsm+branch:master+topic:ubuntu,n,z . Some is merged. I change the order of the udev rules because there is a 85-lvm2.rules under Ubuntu /lib/udev/rules.d/, it sets the owner group of LV to disk', preventing VDSM from chown-ing, reading, writing the LV. When I ran the functional tests, vdsm failed to access the LV. After I change the order of VDSM udev rules, the functional test works. Maybe you can give me some comments on http://gerrit.ovirt.org/#/c/12816/ . Thank you very much! - Original Message - From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com To: VDSM Project Development vdsm-devel@lists.fedorahosted.org Sent: Wednesday, March 20, 2013 12:00:57 PM Subject: [vdsm] Porting vdsmd.init.in and vdsm.spec.in to Ubuntu Hi guys, You may notice recently I work on porting VDSM to Ubuntu. I managed to run VDSM functional tests with some hacks in the code, then I documented the hacks and try to create long term solution patches for them [1]. I'm a bit stuck on the two files, vdsmd.init.in and vdsm.spec.in. I made a lot of small modifications to vdsmd.init.in to make it work in Ubuntu. Firstly I try to do it in the GNU autotools way to add more macros to this file, and make it adapt to Ubuntu. Then I find this file is already somewhat complicated, adding more macros to configure.ac and this file would make it worse. So I created a separate init file for Ubuntu based on vdsmd.init.in, but I find the structure and most part of the new file is the same as the original one. Dan suggests me break the init file into small functions and put the functions in vdsm-tool. I think this job may take long time and delay the process of porting VDSM. You can have a look at my modifications to this file at [2]. I have another idea. I can create a vdsm-contrib sub-directory under the VDSM source directory. Then put a vdsm-contrib/ubuntu/vdsmd.init.in.patch file in it. When the user build VDSM in Ubuntu, he can firstly patch the vdsmd.init.in. In this way we avoid creating more complicated macros and re-use the existing code. We do not have to maintain vdsmd.init.in.patch. There is no promise on this file to be patched cleanly in the long term. For now I can write new upates for vdsm-contrib/ubuntu temporarily. Once Ubuntu accept VDSM, we have them maintain the init file. How does this plan sound like? As regarding to vdsm.spec.in, I find there are some shell scripts to be executed during/after the installation. When I'm on Ubuntu, I grab the scripts out of vdsm.spec and run them to make VDSM work. If I want to create .deb files, these scripts should be re-used. Otherwise the spec file for .deb contains duplicated content from spec file for .rpm. My idea is crab these script out of .spec.in and create standalone scripts like vdsm_postInstall.sh. Then we distribute these script in vdsm package and invoke them after installation. If I could successfully port these two files, it would be a big step closer to my final goal. Could anyone gives me some suggestions? Really thanks. [1] http://www.ovirt.org/VDSM_on_Ubuntu [2] https://github.com/edwardbadboy/vdsm-ubuntu/commit/0b23299f48c5a341b73f2bb2b3dfdb58e82f5459 -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] VDSM test unit running path
on 03/13/2013 19:36, Yaniv Bronheim wrote: Hi, I never used the autobuilder.sh script so i never really saw or used the ~/builder directory.. how does running the tests from there supposed to work? how does it help to omit the hack? and how does the re-organization effect here? autobuild.sh invokes autogen.sh to prepare the project to be installed under the prefix of ~/builder. Then make and make install will actually generate and copy the files there. The directory structure under the ~/builder is the same as when it is installed to / . You can find ~/builder/bin, ~/builder/etc, ~/builder/lib64, ~/builder/var and so on. For unit test, we needn't start VDSM from ~/builder, we just need a directory tree that is similar to /. In this way hackVDSM can be removed. The re-organization is not a problem. After the re-organization, VDSM installed under / should work of cause, so ut on the ~/builder will work as well, because the structure under the ~/builder is the same as /. - Original Message - From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com To: VDSM Project Development vdsm-devel@lists.fedorahosted.org Cc: Yaniv Bronheim ybron...@redhat.com Sent: Wednesday, March 13, 2013 6:25:46 AM Subject: Re: [vdsm] VDSM test unit running path on 03/11/2013 14:56, Yaniv Bronheim wrote: Hey Shu, Personally when I run specific unit test I prefer to run it without installing vdsm package, means locally from my development directory. I develop on my major machine and install the package on testing environments for verification, so during the development if I write some uts, I want to be able to run them fast and easy and continue. Using installed vdsm is more for functional tests in my opinion. By the way, currently the tests can run both in installed and uninstalled environment.. and I agree about the trickiness of hackVDSM Maybe running only after install is better practice but I'm not sure, we need more opinions about that issue.. Hi Yaniv. I agree with you on running tests fast and easy. I also think make install to ~/builder is fast and easy enough for me. I can accept run unit test in ~/builder and we can create script to automate this process. Before the git repo re-organization is ready, we can get rid of hackVDSM by running tests in ~/builder. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] VDSM test unit running path
on 03/11/2013 14:56, Yaniv Bronheim wrote: Hey Shu, Personally when I run specific unit test I prefer to run it without installing vdsm package, means locally from my development directory. I develop on my major machine and install the package on testing environments for verification, so during the development if I write some uts, I want to be able to run them fast and easy and continue. Using installed vdsm is more for functional tests in my opinion. By the way, currently the tests can run both in installed and uninstalled environment.. and I agree about the trickiness of hackVDSM Maybe running only after install is better practice but I'm not sure, we need more opinions about that issue.. Hi Yaniv. I agree with you on running tests fast and easy. I also think make install to ~/builder is fast and easy enough for me. I can accept run unit test in ~/builder and we can create script to automate this process. Before the git repo re-organization is ready, we can get rid of hackVDSM by running tests in ~/builder. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [Users] latest vdsm cannot read ib device speeds causing storage attach fail
on 01/25/2013 15:31, Mark Wu wrote: On 01/25/2013 03:20 PM, Mark Wu wrote: Great work! The default action for SIGCHLD is ignore, so there's no problems reported before a signal handler is installed by zombie reaper. But I still have one problem: the python multiprocessing.manager code is running a new thread and according to the implementation of python's signal, only the main thread can receive the signal. So how is the signal delivered to the server thread? Is it possible to reap the zombie process in the main thread asynchronously and periodically? It could be more safe than using the signal handler. It seem add this line signal.siginterrupt(signal.SIGCHLD, False) after registering the SIGCHLD handler can solve the problem and pass the unit test in Royce's attachment. Another idea is to use signalfd. When using a signalfd, the targeted signal is blocked using sigprocmask, then another thread can read the signalfd to get the targeted signal. If the signalfd is created or set to non-blocking mode, we can select/poll on the signalfd for reading. I write code snippet to demo it at https://gist.github.com/4661516 Firstly yum install python-signalfd. Then run python signalfdReaper.py to test the demo code. In this way, SIGCHLD will not lead to interrupted calls since it's blocked. I think both Python manager and our zombie reaper both should be patched. For Python manager, it should restart interrupted calls. For zombie reaper is should avoid SIGCHLD leading to interrupted calls. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] starting up vdsm and svdsm
on 01/04/2013 21:08, Royce Lv wrote:
I'm also agree with breaking these two as dependent services just as
vdsmd and libvirtd. These will involve supervdsm died
and vdsm reconnect, if we use key this will be security issue. So now
we use key as local variable.If just use child/parent
pipe, here is similar restart logic to handle.
The auth key of Python manager mechanism is good, but currently when
VDSM starts super VDSM, it passes the key in the command line. You can
get the key from ps aux | grep supervdsmServer. This is not secure at
all. You can just start a python and try this.
# PYTHONPATH=/usr/share/vdsm python
Python 2.7.3 (default, Jul 24 2012, 10:05:38)
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2
Type help, copyright, credits or license for more information.
import supervdsm
m = supervdsm._SuperVdsmManager(address='/var/run/vdsm/svdsm.sock',
authkey='5e21c5e1-0050-4eca-85a0-098433f3a820')
m.register('instance')
m.register('open')
m.connect()
s = m.instance()
s.ping()
True
Here auth key is copied from the output of the ps programme. In fact the
super VDSM server is not protected by the auth key, but the srwxr-xr-x
vdsm:kvm of the /var/run/vdsm/svdsm.sock . Ordinary users can not write
to this socket. The auth key is generated by VDSM each time it launches
a new super VDSM server instance. I think the most useful thing of this
auth key is that, after a totally restart, to prevent VDSM to connect to
a previous stuck but not yet died super VDSM server, in this case we
will get Authentication Error and kill it explicitly.
The Python manager framework starts a new thread for each request, that
means the super VDSM server serve each call in a new thread. So even a
operation cause a thread to stuck, the super VDSM server is still
functional. In case that some operation causing the whole process of the
super VDSM server stuck, the super VDSM server should fork a child
itself and delegate the operation to the child. Anyway what operation
will stuck the whole process?
I think splitting VDSM and super VDSM into two services and delegate
everything to systemd is simple and robust, just like libvirtd and VDSM.
The auth key problem you mentioned also applies to connecting libvirtd,
we can just follow the existing solution for it.
--
Thanks and best regards!
Zhou Zheng Sheng / 周征晟
E-mail: zhshz...@linux.vnet.ibm.com
Telephone: 86-10-82454397
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] starting up vdsm and svdsm
on 01/06/2013 17:07, Alon Bar-Lev wrote: - Original Message - From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com To: vdsm-devel@lists.fedorahosted.org Sent: Sunday, January 6, 2013 11:03:59 AM Subject: Re: [vdsm] starting up vdsm and svdsm I think splitting VDSM and super VDSM into two services and delegate everything to systemd is simple and robust, just like libvirtd and VDSM. The auth key problem you mentioned also applies to connecting libvirtd, we can just follow the existing solution for it. I don't understand this auth key thing. Why is it required? Shouldn't it be sufficient to allow only vdsm user to interact with svdsm? Thanks, Alon. The auth key is not very useful. It is passed in the command arguments of super VDSM server, very insecure. By writing follow the existing solution, I mean libvirtd refer to a SASL DB for password and VDSM refer to /etc/pki/vdsm/keys/libvirt_password when connecting to libvirtd. I agree to allow only vdsm user to access the svdsm.sock and forget the auth key thing because saving the auth key in a vdsm user readonly file does not improve any security level. If the some one can access svdsm.sock, he can always access libvirt_password. libvirtd is mean to be used by many clients so its unix socket file can not be restricted to vdsm user only, it needs a password for each user in the SASL DB. The super VDSM server is only for VDSM itself, so restricting access svdsm.sock is enough, no auth key needed. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC]about the implement of text-based console
Hi all, in this mail I further explain how solution 5 (console streaming API) works, and propose a virtual HTTP server live inside existing XMLRPC server with a request router. You can have a look at http://gerrit.ovirt.org/9605 on 11/28/2012 01:09, Adam Litke wrote: One issue that was raised is console buffering. What happens if a client does not call getConsoleReadStream() fast enough? Will characters be dropped? This could create a reliability problem and would make scripting against this interface risky at best. on 11/28/2012 01:45, Saggi Mizrahi wrote: I don't really understand 5. What does those methods return the virtio dev path? As I know, HTTP supports persistent connection and data streaming, this is popular for AJAX applications and live video broadcasting servers. The client sends one GET request to server, and server returns a data stream, then the client reads the stream continuously. XMLRPC and REST calls relies on HTTP, so I was considering getConsoleReadStream() can utilize streaming feature in HTTP, and VDSM just forwards the console data when it is called. Unfortunately I can not find out how XMLRPC and REST supports data streaming, because XML and JSON do not support containing a continuous stream object. It seems that to get the continuous stream data, the client must call getConsoleReadStream() again and again. I think it's expensive to call getConsoleReadStream() very frequently to get the data, and it may cause a notable delay, which is not acceptable for interactive console. I am thinking of providing console stream through HTTP(s) directly. A virtual server can forward the data from guest serial console by traditional HTTP streaming method (GET /consoleStream/vmid HTTP/1.0), and can forward the input data from the user by POST method as well(POST /consoleStream/vmid HTTP/1.0), or forward input and output stream at the same time in a POST request. This virtual server can be further extended to serve downloading guest crash core dump, and we can provide flexible authentication policies in this server. The auth for HTTP requests can be different from the XMLRPC request. The normal XMLRPC requests are always POST / HTTP/1.0 or POST /RPC2 HTTP/1.0. So this virtual server can live inside the existing XMLRPC server, just with a request router. I read the code implementing the XMLRPC binding and find that implementing a request router is not very complex. We can multiplex the port 54321, and route the raw HTTP request to the virtual server while normal XMLRPC request still goes to XMLRPC handler. This means it can serve XMLRPC request as vdsClient -s localhost getVdsCaps at the same time it can serve a wget client as wget --no-check-certificate \ --certificate=/etc/pki/vdsm/certs/vdsmcert.pem \ --private-key=/etc/pki/vdsm/keys/vdsmkey.pem \ --ca-certificate=/etc/pki/vdsm/certs/cacert.pem \ https://localhost:54321/console/vmid I try to implement a simple request router at http://gerrit.ovirt.org/9605 If interested, you can have a look it. It can pass the recently add VDSM functional tests, and can serve wget requests at the same time. If we do not like this idea, I think only the solution of extending spice will fulfill our requirements. There are obvious problems in other solutions. - Original Message - From: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com To: VDSM Project Development vdsm-devel@lists.fedorahosted.org Sent: Tuesday, November 27, 2012 4:22:20 AM Subject: Re: [vdsm] [RFC]about the implement of text-based console Hi all, For now in there is no agreement on the remote guest console solution, so I decide to do some investigation continue the discussion. Our goal VM serial console remote access in CLI mode. That means the client runs without X environment. Do you mean like running qemu with -curses? I mean like virsh console -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC]about the implement of text-based console
/wiki/Features/Serial_Console_in_CLI#Currently_operational_workaround The design is good but I do not like Engine talking to libvirtd directly, thus comes the VDSM console streaming API below. Work to do Provide console streaming API from Engine to be invoked in oVirt shell. Implement the serial-console command in oVirt shell. pros Support migration. Engine can reconnect to the guest automatically after migration while keeping the connection from oVirt shell. Fit well in the current oVirt architecture: no new server process introduced, no new VM introduced, easy to maintain and manage. cons Engine talking to libvirtd directly breaks the encapsulation of VDSM. Users only can get the console stream from Engine, they can not directly connect to the host as VNC and the above two sshd solutions do. 5. VDSM console streaming API Implement new APIs in VDSM to forward the raw data from console. It exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC binding. Then Engine can get the console data stream via API instead of directly connecting to libvirtd. Other things will be the same as solution 4. Work to do Implement getConsoleReadStream() and getConsoleWriteStream() in VDSM. Provide console streaming API from Engine to be invoked in oVirt shell. Implement the serial-console command in oVirt shell. Optional: Implement a client program in vdsClient to consume the stream API. pros Same as solution 4 cons We can not allow ordinary user directly connect to VDSM and invoke the stream API, because there is no ACL in VDSM, once a client cert is setup for the ordinary user, he can call all the APIs in VDSM and get total control. So the ordinary user can only get the stream from Engine, and we leave Engine to do the ACL. I like solution 4 best. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] Functional Tests: Submit Some Tests for Review
Hi everyone, Recently I submit some vdsm functional tests for localfs storage and VM creation. If you have time, may I get some code review attention for these tests? They are under topic branch VMTests. You can find them on gerrit from the following URL. http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:VMTests,n,z The main idea of these patches is, firstly create a local storage backend, then create storage domains and pools, then become spm and create volumes, at last destroy them. The code can be extend to test NFS and iscsi storage backend. The VM creation test reuses the code from storage tests to create a storage layout, then boots the VM using host kernel and initramfs with the storage layout. How to verify 1 Build and install vdsm rpms 2 su 3 cd /usr/share/vdsm/tests 4 ./run_tests.sh functional/xmlrpcTests.py Note: Because of a bug https://bugzilla.redhat.com/show_bug.cgi?id=806774 storage domain UUID can not be reused even though the domain is destroyed. So if you run the tests twice, it will fail. VM creation test over localfs, and storage creation test over localfs use the same storage layout, same domain UUID, so we have to restart vdsmd manually between these two tests. I submit a patch to fix this bug at http://gerrit.ovirt.org/#/c/8144/ Thanks a lot! -- Best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [RFC]about the implement of text-based console
on 09/04/2012 22:19, Ryan Harper wrote: * Dan Kenigsberg dan...@redhat.com [2012-09-04 05:53]: On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: Hi, I submited a patch for text-based console http://gerrit.ovirt.org/#/c/7165/ the issue I want to discussing as below: 1. fix port VS dynamic port Use fix port for all VM's console. connect console with 'ssh vmUUID@ip -p port'. Distinguishing VM by vmUUID. The current implement was vdsm will allocated port for console dynamically and spawn sub-process when VM creating. In sub-process the main thread responsible for accept new connection and dispatch output of console to each connection. When new connection is coming, main processing create new thread for each new connection. Dynamic port will allocated port for each VM and use range port. It isn't good for firewall rules. so I got a suggestion that use fix port. and connect console with 'ssh vmuuid@hostip -p fixport'. this is simple for user. We need one process for accept new connection from fix port and when new connection is coming, spawn sub-process for each vm. But because the console only can open by one process, main process need responsible for dispatching console's output of all vms and all connection. So the code will be a little complex then dynamic port. So this is dynamic port VS fix port and simple code VS complex code. From a usability point of view, I think the fixed port suggestion is nicer. This means that a system administrator needs only to open one port to enable remote console access. If your initial implementation limits console access to one connection per VM would that simplify the code? Yes, using a fixed port for all consoles of all VMs seems like a cooler idea. Besides the firewall issue, there's user experience: instead of calling getVmStats to tell the vm port, and then use ssh, only one ssh call is needed. (Taking this one step further - it would make sense to add another layer on top, directing console clients to the specific host currently running the Vm.) I did not take a close look at your implementation, and did not research this myself, but have you considered using sshd for this? I suppose you can configure sshd to collect the list of known users from `getAllVmStats`, and force it to run a command that redirects VM's console to the ssh client. It has a potential of being a more robust implementation. I have considered using sshd and ssh tunnel. They can't implement fixed port and share console. Would you elaborate on that? Usually sshd listens to a fixed port 22, and allows multiple users to have independet shells. What do you mean by share console? Current implement we can do anything that what we want. Yes, it is completely under our control, but there are down sides, too: we have to maintain another process, and another entry point, instead of configuring a universally-used, well maintained and debugged application. Think of the security implications of having another remote shell access point to a host. I'd much rather trust sshd if we can make it work. Dan. At first glance, the standard sshd on the host is stronger and more robust than a custom ssh server, but the risk using the host sshd is high. If we implement this feature via host ssd, when a hacker attacks the sshd successfully, he will get access to the host shell. After all, the custom ssh server is not for accessing host shell, but just for forwarding the data from the guest console (a host /dev/pts/X device). If we just use a custom ssh server, the code in this server only does 1. auth, 2. data forwarding, when the hacker attacks, he just gets access to that virtual machine. Notice that there is no code written about login to the host in the custom ssh server, and the custom ssh server can be protected under selinux, only allowing it to access /dev/pts/X. In fact using a custom VNC server in qemu is as risky as a custom ssh server in vdsm. If we accepts the former one, then I can accepts the latter one. The consideration is how robust of the custom ssh server, and the difficulty to maintain it. In He Jie's current patch, the ssh auth and transport library is an open-source third-party project, unless the project is well maintained and well proven, using it can be risky. So my opinion is using neither the host sshd, nor a custom ssh server. Maybe we can apply the suggestion from Dan Yasny, running a standard sshd in a very small VM in every host, and forward data from this VM to other guest consoles. The ssh part is in the VM, then our work is just forwarding data from the VM via virto serial channels, to the guest via the pty. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397
[vdsm] Avoid testStressTest Failing in Jenkins VDSM Unit Tests Job
Hi all, Recently the oVirt Jenkins runs unit test for every new patch set in Gerrit. There are a lot of new patch sets every day, so the Jenkins may run the unit tests in parallel. I notice that most of the unit tests can be run in parallel except testStressTest. testStressTest creates lots of threads nearly to the system limit, so if we run two or more testStressTest, it will fail, giving false positives. So I suggest the oVirt Jenkins run most of the tests in parellel, but run testStressTest exlusively. With the help of the Exclusion-Plugin, it its possible to configure Jenkins to run some steps in parellel while some steps exclusive in one job. Firstly, add a resource with the help of Exclusion-Plugin. Give a meaningful name to that resource, and assign the resource to this Job. Secondly, add a build step of Execute Shell, in this step, run all the tests other than testStressTest. So the shell script can be as follow. cd tests NOSE_EXCLUDE=testStressTest \ ./run_tests_local.sh \ --with-xunit --xunit-file=nosetests0.xml \ *.py Then add a build step of Critical Block Start. Then add a build step of Execute Shell, in this step, only run testStressTest as follow. cd tests ./run_tests_local.sh -m testStressTest \ --with-xunit --xunit-file=nosetests1.xml \ resourceManagerTests.py Then add a build step of Critical BlockEnd. At last, in post-build actions, in the Publish Test Result Report section, modify the test report XMLs to tests/nosetests*.xml. Jenkins will merge the test reports. Some patch sets are marked as verify failure by oVirt Jenkins. Most of them fails in testStressTest. I hope this can help the oVirt Jenkins a little bit. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Jenkins failures due to vdscli.py missing
Hi, I will look into the problem. The build system is a bit complicated for me. Will try best. @shaohe feng: Could you give me any advice on this problem? on 06/28/2012 16:05, Dan Kenigsberg wrote: Since I've taken http://gerrit.ovirt.org/5093 Properly parse configurations in function do_create in vdsClient Jenkins's unit tests jobs are failing. See http://jenkins.ovirt.org/job/vdsm_unit_tests/242/console The reason is that `make check` does not create vdsm_cli/vdscli.py out of vdsm_cli/vdscli.py.in. Manually, things can be worked-around with make vdsm_cli/vdscli.py Could you try to fix automake so plain `make check` works? I would not like to revert the patch just for that. Regards, Dan. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] Jenkins failures due to vdscli.py missing
Hi, Yes. If a .py file is cleaned but the .pyc exists, the unit test still runs OK using the .pyc file. Those .pyc files will hide some problems in the patch. Cleaning .pyc files can strengthen the build system. I can have a try on writing a new patch to do this. on 06/28/2012 18:14, Dan Kenigsberg wrote: On Thu, Jun 28, 2012 at 05:36:22PM +0800, Zhou Zheng Sheng wrote: Hi, I submit a fix at http://gerrit.ovirt.org/5765 Thanks for the quick fix! Taken. I suppose we should clean up those .pyc files on `make clean`. Right? Dan. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] vdsm vs selinux
Hi,
I had the same problem before. I investigated the problem, and found the
selinux label for some of the directories under /rhev/data-center
hierarchy were wrong. Then I inspected the policies for
/rhev/data-center hierachy and found no problem. So I uninstalled vdsm,
delete /rhev, reinstalled vdsm, then the problem was gone. Since
policies for /rhev/data-center are correct, I have no ideas on how the
wrong labels were tagged. I examined log files of selinux, libvirt and
vdsm, still could not find out any clues.
You can examine the policies on /rhev by running the following command:
semanage fcontext -l | grep '^/rhev'
This should give the result as:
/rhev directory
system_u:object_r:mnt_t:s0
/rhev(/[^/]*)? directory
system_u:object_r:mnt_t:s0
/rhev/[^/]*/.* all filesNone
Then you can examine the actual labels of subdirs under
/rhev/data-center by running:
ls -dZ /rhev/data-center/8c369da4-b3a0-11e1-9db0-273609afe0b1
ls -dZ
/rhev/data-center/8c369da4-b3a0-11e1-9db0-273609afe0b1/efef4a96-16b1-4f14-a252-f33c7a8ce52b
ls -dZ /rhev/data-center/mnt
ls -Z /rhev/data-center/mnt/*
You should see mnt_t label on the directories and the soft links. The
labels of the soft links are inherited from its parent dir. In my
previous selinux problem, some of the dirs and soft links were tagged
with default_t. So selinux will prevent qemu from visiting the soft links.
Anyway, deleting /rhev and re-installing vdsm solved my previous
problem, because vdsm will create /rhev automatically with the right
labels. Hope it works for you too.
on 06/18/2012 23:37, Saggi Mizrahi wrote:
Do you have an AVC denial in the audit log? What does it say?
(Please run sealert -a FILE and put the resolved text along with the original
AVC denail)
Are you using NFS\localfs\SAN?
What are the credentials and contexts of the files in question?
Have you recently turned selinux on\off?
Did you upgrade the OS or selinux policy?
What is the libvirt version?
- Original Message -
From: Laszlo Hornyaklhorn...@redhat.com
To: vdsm-devel@lists.fedorahosted.org
Sent: Monday, June 18, 2012 11:13:37 AM
Subject: [vdsm] vdsm vs selinux
hi,
I am running the latest VDSM (built from git repo) on rhel 6.2 and
looks like it has some issues with selinux. setenforce 0 solves the
problem, but is there a proper solution under way?
Traceback (most recent call last):
File /usr/share/vdsm/vm.py, line 570, in _startUnderlyingVm
self._run()
File /usr/share/vdsm/libvirtvm.py, line 1364, in _run
self._connection.createXML(domxml, flags),
File
/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py,
line 82, in wrapper
ret = f(*args, **kwargs)
File /usr/lib64/python2.6/site-packages/libvirt.py, line 2490, in
createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed',
conn=self)
libvirtError: internal error Process exited while reading console log
output: char device redirected to /dev/pts/2
qemu-kvm: -drive
file=/rhev/data-center/8c369da4-b3a0-11e1-9db0-273609afe0b1/efef4a96-16b1-4f14-a252-f33c7a8ce52b/images/40d2cc3a-9e9c-4224-af6f-2450efc883ca/e84617c5-8073-46de-85bd-2497235a5ba2,if=none,id=drive-virtio-disk0,format=raw,serial=40d2cc3a-9e9c-4224-af6f-2450efc883ca,cache=none,werror=stop,rerror=stop,aio=threads:
could not open disk image
/rhev/data-center/8c369da4-b3a0-11e1-9db0-273609afe0b1/efef4a96-16b1-4f14-a252-f33c7a8ce52b/images/40d2cc3a-9e9c-4224-af6f-2450efc883ca/e84617c5-8073-46de-85bd-2497235a5ba2:
Permission denied
Thank you,
Laszlo
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
--
Thanks and best regards!
Zhou Zheng Sheng / 周征晟
E-mail: zhshz...@linux.vnet.ibm.com
Telephone: 86-10-82454397
___
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] some questions about selinux policy on VDSM storage
the least privilege it needs, but I'm not sure if the policies are too permissive. 2. Does anyone ever have the same problem? Am I solving it in a right way? 3. Since the soft links are created by vdsm, this seems like a bug. If I want to submit a patch for this bug, which project should I submit those new policies: selinux, libvirt or vdsm? -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] SSLError with vdsm
Hi, It is because normal user do not have the privilege to access the keys in /etc/pki/vdsm/keys/ and certificates in /etc/pki/vdsm/certs/. You can su to root or sudo vdsClient to use SSL connection. 于 2012年06月07日 13:03, Wenyi Gao 写道: Hi guys, When I ran the cmmand vdsClient -s 0 getVdsCaps, I got the following error: $ vdsClient -s 0 getVdsCaps Traceback (most recent call last): File /usr/share/vdsm/vdsClient.py, line 2275, in module code, message = commands[command][0](commandArgs) File /usr/share/vdsm/vdsClient.py, line 403, in do_getCap return self.ExecAndExit(self.s.getVdsCapabilities()) File /usr/lib64/python2.7/xmlrpclib.py, line 1224, in __call__ return self.__send(self.__name, args) File /usr/lib64/python2.7/xmlrpclib.py, line 1578, in __request verbose=self.__verbose File /usr/lib64/python2.7/xmlrpclib.py, line 1264, in request return self.single_request(host, handler, request_body, verbose) File /usr/lib64/python2.7/xmlrpclib.py, line 1292, in single_request self.send_content(h, request_body) File /usr/lib64/python2.7/xmlrpclib.py, line 1439, in send_content connection.endheaders(request_body) File /usr/lib64/python2.7/httplib.py, line 954, in endheaders self._send_output(message_body) File /usr/lib64/python2.7/httplib.py, line 814, in _send_output self.send(msg) File /usr/lib64/python2.7/httplib.py, line 776, in send self.connect() File /usr/lib/python2.7/site-packages/vdsm/SecureXMLRPCServer.py, line 98, in connect cert_reqs=self.cert_reqs) File /usr/lib64/python2.7/ssl.py, line 381, in wrap_socket ciphers=ciphers) File /usr/lib64/python2.7/ssl.py, line 141, in __init__ ciphers) SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib But if I set ssl = false in /etc/vdsm/vdsm.conf, then run vdsClient 0 getVdsCaps, the problem goes away. Does anyone know what causes the problem above? Thanks. Wenyi Gao ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] pep8 questions
Hi, I think a space is needed after %s/%s, just like follow: cls.log.warn(Could not get size for vol %s/%s using optimized methods, sdobj.sdUUID, volUUID, exc_info=True) 于 2012年06月06日 04:11, Saggi Mizrahi 写道: cls.log.warn(Could not get size for vol %s/%s using optimized methods, sdobj.sdUUID, volUUID, exc_info=True) -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshz...@linux.vnet.ibm.com Telephone: 86-10-82454397 ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
