Re: [openstack-dev] Make libguestfs available on pypi
On Fri, Oct 16, 2015 at 01:40:56PM +0100, Richard W.M. Jones wrote: > On Fri, Oct 16, 2015 at 12:58:20PM +0100, Matt Thompson wrote: > > Hi All, > > > > Does anyone know if any progress has been made here? We're doing something > > similar to Kris and it'd be great if libguestfs could be installed directly > > from pypi. > > Good and bad news. No movement on the PyPi sign-up / licensing issue > as far as I'm aware, so I'm still unable to agree to uploading the > Python bindings to PyPi. [I tried to sign up again just now, but the > email is taking a very long time to come through - will update here if > there is any change] Nope - the soul-sucking registration page is still there, so I'm unable to agree to uploading the python bindings to PyPi. The onus is still (for > 1 year) on the Python Foundation to fix this. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Make libguestfs available on pypi
On Fri, Oct 16, 2015 at 01:40:56PM +0100, Richard W.M. Jones wrote: > On Fri, Oct 16, 2015 at 12:58:20PM +0100, Matt Thompson wrote: > > Hi All, > > > > Does anyone know if any progress has been made here? We're doing something > > similar to Kris and it'd be great if libguestfs could be installed directly > > from pypi. > > Good and bad news. No movement on the PyPi sign-up / licensing issue > as far as I'm aware, so I'm still unable to agree to uploading the > Python bindings to PyPi. [I tried to sign up again just now, but the > email is taking a very long time to come through - will update here if > there is any change] It looks as if I managed to sign up to the PyPi account without the soul-sucking licensing agreement. As a result, I'll be able to upload the Python bindings in the next few days. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Make libguestfs available on pypi
On Fri, Oct 16, 2015 at 12:58:20PM +0100, Matt Thompson wrote: > Hi All, > > Does anyone know if any progress has been made here? We're doing something > similar to Kris and it'd be great if libguestfs could be installed directly > from pypi. Good and bad news. No movement on the PyPi sign-up / licensing issue as far as I'm aware, so I'm still unable to agree to uploading the Python bindings to PyPi. [I tried to sign up again just now, but the email is taking a very long time to come through - will update here if there is any change] However the good news is I fixed libguestfs so that you can now build the Python bindings separate from the C library. See this commit for details: https://github.com/libguestfs/libguestfs/commit/bb7d088edaa2ea3a664ae703e002f6297dd1844f That commit was also backported to 1.30.3 on the stable-1.30 branch. For instructions on how to actually build a PIP module, see: https://github.com/libguestfs/libguestfs/commit/fcbfc4775fa2a44020974073594a745ca420d614 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Make libguestfs available on pypi
On Thu, Jul 30, 2015 at 11:14:20AM +1000, Ian Wienand wrote: > On 07/30/2015 04:55 AM, Kris G. Lindgren wrote: > > The following bug has already been created over a year ago [1], and > > it looks like most of the work on the libguestfs side is already > > done [2]. It seems something about a complaint of licensing per > > the bug report. > > I think best to follow up in that bug > > On the license front, to quote from an internal email I > saw fly-by about the pypi sign-up terms under question from Nick on > 20-Jul-2015: > > --- > Van started drafting some amendments back in February: > https://bitbucket.org/vanl/pypi/commits/all > > Key changes are here: > > https://bitbucket.org/vanl/pypi/commits/8df8e0295c0a719e963f7c3ce430284179f03b1f > > Further clarifications at > > https://bitbucket.org/vanl/pypi/commits/734b1f49776d1f7f5d0671306f61a90aad713e5d > and > > https://bitbucket.org/vanl/pypi/commits/0e94b169e81306607936912ecc3c42312aac5eb7 > > I'll ping the Board list about next steps in getting those amendments > formally approved and submitted as a PR to the main PyPI repo. > --- > > So it is being looked at, but I'm not sure of the time-frame. Yup, you beat me to it. Earlier this month I asked the Python Software Foundation to look again at their terms, and they have agreed to make some changes. It's my understanding that it is waiting on the PSF Board to approve changes. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?
On Mon, Feb 23, 2015 at 11:08:31AM +0100, Raphael Glon wrote: > sudo sysctl -w fs.protected_hardlinks=0 + common user nova/qemu We fixed this a while back (in July 2013 in fact). I think if you forked at Fedora 19 then you're probably using libguestfs 1.24 + supermin 4. I'd definitely recommend updating to at least libguestfs >= 1.26 + supermin 5 since that fixes a bunch of bugs around building the appliance, and is also faster. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?
On Wed, Feb 18, 2015 at 07:23:52PM +0100, Raphael Glon wrote: > I entcountered a similar case more recently on powerkvm 2.1.0 > (defect with the libguestfs) What's the actual bug? We've worked hard, with IBM, to make libguestfs work on POWER 7 and POWER 8 systems. I have full access to those systems through Red Hat. If there's a new bug I'm sure we'll be able to fix it. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Deprecating localfs?
On Tue, Sep 23, 2014 at 09:53:36AM +1000, Michael Still wrote: > Hi. > > I know we've been talking about deprecating nova.virt.disk.vfs.localfs > for a long time, in favour of wanting people to use libguestfs > instead. However, I can't immediately find any written documentation > for when we said we'd do that thing. > > Additionally, this came to my attention because Ubuntu 14.04 is > apparently shipping a libguestfs old enough to cause us to emit the > "falling back to localfs" warning, so I think we need Ubuntu to catch > up before we can do this thing. > > So -- how about we remove localfs early in Kilo to give Canonical a > release to update libguestfs? A few randomly related points: - libguestfs 1.26 in Debian (and eventually in Ubuntu) finally gets rid of 'update-guestfs-appliance'. - Unfortunately Ubuntu still has the kernel permissions bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725 Fedora/RHEL has none of these issues. - There are a couple of easy to fix bugs that would greatly improve libguestfs usability in OpenStack: (1) Don't throw away debugging information: https://bugs.launchpad.net/nova/+bug/1279857 (2) [Don't think there's a bug# for this] The libvirt_inject_partition parameter doesn't adequately model what libguestfs can do for guests. Plus it's a global setting and ought to be a glance setting (or per disk/per template anyway). libguestfs has a rich API for inspecting guests, and that cannot be modelled in a single integer. http://libguestfs.org/guestfs.3.html#inspection Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction as part of resize ?
On Wed, Jun 18, 2014 at 11:05:01AM +, Day, Phil wrote: > > -Original Message- > > From: Russell Bryant [mailto:rbry...@redhat.com] > > Sent: 17 June 2014 15:57 > > To: OpenStack Development Mailing List (not for usage questions) > > Subject: Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction > > as part of resize ? > > > > On 06/17/2014 10:43 AM, Richard W.M. Jones wrote: > > > On Fri, Jun 13, 2014 at 06:12:16AM -0400, Aryeh Friedman wrote: > > >> Theoretically impossible to reduce disk unless you have some really > > >> nasty guest additions. > > > > > > True for live resizing. > > > > > > For "dead" resizing, libguestfs + virt-resize can do it. Although I > > > wouldn't necessarily recommend it. In almost all cases where someone > > > wants to shrink a disk, IMHO it is better to sparsify it instead (ie. > > > virt-sparsify). > > > > FWIW, the resize operation in OpenStack is a dead one. > > > Dead as in "not supported in V3" ? "dead" as in not live resizing, ie. it happens only on offline disk images. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction as part of resize ?
On Tue, Jun 17, 2014 at 10:56:36AM -0400, Russell Bryant wrote: > On 06/17/2014 10:43 AM, Richard W.M. Jones wrote: > > On Fri, Jun 13, 2014 at 06:12:16AM -0400, Aryeh Friedman wrote: > >> Theoretically impossible to reduce disk unless you have some really nasty > >> guest additions. > > > > True for live resizing. > > > > For "dead" resizing, libguestfs + virt-resize can do it. Although I > > wouldn't necessarily recommend it. In almost all cases where someone > > wants to shrink a disk, IMHO it is better to sparsify it instead > > (ie. virt-sparsify). > > FWIW, the resize operation in OpenStack is a dead one. In >= 1.26, `virt-sparsify --in-place' is very fast, doesn't copy, and doesn't need mountains of temporary space (unlike the copying mode virt-sparsify). http://libguestfs.org/virt-sparsify.1.html#in-place-sparsification Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction as part of resize ?
On Fri, Jun 13, 2014 at 06:12:16AM -0400, Aryeh Friedman wrote: > Theoretically impossible to reduce disk unless you have some really nasty > guest additions. True for live resizing. For "dead" resizing, libguestfs + virt-resize can do it. Although I wouldn't necessarily recommend it. In almost all cases where someone wants to shrink a disk, IMHO it is better to sparsify it instead (ie. virt-sparsify). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] nova-compute vfsguestfs
On Fri, Jun 13, 2014 at 03:06:25PM +0530, abhishek jain wrote: > Hi Rich > > Can you help me regarding the possible cause for VM stucking at spawning > state on ubuntu powerpc compute node in openstack using devstack. Did you solve this one? It's impossible to debug unless you collect the full debugging information. See also: http://libguestfs.org/guestfs-faq.1.html#how-do-i-debug-when-using-the-api https://bugs.launchpad.net/nova/+bug/1279857 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] nova-compute deadlock
On Sat, May 31, 2014 at 01:25:04AM +0800, Qin Zhao wrote: > Hi all, > > When I run Icehouse code, I encountered a strange problem. The nova-compute > service becomes stuck, when I boot instances. I report this bug in > https://bugs.launchpad.net/nova/+bug/1313477. > > After thinking several days, I feel I know its root cause. This bug should > be a deadlock problem cause by pipe fd leaking. I draw a diagram to > illustrate this problem. > https://docs.google.com/drawings/d/1pItX9urLd6fmjws3BVovXQvRg_qMdTHS-0JhYfSkkVc/pub?w=960&h=720 > > However, I have not find a very good solution to prevent this deadlock. > This problem is related with Python runtime, libguestfs, and eventlet. The > situation is a little complicated. Is there any expert who can help me to > look for a solution? I will appreciate for your help! Thanks for the useful diagram. libguestfs itself is very careful to open all file descriptors with O_CLOEXEC (atomically if the OS supports that), so I'm fairly confident that the bug is in Python 2, not in libguestfs. Another thing to say is that g.shutdown() sends a kill 9 signal to the subprocess. Furthermore you can obtain the qemu PID (g.get_pid()) and send any signal you want to the process. I wonder if a simpler way to fix this wouldn't be something like adding a tiny C extension to the Python code to use pipe2 to open the Python pipe with O_CLOEXEC atomically? Are we allowed Python extensions in OpenStack? BTW do feel free to CC libgues...@redhat.com on any libguestfs problems you have. You don't need to subscribe to the list. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] nova-compute vfsguestfs
On Tue, May 27, 2014 at 03:25:10PM +0530, abhishek jain wrote:
> Hi Daniel
>
> Thanks for the help.
> The end result of my setup is that the VM is stucking at Spawning state on
> my compute node whereas it is working fine on the controller node.
> Therefore I'm comparing nova-compute logs of both compute node as well as
> controller node and trying to proceed step by step.
> I'm having all the above packages enabled
>
> Do you have any idea regarding reason for VM stucking at spawning state.
The most common reason is that nested virt is broken. libguestfs is the canary
in the mine here, not the cause of the problem.
Rich.
>
>
> On Tue, May 27, 2014 at 2:38 PM, Daniel P. Berrange
> wrote:
>
> > On Tue, May 27, 2014 at 12:04:23PM +0530, abhishek jain wrote:
> > > Hi
> > > Below is the code to which I'm going to reffer to..
> > >
> > > vim /opt/stack/nova/nova/virt/disk/vfs/api.py
> > >
> > > #
> > >
> > > try:
> > > LOG.debug(_("Trying to import guestfs"))
> > > importutils.import_module("guestfs")
> > > hasGuestfs = True
> > > except Exception:
> > > pass
> > >
> > > if hasGuestfs:
> > > LOG.debug(_("Using primary VFSGuestFS"))
> > > return importutils.import_object(
> > > "nova.virt.disk.vfs.guestfs.VFSGuestFS",
> > > imgfile, imgfmt, partition)
> > > else:
> > > LOG.debug(_("Falling back to VFSLocalFS"))
> > > return importutils.import_object(
> > > "nova.virt.disk.vfs.localfs.VFSLocalFS",
> > > imgfile, imgfmt, partition)
> > >
> > > ###
> > >
> > > When I'm launching VM from the controller node onto compute node,the
> > > nova compute logs on the compute node displays...Falling back to
> > > VFSLocalFS and the result is that the VM is stuck in spawning state.
> > > However When I'm trying to launch a VM onto controller node form the
> > > controller node itself,the nova compute logs on the controller node
> > > dislpays ...Using primary VFSGuestFS and I'm able to launch VM on
> > > controller node.
> > > Is there any module in the kernel or any package that i need to
> > > enable.Please help regarding this.
> >
> > VFSGuestFS requires the libguestfs python module & corresponding native
> > package to be present, and only works with KVM/QEMU enabled hosts.
> >
> > VFSLocalFS requires loopback module, nbd module, qemu-nbd, kpartx and
> > a few other misc host tools
> >
> > Neither of these should cause a VM getting stuck in the spawning
> > state, even if stuff they need is missing.
> >
> > Regards,
> > Daniel
> > --
> > |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/:|
> > |: http://libvirt.org -o- http://virt-manager.org:|
> > |: http://autobuild.org -o- http://search.cpan.org/~danberr/:|
> > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc:|
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bp proposal: libvirt-resize-disk-down
On Thu, Jan 30, 2014 at 02:59:45PM +, sahid wrote: > Greetings, > > A blueprint is being discussed about the disk resize down feature of libvirt > driver. > https://blueprints.launchpad.net/nova/+spec/libvirt-resize-disk-down > > The current implementation does not handle disk resize down and just skips the > step during a resize down of the instance. I'm really convinced we can > implement > this feature by using the good job of disk resize down of the driver xenapi. resize2fs -M is problematic as another reply mentions. virt-sparsify is designed to handle this case properly. It currently works by copying the disk image, but it should soon work in-place too (waiting on some qemu command line changes). And incidentally, virt-resize can handle the offline growing case well too. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] log message translations
On Mon, Jan 27, 2014 at 05:58:20PM +, Daniel P. Berrange wrote: > On Mon, Jan 27, 2014 at 12:42:28PM -0500, Doug Hellmann wrote: > > We have a blueprint open for separating translated log messages into > > different domains so the translation team can prioritize them differently > > (focusing on errors and warnings before debug messages, for example) [1]. > > > Feedback? > > > [1] > > https://blueprints.launchpad.net/oslo/+spec/log-messages-translation-domain > > IMHO we've created ourselves a problem we don't need to have in the first > place by trying to translate every single log message. It causes pain for > developers & vendors because debug logs from users can in any language > which the person receiving will often not be able to understand. It creates > pain for translators by giving them an insane amount of work todo, which > never ends since log message text is changed so often. Now we're creating > yet more pain & complexity by trying to produce multiple log domains to solve > a problem of havin some many msgs to translate. I accept that some people will > like translated log messages, but I don't think this is a net win when you > look at the overall burden they're imposing. Also it impedes using search engines to look up the causes of error messages. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Disabling file injection *by default*
On Wed, Jan 22, 2014 at 10:57:29AM +1300, Robert Collins wrote: [...] I'm a bit surprised that file injection is on by default. I thought it was disabled by default upstream. (Just checked and it is enabled as you say.) So yes, file injection should be off by default, but read below. > There's nothing wrong with libguestfs, this is about the feature which > has been discussed, here, a lot :) - for delivering metadata to > images, config-drive || metadata service are much better. I generally agree that it's nicer to use cloud-init etc instead of injection, although some (declining) number of guests that people want to run might not have cloud-init. The current file injection setting is tricky from the libguestfs point of view because all we have is this "inject_partition" integer ... per Nova instance! The knob makes no sense since libguestfs can inspect guests, and it definitely makes no sense that the user can't set it when uploading a guest to glance or starting a guest. [Or is this possible? I've never found a way] This single "partition" setting is a hang-over from some really ancient code that predates libguestfs file injection, and we just reused and overloaded the same setting. "inject_password" similarly. > Hypervisors shouldn't be in the business of tinkering inside VM file > systems at all. Yes and no. In theory there should be a clean separation. In practice libguestfs lets you do some wonderful things based on tinkering inside VMs :-) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [libvirt] [nova] live-snapshot/cloning of virtual machines
On Fri, Aug 16, 2013 at 11:05:19AM +0100, Daniel P. Berrange wrote: > On Wed, Aug 14, 2013 at 04:53:01PM -0700, Vishvananda Ishaya wrote: > > Hi Everyone, > > > > I have been trying for some time to get the code for the live-snapshot > > blueprint[1] > > in. Going through the review process for the rpc and interface code[2] was > > easy. I > > suspect the api-extension code[3] will also be relatively trivial to get > > in. The > > main concern is with the libvirt driver implementation[4]. I'd like to > > discuss the > > concerns and see if we can make some progress. > > > > Short Summary (tl;dr) > > = > > > > I propose we merge live-cloning as an experimental feature for havanna and > > have the > > api extension disabled by default. > > > > Overview > > > > > > First of all, let me express the value of live snapshoting. The > > slowest part of the vm provisioning process is generally booting > > of the OS. Like Dan I'm dubious about this whole plan. But this ^^ statement in particular. I would like to see hard data to back this up. You should be able to boot an OS pretty quickly, and furthermore it's (a) much safer for all the reasons Dan outlines, and (b) improvements that you make to boot times help everyone. [...] > > 2. Security Concerns > > > > > > There are a number of security issues with loading state from another vm. > > Here is a > > short list of things that need to be done just to make a cloned vm usable: > > > > a) mac address needs to be recreated > > b) entropy pool needs to be reset > > c) host name must be reset > > d) host keys bust be regenerated > > > > There are others, and trying to clone a running application as well may > > expose other > > sensitive data, especially if users are snaphsoting vms and making them > > public. Are we talking about cloning VMs that you already trust, or cloning random VMs and allowing random other users to use them? These would lead to very different solutions. In the first case, you only care about correctness, not security. In the second case, you care about security as well as correctness. I highly doubt the second case is possible because scrubbing the disk is going to take far too long for any supposed time-saving to matter. As Dan says, even the first case is dubious because it won't be correct. > The libguestfs project provide tools to perform offline cloning of > VM disk images. Its virt-sysprep knows how to delete alot (but by > no means all possible) sensitive file data for common Linux & > Windows OS. It still has to be combined with use of the > virt-sparsify tool though, to ensure the deleted data is actually > purged from the VM disk image as well as the filesystem, by > releasing all unused VM disk sectors back to the host storage (and > not all storage supports that). Links to the tools that Dan mentions: http://libguestfs.org/virt-sysprep.1.html http://libguestfs.org/virt-sparsify.1.html Note these tools can only be used on offline machines. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
