Hi Daniel, On Fri, 2015-10-23 at 12:05 +0100, Daniel P. Berrange wrote: > On Thu, Oct 22, 2015 at 06:01:42PM +0000, Marc Heckmann wrote: > > Hi, > > > > On Thu, 2015-10-22 at 08:17 -0700, Abel Lopez wrote: > > > I've actually looked for this for our RBD backed ephemeral instances, > > > but found the options lacking. I last looked in Juno. > > > > > > On Thursday, October 22, 2015, Tim Bell <tim.b...@cern.ch> wrote: > > > > > > > > > Has anyone had experience with setting up Nova with KVM so it > > > has raw ephemeral disks but qcow2 images for the VMs ? We’ve > > > got very large ephemeral disks and could benefit from the > > > performance of raw volumes for this. > > > > We looked into this for the very same reasons and it doesn't seem to be > > supported. > > > > That being said, I'm fearful of the boot time performance impact of > > using RAW for ephemeral. > > There should be no performance impact of using fully pre-allocated > raw images. Any decent modern filesystem (ext4, xfs) supports fallocate > which allows you to pre-allocate an arbitrary sizes plain file in > constant time.
I was actually more afraid of the time it would take mkfs.<ext4|ntfs> to create the filesystem. I was thinking that it might be non-negligible on larger ephemeral sizes (approaching 1TB) going up to a minute or more. I just tested this and it doesn't seem to be an issue at all (obviously --fast is used for NTFS). This would seem to make a setting to use raw only for ephemeral slightly more attractive. That being said, if Nova gets fixed to pre-initialize qcow2 internally, then the raison-d'être for raw might become less important. I'm assuming the Nova fix for qcow2 would be easier to implement? Any downsides to pre-initialized qcow2? In any case, thanks for the useful info! > > By default Nova does *not* preallocate images - so both raw & qcow2 > will grow-on-demand as guest writes to sectors. > > If the "preallocate_images" nova.conf option is set to "space", then > Nova will call fallocate for both raw & qcow2 images to fully allocate > the maximum space they require. There's no appreciable time overhead > for this - it just prevents you overcommitting disk obviously. > > If you fully preallocate a qcow2 image its performance should pretty > much match raw images (modulo the l2-cache-size item mentioned > below), unfortunately, the way Nova is preallocating qcow2 images is > wrong - it preallocates the space on disk, but does not pre-initialize > the internal qcow2 data structures to match :-( So we need to fix > that for qcow2 in Nova. > > > I suggest you check out the following presentation about qcow2 > > performance if you haven't already done so: > > > > http://events.linuxfoundation.org/sites/events/files/slides/p0.pp_.pdf > > > > I think it would be worthwhile for Openstack (and libvirt if required) > > to support the "l2-cache-size" option for qcow2. > > Yep, we should look at supporting that. > > > Regards, > Daniel _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators