Wido,
Thanks, i will give it a try when I have a moment. Andrei ----- Original Message ----- From: "Wido den Hollander" <w...@widodh.nl> To: dev@cloudstack.apache.org Sent: Monday, 24 March, 2014 3:38:57 PM Subject: Re: ACS and KVM uses /tmp for volumes migration and templates On 03/24/2014 03:22 PM, Andrei Mikhailovsky wrote: > > Do you think I can apply the patch manually to the 4.3 branch? I would love > to try it with 4.3, but not too adventitious to upgrade my setup to 4.4 yet > )) > Yes! I just pushed a commit to the master branch: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=9763faf85e3f54ac84d5ca1d5ad6e89c7fcc87ee To build 4.3 $ git checkout 4.3 $ git cherry-pick 9763faf85e3f54ac84d5ca1d5ad6e89c7fcc87ee $ dpkg-buildpackage Now you only have to update the cloudstack-agent package on the hypervisors. Wido > > Andrei > ----- Original Message ----- > > From: "Wido den Hollander" <w...@widodh.nl> > To: dev@cloudstack.apache.org > Sent: Monday, 24 March, 2014 12:29:36 PM > Subject: Re: ACS and KVM uses /tmp for volumes migration and templates > > On 03/23/2014 08:01 PM, Andrei Mikhailovsky wrote: >> Wido, >> >> Could you please let me know when you've done this so I could try it out. >> Would it be a part of the 4.3 branch or 4.4? >> > > I'll do that. It will go into master which is 4.4 and I'm not sure if > this will be backported to 4.3.1 > > Wido > >> Thanks >> ----- Original Message ----- >> >> From: "Wido den Hollander" <w...@widodh.nl> >> To: dev@cloudstack.apache.org >> Sent: Sunday, 23 March, 2014 3:56:44 PM >> Subject: Re: ACS and KVM uses /tmp for volumes migration and templates >> >> >> >> On 03/21/2014 02:23 PM, Andrei Mikhailovsky wrote: >>> >>> Wido, >>> >>> >>> i would be happy to try the custom ACS build unless 4.3 comes out soon. It >>> has been overdue for sometime now )). Has this feature been addressed in >>> the 4.3 release? >>> >> >> No, it hasn't been fixed yet. I have to admit, I forgot about this until >> you sent this e-mail to the list. >> >> I'll fix this in master later this week. >> >>> >>> I can leave with this feature for the time being, but i do see a longer >>> term issue when my volumes become large as i've only got about 100gb free >>> space on my host servers. >>> >>> >> >> I fully agree. While writing this code I was aware of this. See my >> comments in the code: >> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java;h=5de8bd26ae201187f5db5fd16b7e3ca157cab53a;hb=master#l1087 >> >> >>> From what i can tell by looking at the rbd ls -l info all of my volumes are >>> done in Format 2 >>> >> >> Correct, because I by-pass libvirt and Qemu at some places right now. >> >>> >>> Cheers, >>> >>> >>> Andrei >>> >>> >>> >>> >>> ----- Original Message ----- >>> >>> From: "Wido den Hollander" <w...@widodh.nl> >>> To: dev@cloudstack.apache.org >>> Sent: Thursday, 20 March, 2014 9:40:29 AM >>> Subject: Re: ACS and KVM uses /tmp for volumes migration and templates >>> >>> On 03/20/2014 12:59 AM, Andrei Mikhailovsky wrote: >>>> Hi guys, >>>> >>>> I was wondering if this is a bug? >>>> >>> >>> No, it's a "feature". >>> >>>> I've noticed that during volume migration from NFS to RBD primary storage >>>> the volume image is first copied to /tmp and only then to the RBD storage. >>>> This seems silly to me as one would expect a typical volume to be larger >>>> than the host's hard disk. Also, it is a common practice to use tmpfs as >>>> /tmp for performance reasons. Thus, a typical host server will have far >>>> smaller /tmp folder than the size of an average volume. As a result, >>>> volume migration would break after filling the /tmp and could probably >>>> cause a bunch of issue for the KVM host itself as well as any vms running >>>> on the server. >>>> >>> >>> Correct. The problem was that RBD images know two formats. Format 1 >>> (old/legacy) and format 2. >>> >>> In order to perform cloning images should be in RBD format 2. >>> >>> When running qemu-img convert with a RBD image as a destination qemu-img >>> will create a RBD image in format 1. >>> >>> That's due to this piece of code in block/rbd.c in Qemu: >>> >>> ret = rbd_create(io_ctx, name, bytes, &obj_order); >>> >>> rbd_create() creates images in format 1. To use format 2 you should use >>> rbd_create2() or rbd_create3(). >>> >>> With RBD format 1 we can't do snapshotting or cloning, which we require >>> in ACS. >>> >>> So I had to do a intermediate step where I first wrote the RAW image >>> somewhere and afterwards write it to RBD. >>> >>> After some discussion a config option has been added to Ceph: >>> >>> OPTION(rbd_default_format, OPT_INT, 1) >>> >>> This allows me to do this: >>> >>> qemu-img convert .. -O raw .. rbd:rbd/myimage:rbd_default_format=2 >>> >>> This causes librbd/RBD to create a format 2 image and we can skip the >>> convert step to /tmp. >>> >>> This option is available since Ceph Dumpling 0.67.5 and was not >>> available when ACS 4.2 was written. >>> >>> I'm going to make changes in master which skip the step with /tmp. >>> >>> Technically this can be backported to 4.2, but then you would have to >>> run your own homebrew version of 4.2 >>> >>>> It also seems that the /tmp is temporarily used during a template creation >>>> . >>>> >>> >>> Same story as above. >>> >>>> My setup: >>>> >>>> ACS 4.2.1 >>>> Ubuntu 12.04 with KVM >>>> RBD + NFS for Primary storage >>>> NFS for Staging and Secondary storage >>>> >>>> >>>> Thanks >>>> >>>> Andrei >>>> >>> >>> >>> >> >> > > >
