Hi, Indeed, you fill up the last blocks that are not big enough for my defined blocksize. But those blocks are very likely to be empty, also the loss of max 1024k - 1 of datastorage isn't really bothering me.
The requirement to fill out the empty space with zeroes and then take down the vm to create a new qcow2 image is what bothers me. I wish to reclaim the empty space from a qcow2 image in a more efficient way, preferably without creating a new qcow2 image. (no downtime would be a nice bonus) Rob 2013/8/20 Tony Su <[email protected]>: > Haven't tried the following to zero a virtual disk, but the following should > be fast, make the bs as big as you want (maybe 10x for very large disks). Of > course, the bigger you make zero.big.file the longer it <might> take to > create zero.file. My code zeroes all bytes in 2 steps whereas yours doesn't > zero all. > > dd if=/dev/zero of=zero.big.file bs=1024 count=102400 > cat /dev/zero > zero.file > rm zero.big.file > rm zero.file > > Tony > > On Aug 20, 2013 1:39 AM, "Rob Verduijn" <[email protected]> wrote: >> >> Hi, >> >> Using separate images for partitions makes the script a tiny bit less >> complex. >> >> I've already been scripting the exercise, the use off multiple >> partition images makes it only a tiny bit more complex. >> qemu-nbd simply adds another device for each additional partition. >> >> #!/binbash >> modprobe nbd max_part=16 #number off partitions >> qemu-nbd -c --nocache --aio=native /path/to/image.qcow2 >> mount /dev/nbd0 /mnt # I didn't partiotion the image, just >> formated the image as ext4 >> dd if=/dev/zero of=/mnt/bigfile bs=1024k; sync; rm /mnt/bigfile; rm >> bigfile; sync; rm /mnt/bigfile; sync >> sleep 2 >> umount /dev/nbd0 # umount the device >> qemu-nbd /dev/nbd0 # clean up nbd devices or they will bite you >> qemu-img convert -c -O qcow2 /path/to/image.qcow2 /path/to/shrunk.qcow2 >> mv /path/to/image.qcow2 /path/to/image.qcow2.bak >> mv /path/to/shrunk.qcow2 /path/to/image.qcow2 >> >> The performance of qemu-nbd is rather poor, using writeback cache is >> not really an option since you always need to wait for the zeroes to >> be actually written to the hd. >> Also writeback is very hazardous if you use a script to umount and >> disconnect the nbd device, image corruption is very likely to happen >> since sync doesn't apply to nbd0 devices and blockdev --flushbufs >> /dev/nbd0 isn't foolproof either when scripting. >> (ctrl-c the script at the wrong time and you are in for a recovery) >> >> So the script isn't that difficult, neither is the use of multiple >> partitions in an image. >> The biggest drawback is, downtime, filling the image with zeroes takes >> even longer offline than online. >> Ok you could have a cronjob in the vm that does that nightly, but I >> can imagine issues when the hd of a vm is at filled to the max at >> regular intervals (once a month ??), even at expected times and only >> for a short moment. >> Also that would mean the shrinking cronjob is required on the host (I >> want the shrinking done as soon as possible after the zeroing) >> This has to be timed properly with the zero cronjob of the guest, this >> becomes rather complex with every additional guest. >> >> Regards >> Rob >> >> >> 2013/8/19 Tony Su <[email protected]>: >> > Not specific to qcow >> > In the past if I wanted to partition files I'd deploy multiple disks >> > instead >> > of partitions. I don't knowi if there a significant overhead diff but I >> > found performance did not suffer. Once on separate disks, should be >> > trivial >> > to script the procedure. >> > >> > You can execute your conversion on any storage, just as fast as >> > possible. >> > So, for instance can even be cheap temporary attached storage. >> > >> > And AFAIK has to be done offline although I suppose a fancy live >> > migraation >> > could be implemented so you're not offline. >> > Tony >> > >> > On Aug 19, 2013 11:50 AM, "Rob Verduijn" <[email protected]> wrote: >> >> >> >> Hello all, >> >> >> >> I'm looking for a clever way to shrink qcow2 images. >> >> >> >> what I do now is : >> >> >> >> 1 in the vm delete the files I don't need (tempfiles, spoolfiles, >> >> junkfiles, etc, etc) >> >> 2 fill the empty space with zeros >> >> dd if=/dev/zero of=bigfile bs=1024k ; sync ; rm bigfile ; sync >> >> 3 shut down the vm >> >> 4 convert the qcow2 to a fresh new qcow2 >> >> qemu-img convert -c -O qcow2 orig.qcow2 shrunk.qcow2 >> >> mv orig.qcow2 orig.qcow2.bak >> >> mv shrunk.qcow2 orig.qcow2 >> >> 5 test the vm >> >> 6 delete the backup if it works >> >> >> >> Now this is fine if you got plenty space and just want to tidy up a >> >> bit before it turns bad. >> >> >> >> But what if there is no space, and adding extra partitions is not that >> >> easy. >> >> (imagine an offsite, far off location that is a real pain to get into >> >> due to serious security hassle) >> >> >> >> Or if you are me and are seriously annoyed by the cumbersome exercise >> >> above and have a gut feeling that there should be a more elegant way >> >> (preferably one without downtime) >> >> >> >> The above exercise requires to be done on all the partitions of the >> >> qcow2 image if there is more than one, adding to my irritation of the >> >> exercise. >> >> >> >> Does anybody have any ideas on a more elegant solution ? >> >> >> >> cheers >> >> Rob >> >> -- >> >> To unsubscribe, e-mail: [email protected] >> >> To contact the owner, e-mail: [email protected] >> >> >> > -- To unsubscribe, e-mail: [email protected] To contact the owner, e-mail: [email protected]
