oops I meant ofcourse online defragmenting filesystem like btrfs. silly me
2013/8/20 Tony Su <[email protected]>: > But as I noted you can write zeroes in blocks bigger than 1MB and as you > make your blocks bigger that last unwritten block will also get bigger and > possibly contain data which isn't compressable. > > Tony > > On Tuesday, August 20, 2013, Rob Verduijn <[email protected]> wrote: >> Hi, >> >> Indeed, you fill up the last blocks that are not big enough for my >> defined blocksize. >> But those blocks are very likely to be empty, also the loss of max >> 1024k - 1 of datastorage isn't really bothering me. >> >> The requirement to fill out the empty space with zeroes and then take >> down the vm to create a new qcow2 image is what bothers me. >> I wish to reclaim the empty space from a qcow2 image in a more >> efficient way, preferably without creating a new qcow2 image. >> (no downtime would be a nice bonus) >> >> Rob >> >> 2013/8/20 Tony Su <[email protected]>: >>> Haven't tried the following to zero a virtual disk, but the following >>> should >>> be fast, make the bs as big as you want (maybe 10x for very large disks). >>> Of >>> course, the bigger you make zero.big.file the longer it <might> take to >>> create zero.file. My code zeroes all bytes in 2 steps whereas yours >>> doesn't >>> zero all. >>> >>> dd if=/dev/zero of=zero.big.file bs=1024 count=102400 >>> cat /dev/zero > zero.file >>> rm zero.big.file >>> rm zero.file >>> >>> Tony >>> >>> On Aug 20, 2013 1:39 AM, "Rob Verduijn" <[email protected]> wrote: >>>> >>>> Hi, >>>> >>>> Using separate images for partitions makes the script a tiny bit less >>>> complex. >>>> >>>> I've already been scripting the exercise, the use off multiple >>>> partition images makes it only a tiny bit more complex. >>>> qemu-nbd simply adds another device for each additional partition. >>>> >>>> #!/binbash >>>> modprobe nbd max_part=16 #number off partitions >>>> qemu-nbd -c --nocache --aio=native /path/to/image.qcow2 >>>> mount /dev/nbd0 /mnt # I didn't partiotion the image, just >>>> formated the image as ext4 >>>> dd if=/dev/zero of=/mnt/bigfile bs=1024k; sync; rm /mnt/bigfile; rm >>>> bigfile; sync; rm /mnt/bigfile; sync >>>> sleep 2 >>>> umount /dev/nbd0 # umount the device >>>> qemu-nbd /dev/nbd0 # clean up nbd devices or they will bite you >>>> qemu-img convert -c -O qcow2 /path/to/image.qcow2 /path/to/shrunk.qcow2 >>>> mv /path/to/image.qcow2 /path/to/image.qcow2.bak >>>> mv /path/to/shrunk.qcow2 /path/to/image.qcow2 >>>> >>>> The performance of qemu-nbd is rather poor, using writeback cache is >>>> not really an option since you always need to wait for the zeroes to >>>> be actually written to the hd. >>>> Also writeback is very hazardous if you use a script to umount and >>>> disconnect the nbd device, image corruption is very likely to happen >>>> since sync doesn't apply to nbd0 devices and blockdev --flushbufs >>>> /dev/nbd0 isn't foolproof either when scripting. >>>> (ctrl-c the script at the wrong time and you are in for a recovery) >>>> >>>> So the script isn't that difficult, neither is the use of multiple >>>> partitions in an image. >>>> The biggest drawback is, downtime, filling the image with zeroes takes >>>> even longer offline than online. >>>> Ok you could have a cronjob in the vm that does that nightly, but I >>>> can imagine issues when the hd of a vm is at filled to the max at >>>> regular intervals (once a month ??), even at expected times and only >>>> for a short moment. >>>> Also that would mean the shrinking cronjob is required on the host (I >>>> want the shrinking done as soon as possible after the zeroing) >>>> This has to be timed properly with the zero cronjob of the guest, this >>>> becomes rather complex with every additional guest. >>>> >>>> Regards >>>> Rob >>>> >>>> >>>> 2013/8/19 Tony Su <[email protected]>: >>>> > Not specific to qcow >>>> > In the past if I wanted to partition files I'd deploy multiple disks >>>> > instead >>>> > of partitions. I don't knowi if there a significant overhead diff but >>>> > I >>>> > found performance did not suffer. Once on separate disks, should be >>>> > trivial >>>> > to script the procedure. >>>> > >>>> > You can execute your conversion on any storage, just as fast as >>>> > possible. >>>> > So, for instance can even be cheap temporary attached storage. >>>> > >>>> > And AFAIK has to be done offline although I suppose a fancy live >>>> > migraation >>>> > could be implemented so you're not offline. >>>> > Tony >>>> > >>>> > On Aug 19, 2013 11:50 AM, "Rob Verduijn" <[email protected]> >>>> > wrote: >>>> >> >>>> >> Hello all, >>>> >> >>>> >> I'm looking for a clever way to shrink qcow2 images. >>>> >> >>>> >> what I do now is : >>>> >> >>>> >> 1 in the vm delete the files I don't need (tempfiles -- To unsubscribe, e-mail: [email protected] To contact the owner, e-mail: [email protected]
