[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
[Expired for QEMU because there has been no activity for 60 days.] ** Changed in: qemu Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
[Expired for qemu-kvm (Ubuntu) because there has been no activity for 60 days.] ** Changed in: qemu-kvm (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Looking through old bug tickets... is there anything left to do here? Or should we rather close this ticket nowadays? ** Changed in: qemu Status: New => Incomplete ** Changed in: qemu-kvm (Ubuntu) Status: Triaged => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, the external snapshots have apparently been around a long time. The ability to create external snapshots from running vms is newer, but it appears to exist evn in qemu-kvm 1.0. So all versions in Debian and Ubuntu should support them. http://wiki.qemu.org/Features/Snapshots#Snapshot_command_flow -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, the external snapshots have apparently been around a long time. The ability to create external snapshots from running vms is newer, but it appears to exist evn in qemu-kvm 1.0. So all versions in Debian and Ubuntu should support them. http://wiki.qemu.org/Features/Snapshots#Snapshot_command_flow -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, in theory an image that should be taking up 30 GB with four snapshots should be taking up at most about 150 GB, of course. Now the question is what you mean by should be taking up 30 GB and by is taking 600+ GB. For the latter, did you query the file length (ls -l) or the actual size (qemu-img info, disk size)? For the former, if you have a virtual disk size of 1 TB and the guest reports 30 GB are used, that doesn't mean that qemu knows that only 30 GB are used. If you delete a file in the guest, it will report less space being used; however, qemu doesn't know about that unless the guest bothers to discard the now unused sectors. If it doesn't (and I don't see a reason why a guest should discard sectors on an HDD), the guest will just remove the file metadata but the data will stay there (and may be overwritten later by the guest when creating new files etc.). qemu has no idea that that data is now unused, therefore it must treat those sectors as being in use. If your image indeed has a virtual disk size of 30 GB, has four snapshots, is clean (qemu-img check) and does take up 600+ GB of actual disk space, that should indeed not be happening (unless there's some case I forgot to consider). -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@serge, what version would I need to upgrade to be able to use the external snapshots? that sounds like it would solve my problems -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Mario, in theory an image that should be taking up 30 GB with four snapshots should be taking up at most about 150 GB, of course. Now the question is what you mean by should be taking up 30 GB and by is taking 600+ GB. For the latter, did you query the file length (ls -l) or the actual size (qemu-img info, disk size)? For the former, if you have a virtual disk size of 1 TB and the guest reports 30 GB are used, that doesn't mean that qemu knows that only 30 GB are used. If you delete a file in the guest, it will report less space being used; however, qemu doesn't know about that unless the guest bothers to discard the now unused sectors. If it doesn't (and I don't see a reason why a guest should discard sectors on an HDD), the guest will just remove the file metadata but the data will stay there (and may be overwritten later by the guest when creating new files etc.). qemu has no idea that that data is now unused, therefore it must treat those sectors as being in use. If your image indeed has a virtual disk size of 30 GB, has four snapshots, is clean (qemu-img check) and does take up 600+ GB of actual disk space, that should indeed not be happening (unless there's some case I forgot to consider). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@serge, what version would I need to upgrade to be able to use the external snapshots? that sounds like it would solve my problems -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Changing priority given workarounds. ** Changed in: qemu-kvm (Ubuntu) Importance: High = Low -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@michael, so you do that once, after some time the machine keeps growing, and growing and growing... and you have to redo that every so often... I have a machine that should be taking up 30 gb yet is taking 600+ GB with 4 snapshots... but yeah... I'll just plug in another 1tb hard drive so that i can free up the space only for it to happen again in a near future... Seems a great workaround! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
For the record, the workaround is deleting old snapshots in shutdown mode as per comment #14. Upstream has moved toward external snapshots as the way forward, so while I don't argue that this is a bug, it seems unlikely to receive a fix from upstream. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Changing priority given workarounds. ** Changed in: qemu-kvm (Ubuntu) Importance: High = Low -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@michael, so you do that once, after some time the machine keeps growing, and growing and growing... and you have to redo that every so often... I have a machine that should be taking up 30 gb yet is taking 600+ GB with 4 snapshots... but yeah... I'll just plug in another 1tb hard drive so that i can free up the space only for it to happen again in a near future... Seems a great workaround! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
For the record, the workaround is deleting old snapshots in shutdown mode as per comment #14. Upstream has moved toward external snapshots as the way forward, so while I don't argue that this is a bug, it seems unlikely to receive a fix from upstream. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Looking at what? At the lack of problems as comment #14 says? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Looking at what? At the lack of problems as comment #14 says? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Is anyone even looking at this? been years and the problem still persists! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Is anyone even looking at this? been years and the problem still persists! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Thanks for your advices. I have no more problems with VM-size since deleting snapshot in shutdown-mode. I reduced the overlarge qcow2-images by converting in qcow2 again (that detects unused sectors and omits this). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. Stefan -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On 01/02/2013 08:50 AM, Stefan Hajnoczi wrote: On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. You might also want to try modifying step 5 to use the HMP delvm monitor command from within the running qemu rather than going behind qemu's back with a qemu-img invocation. That's how libvirt deletes internal snapshots from a running qemu. Also, there are patches currently under review that are talking about creating a QMP counterpart to the delvm monitor command. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. Stefan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Qemu-devel] [Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
On 01/02/2013 08:50 AM, Stefan Hajnoczi wrote: On Tue, Dec 18, 2012 at 10:18:20AM -, Andy Menzel wrote: Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... It's not safe to modify the qcow2 file while the guest is running. This means Step 5 is not really safe and could result in an inconsistent image. This may also be causing the problem: the QEMU process has a variable with the next free cluster index. Since Step 5 runs as a separate process it does not update the QEMU process' next free cluster index variable. QEMU doesn't know that there are now free clusters within the image file because you updated the file behind QEMU's back - the result is that it grows the file. Please try deleting the last backup snapshot between Step 1 and Step 2. This way you'll free the space while QEMU isn't accessing the image file. When you boot up the image file again QEMU should reuse the freed clusters. You might also want to try modifying step 5 to use the HMP delvm monitor command from within the running qemu rather than going behind qemu's back with a qemu-img invocation. That's how libvirt deletes internal snapshots from a running qemu. Also, there are patches currently under review that are talking about creating a QMP counterpart to the delvm monitor command. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I don't know of any qcow2-based workaround. Is anyone actively working on fixing the qcow2 code? In particular, the fact that after removing snapshots, un-used blocks are not reclaimed and disk size is never reduced? One possible workaround (the one I would use) would be to use lvm-based snapshotting instead. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I don't know of any qcow2-based workaround. Is anyone actively working on fixing the qcow2 code? In particular, the fact that after removing snapshots, un-used blocks are not reclaimed and disk size is never reduced? One possible workaround (the one I would use) would be to use lvm-based snapshotting instead. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... But the problem is, that our original VM-size growing steadily (although few changes were made) ?! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Any solution right now? I have a similar problem like Todor Andreev; Our daily backup of some virtual machines (qcow2) looks like that: 1. shutdown the VM 2. create a snapshot via: qemu-img snapshot -c nameofsnapshot... 3. boot the VM 4. backup the snapshot to another virtual disk via: qemu-img convert -f qcow2 -O qcow2 -s nameofsnapshot... 5. DELETE the snapshot from VM via: qemu-img snapshot -d nameofsnapshot... But the problem is, that our original VM-size growing steadily (although few changes were made) ?! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Yes, I have created one snapshot and did fallocate in the beginning. The other image, which I have problems with, also has snapshots. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
First going back to the original bug, in that instance you kept around many snapshots. In that case there is no way to avoid having many snapshots of, say, a 2G disk, taking much more space than 2G. The thing that concerned me in this bug was that disk space was never reclaimed. I don't believe that this is the case when no snapshots are used. If you create a new qcow2 image with 2G size, then that image will not exceed 2G on disk. Once you introduce snapshots, this appears to complicate the bookkeeping such that automatic resizing of the disk image is not done. The data *is* reference counted however, This means that you can create a new, trimmed qcow2 image based on the original by doing qemu-img convert -f qcow2 -O qcow2 original.qcow2 new.qcow2 As I don't believe the automatic freeing of disk space was ever implemented, I am going to mark this bug Triaged/Low, to mark it as a desirable feature. I'll also mark it as affecting the upstream project. ** Changed in: qemu-kvm (Ubuntu) Status: Confirmed = Triaged ** Changed in: qemu-kvm (Ubuntu) Importance: High = Low -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
May be here will be more convenient for reading: http://www.linuxquestions.org/questions/linux-virtualization-90/disk-physical-size-more-than-virtual-size-qcow2-image-4175416848/#post4730524 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I did some testing with a WindowsXP guest, that I have and could test on. It seems that this behavior is not present at the beginning. But at the moment we create a snapshot it is starting to write on top of the current size. So, it is like this: 1. Image is: Code: qemu-img info WindowsXP.img image: WindowsXP.img file format: qcow2 virtual size: 15G (1572864 bytes) disk size: 13G cluster_size: 65536 I write some files and it doesn't become more than that. 2. I delete some files, then write again and it doesn't changes size. 3. I create a snapshot: Code: qemu-img info WindowsXP.img image: WindowsXP.img file format: qcow2 virtual size: 15G (1572864 bytes) disk size: 13G cluster_size: 65536 Snapshot list: IDTAG VM SIZEDATE VM CLOCK 1 test 0 2012-07-17 09:52:25 00:00:00.000 4. I write something with a size of 587MB and it becomes larger (with 587MB) Code: du -sm WindowsXP.img 14102 WindowsXP.img 5. I delete it and then write it again. It becomes more larger... (again with another 587MB) Code: du -sm WindowsXP.img 14703 WindowsXP.img 6. I delete it and then write it again. It doesn't change this time. 7. I write a copy of it. It becomes larger (with 587MB) Code: du -sm WindowsXP.img 15309 WindowsXP.img 8. I delete it and then write it again. It doesn't change this time. 9. I write a copy of it. It becomes larger again (with 587MB) Code: du -sm WindowsXP.img 16010 WindowsXP.img 10. I write another copy of it and it stays the same. Code: du -sm WindowsXP.img 16010 WindowsXP.img 11. I write another copy of it and becomes larger again. Code: du -sm WindowsXP.img 16913 WindowsXP.img Code: qemu-img info WindowsXP.img image: WindowsXP.img file format: qcow2 virtual size: 15G (1572864 bytes) disk size: 17G cluster_size: 65536 Snapshot list: IDTAG VM SIZEDATE VM CLOCK 1 test 0 2012-07-17 09:52:25 00:00:00.000 12. Create a snapshot. 13. I write a copy of it. It becomes larger again (with 587MB) Code: du -sm WindowsXP.img 17572 WindowsXP.img 14. I delete both of the snapshots. 15. I delete the file (in guest with the size of 587MB) and write it again. No change in size. 16. I delete the file again and write it again. No change in size. 17. Create a snapshot. 18. I delete the file again and write it again. No change in size. 19. Delete the file. 20. Create a snapshot. 21. Write the file again. No change in size. Well from all this, I can conclude that most probably: 1. The problem occurs only when there is an internal snapshot present. 2. The problem is not by design because the behavior is not consistent (for example, 13. and 21. should be with the same result, but they arent't).. At the end of the day, after these procedures (4 creations of snapshots, 2 deletions and some writing and deleting of internal guest files) the result is this: Code: qemu-img info WindowsXP.img image: WindowsXP.img file format: qcow2 virtual size: 15G (1572864 bytes) disk size: 17G cluster_size: 65536 Snapshot list: IDTAG VM SIZEDATE VM CLOCK 1 test2 0 2012-07-17 14:55:17 00:00:00.000 2 test3 0 2012-07-17 15:27:43 00:00:00.000 and Code: ls -lsh WindowsXP.img 18G -rw--- 1 libvirt-qemu kvm 18G Jul 17 15:43 WindowsXP.img -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Serge, thank you for answering. Well, I think this is not entirely true. I think that from my above post, 13. and 20. should be with the same results, if it were true. The truth is that when a snapshot is present, sometimes it uses the available space, sometimes it doesn't. (!???) I think there is something wrong here and everyone should be able to confirm it, because I did it on 2 different hosts, with 2 different guest images. Apart from this, I now understand that when I use internal snapshots, the size will be larger (it is logical). Also (apart from my different 13. and 20. results, pointing that there is something wrong), I think that this is a HIGH priority, because the problem is with an image of almost 2TB. So, what people should buy another 2TB, so that they could convert the image somewhere? I don't think this is reasonable. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I'm sorry, I meant 21. not 20. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
@Todor, Thanks, you might be right. It sounds like it's not a missing feature but a bug. I'll re-raise the priority. ** Also affects: qemu Importance: Undecided Status: New ** Changed in: qemu-kvm (Ubuntu) Importance: Low = High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
Thanks for filing this bug, Todor. I'll try figure out whether this is still the case in the upstream git HEAD. ** Changed in: libvirt (Ubuntu) Status: New = Confirmed ** Changed in: libvirt (Ubuntu) Importance: Undecided = High ** Package changed: libvirt (Ubuntu) = qemu-kvm (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1025244] Re: qcow2 image increasing disk size above the virtual limit
I started playing with this by just doing: qemu-img create -f qcow2 x.img 2G (boot a vm from a cdrom/iso into rescue mode with x.img as a drive, and there do): dd if=/dev/zero of=/mnt/zero1 bs=1M count=1000 and then cp /mnt/zero1 /mnt/zero2 rm /mnt/zero2 cp /mnt/zero1 /mnt/zero3 (etc) Here the volume doesn't exceed it's allocation (which was 2G). I created snapshots but still did not exceed 2G. When I started adding more real data (booted from an installed server virtual disk, but I don't believe that made a difference) as well as creating snapshots, I worked up to a 3.2G real disk. In the disk you showed in the bug description, had you created any snapshots? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1025244 Title: qcow2 image increasing disk size above the virtual limit To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/1025244/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
