Re: [qubes-users] Shrinking a private volume
On 3/24/19 4:15 AM, 799 wrote: Hello Unman, unman mailto:un...@thirdeyesecurity.org>> schrieb am So., 24. März 2019, 03:17: [...] We're all working on the assumption that you have Qubes 4.0 - can you confirm that? Yes, of course - sorry for not mentioning this, but I don't see why someone would run Q3.2, I have switched to 4.0 shortly after the release and never looked back. You can see what size is actually used on disk by using s flag to ls: so 'ls -lsS' will show you the apparent size and the actual size. Thanks, I wasn't aware of this command option and will review the man page for ls, to see it there are more interesting options In 3.2, to attach a private.img try: qvm-block -A dom0:/var/. That's 3.x specific. One way to do it in 4.0 (losetup assigns a device to the img file then prints the device name): $ sudo losetup -f /var/lib/qubes/...img loop0 $ qvm-block attach newvm dom0:loop0 As for backup and restore, you can do this just fine between non-lvm and lvm systems. If you do create a backup you may want to copy it to a third device so there is an extra copy, then do a verify on the extra copy just in case. -- Chris Laprise, tas...@posteo.net https://github.com/tasket https://twitter.com/ttaskett PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886 -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/9374f0ab-b020-ca92-d08d-dc0b2663c6a3%40posteo.net. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
Hello Unman, unman schrieb am So., 24. März 2019, 03:17: > [...] > We're all working on the assumption that you have Qubes 4.0 - can you > confirm that? > Yes, of course - sorry for not mentioning this, but I don't see why someone would run Q3.2, I have switched to 4.0 shortly after the release and never looked back. You can see what size is actually used on disk by using s flag to ls: > so 'ls -lsS' will show you the apparent size and the actual size. > Thanks, I wasn't aware of this command option and will review the man page for ls, to see it there are more interesting options In 3.2, to attach a private.img try: > qvm-block -A dom0:/var/. > Will this also work under Qubes 4, as you are mentioning 3.2 specifically. What are the command to do so in 4.0? - O > -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/CAJ3yz2uqMgniDVmMQidau0ij2a1z0BkxTED_UeNLg1CQNXF0tA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
On Sat, Mar 23, 2019 at 01:59:27PM +0100, onw7two99 wrote: > Hello Chris, > > thank you once again for your feedback. > > On 3/23/19 11:31 AM, Chris Laprise wrote: > > On 3/23/19 5:34 AM, onw7two99 wrote: > >> I run sudo fstrim -av from within the AppVM (as root) but I haven't > >> notive any change. > > > > The difference doesn't show until you shutdown the vm for the second > > time (the current .old volume gets deleted). Also the Qubes disk > > widget and Qubes Manager take some time to report the new values. But > > running fstrim may not make any difference if you haven't been > > deleting many files. > > if I run ls -lah /var/lib/qubes/appvms/ the size listed is the > "thick provisioned", is there a way to show the "real" file size in dom0 > whch should be the used size which is reported from within the AppVM > when using "df -h" ? > > > >> Is there a way to move this private disk to the new storage-qube? > > > > Yes, you can use the 'mv' command while the two vms are not running. > > Keep in mind this can erase the existing private image of the > > new/destination vm unless you rename that first (but it won't be > > accessible after renaming it, unless you run 'qvm-block attach'). > > I tried to mount the private.img from the old appvm to the new one, but > I think I don't have the correct syntax. > > I tried: > > qvm-block attach :private.img > qvm-block attach /var/lib/qubes/appvms//private.img > > How can I attach a private.img from one AppVM to another? Maybe even > once in read/write and in another scenario read only. > > This could be usefull for doing forensics on an AppVM or clam av scans etc. > > >> Neither in the AppVM nor in dom0 I see something like LVM, but as far as > >> I remember LVM was available in Qubes (I think have looked into it 1 or > >> 2 years ago. > > > > Since you have -cow.img files in /var/lib/qubes/appvms, your system > > seems to have been installed with one of the non-LVM options. This > > could be either Ext4 (rather limited) or Btrfs (flexible). Looking at > > the output of the 'mount' command in dom0 will tell you which. > > Stupid me, I wasn't aware that I installed Qubes without LVM ... why > should I do so? > > The only reason to fix this is likely to reinstall Qubes with LVM, but > what happens if I restore my old AppVMs from the non-LVM-setup? > Can you give some more details about the private.img vs private-cow.img > vs private-cow.img.old > > >From the Qubes docs > https://www.qubes-os.org/doc/template-implementation/ I learned that the > root.cow.img will hold the differences (from the appVM perspective) to > root.img (the root image of the template which is read only). > > Still I haven't understand what the private.cow.img and > private.cow.img.old do. > > Is there some more information regarding this topic? > We're all working on the assumption that you have Qubes 4.0 - can you confirm that? You can see what size is actually used on disk by using s flag to ls: so 'ls -lsS' will show you the apparent size and the actual size. In 3.2, to attach a private.img try: qvm-block -A dom0:/var/. -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/20190324021744.hzd3lgkenlsmn57m%40thirdeyesecurity.org. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
Hello Chris, thank you once again for your feedback. On 3/23/19 11:31 AM, Chris Laprise wrote: > On 3/23/19 5:34 AM, onw7two99 wrote: >> I run sudo fstrim -av from within the AppVM (as root) but I haven't >> notive any change. > > The difference doesn't show until you shutdown the vm for the second > time (the current .old volume gets deleted). Also the Qubes disk > widget and Qubes Manager take some time to report the new values. But > running fstrim may not make any difference if you haven't been > deleting many files. if I run ls -lah /var/lib/qubes/appvms/ the size listed is the "thick provisioned", is there a way to show the "real" file size in dom0 whch should be the used size which is reported from within the AppVM when using "df -h" ? >> Is there a way to move this private disk to the new storage-qube? > > Yes, you can use the 'mv' command while the two vms are not running. > Keep in mind this can erase the existing private image of the > new/destination vm unless you rename that first (but it won't be > accessible after renaming it, unless you run 'qvm-block attach'). I tried to mount the private.img from the old appvm to the new one, but I think I don't have the correct syntax. I tried: qvm-block attach :private.img qvm-block attach /var/lib/qubes/appvms//private.img How can I attach a private.img from one AppVM to another? Maybe even once in read/write and in another scenario read only. This could be usefull for doing forensics on an AppVM or clam av scans etc. >> Neither in the AppVM nor in dom0 I see something like LVM, but as far as >> I remember LVM was available in Qubes (I think have looked into it 1 or >> 2 years ago. > > Since you have -cow.img files in /var/lib/qubes/appvms, your system > seems to have been installed with one of the non-LVM options. This > could be either Ext4 (rather limited) or Btrfs (flexible). Looking at > the output of the 'mount' command in dom0 will tell you which. Stupid me, I wasn't aware that I installed Qubes without LVM ... why should I do so? The only reason to fix this is likely to reinstall Qubes with LVM, but what happens if I restore my old AppVMs from the non-LVM-setup? Can you give some more details about the private.img vs private-cow.img vs private-cow.img.old >From the Qubes docs https://www.qubes-os.org/doc/template-implementation/ I learned that the root.cow.img will hold the differences (from the appVM perspective) to root.img (the root image of the template which is read only). Still I haven't understand what the private.cow.img and private.cow.img.old do. Is there some more information regarding this topic? - O -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/75368c21-ba56-e565-6cf1-fef15a3924b8%40gmail.com. For more options, visit https://groups.google.com/d/optout. pEpkey.asc Description: application/pgp-keys
Re: [qubes-users] Shrinking a private volume
On 3/23/19 5:34 AM, onw7two99 wrote: Hello Brendan On 2/28/19 3:14 AM, brendan.h...@gmail.com wrote: I’ve found it necessary sometimes to sudo fstrim -av from inside the VM in order for the dom0 lvm thin provisioned volumes to return the unallocated space back to the pool. This doesn’t shrink the volume but it does tell the system that it doesn’t need to store anything there for that VM currently so other VMs can use the space. Welcome to thin provisioning! I run sudo fstrim -av from within the AppVM (as root) but I haven't notive any change. The difference doesn't show until you shutdown the vm for the second time (the current .old volume gets deleted). Also the Qubes disk widget and Qubes Manager take some time to report the new values. But running fstrim may not make any difference if you haven't been deleting many files. The AppVM has currently a 145GB private disk attached, from which 91GB is used, 44GB is free. I would like to move this data to another storage qube, but I have only 58GB available (df -h) in dom0. Therefore I thought that I can someone move the private disk of "storage-qube-old" to "storage-qube-new". I have looked into /var/lib/qubes/appvms/storage-qube-old and see 3 private images: private-cow.img -> 147 GB private-cow.img.old -> 147 GB private-cow.img -> 147 GB Is there a way to move this private disk to the new storage-qube? Yes, you can use the 'mv' command while the two vms are not running. Keep in mind this can erase the existing private image of the new/destination vm unless you rename that first (but it won't be accessible after renaming it, unless you run 'qvm-block attach'). Also keep in mind that any problems the old vm had arising from anything in /rw or /home will be carried over to the new vm. Slightly offtopic: I haven't understand at which layer thin provisioning works in Qubes. Neither in the AppVM nor in dom0 I see something like LVM, but as far as I remember LVM was available in Qubes (I think have looked into it 1 or 2 years ago. Since you have -cow.img files in /var/lib/qubes/appvms, your system seems to have been installed with one of the non-LVM options. This could be either Ext4 (rather limited) or Btrfs (flexible). Looking at the output of the 'mount' command in dom0 will tell you which. -- Chris Laprise, tas...@posteo.net https://github.com/tasket https://twitter.com/ttaskett PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886 -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/1f38cbf1-482c-2c57-55a7-0108af8632f7%40posteo.net. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
Hello Brendan On 2/28/19 3:14 AM, brendan.h...@gmail.com wrote: > I’ve found it necessary sometimes to sudo fstrim -av from inside the VM in > order for the dom0 lvm thin provisioned volumes to return the unallocated > space back to the pool. > This doesn’t shrink the volume but it does tell the system that it doesn’t > need to store anything there for that VM currently so other VMs can use the > space. > Welcome to thin provisioning! I run sudo fstrim -av from within the AppVM (as root) but I haven't notive any change. The AppVM has currently a 145GB private disk attached, from which 91GB is used, 44GB is free. I would like to move this data to another storage qube, but I have only 58GB available (df -h) in dom0. Therefore I thought that I can someone move the private disk of "storage-qube-old" to "storage-qube-new". I have looked into /var/lib/qubes/appvms/storage-qube-old and see 3 private images: private-cow.img -> 147 GB private-cow.img.old -> 147 GB private-cow.img -> 147 GB Is there a way to move this private disk to the new storage-qube? Slightly offtopic: I haven't understand at which layer thin provisioning works in Qubes. Neither in the AppVM nor in dom0 I see something like LVM, but as far as I remember LVM was available in Qubes (I think have looked into it 1 or 2 years ago. - O -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/80198411-b1f9-c56d-3151-4ccf86cf6e90%40gmail.com. For more options, visit https://groups.google.com/d/optout. pEpkey.asc Description: application/pgp-keys
Re: [qubes-users] Shrinking a private volume
On 2/27/19 9:14 PM, brendan.h...@gmail.com wrote: I’ve found it necessary sometimes to sudo fstrim -av from inside the VM in order for the dom0 lvm thin provisioned volumes to return the unallocated space back to the pool. This doesn’t shrink the volume but it does tell the system that it doesn’t need to store anything there for that VM currently so other VMs can use the space. Welcome to thin provisioning! I agree. The default 'discard' config used by Qubes templates should already have reclaimed most of the space. However, I found it doesn't recover 100% of the deleted space, so fstrim is sometimes necessary to get everything back. OTOH, the upper limit for the volume's size remains the same. If that is somehow an issue, the are options for shrinking it: * If the remaining data is not large, you can qvm-copy it to a new VM and use that instead. * Otherwise, its possible to shrink the upper limit using a combination of qvm-block attach, resize2fs (in a VM), and lvresize. This is one of those "if you know really what you're doing" options. -- Chris Laprise, tas...@posteo.net https://github.com/tasket https://twitter.com/ttaskett PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886 -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/3ef2c3fc-4614-4fe0-373c-2671b7ce7cac%40posteo.net. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
I’ve found it necessary sometimes to sudo fstrim -av from inside the VM in order for the dom0 lvm thin provisioned volumes to return the unallocated space back to the pool. This doesn’t shrink the volume but it does tell the system that it doesn’t need to store anything there for that VM currently so other VMs can use the space. Welcome to thin provisioning! B -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/569dc447-166d-49fd-b475-aa806839e2c4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
On Tue, Feb 26, 2019 at 10:30:35PM +0100, 799 wrote: > Hello, > > I've migrated ~150gb of data into a Qubes Storage Qube. > After cleaning up older files I have reduced the data to 100gb. > Now I'd like to free the additional 50gb so that dom0 can use this capacity > for other qubes. > You dont need to do anything. In 4.0 dom0 will simply use the unallocated space. (Well, sometimes I restart a qube for this purpose.) That 50GB is *already* available to other qubes. -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/20190227131701.aj5dpr5uxkda27d6%40thirdeyesecurity.org. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-users] Shrinking a private volume
Hello Stuart, On Wed, 27 Feb 2019 at 01:57, Stuart Perkins wrote: > > On Tue, 26 Feb 2019 22:30:35 +0100 > 799 wrote: > >Honestly I don't understand why it is not possible to shrink a volume as > >this is something which can done in linux. > >Does someone has an idea how to "reclaim" the free capacity from an AppVM? > > [...] > Can you mount an additional "drive" in the storage qube? Copy all the > files with cp -r or rsync then rearrange things so the new disk image is > the main one and delete the overgrown one? Or, conversely, mount the > storage qube's main drive in a regular appVM and do the same thing there? > Of course I could copy data from the Old AppVM to an external HDD, create a smaller Qube and then copy the data back. But this doesn't feel like how it should be done in 2019 ;-) I thought that there might something like using resize2fs to make the mage smaller from within the AppVM and then use some magic to tell Qubes that it can free up space. But, maybe another good idea which you have brought up is, to mount the old private.img and copy data to the new private.img of the new qube: mkdir /tmp/old-private.img sudo mount /var/lib/qubes/appvms//private.img mkdir /tmp/new-private.img sudo mount /var/lib/qubes/appvms//private.img Then I could copy data from /tmp/old-private.img to /tmp/new-private.img While this could be done from dom0, I don't want to do it from there, but I'd prefer to mount the old private.img into the new AppVM. I think this something which can be done, from the new AppVM Qubes, to which I mount the old private.img additionally. I just have to figure out how to do so, haven't found something yet, but afaik I've read it some time ago. If someone has guidance, this would be helpfull and could work as a "poor man's" reduce disk alternative ;-) - O -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/CAJ3yz2vSsUapVWLuH0LjpwE6nZSk%3DdwkhPtbfgUHCjToGyaXcg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
