Re: [EXT] [qubes-users] How to attach private storage from one AppVM to another AppVM (LVM)?

2020-12-13 Thread 'heinrich...@googlemail.com' via qubes-users
Thank you all for describing the various approaches, this is great feedback 
and all sound feasible. I'll start testing soon. Ultimately I'll probably 
move all the files over to some network location if it proves fast enough. 
The BIG VM so far caused nothing but problems anyway. But currently it is 
there so I have to cope with it.

On Friday, December 11, 2020 at 5:28:12 AM UTC+1 Stuart Perkins wrote:

>
>
> On Thu, 10 Dec 2020 20:43:27 +0100
> Ulrich Windl  wrote:
>
> >On 12/7/20 3:56 PM, 'heinrich...@googlemail.com' via qubes-users wrote:
> >> From one AppVM I need to temporarily access a large amount of files 
> >> from another AppVM. Can this be done without copying the files around?
> >> 
> >> _Background: _
> >> I have a large amount of files stored in AppVM "BIG". That's hundreds 
> of 
> >> GB in a separate pool on a spinning HDD.
> >> I also have a small AppVM "SMALL" running a program that needs to 
> access 
> >> files from "BIG". This AppVM resides on a small SSD.
> >> 
> >> In the past I copied files from BIG to SMALL. But this takes time and I 
> >> need to sort the files beforehand because there is not enough space on 
> >> the SSD. I don't want to do that anymore. It would be okay to allow 
> >> AppVM "SMALL" to access files from "BIG"'s private storage directly.
> >> 
> >> Googling around tells me to mount "private.img", but I'm using LVM so 
> >> that's not an option. But how can this be done? Can it be done? (Or is 
> >> there even a better "file sharing" approach for this amount of data 
> >> without having to revert to a NAS?)
> >> 
> >> Any tips are appreciated. 
> >
> >Actually I have not done it, but it feels like you should have an NFS 
> >server on BIG with a network only accessible from inside qubes, and 
> >specifically from SMALL. Still it will have to transfer the file 
> >contents, but you benefit from any application that only reads parts of 
> >the files.
> >
> >When not wanting to copy I guess you'll have to mount a snapshot of 
> >BIG's data as the LV should be mounted only once (AFAIK).
> >
> What I have done is...
>
> created a large disk image (600g or so) on the mail dom0 drive space
>
> written a script in dom0 to attach the image to whichever machine I want 
> to access it from.
>
> This script..
> attaches the image to a loop device
> mounts the image to the machine I desire
> added /etc/hosts entries to each app vm to mount to a dedicated directory 
> when doing "sudo mount -a" by uuid
> script ends with an execution of "sudo mount -a" on the target vm
>
> there is also a corresponding unmount script, so if it is mounted to 
> "mail" I can easily unmount it and mount it to "money".
>
>
> This lets me treat the 600g.img file like a removable media which can be 
> attached to any app vm. I also back that drive image up on my network 
> server.
>
> Stuart
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/2197e4ca-903a-41b6-b5e5-bb2cd1bbd878n%40googlegroups.com.


Re: [qubes-users] How to attach private storage from one AppVM to another AppVM (LVM)?

2020-12-07 Thread 'heinrich...@googlemail.com' via qubes-users
sshfs sounds great since this probably allows me to also select the folders 
I want to share with the SMALL AppVM instead of revealing all files.

Thanks for taking the time to look into this and I'm looking forward to the 
push :)

On Monday, December 7, 2020 at 4:30:11 PM UTC+1 unman wrote:

> On Mon, Dec 07, 2020 at 06:56:47AM -0800, 'heinrich...@googlemail.com' 
> via qubes-users wrote:
> > >From one AppVM I need to temporarily access a large amount of files 
> from 
> > another AppVM. Can this be done without copying the files around?
> > 
> > *Background: *
> > I have a large amount of files stored in AppVM "BIG". That's hundreds of 
> GB 
> > in a separate pool on a spinning HDD.
> > I also have a small AppVM "SMALL" running a program that needs to access 
> > files from "BIG". This AppVM resides on a small SSD.
> > 
> > In the past I copied files from BIG to SMALL. But this takes time and I 
> > need to sort the files beforehand because there is not enough space on 
> the 
> > SSD. I don't want to do that anymore. It would be okay to allow AppVM 
> > "SMALL" to access files from "BIG"'s private storage directly.
> > 
> > Googling around tells me to mount "private.img", but I'm using LVM so 
> > that's not an option. But how can this be done? Can it be done? (Or is 
> > there even a better "file sharing" approach for this amount of data 
> without 
> > having to revert to a NAS?) 
> > 
> > Any tips are appreciated.
> > 
> > (I'm on Qubes OS v4 latest)
> > 
>
> Take a look at https://qubes-os.org/doc/mount-lvm-image/
> That explains how to mount an lvm image.
>
> Alternatively you could look at https://github.com/unman/qubes-sync
> where I outline how to rsync data over qrexec.
> I've updated that to include sshfs over qrexec, but don't seem to have
> pushed it up yet. That'll have to wait until the morning.
> But the principle is simple - run sshd on the target
> instead of rsyncd: use a forwarder, and then mount the remote directory
> using sshfs on the client. That removes the need to copy files around,
> and keeps a single archive accessible from other qubes.
> That should give you idea of how to get started - if you need help let
> me know and I'll try to help in the morning.
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/909961b5-b4fc-4a34-90c0-f4614a4bede4n%40googlegroups.com.


Re: [qubes-users] How to attach private storage from one AppVM to another AppVM (LVM)?

2020-12-07 Thread 'heinrich...@googlemail.com' via qubes-users
sshfs sounds great since this probably allows me to also select the folders 
I want to share with the SMALL AppVM instead of revealing all files.

Thanks for taking the time to look into this and I'm looking forward to the 
push :)

On Monday, December 7, 2020 at 4:30:11 PM UTC+1 unman wrote:

> On Mon, Dec 07, 2020 at 06:56:47AM -0800, 'heinrich...@googlemail.com' 
> via qubes-users wrote:
> > >From one AppVM I need to temporarily access a large amount of files 
> from 
> > another AppVM. Can this be done without copying the files around?
> > 
> > *Background: *
> > I have a large amount of files stored in AppVM "BIG". That's hundreds of 
> GB 
> > in a separate pool on a spinning HDD.
> > I also have a small AppVM "SMALL" running a program that needs to access 
> > files from "BIG". This AppVM resides on a small SSD.
> > 
> > In the past I copied files from BIG to SMALL. But this takes time and I 
> > need to sort the files beforehand because there is not enough space on 
> the 
> > SSD. I don't want to do that anymore. It would be okay to allow AppVM 
> > "SMALL" to access files from "BIG"'s private storage directly.
> > 
> > Googling around tells me to mount "private.img", but I'm using LVM so 
> > that's not an option. But how can this be done? Can it be done? (Or is 
> > there even a better "file sharing" approach for this amount of data 
> without 
> > having to revert to a NAS?) 
> > 
> > Any tips are appreciated.
> > 
> > (I'm on Qubes OS v4 latest)
> > 
>
> Take a look at https://qubes-os.org/doc/mount-lvm-image/
> That explains how to mount an lvm image.
>
> Alternatively you could look at https://github.com/unman/qubes-sync
> where I outline how to rsync data over qrexec.
> I've updated that to include sshfs over qrexec, but don't seem to have
> pushed it up yet. That'll have to wait until the morning.
> But the principle is simple - run sshd on the target
> instead of rsyncd: use a forwarder, and then mount the remote directory
> using sshfs on the client. That removes the need to copy files around,
> and keeps a single archive accessible from other qubes.
> That should give you idea of how to get started - if you need help let
> me know and I'll try to help in the morning.
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/b6366b52-eba5-4e9e-b214-c41ef0b0bf9en%40googlegroups.com.


[qubes-users] How to attach private storage from one AppVM to another AppVM (LVM)?

2020-12-07 Thread 'heinrich...@googlemail.com' via qubes-users
>From one AppVM I need to temporarily access a large amount of files from 
another AppVM. Can this be done without copying the files around?

*Background: *
I have a large amount of files stored in AppVM "BIG". That's hundreds of GB 
in a separate pool on a spinning HDD.
I also have a small AppVM "SMALL" running a program that needs to access 
files from "BIG". This AppVM resides on a small SSD.

In the past I copied files from BIG to SMALL. But this takes time and I 
need to sort the files beforehand because there is not enough space on the 
SSD. I don't want to do that anymore. It would be okay to allow AppVM 
"SMALL" to access files from "BIG"'s private storage directly.

Googling around tells me to mount "private.img", but I'm using LVM so 
that's not an option. But how can this be done? Can it be done? (Or is 
there even a better "file sharing" approach for this amount of data without 
having to revert to a NAS?) 

Any tips are appreciated.

(I'm on Qubes OS v4 latest)

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/5d42e07f-9170-4504-bbc4-d638d2403cfcn%40googlegroups.com.


Re: [qubes-users] Qubes4: Cannot increase size of VM private storage

2020-09-07 Thread 'heinrich...@googlemail.com' via qubes-users
This exact problem happened to me recently in Qubes 4. The steps outlined 
here worked.

The commands for me were along the lines of: sudo xl console, pkill -u 
1000, umount, e2fsck, resize2fs.

A HUGE THANK YOU FOR DOCUMENTING ALL OF THIS HERE! Excuse the yelling, this 
is my excitement about having this resolved :)

Additional note: Since I had snapd installed in the TemplateVM I needed to 
remove this too to get rid of its mounted file systems in the AppVM.
On Saturday, July 7, 2018 at 12:10:22 PM UTC+2 RSS wrote:

>
> > > resize2fs /dev/xvdb
> > > then prompted to first run fsck on /dev/xvdb. After this 
> > 
> > Usually Qubes handles that resize2fs automatically; I suspect whatever
> > caused the need for fsck was preventing that.
>
> I wish I could remember if anything odd happened at the time, that is
> already several days ago and there has been a lot of water under the
> bridge since then.
>
> > > If someone believes there is a natural and desirable home for this
> > > tidbit somewhere in the Qubes Docs, I would be happy to put in
> > > there. 
> > 
> > There is https://www.qubes-os.org/doc/resize-disk-image/, but I don't
> > think the process you described is intended. Does it happen every
> > time you resize a Linux AppVM's volumes (i.e. can you recreate it)?
> > If so, might be better to file an Issue instead. Anyways, glad it
> > finally worked for you!
>
> No, actually this is I believe the first time on either Qubes 3 or 4
> that this has happened to me, and I tend to be resizing private storage
> quite regularly. So it's basically a one-off that I have no way of
> reproducing, and I don't recall many (any?) other people talking about
> this on the list, so perhaps we can just let the mail archive take care
> of it.
>
> It might be worth noting here that after using Qubes 4 for several
> months, I have had two significant glitches, this and another, and they
> both were storage issues. Everything else has worked perfectly so far.
>
> Thanks for the help!
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/fd05e798-7275-4e27-9a3c-6deec546b29fn%40googlegroups.com.