Re: [atomic-devel] Storage for system containers

[Daniel Walsh]

On 04/28/2017 01:09 PM, Giuseppe Scrivano wrote:

Hi,

Dusty Mabe  writes:


i'm going to show how little I know with this question, but would it be possible
to have a separate partition for system containers that was essentially xfs + an
overlayfs of the host filesystem?

yes we could do that, we will just need to use a separate OSTree
repository, so that it won't be shared with the OS.  This is possible
already today, as the OSTree storage to use is configurable.
The disadvantage is that files in common with the host will still need
to be copied in the new repository.

Regards,
Giuseppe


Ok, We should be able to support both environments, where you want to 
maximize disk space savings by taking advantage of sharing OS Content 
with the Host OS by running your system container storage on the same 
disk as /usr.  If you want to totally isolate your container content 
from the host, we can mount /var/lib/containers on an separate 
partition/disk and keep storage isolated.  The only problem here is that 
/var/lib/docker would need to be modified to use 
/var/lib/containers/docker.  From CRI-O/buildah and all container 
storage we store by default in /var/lib/containers/storage.  With the 
move to ThePackageFormallyKnownAsDocker (TPFKAD), we should move the 
default storage to /var/lib/containers/TPFKAD.


___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

Re: [atomic-devel] Storage for system containers

[Dusty Mabe]

On 04/27/2017 08:51 AM, Daniel Walsh wrote:
> On 04/27/2017 06:44 AM, Giuseppe Scrivano wrote:

>> Mounting a partition at /var wouldn't work with system containers.
>>
>> System containers are stored in the OSTree storage and on Atomic Host
>> they are checked out to /ostree/deploy/$OS/var/lib/containers/atomic/ so
>> that the checkout and the OSTree storage are on the same file system.
>> This is required to use hard links instead of copying files from OSTree.
>>
>> Regards,
>> Giuseppe
> 
> Thanks for giving us a clue.   This breaks the assumptions that spawned 
> this conversation.

yes it does break the assumptions

> 
> We want to keep system containers on the same file system as /usr, and 
> since we use OSTRee
> 
> and most system containers will match the arch, then we should see a lot 
> of sharing so much smaller
> 
> disk usage then if they were standard docker images.   Dusty what do you 
> think?  I guess we should think about increasing the size of the "root" 
> file system to handle the need of system containers.
> 

hmm, i still don't like having possibly unbounded use of root FS for system
containers - if not much is shared then a lot more space could be used.

i'm going to show how little I know with this question, but would it be possible
to have a separate partition for system containers that was essentially xfs + 
an 
overlayfs of the host filesystem?
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

Re: [atomic-devel] Storage for system containers

[Josh Berkus]

On 04/25/2017 12:08 PM, Dusty Mabe wrote:
> 
> 
> On 04/25/2017 08:25 AM, Daniel Walsh wrote:
>> On 04/24/2017 01:56 PM, Dusty Mabe wrote:

>> Perhaps we should just mount a partition at /var or move /var/lib/docker 
>> to /var/lib/containers/docker and make a symbolic link from 
>> /var/lib/docker-> /var/lib/containers/docker.
>>
> 
> I like the approach of /var/lib/docker -> /var/lib/containers/docker.
> 
> Is this something we should pursue?

Aren't we going to have to call it /var/lib/containers/moby pretty soon?

-- 
--
Josh Berkus
Project Atomic
Red Hat OSAS
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

Re: [atomic-devel] Storage for system containers

[Daniel Walsh]

Also rm -rf /var/lib/docker in a devicemapper world is not a good idea. 
You end up in a strange world which could leak devices and resources.


atomic storage reset

Is the preferred way.

On 04/24/2017 11:57 PM, Ben Breard wrote:
The only issue I have with using the same location is that when 
troubleshooting, it's fairly common to wipe the storage pool. I think 
we'd want users to rm -rf /var/lib/docker without worrying about 
removing system containers.


Is this still an issue after the current partition scheme moves to 
OverlayFS?


On Mon, Apr 24, 2017 at 12:56 PM, Dusty Mabe > wrote:


NOTE: please reply-all when responding to this message


In Fedora Atomic Host if we use system containers as advertised
we end up using `atomic pull --storage ostree` which by default
throws images into /var/lib/containers/atomic/. This is on the
root filesystem which may be undesirable.

Since in Fedora 26 the new version of container-storage-setup allows
us greater control over a "CONTAINER_ROOT" should we consider trying
to make sure both ostree storage and docker storage get placed under
that CONTAINER_ROOT?

The current default [1] is to just mount the CONTAINER_ROOT on
/var/lib/docker.

Dusty

[1]

https://src.fedoraproject.org/cgit/rpms/docker.git/tree/docker.spec?h=f26#n535






--

Ben Breard
Sr Technology Product Manager - Linux Containers
Mobile: 972-816-9081



___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

Re: [atomic-devel] Storage for system containers

[Daniel Walsh]

If we move the link under /var/lib/containers/docker then removing this 
would not affect /var/lib/containers/ostree or /var/lib/containers/storage



On 04/24/2017 11:57 PM, Ben Breard wrote:
The only issue I have with using the same location is that when 
troubleshooting, it's fairly common to wipe the storage pool. I think 
we'd want users to rm -rf /var/lib/docker without worrying about 
removing system containers.


Is this still an issue after the current partition scheme moves to 
OverlayFS?


On Mon, Apr 24, 2017 at 12:56 PM, Dusty Mabe > wrote:


NOTE: please reply-all when responding to this message


In Fedora Atomic Host if we use system containers as advertised
we end up using `atomic pull --storage ostree` which by default
throws images into /var/lib/containers/atomic/. This is on the
root filesystem which may be undesirable.

Since in Fedora 26 the new version of container-storage-setup allows
us greater control over a "CONTAINER_ROOT" should we consider trying
to make sure both ostree storage and docker storage get placed under
that CONTAINER_ROOT?

The current default [1] is to just mount the CONTAINER_ROOT on
/var/lib/docker.

Dusty

[1]

https://src.fedoraproject.org/cgit/rpms/docker.git/tree/docker.spec?h=f26#n535






--

Ben Breard
Sr Technology Product Manager - Linux Containers
Mobile: 972-816-9081



___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

Re: [atomic-devel] Storage for system containers

[Daniel Walsh]

On 04/24/2017 01:56 PM, Dusty Mabe wrote:

NOTE: please reply-all when responding to this message


In Fedora Atomic Host if we use system containers as advertised
we end up using `atomic pull --storage ostree` which by default
throws images into /var/lib/containers/atomic/. This is on the
root filesystem which may be undesirable.

Since in Fedora 26 the new version of container-storage-setup allows
us greater control over a "CONTAINER_ROOT" should we consider trying
to make sure both ostree storage and docker storage get placed under
that CONTAINER_ROOT?

The current default [1] is to just mount the CONTAINER_ROOT on
/var/lib/docker.

Dusty

[1] 
https://src.fedoraproject.org/cgit/rpms/docker.git/tree/docker.spec?h=f26#n535

Perhaps we should just mount a partition at /var or move /var/lib/docker 
to /var/lib/containers/docker and make a symbolic link from 
/var/lib/docker-> /var/lib/containers/docker.

___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org