Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Wed, Nov 16, 2016 at 02:56:35PM -0500, Colin Walters wrote:
> On Wed, Nov 16, 2016, at 02:49 PM, Stephen Gallagher wrote:
> 
> > Today, Fedora Server relies on whatever is the default for 
> > docker-storage-setup.
> > We just tell Anaconda to reserve up to 15GiB by default for the / partition 
> > and
> > then it puts all remaining free space (on drives selected to be used by
> > Anaconda) into a single logical volume with no partitions.
> > 
> > It's a very easy thing for us to drop a different config file for
> > docker-storage-setup into place for Server. So if that's all we need to do, 
> > let
> > me know and I'll work it up.
> 
> If anyone has time to work on this, I don't see a need for Atomic Host
> and Server to have different partitioning defaults, so if we can just
> merge them, that'd be nice.

Agreed. 

Vivek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Tue, Nov 22, 2016 at 11:24:04PM -, Josh Berkus wrote:
> Vivek, Dan,
> 
> > - Now when docker users overlay2 graph driver, all the images, containers
> >   and associated metadata will be stored outside the root filesystem and
> >   onto /dev/docker-vg/foo logical volume.
> 

[ Please don't remove emaild ids from oringal to, cc list. My emails
  get filtered in a folder and I don't scan it often ]

> This is a change from current storage setup?  Right now, containers go in the 
> docker volume, but images do not.  If we can do that, it would be worth 
> having overlay right there.

Right now, with lvm thin pool, both images and containers get stored in
lvm thin pool. Some metadata associated with images and containers remains
on rootfs (in /var/lib/docker/ dir).

> 
> My one concern for Atomic is ... how do existing users upgrade when we make 
> this change?  Does their devicemapper config still just work when they pull 
> the new tree?

There will be no change to existing devicemapper functionality and upgrade
should continue to work fine. 

This new functionality should kick in only if there is no existing setup.
(Signalled by the absense of /etc/sysconfig/docker-storage file).

Otherwise docker-storage-setup can detect that already there is a
configured driver and it will not setup a new one.

Vivek
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Daniel J Walsh]

On 11/22/2016 06:24 PM, Josh Berkus wrote:
> Vivek, Dan,
>
>> - Now when docker users overlay2 graph driver, all the images, containers
>>   and associated metadata will be stored outside the root filesystem and
>>   onto /dev/docker-vg/foo logical volume.
> This is a change from current storage setup?  Right now, containers go in the 
> docker volume, but images do not.  If we can do that, it would be worth 
> having overlay right there.
>
> My one concern for Atomic is ... how do existing users upgrade when we make 
> this change?  Does their devicemapper config still just work when they pull 
> the new tree?
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
>
They don't we to have tools to allow a user to convert from devicemapper
backend to overlay.  And we can document that in a blog once we make a
change.
I think there is an existing blog that covers it.

Sadly I can not find it right now.

But you can do something like:

atomic storage export
atomic storage reset
atomic storage modify --driver overlay2
atomic storage import

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Josh Berkus]

Vivek, Dan,

> - Now when docker users overlay2 graph driver, all the images, containers
>   and associated metadata will be stored outside the root filesystem and
>   onto /dev/docker-vg/foo logical volume.

This is a change from current storage setup?  Right now, containers go in the 
docker volume, but images do not.  If we can do that, it would be worth having 
overlay right there.

My one concern for Atomic is ... how do existing users upgrade when we make 
this change?  Does their devicemapper config still just work when they pull the 
new tree?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Daniel J Walsh]

No as long as the storage is modified before the first start of the
docker daemon and the first execution of docker-storage-setup, it should
be fine.


On 11/17/2016 07:54 PM, Subhendu Ghosh wrote:
>
> Assuming cloud-init can also select the storage or is that too late in
> the process?
>
>
> On Nov 16, 2016 15:58, "Vivek Goyal"  > wrote:
>
> On Wed, Nov 16, 2016 at 03:19:06PM -0500, Stephen Gallagher wrote:
> > On 11/16/2016 03:09 PM, Vivek Goyal wrote:
> > > On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
> > >> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
> > >>> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher
> wrote:
> >  On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> > > On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh
> wrote:
> > >> We would like to change the docker container storage to
> default to
> > >> Overlayfs2 in Fedora 26.  But we have a problem on Atomic
> Host and
> > >> Fedora Server distributions.
> > >>
> > >>
> > >> Currently docker-storage-setup defaults to devicemapper
> and is hard
> > >> coded to setup a thinpool of 40% of remaining disk. 
> Otherwise it sets
> > >> up loopback devices on the root file system. 
>  Devicemapper is nice
> > >> since it works with thinpools and can automatically
> expand the storage
> > >> if the disk space is getting used up.
> > >>
> > >> Moving to Overlay, we can more easily use the root file
> system directly,
> > >> which would be fine for Fedora Workstation.  We want to
> preserve the use
> > >> of the remaining storage for Overlay on AH and Fedora
> Server,  since
> > >> this would give a user flexibility to switch back to
> using devicemapper
> > >> if they had problems with the Overlay driver.
> > >
> > > And being able to do so basically involves following.
> > >
> > > - docker-storage-setup creates a logical volume from free
> space
> > > - Creates a filesystem on that logical volume
> > > - Mounts that logical volume on the directory which docker
> is going to
> > >   use.
> > >
> > >   mount /dev/docker-vg/foo /var/lib/docker/
> > >
> > > - Now when docker users overlay2 graph driver, all the
> images, containers
> > >   and associated metadata will be stored outside the root
> filesystem and
> > >   onto /dev/docker-vg/foo logical volume.
> > >
> > >> We can not as easily
> > >> support the expanding disk for Overlay since we will not
> use using thinpool.
> > >
> > >>
> > >> We have looked at options to hard code OverlayFS with the
> defaults,
> > >
> > > If we always mount /var/lib/docker on on /dev/vg/foo for
> overlay2 driver
> > > this will be a regression w.r.t current behavior. So I
> would not
> > > recommend changing current behavior. I think this should
> be an opt-in.
> > > We are working on provide a config knob to elect this
> behavior and
> > > atomic host and fedora server will have to opt-in somehow.
> > >
> > > I think it will be easy for atomic host as they already
> drop something
> > > in /etc/sysconfig/docker-storage-setup. Not sure how
> fedora server
> > > variant will do it.
> > >
> > 
> > 
> >  Today, Fedora Server relies on whatever is the default for
> docker-storage-setup.
> >  We just tell Anaconda to reserve up to 15GiB by default for
> the / partition and
> >  then it puts all remaining free space (on drives selected
> to be used by
> >  Anaconda) into a single logical volume with no partitions.
> > 
> >  It's a very easy thing for us to drop a different config
> file for
> >  docker-storage-setup into place for Server. So if that's
> all we need to do, let
> >  me know and I'll work it up.
> > >>>
> > >>> Ok, that sounds good. We are working on providing a knob to
> opt-in new
> > >>> behavior. I think all you have to drop in config file will
> be something
> > >>> like.
> > >>>
> > >>> /etc/sysconfig/docker-storage-setup
> > >>>
> > >>> STORAGE_DRIVER=overlay2
> > >>> YET_TO_BE_NAMED_OPTION=VAL
> > >>>
> > >>> So upstream default will continue to be devicemapper. We
> will have to
> > >>> modify fedora workstation, fedora server and atomic host
> infrastructure
> > >>> to opt-in for overlay2.
> > >>>
> > >>
> > >> Why exactly does this need to be opt-in? Why wouldn't we just
> change the default
> > >> on Fedora Server to use overlay2 instead of 

Re: Modifying container storage for Fedora 26.

[Denise Dumas]

So are we going to see you tomorrow at BoA? How is it going? 
Denise 

> On Nov 17, 2016, at 7:55 PM, Subhendu Ghosh  wrote:
> 
> Assuming cloud-init can also select the storage or is that too late in the 
> process?
> 
> 
>> On Nov 16, 2016 15:58, "Vivek Goyal"  wrote:
>> On Wed, Nov 16, 2016 at 03:19:06PM -0500, Stephen Gallagher wrote:
>> > On 11/16/2016 03:09 PM, Vivek Goyal wrote:
>> > > On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
>> > >> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
>> > >>> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
>> >  On 11/16/2016 02:40 PM, Vivek Goyal wrote:
>> > > On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
>> > >> We would like to change the docker container storage to default to
>> > >> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
>> > >> Fedora Server distributions.
>> > >>
>> > >>
>> > >> Currently docker-storage-setup defaults to devicemapper and is hard
>> > >> coded to setup a thinpool of 40% of remaining disk.  Otherwise it 
>> > >> sets
>> > >> up loopback devices on the root file system.   Devicemapper is nice
>> > >> since it works with thinpools and can automatically expand the 
>> > >> storage
>> > >> if the disk space is getting used up.
>> > >>
>> > >> Moving to Overlay, we can more easily use the root file system 
>> > >> directly,
>> > >> which would be fine for Fedora Workstation.  We want to preserve 
>> > >> the use
>> > >> of the remaining storage for Overlay on AH and Fedora Server,  since
>> > >> this would give a user flexibility to switch back to using 
>> > >> devicemapper
>> > >> if they had problems with the Overlay driver.
>> > >
>> > > And being able to do so basically involves following.
>> > >
>> > > - docker-storage-setup creates a logical volume from free space
>> > > - Creates a filesystem on that logical volume
>> > > - Mounts that logical volume on the directory which docker is going 
>> > > to
>> > >   use.
>> > >
>> > >   mount /dev/docker-vg/foo /var/lib/docker/
>> > >
>> > > - Now when docker users overlay2 graph driver, all the images, 
>> > > containers
>> > >   and associated metadata will be stored outside the root filesystem 
>> > > and
>> > >   onto /dev/docker-vg/foo logical volume.
>> > >
>> > >> We can not as easily
>> > >> support the expanding disk for Overlay since we will not use using 
>> > >> thinpool.
>> > >
>> > >>
>> > >> We have looked at options to hard code OverlayFS with the defaults,
>> > >
>> > > If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 
>> > > driver
>> > > this will be a regression w.r.t current behavior. So I would not
>> > > recommend changing current behavior. I think this should be an 
>> > > opt-in.
>> > > We are working on provide a config knob to elect this behavior and
>> > > atomic host and fedora server will have to opt-in somehow.
>> > >
>> > > I think it will be easy for atomic host as they already drop 
>> > > something
>> > > in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
>> > > variant will do it.
>> > >
>> > 
>> > 
>> >  Today, Fedora Server relies on whatever is the default for 
>> >  docker-storage-setup.
>> >  We just tell Anaconda to reserve up to 15GiB by default for the / 
>> >  partition and
>> >  then it puts all remaining free space (on drives selected to be used 
>> >  by
>> >  Anaconda) into a single logical volume with no partitions.
>> > 
>> >  It's a very easy thing for us to drop a different config file for
>> >  docker-storage-setup into place for Server. So if that's all we need 
>> >  to do, let
>> >  me know and I'll work it up.
>> > >>>
>> > >>> Ok, that sounds good. We are working on providing a knob to opt-in new
>> > >>> behavior. I think all you have to drop in config file will be something
>> > >>> like.
>> > >>>
>> > >>> /etc/sysconfig/docker-storage-setup
>> > >>>
>> > >>> STORAGE_DRIVER=overlay2
>> > >>> YET_TO_BE_NAMED_OPTION=VAL
>> > >>>
>> > >>> So upstream default will continue to be devicemapper. We will have to
>> > >>> modify fedora workstation, fedora server and atomic host infrastructure
>> > >>> to opt-in for overlay2.
>> > >>>
>> > >>
>> > >> Why exactly does this need to be opt-in? Why wouldn't we just change 
>> > >> the default
>> > >> on Fedora Server to use overlay2 instead of devicemapper?
>> > >>
>> > >> I think I'm missing some key part of the problem here.
>> > >
>> > > I mean it will be devicemapper in upstream project. And distributions 
>> > > will
>> > > have to opt-in for overlay2.
>> > >
>> > > And I think one reason being that rhel uses same git tree and we don't
>> > > want to 

Re: Modifying container storage for Fedora 26.

[Subhendu Ghosh]

Assuming cloud-init can also select the storage or is that too late in the
process?

On Nov 16, 2016 15:58, "Vivek Goyal"  wrote:

> On Wed, Nov 16, 2016 at 03:19:06PM -0500, Stephen Gallagher wrote:
> > On 11/16/2016 03:09 PM, Vivek Goyal wrote:
> > > On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
> > >> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
> > >>> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
> >  On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> > > On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
> > >> We would like to change the docker container storage to default to
> > >> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
> > >> Fedora Server distributions.
> > >>
> > >>
> > >> Currently docker-storage-setup defaults to devicemapper and is
> hard
> > >> coded to setup a thinpool of 40% of remaining disk.  Otherwise it
> sets
> > >> up loopback devices on the root file system.   Devicemapper is
> nice
> > >> since it works with thinpools and can automatically expand the
> storage
> > >> if the disk space is getting used up.
> > >>
> > >> Moving to Overlay, we can more easily use the root file system
> directly,
> > >> which would be fine for Fedora Workstation.  We want to preserve
> the use
> > >> of the remaining storage for Overlay on AH and Fedora Server,
> since
> > >> this would give a user flexibility to switch back to using
> devicemapper
> > >> if they had problems with the Overlay driver.
> > >
> > > And being able to do so basically involves following.
> > >
> > > - docker-storage-setup creates a logical volume from free space
> > > - Creates a filesystem on that logical volume
> > > - Mounts that logical volume on the directory which docker is
> going to
> > >   use.
> > >
> > >   mount /dev/docker-vg/foo /var/lib/docker/
> > >
> > > - Now when docker users overlay2 graph driver, all the images,
> containers
> > >   and associated metadata will be stored outside the root
> filesystem and
> > >   onto /dev/docker-vg/foo logical volume.
> > >
> > >> We can not as easily
> > >> support the expanding disk for Overlay since we will not use
> using thinpool.
> > >
> > >>
> > >> We have looked at options to hard code OverlayFS with the
> defaults,
> > >
> > > If we always mount /var/lib/docker on on /dev/vg/foo for overlay2
> driver
> > > this will be a regression w.r.t current behavior. So I would not
> > > recommend changing current behavior. I think this should be an
> opt-in.
> > > We are working on provide a config knob to elect this behavior and
> > > atomic host and fedora server will have to opt-in somehow.
> > >
> > > I think it will be easy for atomic host as they already drop
> something
> > > in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> > > variant will do it.
> > >
> > 
> > 
> >  Today, Fedora Server relies on whatever is the default for
> docker-storage-setup.
> >  We just tell Anaconda to reserve up to 15GiB by default for the /
> partition and
> >  then it puts all remaining free space (on drives selected to be
> used by
> >  Anaconda) into a single logical volume with no partitions.
> > 
> >  It's a very easy thing for us to drop a different config file for
> >  docker-storage-setup into place for Server. So if that's all we
> need to do, let
> >  me know and I'll work it up.
> > >>>
> > >>> Ok, that sounds good. We are working on providing a knob to opt-in
> new
> > >>> behavior. I think all you have to drop in config file will be
> something
> > >>> like.
> > >>>
> > >>> /etc/sysconfig/docker-storage-setup
> > >>>
> > >>> STORAGE_DRIVER=overlay2
> > >>> YET_TO_BE_NAMED_OPTION=VAL
> > >>>
> > >>> So upstream default will continue to be devicemapper. We will have to
> > >>> modify fedora workstation, fedora server and atomic host
> infrastructure
> > >>> to opt-in for overlay2.
> > >>>
> > >>
> > >> Why exactly does this need to be opt-in? Why wouldn't we just change
> the default
> > >> on Fedora Server to use overlay2 instead of devicemapper?
> > >>
> > >> I think I'm missing some key part of the problem here.
> > >
> > > I mean it will be devicemapper in upstream project. And distributions
> will
> > > have to opt-in for overlay2.
> > >
> > > And I think one reason being that rhel uses same git tree and we don't
> > > want to switch to overlay2 by default for rhel yet.
> > >
> > > overlay2 will be an experiment on fedora first as default and if it
> works
> > > well, then change default upstream too.
> >
> > OK, when you said "opt-in", I was assuming you meant that from the user's
> > perspective (as in having the user make an explicit choice). What you
> meant was
> > that Server would carry a different default from the upstream 

Re: Modifying container storage for Fedora 26.

[Chris Murphy]

Does it make sense to move all the products to XFS? An exception may
be possible for cloud images [1]

Server uses XFS. Cloud/Atomic have expressed a preference for it. I
found that ext4 allocates inodes at mkfs time, and there's no way to
get more later, and there's a real possibility with overlayfs that
there can be inode exhaustion. [2] Yes, you can just use non-default
mkfs options to create more inodes, but installer folks are typically
reluctant to use non-default mkfs options. XFS and Btrfs have
dynamically allocated inodes.

Is there any advantage of LVM for Cloud/Atomic? What about
Workstation? Dropping it for cloud/atomic may be an advantage, more
compact, less wasted space, less overhead, one less layer. For
Workstation it's probably a wash. Many users are confused by it, few
benefit from it. But the documentation refers to it.


---
Chris Murphy


[1]
Syslinux is preferred because it significantly reduces cloud image
size. The problem is this extlinux + ext4 bug. While syslinux 6.04
supports XFS, and builds are in koji, it's not finalized yet upstream.
So if this syslinux + ext4 bug gets fixed, then use use that
combination. Otherwise, GRUB + XFS.
Fedora 25 cloud images built with syslinux do not boot
https://bugzilla.redhat.com/show_bug.cgi?id=1369934

[2]
https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/message/47PC3YHBR7LTIDSDCZERROWRLEFOSVP7/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Wed, Nov 16, 2016 at 03:19:06PM -0500, Stephen Gallagher wrote:
> On 11/16/2016 03:09 PM, Vivek Goyal wrote:
> > On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
> >> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
> >>> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
>  On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> > On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
> >> We would like to change the docker container storage to default to
> >> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
> >> Fedora Server distributions.
> >>
> >>
> >> Currently docker-storage-setup defaults to devicemapper and is hard
> >> coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
> >> up loopback devices on the root file system.   Devicemapper is nice
> >> since it works with thinpools and can automatically expand the storage
> >> if the disk space is getting used up. 
> >>
> >> Moving to Overlay, we can more easily use the root file system 
> >> directly,
> >> which would be fine for Fedora Workstation.  We want to preserve the 
> >> use
> >> of the remaining storage for Overlay on AH and Fedora Server,  since
> >> this would give a user flexibility to switch back to using devicemapper
> >> if they had problems with the Overlay driver.
> >
> > And being able to do so basically involves following.
> >
> > - docker-storage-setup creates a logical volume from free space
> > - Creates a filesystem on that logical volume
> > - Mounts that logical volume on the directory which docker is going to
> >   use.
> >
> >   mount /dev/docker-vg/foo /var/lib/docker/
> >
> > - Now when docker users overlay2 graph driver, all the images, 
> > containers
> >   and associated metadata will be stored outside the root filesystem and
> >   onto /dev/docker-vg/foo logical volume.
> >
> >> We can not as easily
> >> support the expanding disk for Overlay since we will not use using 
> >> thinpool.
> >
> >>
> >> We have looked at options to hard code OverlayFS with the defaults,
> >
> > If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
> > this will be a regression w.r.t current behavior. So I would not
> > recommend changing current behavior. I think this should be an opt-in.
> > We are working on provide a config knob to elect this behavior and
> > atomic host and fedora server will have to opt-in somehow.
> >
> > I think it will be easy for atomic host as they already drop something
> > in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> > variant will do it.
> >
> 
> 
>  Today, Fedora Server relies on whatever is the default for 
>  docker-storage-setup.
>  We just tell Anaconda to reserve up to 15GiB by default for the / 
>  partition and
>  then it puts all remaining free space (on drives selected to be used by
>  Anaconda) into a single logical volume with no partitions.
> 
>  It's a very easy thing for us to drop a different config file for
>  docker-storage-setup into place for Server. So if that's all we need to 
>  do, let
>  me know and I'll work it up.
> >>>
> >>> Ok, that sounds good. We are working on providing a knob to opt-in new
> >>> behavior. I think all you have to drop in config file will be something
> >>> like.
> >>>
> >>> /etc/sysconfig/docker-storage-setup
> >>>
> >>> STORAGE_DRIVER=overlay2
> >>> YET_TO_BE_NAMED_OPTION=VAL
> >>>
> >>> So upstream default will continue to be devicemapper. We will have to
> >>> modify fedora workstation, fedora server and atomic host infrastructure
> >>> to opt-in for overlay2.
> >>>
> >>
> >> Why exactly does this need to be opt-in? Why wouldn't we just change the 
> >> default
> >> on Fedora Server to use overlay2 instead of devicemapper?
> >>
> >> I think I'm missing some key part of the problem here.
> > 
> > I mean it will be devicemapper in upstream project. And distributions will
> > have to opt-in for overlay2. 
> > 
> > And I think one reason being that rhel uses same git tree and we don't
> > want to switch to overlay2 by default for rhel yet.
> > 
> > overlay2 will be an experiment on fedora first as default and if it works
> > well, then change default upstream too.
> 
> OK, when you said "opt-in", I was assuming you meant that from the user's
> perspective (as in having the user make an explicit choice). What you meant 
> was
> that Server would carry a different default from the upstream project. That's
> easy enough to accomplish (we can do the same for Atomic as well).
> 
> The packaging guidelines for this are here:
> https://fedoraproject.org/wiki/Packaging:Per-Product_Configuration
> 
> Of course, that's only necessary if we want Server (and Atomic) to differ from
> Workstation and/or 

Re: Modifying container storage for Fedora 26.

[Stephen Gallagher]

On 11/16/2016 03:09 PM, Vivek Goyal wrote:
> On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
>> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
>>> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
 On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
>> We would like to change the docker container storage to default to
>> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
>> Fedora Server distributions.
>>
>>
>> Currently docker-storage-setup defaults to devicemapper and is hard
>> coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
>> up loopback devices on the root file system.   Devicemapper is nice
>> since it works with thinpools and can automatically expand the storage
>> if the disk space is getting used up. 
>>
>> Moving to Overlay, we can more easily use the root file system directly,
>> which would be fine for Fedora Workstation.  We want to preserve the use
>> of the remaining storage for Overlay on AH and Fedora Server,  since
>> this would give a user flexibility to switch back to using devicemapper
>> if they had problems with the Overlay driver.
>
> And being able to do so basically involves following.
>
> - docker-storage-setup creates a logical volume from free space
> - Creates a filesystem on that logical volume
> - Mounts that logical volume on the directory which docker is going to
>   use.
>
>   mount /dev/docker-vg/foo /var/lib/docker/
>
> - Now when docker users overlay2 graph driver, all the images, containers
>   and associated metadata will be stored outside the root filesystem and
>   onto /dev/docker-vg/foo logical volume.
>
>> We can not as easily
>> support the expanding disk for Overlay since we will not use using 
>> thinpool.
>
>>
>> We have looked at options to hard code OverlayFS with the defaults,
>
> If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
> this will be a regression w.r.t current behavior. So I would not
> recommend changing current behavior. I think this should be an opt-in.
> We are working on provide a config knob to elect this behavior and
> atomic host and fedora server will have to opt-in somehow.
>
> I think it will be easy for atomic host as they already drop something
> in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> variant will do it.
>


 Today, Fedora Server relies on whatever is the default for 
 docker-storage-setup.
 We just tell Anaconda to reserve up to 15GiB by default for the / 
 partition and
 then it puts all remaining free space (on drives selected to be used by
 Anaconda) into a single logical volume with no partitions.

 It's a very easy thing for us to drop a different config file for
 docker-storage-setup into place for Server. So if that's all we need to 
 do, let
 me know and I'll work it up.
>>>
>>> Ok, that sounds good. We are working on providing a knob to opt-in new
>>> behavior. I think all you have to drop in config file will be something
>>> like.
>>>
>>> /etc/sysconfig/docker-storage-setup
>>>
>>> STORAGE_DRIVER=overlay2
>>> YET_TO_BE_NAMED_OPTION=VAL
>>>
>>> So upstream default will continue to be devicemapper. We will have to
>>> modify fedora workstation, fedora server and atomic host infrastructure
>>> to opt-in for overlay2.
>>>
>>
>> Why exactly does this need to be opt-in? Why wouldn't we just change the 
>> default
>> on Fedora Server to use overlay2 instead of devicemapper?
>>
>> I think I'm missing some key part of the problem here.
> 
> I mean it will be devicemapper in upstream project. And distributions will
> have to opt-in for overlay2. 
> 
> And I think one reason being that rhel uses same git tree and we don't
> want to switch to overlay2 by default for rhel yet.
> 
> overlay2 will be an experiment on fedora first as default and if it works
> well, then change default upstream too.

OK, when you said "opt-in", I was assuming you meant that from the user's
perspective (as in having the user make an explicit choice). What you meant was
that Server would carry a different default from the upstream project. That's
easy enough to accomplish (we can do the same for Atomic as well).

The packaging guidelines for this are here:
https://fedoraproject.org/wiki/Packaging:Per-Product_Configuration

Of course, that's only necessary if we want Server (and Atomic) to differ from
Workstation and/or non-productized Fedora. If they're all going to switch to
overlay2, then it's just a trivial matter.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to 

Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Wed, Nov 16, 2016 at 03:01:06PM -0500, Stephen Gallagher wrote:
> On 11/16/2016 02:56 PM, Vivek Goyal wrote:
> > On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
> >> On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> >>> On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
>  We would like to change the docker container storage to default to
>  Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
>  Fedora Server distributions.
> 
> 
>  Currently docker-storage-setup defaults to devicemapper and is hard
>  coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
>  up loopback devices on the root file system.   Devicemapper is nice
>  since it works with thinpools and can automatically expand the storage
>  if the disk space is getting used up. 
> 
>  Moving to Overlay, we can more easily use the root file system directly,
>  which would be fine for Fedora Workstation.  We want to preserve the use
>  of the remaining storage for Overlay on AH and Fedora Server,  since
>  this would give a user flexibility to switch back to using devicemapper
>  if they had problems with the Overlay driver.
> >>>
> >>> And being able to do so basically involves following.
> >>>
> >>> - docker-storage-setup creates a logical volume from free space
> >>> - Creates a filesystem on that logical volume
> >>> - Mounts that logical volume on the directory which docker is going to
> >>>   use.
> >>>
> >>>   mount /dev/docker-vg/foo /var/lib/docker/
> >>>
> >>> - Now when docker users overlay2 graph driver, all the images, containers
> >>>   and associated metadata will be stored outside the root filesystem and
> >>>   onto /dev/docker-vg/foo logical volume.
> >>>
>  We can not as easily
>  support the expanding disk for Overlay since we will not use using 
>  thinpool.
> >>>
> 
>  We have looked at options to hard code OverlayFS with the defaults,
> >>>
> >>> If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
> >>> this will be a regression w.r.t current behavior. So I would not
> >>> recommend changing current behavior. I think this should be an opt-in.
> >>> We are working on provide a config knob to elect this behavior and
> >>> atomic host and fedora server will have to opt-in somehow.
> >>>
> >>> I think it will be easy for atomic host as they already drop something
> >>> in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> >>> variant will do it.
> >>>
> >>
> >>
> >> Today, Fedora Server relies on whatever is the default for 
> >> docker-storage-setup.
> >> We just tell Anaconda to reserve up to 15GiB by default for the / 
> >> partition and
> >> then it puts all remaining free space (on drives selected to be used by
> >> Anaconda) into a single logical volume with no partitions.
> >>
> >> It's a very easy thing for us to drop a different config file for
> >> docker-storage-setup into place for Server. So if that's all we need to 
> >> do, let
> >> me know and I'll work it up.
> > 
> > Ok, that sounds good. We are working on providing a knob to opt-in new
> > behavior. I think all you have to drop in config file will be something
> > like.
> > 
> > /etc/sysconfig/docker-storage-setup
> > 
> > STORAGE_DRIVER=overlay2
> > YET_TO_BE_NAMED_OPTION=VAL
> > 
> > So upstream default will continue to be devicemapper. We will have to
> > modify fedora workstation, fedora server and atomic host infrastructure
> > to opt-in for overlay2.
> > 
> 
> Why exactly does this need to be opt-in? Why wouldn't we just change the 
> default
> on Fedora Server to use overlay2 instead of devicemapper?
> 
> I think I'm missing some key part of the problem here.

I mean it will be devicemapper in upstream project. And distributions will
have to opt-in for overlay2. 

And I think one reason being that rhel uses same git tree and we don't
want to switch to overlay2 by default for rhel yet.

overlay2 will be an experiment on fedora first as default and if it works
well, then change default upstream too.

Vivek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Stephen Gallagher]

On 11/16/2016 02:56 PM, Vivek Goyal wrote:
> On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
>> On 11/16/2016 02:40 PM, Vivek Goyal wrote:
>>> On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
 We would like to change the docker container storage to default to
 Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
 Fedora Server distributions.


 Currently docker-storage-setup defaults to devicemapper and is hard
 coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
 up loopback devices on the root file system.   Devicemapper is nice
 since it works with thinpools and can automatically expand the storage
 if the disk space is getting used up. 

 Moving to Overlay, we can more easily use the root file system directly,
 which would be fine for Fedora Workstation.  We want to preserve the use
 of the remaining storage for Overlay on AH and Fedora Server,  since
 this would give a user flexibility to switch back to using devicemapper
 if they had problems with the Overlay driver.
>>>
>>> And being able to do so basically involves following.
>>>
>>> - docker-storage-setup creates a logical volume from free space
>>> - Creates a filesystem on that logical volume
>>> - Mounts that logical volume on the directory which docker is going to
>>>   use.
>>>
>>>   mount /dev/docker-vg/foo /var/lib/docker/
>>>
>>> - Now when docker users overlay2 graph driver, all the images, containers
>>>   and associated metadata will be stored outside the root filesystem and
>>>   onto /dev/docker-vg/foo logical volume.
>>>
 We can not as easily
 support the expanding disk for Overlay since we will not use using 
 thinpool.
>>>

 We have looked at options to hard code OverlayFS with the defaults,
>>>
>>> If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
>>> this will be a regression w.r.t current behavior. So I would not
>>> recommend changing current behavior. I think this should be an opt-in.
>>> We are working on provide a config knob to elect this behavior and
>>> atomic host and fedora server will have to opt-in somehow.
>>>
>>> I think it will be easy for atomic host as they already drop something
>>> in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
>>> variant will do it.
>>>
>>
>>
>> Today, Fedora Server relies on whatever is the default for 
>> docker-storage-setup.
>> We just tell Anaconda to reserve up to 15GiB by default for the / partition 
>> and
>> then it puts all remaining free space (on drives selected to be used by
>> Anaconda) into a single logical volume with no partitions.
>>
>> It's a very easy thing for us to drop a different config file for
>> docker-storage-setup into place for Server. So if that's all we need to do, 
>> let
>> me know and I'll work it up.
> 
> Ok, that sounds good. We are working on providing a knob to opt-in new
> behavior. I think all you have to drop in config file will be something
> like.
> 
> /etc/sysconfig/docker-storage-setup
> 
> STORAGE_DRIVER=overlay2
> YET_TO_BE_NAMED_OPTION=VAL
> 
> So upstream default will continue to be devicemapper. We will have to
> modify fedora workstation, fedora server and atomic host infrastructure
> to opt-in for overlay2.
> 

Why exactly does this need to be opt-in? Why wouldn't we just change the default
on Fedora Server to use overlay2 instead of devicemapper?

I think I'm missing some key part of the problem here.




signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Colin Walters]

On Wed, Nov 16, 2016, at 02:49 PM, Stephen Gallagher wrote:

> Today, Fedora Server relies on whatever is the default for 
> docker-storage-setup.
> We just tell Anaconda to reserve up to 15GiB by default for the / partition 
> and
> then it puts all remaining free space (on drives selected to be used by
> Anaconda) into a single logical volume with no partitions.
> 
> It's a very easy thing for us to drop a different config file for
> docker-storage-setup into place for Server. So if that's all we need to do, 
> let
> me know and I'll work it up.

If anyone has time to work on this, I don't see a need for Atomic Host
and Server to have different partitioning defaults, so if we can just
merge them, that'd be nice.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Wed, Nov 16, 2016 at 02:49:25PM -0500, Stephen Gallagher wrote:
> On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> > On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
> >> We would like to change the docker container storage to default to
> >> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
> >> Fedora Server distributions.
> >>
> >>
> >> Currently docker-storage-setup defaults to devicemapper and is hard
> >> coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
> >> up loopback devices on the root file system.   Devicemapper is nice
> >> since it works with thinpools and can automatically expand the storage
> >> if the disk space is getting used up. 
> >>
> >> Moving to Overlay, we can more easily use the root file system directly,
> >> which would be fine for Fedora Workstation.  We want to preserve the use
> >> of the remaining storage for Overlay on AH and Fedora Server,  since
> >> this would give a user flexibility to switch back to using devicemapper
> >> if they had problems with the Overlay driver.
> > 
> > And being able to do so basically involves following.
> > 
> > - docker-storage-setup creates a logical volume from free space
> > - Creates a filesystem on that logical volume
> > - Mounts that logical volume on the directory which docker is going to
> >   use.
> > 
> >   mount /dev/docker-vg/foo /var/lib/docker/
> > 
> > - Now when docker users overlay2 graph driver, all the images, containers
> >   and associated metadata will be stored outside the root filesystem and
> >   onto /dev/docker-vg/foo logical volume.
> > 
> >> We can not as easily
> >> support the expanding disk for Overlay since we will not use using 
> >> thinpool.
> > 
> >>
> >> We have looked at options to hard code OverlayFS with the defaults,
> > 
> > If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
> > this will be a regression w.r.t current behavior. So I would not
> > recommend changing current behavior. I think this should be an opt-in.
> > We are working on provide a config knob to elect this behavior and
> > atomic host and fedora server will have to opt-in somehow.
> > 
> > I think it will be easy for atomic host as they already drop something
> > in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> > variant will do it.
> > 
> 
> 
> Today, Fedora Server relies on whatever is the default for 
> docker-storage-setup.
> We just tell Anaconda to reserve up to 15GiB by default for the / partition 
> and
> then it puts all remaining free space (on drives selected to be used by
> Anaconda) into a single logical volume with no partitions.
> 
> It's a very easy thing for us to drop a different config file for
> docker-storage-setup into place for Server. So if that's all we need to do, 
> let
> me know and I'll work it up.

Ok, that sounds good. We are working on providing a knob to opt-in new
behavior. I think all you have to drop in config file will be something
like.

/etc/sysconfig/docker-storage-setup

STORAGE_DRIVER=overlay2
YET_TO_BE_NAMED_OPTION=VAL

So upstream default will continue to be devicemapper. We will have to
modify fedora workstation, fedora server and atomic host infrastructure
to opt-in for overlay2.

Vivek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Stephen Gallagher]

On 11/16/2016 02:40 PM, Vivek Goyal wrote:
> On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
>> We would like to change the docker container storage to default to
>> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
>> Fedora Server distributions.
>>
>>
>> Currently docker-storage-setup defaults to devicemapper and is hard
>> coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
>> up loopback devices on the root file system.   Devicemapper is nice
>> since it works with thinpools and can automatically expand the storage
>> if the disk space is getting used up. 
>>
>> Moving to Overlay, we can more easily use the root file system directly,
>> which would be fine for Fedora Workstation.  We want to preserve the use
>> of the remaining storage for Overlay on AH and Fedora Server,  since
>> this would give a user flexibility to switch back to using devicemapper
>> if they had problems with the Overlay driver.
> 
> And being able to do so basically involves following.
> 
> - docker-storage-setup creates a logical volume from free space
> - Creates a filesystem on that logical volume
> - Mounts that logical volume on the directory which docker is going to
>   use.
> 
>   mount /dev/docker-vg/foo /var/lib/docker/
> 
> - Now when docker users overlay2 graph driver, all the images, containers
>   and associated metadata will be stored outside the root filesystem and
>   onto /dev/docker-vg/foo logical volume.
> 
>> We can not as easily
>> support the expanding disk for Overlay since we will not use using thinpool.
> 
>>
>> We have looked at options to hard code OverlayFS with the defaults,
> 
> If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
> this will be a regression w.r.t current behavior. So I would not
> recommend changing current behavior. I think this should be an opt-in.
> We are working on provide a config knob to elect this behavior and
> atomic host and fedora server will have to opt-in somehow.
> 
> I think it will be easy for atomic host as they already drop something
> in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
> variant will do it.
> 


Today, Fedora Server relies on whatever is the default for docker-storage-setup.
We just tell Anaconda to reserve up to 15GiB by default for the / partition and
then it puts all remaining free space (on drives selected to be used by
Anaconda) into a single logical volume with no partitions.

It's a very easy thing for us to drop a different config file for
docker-storage-setup into place for Server. So if that's all we need to do, let
me know and I'll work it up.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modifying container storage for Fedora 26.

[Vivek Goyal]

On Wed, Nov 16, 2016 at 02:32:46PM -0500, Daniel J Walsh wrote:
> We would like to change the docker container storage to default to
> Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
> Fedora Server distributions.
> 
> 
> Currently docker-storage-setup defaults to devicemapper and is hard
> coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
> up loopback devices on the root file system.   Devicemapper is nice
> since it works with thinpools and can automatically expand the storage
> if the disk space is getting used up. 
> 
> Moving to Overlay, we can more easily use the root file system directly,
> which would be fine for Fedora Workstation.  We want to preserve the use
> of the remaining storage for Overlay on AH and Fedora Server,  since
> this would give a user flexibility to switch back to using devicemapper
> if they had problems with the Overlay driver.

And being able to do so basically involves following.

- docker-storage-setup creates a logical volume from free space
- Creates a filesystem on that logical volume
- Mounts that logical volume on the directory which docker is going to
  use.

  mount /dev/docker-vg/foo /var/lib/docker/

- Now when docker users overlay2 graph driver, all the images, containers
  and associated metadata will be stored outside the root filesystem and
  onto /dev/docker-vg/foo logical volume.

> We can not as easily
> support the expanding disk for Overlay since we will not use using thinpool.

> 
> We have looked at options to hard code OverlayFS with the defaults,

If we always mount /var/lib/docker on on /dev/vg/foo for overlay2 driver
this will be a regression w.r.t current behavior. So I would not
recommend changing current behavior. I think this should be an opt-in.
We are working on provide a config knob to elect this behavior and
atomic host and fedora server will have to opt-in somehow.

I think it will be easy for atomic host as they already drop something
in /etc/sysconfig/docker-storage-setup. Not sure how fedora server
variant will do it.

Thanks
Vivek

> or
> we could just drop a /etc/sysconfig/docker-storage-setup that specified
> Overlay and the percentage of remaining space to use for the
> /var/lib/docker device.  But what is the best way to set different
> defaults for AH, Fedora Server and Fedora Workstation.
> 
> We would like to discuss with you guys about what  you would think is
> the best way to handle. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Modifying container storage for Fedora 26.

[Daniel J Walsh]

We would like to change the docker container storage to default to
Overlayfs2 in Fedora 26.  But we have a problem on Atomic Host and
Fedora Server distributions.


Currently docker-storage-setup defaults to devicemapper and is hard
coded to setup a thinpool of 40% of remaining disk.  Otherwise it sets
up loopback devices on the root file system.   Devicemapper is nice
since it works with thinpools and can automatically expand the storage
if the disk space is getting used up. 

Moving to Overlay, we can more easily use the root file system directly,
which would be fine for Fedora Workstation.  We want to preserve the use
of the remaining storage for Overlay on AH and Fedora Server,  since
this would give a user flexibility to switch back to using devicemapper
if they had problems with the Overlay driver.   We can not as easily
support the expanding disk for Overlay since we will not use using thinpool.

We have looked at options to hard code OverlayFS with the defaults, or
we could just drop a /etc/sysconfig/docker-storage-setup that specified
Overlay and the percentage of remaining space to use for the
/var/lib/docker device.  But what is the best way to set different
defaults for AH, Fedora Server and Fedora Workstation.

We would like to discuss with you guys about what  you would think is
the best way to handle. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org