Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Matt Turner]

On Sun, Jul 29, 2018 at 1:01 PM, Rich Freeman  wrote:
> On Sun, Jul 29, 2018 at 3:56 PM Matt Turner  wrote:
>>
>> On Fri, Jul 27, 2018 at 1:32 AM, Ulrich Mueller  wrote:
>> >
>> > So, considering all the feedback from mailing list and IRC:
>> >
>> >/usr/portage   -> /var/db/repos/gentoo
>> >/usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>> >/usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
>> >
>> > Open question: Should we have the additional "gentoo" path component
>> > for the ones in /var/cache? The tradeoff is between a path that is
>> > easier to type, or slightly easier usage if someone wants to NFS mount
>> > distfiles and binpkgs.
>>
>> That proposal has by vote of support. No strong preference on whether
>> to include gentoo/ or not. It's one NFS mount vs two so not a big deal
>> either way.
>>
>
> Why not stick the repos in /var/repos and not /var/db/repos?  If we're
> just making up paths, why not make up a shorter one?  I don't think
> any other linux distros use /var/db...

That would be fine with me as well :)

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Fri, 27 Jul 2018, Brian Dolbec wrote:

> On Fri, 27 Jul 2018 16:31:15 +0200
> Ulrich Mueller  wrote:

>> > On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:
>>
>> > From the same source
>> > "No other requirements are made on the data format of the cache
>> > directories."
>> > And as you have quoted it, everything under /var/cache is
>> > optional.
>>
>> > So anything which doesn't conflict with another package seems
>> > fine according to FHS.
>>
>> That's how I would read it, too. We could of course invent a
>> package name (like "package-manager" for virtual/package-manager)
>> but it seems cumbersome, and I don't see any benefit of it.
>>
>> There also is /var/cache/fonts, so the FHS itself lists an example
>> of a directory that's not named after a specific package.

> /var/db/repos/gentoo
> /var/cache/distfiles
> /var/cache/binpkgs

For the record, these three paths have been approved in today's
Council meeting.

> Works for me, just please keep "portage" out of it, after all
> distfiles are not restricted to portage use only, and neither are
> binpkgs. There is alternate binpkg installers.

By another Council vote, snapshot names should be changed from
portage-MMDD.tar.{bz2,xz} to gentoo-MMDD.tar.{bz2,xz}.

So, no "portage" any more. ;)

Ulrich


pgpabZ6TuTvWB.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[M. J. Everitt]

On 29/07/18 21:01, Rich Freeman wrote:
> 
>
> Why not stick the repos in /var/repos and not /var/db/repos?  If we're
> just making up paths, why not make up a shorter one?  I don't think
> any other linux distros use /var/db...
>
*BSD I believe ..



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Rich Freeman]

On Sun, Jul 29, 2018 at 3:56 PM Matt Turner  wrote:
>
> On Fri, Jul 27, 2018 at 1:32 AM, Ulrich Mueller  wrote:
> >
> > So, considering all the feedback from mailing list and IRC:
> >
> >/usr/portage   -> /var/db/repos/gentoo
> >/usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
> >/usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
> >
> > Open question: Should we have the additional "gentoo" path component
> > for the ones in /var/cache? The tradeoff is between a path that is
> > easier to type, or slightly easier usage if someone wants to NFS mount
> > distfiles and binpkgs.
>
> That proposal has by vote of support. No strong preference on whether
> to include gentoo/ or not. It's one NFS mount vs two so not a big deal
> either way.
>

Why not stick the repos in /var/repos and not /var/db/repos?  If we're
just making up paths, why not make up a shorter one?  I don't think
any other linux distros use /var/db...

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Matt Turner]

On Fri, Jul 27, 2018 at 1:32 AM, Ulrich Mueller  wrote:
>> On Thu, 19 Jul 2018, Chí-Thanh Christopher Nguyễn wrote:
>
>>> Users must never need to modify files in /var/lib to configure a
>>> package's operation, and _the_specific_file_hierarchy_ used to
>>> store the data _must_not_be_ _exposed_ to regular users."
>
>> One small note, while it is never needed to modify, skel.ebuild
>> would then be a file that is meant to be accessed by users in
>> /var/lib if your proposal is realized.
>
> That's one of the reasons why the proposal prefers /var/db. The other
> reason is existing usage in eselect-repository.
>
>> On Thu, 19 Jul 2018, Ulrich Mueller wrote:
>
>> In my understanding, a cache is typically an open collection of items.
>> Some subset of them can be deleted without much negative consequence,
>> and there may also be surplus items that are no longer necessary and
>> will be expired at some later time in order to reclaim disk space.
>
>> Nothing of this is true for an ebuild repository, which is a closed
>> collection of files: A single file cannot be discarded without
>> invalidating the whole repository. Also there cannot be any stray
>> files which would be expired later. Same as above, a single stray file
>> will invalidate all.
>
>> (A collection of binary packages may qualify as a cache though, by
>> this definition.)
>
> So, considering all the feedback from mailing list and IRC:
>
>/usr/portage   -> /var/db/repos/gentoo
>/usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>/usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
>
> Open question: Should we have the additional "gentoo" path component
> for the ones in /var/cache? The tradeoff is between a path that is
> easier to type, or slightly easier usage if someone wants to NFS mount
> distfiles and binpkgs.

That proposal has by vote of support. No strong preference on whether
to include gentoo/ or not. It's one NFS mount vs two so not a big deal
either way.

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Matt Turner]

On Fri, Jul 27, 2018 at 1:40 AM, Michał Górny  wrote:
> Dnia 27 lipca 2018 10:32:17 CEST, Ulrich Mueller  napisał(a):
>>> On Thu, 19 Jul 2018, Chí-Thanh Christopher Nguyễn wrote:
>>
 Users must never need to modify files in /var/lib to configure a
 package's operation, and _the_specific_file_hierarchy_ used to
 store the data _must_not_be_ _exposed_ to regular users."
>>
>>> One small note, while it is never needed to modify, skel.ebuild
>>> would then be a file that is meant to be accessed by users in
>>> /var/lib if your proposal is realized.
>>
>>That's one of the reasons why the proposal prefers /var/db. The other
>>reason is existing usage in eselect-repository.
>>
>>> On Thu, 19 Jul 2018, Ulrich Mueller wrote:
>>
>>> In my understanding, a cache is typically an open collection of
>>items.
>>> Some subset of them can be deleted without much negative consequence,
>>> and there may also be surplus items that are no longer necessary and
>>> will be expired at some later time in order to reclaim disk space.
>>
>>> Nothing of this is true for an ebuild repository, which is a closed
>>> collection of files: A single file cannot be discarded without
>>> invalidating the whole repository. Also there cannot be any stray
>>> files which would be expired later. Same as above, a single stray
>>file
>>> will invalidate all.
>>
>>> (A collection of binary packages may qualify as a cache though, by
>>> this definition.)
>>
>>So, considering all the feedback from mailing list and IRC:
>>
>>   /usr/portage   -> /var/db/repos/gentoo
>>   /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>>   /usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
>>
>>Open question: Should we have the additional "gentoo" path component
>>for the ones in /var/cache? The tradeoff is between a path that is
>>easier to type, or slightly easier usage if someone wants to NFS mount
>>distfiles and binpkgs.
>
> Note that NFS is not exactly clear cut here since binpkgs are not portable to 
> different hosts, so you can have multiple variants of it.

True, but trivially solvable by configuring like-hosts to share
packages. Skylake packages go here, Sandybridge packages go here, etc.
This is what I do.

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

[Proxying a message from Anders Thomson ]

Hi Ulrich,
As non-devs aren't allowed to post to gentoo-dev, I was hoping that you would
proxy this question/comment for me.

While on the subject of changing defaults, could we consider changing the
(default) location of the pkg db? Roughly everything in /usr (and /(s)bin) is
managed by the package manager, and it would be handy if the db of installed
bits is close to the bits themselves . Keeping it in /var/ makes /var
opinionated on the current set of installed packages and thus another thing to
backup etc of you want "just the os/system stuff, not the applications'
databases". along the same vein, if you want /var to be on another storage
device (NAS/SAN for large databases), you get the OS's pkg db with it. If for
any reason a boot fails to get /var mounted, you're without the pkg db.

Something along the lines of /usr/lib/pkg ?

Best,
/Anders

On Fri, Jul 27, 2018 at 4:31 PM, Ulrich Mueller  wrote:

> On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:
   
> July 27, 2018 4:07 PM, "William Hubbs"  wrote:
   
>> Section 5.5.2 describes the directory structure of /var/cache.
>> These paths are all optional [1].
>>
>> /var/cache/fonts
>> /var/cache/man
>> /var/cache/www
>> /var/cache/
>>
>> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
>> appropriate. Here is my proposal:
>>
>> /usr/portage -> /var/db/repos/gentoo
>> /usr/portage/distfiles -> /var/cache/portage/distfiles
>> /usr/portage/packages -> /var/cache/portage/binpkgs
>>
>> I'm not 100% comfortable with /var/db, but I don't have any better
>> suggestion either.
>>
>> [1] 
http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData
   
> From the same source
> "No other requirements are made on the data format of the cache
> directories."
> And as you have quoted it, everything under /var/cache is optional.
   
> So anything which doesn't conflict with another package seems fine
> according to FHS.
   
That's how I would read it, too. We could of course invent a package
name (like "package-manager" for virtual/package-manager) but it seems
cumbersome, and I don't see any benefit of it.
   
There also is /var/cache/fonts, so the FHS itself lists an example of
a directory that's not named after a specific package.
   
Ulrich


pgpvkqkV7URia.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Michał Górny]

W dniu pią, 27.07.2018 o godzinie 08∶06 -0700, użytkownik Brian Dolbec
napisał:
> On Fri, 27 Jul 2018 16:31:15 +0200
> Ulrich Mueller  wrote:
> 
> > > > > > > On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:  
> > > July 27, 2018 4:07 PM, "William Hubbs" 
> > > wrote:  
> > > > Section 5.5.2 describes the directory structure of /var/cache.
> > > > These paths are all optional [1].
> > > > 
> > > > /var/cache/fonts
> > > > /var/cache/man
> > > > /var/cache/www
> > > > /var/cache/
> > > > 
> > > > Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
> > > > appropriate. Here is my proposal:
> > > > 
> > > > /usr/portage -> /var/db/repos/gentoo
> > > > /usr/portage/distfiles -> /var/cache/portage/distfiles
> > > > /usr/portage/packages -> /var/cache/portage/binpkgs
> > > > 
> > > > I'm not 100% comfortable with /var/db, but I don't have any better
> > > > suggestion either.
> > > > 
> > > > [1]
> > > > http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData
> > > >   
> > > From the same source
> > > "No other requirements are made on the data format of the cache
> > > directories."
> > > And as you have quoted it, everything under /var/cache is
> > > optional.  
> > > So anything which doesn't conflict with another package seems fine
> > > according to FHS.  
> > 
> > That's how I would read it, too. We could of course invent a package
> > name (like "package-manager" for virtual/package-manager) but it seems
> > cumbersome, and I don't see any benefit of it.
> > 
> > There also is /var/cache/fonts, so the FHS itself lists an example of
> > a directory that's not named after a specific package.
> > 
> > Ulrich
> 
> /var/db/repos/gentoo
> /var/cache/distfiles
> /var/cache/binpkgs
> 
> Works for me, just please keep "portage" out of it, after all distfiles
> are not restricted to portage use only, and neither are binpkgs.  There
> is alternate binpkg installers.

Well, technically speaking this specific binary package format is
Portage-specific.  But I don't think we need to go into that kind of
nuances.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Brian Dolbec]

On Fri, 27 Jul 2018 16:31:15 +0200
Ulrich Mueller  wrote:

> > On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:  
> 
> > July 27, 2018 4:07 PM, "William Hubbs" 
> > wrote:  
> 
> >> Section 5.5.2 describes the directory structure of /var/cache.
> >> These paths are all optional [1].
> >> 
> >> /var/cache/fonts
> >> /var/cache/man
> >> /var/cache/www
> >> /var/cache/
> >> 
> >> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
> >> appropriate. Here is my proposal:
> >> 
> >> /usr/portage -> /var/db/repos/gentoo
> >> /usr/portage/distfiles -> /var/cache/portage/distfiles
> >> /usr/portage/packages -> /var/cache/portage/binpkgs
> >> 
> >> I'm not 100% comfortable with /var/db, but I don't have any better
> >> suggestion either.
> >> 
> >> [1]
> >> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData
> >>   
> 
> > From the same source
> > "No other requirements are made on the data format of the cache
> > directories."
> > And as you have quoted it, everything under /var/cache is
> > optional.  
> 
> > So anything which doesn't conflict with another package seems fine
> > according to FHS.  
> 
> That's how I would read it, too. We could of course invent a package
> name (like "package-manager" for virtual/package-manager) but it seems
> cumbersome, and I don't see any benefit of it.
> 
> There also is /var/cache/fonts, so the FHS itself lists an example of
> a directory that's not named after a specific package.
> 
> Ulrich

/var/db/repos/gentoo
/var/cache/distfiles
/var/cache/binpkgs

Works for me, just please keep "portage" out of it, after all distfiles
are not restricted to portage use only, and neither are binpkgs.  There
is alternate binpkg installers.


pgpNgTs22e92A.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

> On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:

> July 27, 2018 4:07 PM, "William Hubbs"  wrote:

>> Section 5.5.2 describes the directory structure of /var/cache.
>> These paths are all optional [1].
>> 
>> /var/cache/fonts
>> /var/cache/man
>> /var/cache/www
>> /var/cache/
>> 
>> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
>> appropriate. Here is my proposal:
>> 
>> /usr/portage -> /var/db/repos/gentoo
>> /usr/portage/distfiles -> /var/cache/portage/distfiles
>> /usr/portage/packages -> /var/cache/portage/binpkgs
>> 
>> I'm not 100% comfortable with /var/db, but I don't have any better
>> suggestion either.
>> 
>> [1] 
>> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

> From the same source
> "No other requirements are made on the data format of the cache
> directories."
> And as you have quoted it, everything under /var/cache is optional.

> So anything which doesn't conflict with another package seems fine
> according to FHS.

That's how I would read it, too. We could of course invent a package
name (like "package-manager" for virtual/package-manager) but it seems
cumbersome, and I don't see any benefit of it.

There also is /var/cache/fonts, so the FHS itself lists an example of
a directory that's not named after a specific package.

Ulrich


pgp_STEQdonoi.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Corentin “Nado” Pazdera]

July 27, 2018 4:07 PM, "William Hubbs"  wrote:

> On Fri, Jul 27, 2018 at 10:32:17AM +0200, Ulrich Mueller wrote:
> 
>> So, considering all the feedback from mailing list and IRC:
>> 
>> /usr/portage -> /var/db/repos/gentoo
>> /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>> /usr/portage/packages -> /var/cache{,/gentoo}/binpkgs
>> 
>> Open question: Should we have the additional "gentoo" path component
>> for the ones in /var/cache? The tradeoff is between a path that is
>> easier to type, or slightly easier usage if someone wants to NFS mount
>> distfiles and binpkgs.
> 
> Section 5.5.2 describes the directory structure of /var/cache. These
> paths are all optional [1].
> 
> /var/cache/fonts
> /var/cache/man
> /var/cache/www
> /var/cache/
> 
> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
> appropriate. Here is my proposal:
> 
> /usr/portage -> /var/db/repos/gentoo
> /usr/portage/distfiles -> /var/cache/portage/distfiles
> /usr/portage/packages -> /var/cache/portage/binpkgs
> 
> I'm not 100% comfortable with /var/db, but I don't have any better
> suggestion either.
> 
> William
> 
> [1] 
> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

From the same source
"No other requirements are made on the data format of the cache directories."
And as you have quoted it, everything under /var/cache is optional.

So anything which doesn't conflict with another package seems fine according to 
FHS.

--
Corentin “Nado” Pazdera

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[William Hubbs]

On Fri, Jul 27, 2018 at 10:32:17AM +0200, Ulrich Mueller wrote:
> So, considering all the feedback from mailing list and IRC:
> 
>/usr/portage   -> /var/db/repos/gentoo
>/usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>/usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
> 
> Open question: Should we have the additional "gentoo" path component
> for the ones in /var/cache? The tradeoff is between a path that is
> easier to type, or slightly easier usage if someone wants to NFS mount
> distfiles and binpkgs.

Section 5.5.2 describes the directory structure of /var/cache. These
paths are all optional [1].

/var/cache/fonts
/var/cache/man
/var/cache/www
/var/cache/

Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
appropriate. Here is my proposal:

/usr/portage   -> /var/db/repos/gentoo
/usr/portage/distfiles -> /var/cache/portage/distfiles
/usr/portage/packages  -> /var/cache/portage/binpkgs

I'm not 100% comfortable with /var/db, but I don't have any better
suggestion either.

William

[1] 
http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData


signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Michał Górny]

Dnia 27 lipca 2018 10:32:17 CEST, Ulrich Mueller  napisał(a):
>> On Thu, 19 Jul 2018, Chí-Thanh Christopher Nguyễn wrote:
>
>>> Users must never need to modify files in /var/lib to configure a
>>> package's operation, and _the_specific_file_hierarchy_ used to
>>> store the data _must_not_be_ _exposed_ to regular users."
>
>> One small note, while it is never needed to modify, skel.ebuild
>> would then be a file that is meant to be accessed by users in
>> /var/lib if your proposal is realized.
>
>That's one of the reasons why the proposal prefers /var/db. The other
>reason is existing usage in eselect-repository.
>
>> On Thu, 19 Jul 2018, Ulrich Mueller wrote:
>
>> In my understanding, a cache is typically an open collection of
>items.
>> Some subset of them can be deleted without much negative consequence,
>> and there may also be surplus items that are no longer necessary and
>> will be expired at some later time in order to reclaim disk space.
>
>> Nothing of this is true for an ebuild repository, which is a closed
>> collection of files: A single file cannot be discarded without
>> invalidating the whole repository. Also there cannot be any stray
>> files which would be expired later. Same as above, a single stray
>file
>> will invalidate all.
>
>> (A collection of binary packages may qualify as a cache though, by
>> this definition.)
>
>So, considering all the feedback from mailing list and IRC:
>
>   /usr/portage   -> /var/db/repos/gentoo
>   /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>   /usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
>
>Open question: Should we have the additional "gentoo" path component
>for the ones in /var/cache? The tradeoff is between a path that is
>easier to type, or slightly easier usage if someone wants to NFS mount
>distfiles and binpkgs.

Note that NFS is not exactly clear cut here since binpkgs are not portable to 
different hosts, so you can have multiple variants of it.

>
>Ulrich


-- 
Best regards,
Michał Górny (by phone)

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

> On Thu, 19 Jul 2018, Chí-Thanh Christopher Nguyễn wrote:

>> Users must never need to modify files in /var/lib to configure a
>> package's operation, and _the_specific_file_hierarchy_ used to
>> store the data _must_not_be_ _exposed_ to regular users."

> One small note, while it is never needed to modify, skel.ebuild
> would then be a file that is meant to be accessed by users in
> /var/lib if your proposal is realized.

That's one of the reasons why the proposal prefers /var/db. The other
reason is existing usage in eselect-repository.

> On Thu, 19 Jul 2018, Ulrich Mueller wrote:

> In my understanding, a cache is typically an open collection of items.
> Some subset of them can be deleted without much negative consequence,
> and there may also be surplus items that are no longer necessary and
> will be expired at some later time in order to reclaim disk space.

> Nothing of this is true for an ebuild repository, which is a closed
> collection of files: A single file cannot be discarded without
> invalidating the whole repository. Also there cannot be any stray
> files which would be expired later. Same as above, a single stray file
> will invalidate all.

> (A collection of binary packages may qualify as a cache though, by
> this definition.)

So, considering all the feedback from mailing list and IRC:

   /usr/portage   -> /var/db/repos/gentoo
   /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
   /usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs

Open question: Should we have the additional "gentoo" path component
for the ones in /var/cache? The tradeoff is between a path that is
easier to type, or slightly easier usage if someone wants to NFS mount
distfiles and binpkgs.

Ulrich


pgp0COMHZDbWr.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Chí-Thanh Christopher Nguyễn]

Ulrich Mueller schrieb:

Users must never
need to modify files in /var/lib to configure a package's operation,
and _the_specific_file_hierarchy_ used to store the data _must_not_be_
_exposed_ to regular users."


One small note, while it is never needed to modify, skel.ebuild would then be 
a file that is meant to be accessed by users in /var/lib if your proposal is 
realized.



Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Christopher Head]

On July 18, 2018 2:55:55 AM PDT, Ulrich Mueller  wrote:
>It was mentioned that all three directories (ebuild repository, binary
>packages, distfiles) have some characteristics of a cache. However, I
>think this is much more true for distfiles than for the other two,
>which cannot be easily restored to a previous state.

Neither can a Web browser’s cache, if the remote page has changed, yet those 
are still called caches. Surely a cache is something that can safely be 
discarded without negative impact (which, for most users, the ebuild tree can 
be, since for most users, getting back today’s tree rather than last week’s is 
not a negative impact), not something that can be restored to exactly its prior 
state automatically.

-- 
Christopher Head

signature.asc
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Rich Freeman]

On Wed, Jul 18, 2018 at 9:30 AM Ulrich Mueller  wrote:
>
> > On Wed, 18 Jul 2018, Rich Freeman wrote:
>
> > On Wed, Jul 18, 2018 at 5:55 AM Ulrich Mueller  wrote:
> >> Also, there is that strange requirement that the
> >> file hierarchy must not be exposed to users. At least for the
> >> make.profile link we rely on a well defined hierarchy, and certainly
> >> expose it to users.
>
> > That isn't true at all.  When you run eselect profile you have no idea
> > what it is looking at.
>
> If I run eselect profile, it shows a list of pathnames like
> default/linux/amd64/17.1/desktop. If that doesn't expose the "specific
> file hierarchy" then I wonder what could ever qualify?

It lists profile names.  The fact that our profiles are implemented
using paths is an implementation detail.  And you would have no idea
that they're stored in /usr or /var from the output of eselect.  If we
modified PMS so that profiles were stored in a mysql database the
current eselect output could work just fine.

> Also eselect profile is a tool for convenience only, and nobody is
> forced to use it. The make.profile symlink and its target are
> mentioned in our documentation and users can set it manually.

So is /var/lib/portage/world.  Are you planning to move that to /etc?
(I wouldn't object to that, actually.)

Personally I think that a lot of this comes from the standpoint of a
typical distro where command lines are things that should be hidden as
deeply in the gnome menus as possible.  But, my point is that if you
want that kind of experience on Gentoo it is certainly possible
insofar as profiles are concerned.

> Hopefully we will keep having such users who want to understand the
> inner workings, instead of being satisfied with a black box. :)
> Ebuild repositories are human readable text, and we shouldn't move
> them to a hidden location.

There is nothing hidden about /var/lib.  And I also prefer editing
text files for the most part.  My point is that FHS wasn't really
written with Gentoo as its main target, and that is likely the reason
for the bit about hiding paths.

>
> >> The same is true for license information in
> >> /usr/portage/licenses.
>
> > You have a fair point there.  Honestly, I don't really see this as a
> > good reason on its own to abandon /var/lib, which is where other
> > distros seem to stick stuff like this.  Also, you don't need to poke
> > around that directory to determine what license a package uses - just
> > to read the text of the license itself (which could arguably just be
> > stored on our webserver unless we're actually redistributing something
> > in the repository under that license, which is unlikely in general
> > since patches/etc are probably fair use).
>
> I totally disagree here. We keep local copies of licenses for good
> reasons, and storing them on our webserver is no substitute.

I'm not sure what you're disagreeing with.  I'm not advocating for not
storing them locally.  I'm just saying it could be done, and I did say
that you had a fair point.

> No, the argument is that we already use /var/db/pkg, and grouping
> other related things with it seems natural.

Perhaps we should be trying to move away from using /var/db, and not
doubling down?

> Apart from this (quoting the devmanual): "Gentoo does not consider
> the Filesystem Hierarchy Standard to be an authoritative standard,
> although much of our policy coincides with it." And the discussion so
> far has shown that the FHS wasn't designed for our use case. We can
> still use it as a guideline, but maybe we shouldn't jump through
> hoops, only to make it completely fit.

Sure, I'd agree with that, but why not use /var/lib then?  Your only
argument against it is based on the FHS, which you freely concede
wasn't written to fit our use case.

> The only thing that seems certain is that regardless of what we will
> do, we cannot make everybody happy. We shouldn't take that as an
> excuse to do nothing, because /usr/portage is no good solution either.

Well, the one thing I like about all these proposals is that it will
result in Gentoo systems with a wide mixture of locations for these
things, which will guarantee that any script that hard-codes paths
will break.  That means that those who exercise their choice to do
things in a more sane way than our defaults will probably run into
less trouble...

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

> On Wed, 18 Jul 2018, Rich Freeman wrote:

> On Wed, Jul 18, 2018 at 5:55 AM Ulrich Mueller  wrote:
>> Also, there is that strange requirement that the
>> file hierarchy must not be exposed to users. At least for the
>> make.profile link we rely on a well defined hierarchy, and certainly
>> expose it to users.

> That isn't true at all.  When you run eselect profile you have no idea
> what it is looking at.

If I run eselect profile, it shows a list of pathnames like
default/linux/amd64/17.1/desktop. If that doesn't expose the "specific
file hierarchy" then I wonder what could ever qualify?

Also eselect profile is a tool for convenience only, and nobody is
forced to use it. The make.profile symlink and its target are
mentioned in our documentation and users can set it manually.

> The fact that some of our users like to poke around the internals of
> the package manager doesn't change the fact that it has interfaces
> that don't require direct modification.

Hopefully we will keep having such users who want to understand the
inner workings, instead of being satisfied with a black box. :)
Ebuild repositories are human readable text, and we shouldn't move
them to a hidden location.

>> The same is true for license information in
>> /usr/portage/licenses.

> You have a fair point there.  Honestly, I don't really see this as a
> good reason on its own to abandon /var/lib, which is where other
> distros seem to stick stuff like this.  Also, you don't need to poke
> around that directory to determine what license a package uses - just
> to read the text of the license itself (which could arguably just be
> stored on our webserver unless we're actually redistributing something
> in the repository under that license, which is unlikely in general
> since patches/etc are probably fair use).

I totally disagree here. We keep local copies of licenses for good
reasons, and storing them on our webserver is no substitute.

>> This follows the existing usage in eselect-repositories. Note that
>> /var/db is not specified by the FHS (though it is mentioned in a
>> footnote [1]) but used by all current BSD variants. Plus, we already
>> have /var/db/pkgs and (as pointed out by antarus) we won't get rid of
>> it anytime soon.

> So, you reject one path on the basis of small conflicts with FHS, and
> then just toss FHS out the window entirely to use a path not specified
> at all?

No, the argument is that we already use /var/db/pkg, and grouping
other related things with it seems natural.

> If you want to appeal to what BSD is doing then /usr/portage belongs
> right where it is today.  Oh, and most of our packages should be
> installing to /usr/local as well.

As I said before, the two important parts are the move from /usr to
/var, and that distfiles and packages are moved out of the repository.

Apart from this (quoting the devmanual): "Gentoo does not consider
the Filesystem Hierarchy Standard to be an authoritative standard,
although much of our policy coincides with it." And the discussion so
far has shown that the FHS wasn't designed for our use case. We can
still use it as a guideline, but maybe we shouldn't jump through
hoops, only to make it completely fit.

> What other linux distro even has /var/db at all?

Many (most?) of those that build from source. Also, Gentoo is not only
a Linux distro.

>> /usr/portage/packages -> /var/db/binpkgs

> Why not put this in /var/cache?  It is completely reproducible and
> exists to save build time.  That's what a cache is.

*shrug* I don't have a strong opinion about this. To me, binpkgs look
less cache-like than distfiles, but more cache-like than ebuild
repositories. So /var/cache/binpkgs would work for me as well.

The only thing that seems certain is that regardless of what we will
do, we cannot make everybody happy. We shouldn't take that as an
excuse to do nothing, because /usr/portage is no good solution either.

Ulrich


pgppLkw8SOPNi.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Rich Freeman]

On Wed, Jul 18, 2018 at 5:55 AM Ulrich Mueller  wrote:
>
> "Pertains to one specific host" doesn't seem to apply to the Gentoo
> repository though.

Sure it does.  The state of the package repository on a Gentoo host
doesn't affect any other host.

Sure, that state is synced from someplace that many other hosts also
sync from, so the state of THAT version of the repository does affect
many hosts.

However, if you do an emerge --sync on one host, and then an hour
later do an emerge --sync on another host, the state of either local
repository won't affect the other.

When they talk about things that pertain to multiple hosts they're
talking about situations where a single path is often mounted across
many hosts simultaneously, eg using NFS.  An example of this given in
FHS is /var/mail.

Now, you could share the repository across multiple hosts, but that is
not a typical configuration.  Really, just about anything on the
filesystem could potentially be shared across multiple hosts.

> Also, there is that strange requirement that the
> file hierarchy must not be exposed to users. At least for the
> make.profile link we rely on a well defined hierarchy, and certainly
> expose it to users.

That isn't true at all.  When you run eselect profile you have no idea
what it is looking at.

The fact that some of our users like to poke around the internals of
the package manager doesn't change the fact that it has interfaces
that don't require direct modification.

> The same is true for license information in
> /usr/portage/licenses.

You have a fair point there.  Honestly, I don't really see this as a
good reason on its own to abandon /var/lib, which is where other
distros seem to stick stuff like this.  Also, you don't need to poke
around that directory to determine what license a package uses - just
to read the text of the license itself (which could arguably just be
stored on our webserver unless we're actually redistributing something
in the repository under that license, which is unlikely in general
since patches/etc are probably fair use).

> This follows the existing usage in eselect-repositories. Note that
> /var/db is not specified by the FHS (though it is mentioned in a
> footnote [1]) but used by all current BSD variants. Plus, we already
> have /var/db/pkgs and (as pointed out by antarus) we won't get rid of
> it anytime soon.

So, you reject one path on the basis of small conflicts with FHS, and
then just toss FHS out the window entirely to use a path not specified
at all?

If you want to appeal to what BSD is doing then /usr/portage belongs
right where it is today.  Oh, and most of our packages should be
installing to /usr/local as well.

What other linux distro even has /var/db at all?

> /usr/portage/packages -> /var/db/binpkgs

Why not put this in /var/cache?  It is completely reproducible and
exists to save build time.  That's what a cache is.

Or put it in /var/lib.  Certainly your objections above don't apply to
binpkgs.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Kristian Fiskerstrand]

On 07/18/2018 11:55 AM, Ulrich Mueller wrote:
> I therefore suggest the following scheme:

The full scheme looks good to me

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

[Moving the thread back from -project to -dev.]

> On Fri, 13 Jul 2018, Ulrich Mueller wrote:

> On Fri, 13 Jul 2018, Rich Freeman wrote:
>> Well, /var/lib/ is 3 right there.  If 5 is no good then you
>> only have one left.  We could just make it /var/lib/repos which seems
>> non-compliant.

> FHS 3.0 says: "An application (or a group of inter-related
> applications) must use a subdirectory of /var/lib for its data".
> Certainly /var/lib/repos is a subdirectory of /var/lib? So why would
> it be non-compliant? And if it was, do we care about non-compliance at
> the third directory level? The important part is that we move it out
> of /usr, and IMHO we should care to get the /var/{lib,cache,db} part
> somewhat right.

In the same section 5.8.1 (emphasis is mine):
"This hierarchy holds state information pertaining to an application
or the system. State information is data that programs modify while
they run, and that _pertains_to_one_specific_host._ Users must never
need to modify files in /var/lib to configure a package's operation,
and _the_specific_file_hierarchy_ used to store the data _must_not_be_
_exposed_ to regular users."

"Pertains to one specific host" doesn't seem to apply to the Gentoo
repository though. Also, there is that strange requirement that the
file hierarchy must not be exposed to users. At least for the
make.profile link we rely on a well defined hierarchy, and certainly
expose it to users. The same is true for license information in
/usr/portage/licenses. So I would conclude that using /var/lib does
not comply with the FHS.

It was mentioned that all three directories (ebuild repository, binary
packages, distfiles) have some characteristics of a cache. However, I
think this is much more true for distfiles than for the other two,
which cannot be easily restored to a previous state.

I therefore suggest the following scheme:

/usr/portage -> /var/db/repos/gentoo

This follows the existing usage in eselect-repositories. Note that
/var/db is not specified by the FHS (though it is mentioned in a
footnote [1]) but used by all current BSD variants. Plus, we already
have /var/db/pkgs and (as pointed out by antarus) we won't get rid of
it anytime soon.

/usr/portage/packages -> /var/db/binpkgs

This keeps binary packages along with ebuilds. As suggested by mgorny,
rename the "packages" directory to "binpkgs", because it is more
specific and avoids confusion with "pkgs".

/usr/portage/distfiles -> /var/cache/distfiles

Omitting the "portage" subdirectory level here, as distfiles aren't
specific to any package manager. We could use an intermediate
"package-manager" directory level, but then again, it would have only
one single subdirectory. The FHS isn't very specific about /var/cache
but mentions /var/cache/fonts as an example which appears to be
similar.

Ulrich

[1] 
https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#ftn.idm236086558032


pgpB26eJcfh4H.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 5:15 PM, Ulrich Mueller  wrote:
>> On Fri, 13 Jul 2018, konsolebox  wrote:
>
>> I don't mind calling ::gentoo as Gentoo's official ebuild repository,
>> but it also has been "a portage tree", and "the portage tree" by
>> default context. If you imply that people should change convention to
>> something more PMS friendly, be explicit, and perhaps make it
>> official, and the let them decide for themselves. Be fair at reminding
>> that it has been there, but it's better be changed for PMS's sake.
>> Don't make it look like the usage has always been wrong.
>
> You may be surprised, but the word "Gentoo" doesn't even occur in the
> main part (chapters 2 to 15) of the PMS document, except for one place
> referring to "Gentoo's Catalyst tool".
>
> Calling it "Gentoo repository" instead of "Portage tree" is purely a
> matter of distro policy and has nothing to do with PMS.

PMS mentions "ebuild repository", hence "Gentoo's official ebuild
repository" vs. "Gentoo Portage Tree"; vs. "Which Portage tree do you
use?".

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Fri, 13 Jul 2018, konsolebox  wrote:

> I don't mind calling ::gentoo as Gentoo's official ebuild repository,
> but it also has been "a portage tree", and "the portage tree" by
> default context. If you imply that people should change convention to
> something more PMS friendly, be explicit, and perhaps make it
> official, and the let them decide for themselves. Be fair at reminding
> that it has been there, but it's better be changed for PMS's sake.
> Don't make it look like the usage has always been wrong.

You may be surprised, but the word "Gentoo" doesn't even occur in the
main part (chapters 2 to 15) of the PMS document, except for one place
referring to "Gentoo's Catalyst tool".

Calling it "Gentoo repository" instead of "Portage tree" is purely a
matter of distro policy and has nothing to do with PMS.

Ulrich


pgpem_gzhZ8LK.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 3:47 AM, Brian Dolbec  wrote:
> On Thu, 12 Jul 2018 11:49:37 -0700
> Raymond Jennings  wrote:
>
>> In that case, I vote for /var/cache/portage, since that's literally
>> what purpose it serves.  Namely, the cache of the gentoo infra's
>> current copy of teh portage tree.
>>
>> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner 
>> wrote:
>> >
>> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
>> >  wrote:
>> >>
>> >> Just for the record, but would putting a setting inside
>> >> /etc/portage/make.conf be the appropriate way to handle this?
>> >>
>> >
>> > The settings already exist (and have existed for 10 years.) This
>> > bikeshed discussion is literally trying to decide what the default
>> > should be.
>> >
>> > -A
>> >
>>
>
> This is not a personal attack against you.  Just picked this one to say
> something again...
>
>
> PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> "gentoo".  "portage is the default package manager, but not the only
> choice.  Far too often, it takes awhile to figure out what someone is
> trying to say because of that confusion between the tree and the
> package manager.
>
> PLUS, it has been decided already long ago that the directory name
> should reflect the repository name.  We have been enforcing that rule
> for overlays for a long time.  It has just been taking a long time to
> get our tooling in order so that we can change our own to follow that
> rule.
>
> So, "portage" should not be a directory name in the new default path.
>
> --
> Brian Dolbec 

I don't mind calling ::gentoo as Gentoo's official ebuild repository,
but it also has been "a portage tree", and "the portage tree" by
default context. If you imply that people should change convention to
something more PMS friendly, be explicit, and perhaps make it
official, and the let them decide for themselves. Be fair at reminding
that it has been there, but it's better be changed for PMS's sake.
Don't make it look like the usage has always been wrong.

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 5:12 AM, William Hubbs  wrote:
> On Thu, Jul 12, 2018 at 10:13:57PM +0200, Michał Górny wrote:
>> W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
>> napisał:
>> > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec  wrote:
>> > >
>> > > So, "portage" should not be a directory name in the new default path.
>> > >
>> >
>> > Well, in my examples I proposed it as that is the software that
>> > created the path, but then again in the spirit of PMS portage isn't
>> > the only PM.
>> >
>> > So:
>> > /var/lib/repos/gentoo ?
>> >
>>
>> Subdirectories of /var/lib should be named after the tool/package name.
>> There's no tool or package called 'repos'.
>
> Technically mgorny is correct here. FHS requires that everything under
> /var/lib be under a directory for the package or for the distro [1].
> Note the comment about packaging support in 5.8.1.
>
> Based on that this is my thought:
>
> * /var/lib/portage is for portage specific stuff -- maybe even /var/db/pkg
> in the future should go to /var/lib/portage/pkg.
>
> * /var/lib/gentoo, on the other hand, could be where repos, distfiles
> and binpkgs go.
> William
>
> [1]
> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation

/var/lib/gentoo/portage then. (Or /var/lib/gentoo/repos/gentoo if you
care about PMS diplomacy.)

People can just move it somewhere and/or use symbolic links if they
want to use a different path.

Besides having /var/lib/gentoo/portage being set as "PORTDIR", I also
have DISTDIR=/var/lib/gentoo/distfiles
and PKGDIR=/var/lib/gentoo/packages.

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

On Thu, 12 Jul 2018 17:35:41 -0700
Raymond Jennings  wrote:

> On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec 
> wrote:
> >
> > On Thu, 12 Jul 2018 11:49:37 -0700
> > Raymond Jennings  wrote:
> >  
> > > In that case, I vote for /var/cache/portage, since that's
> > > literally what purpose it serves.  Namely, the cache of the
> > > gentoo infra's current copy of teh portage tree.
> > >
> > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner 
> > > wrote:  
> > > >
> > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> > > >  wrote:  
> > > >>
> > > >> Just for the record, but would putting a setting inside
> > > >> /etc/portage/make.conf be the appropriate way to handle this?
> > > >>  
> > > >
> > > > The settings already exist (and have existed for 10 years.) This
> > > > bikeshed discussion is literally trying to decide what the
> > > > default should be.
> > > >
> > > > -A
> > > >  
> > >  
> >
> > This is not a personal attack against you.  Just picked this one to
> > say something again...
> >
> >
> > PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> > "gentoo".  "portage is the default package manager, but not the only
> > choice.  Far too often, it takes awhile to figure out what someone
> > is trying to say because of that confusion between the tree and the
> > package manager.  
> 
> Point of order:
> 
> http://distfiles.gentoo.org/snapshots and numerous pieces of
> documentation call it "portage"
> 
> The confusion is ingrained by documentation.
> 

Yes, it is, and well we can't very well change the documentation until
we can get an end to this new default path bikeshed.  Council will
need to make the decision (soon I hope)...  Then we make all the changes
necessary.


-- 
Brian Dolbec 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec  wrote:
>
> On Thu, 12 Jul 2018 11:49:37 -0700
> Raymond Jennings  wrote:
>
> > In that case, I vote for /var/cache/portage, since that's literally
> > what purpose it serves.  Namely, the cache of the gentoo infra's
> > current copy of teh portage tree.
> >
> > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner 
> > wrote:
> > >
> > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> > >  wrote:
> > >>
> > >> Just for the record, but would putting a setting inside
> > >> /etc/portage/make.conf be the appropriate way to handle this?
> > >>
> > >
> > > The settings already exist (and have existed for 10 years.) This
> > > bikeshed discussion is literally trying to decide what the default
> > > should be.
> > >
> > > -A
> > >
> >
>
> This is not a personal attack against you.  Just picked this one to say
> something again...
>
>
> PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> "gentoo".  "portage is the default package manager, but not the only
> choice.  Far too often, it takes awhile to figure out what someone is
> trying to say because of that confusion between the tree and the
> package manager.

Point of order:

http://distfiles.gentoo.org/snapshots and numerous pieces of
documentation call it "portage"

The confusion is ingrained by documentation.

> PLUS, it has been decided already long ago that the directory name
> should reflect the repository name.  We have been enforcing that rule
> for overlays for a long time.  It has just been taking a long time to
> get our tooling in order so that we can change our own to follow that
> rule.
>
> So, "portage" should not be a directory name in the new default path.
>
> --
> Brian Dolbec 
>
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Vadim A. Misbakh-Soloviov]

> I guess /var/portage is not  a terrible choice.

Well, I double that.
I've already use the following structure:
|- /var/portage/
|   |- repos
|   |   |- gentoo
|   |   |- reponame1
|   |   |- reponame2
|   |- distfiles
|   |   |- ...
|   |   |- ...
|   |- packages
|   |   |- ...
|   |   |- ...
|   |- meta
|   |- layman
|   |- ... # (used to be used for layman stuff until 
eselect-repository)


And I think it's pretty fine and usefull, especially because distfiles/
packages does not belong to one specific repo (gentoo) as it does with /usr/
portage.

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Thu, Jul 12, 2018 at 10:13:57PM +0200, Michał Górny wrote:
> W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
> napisał:
> > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec  wrote:
> > > 
> > > So, "portage" should not be a directory name in the new default path.
> > > 
> > 
> > Well, in my examples I proposed it as that is the software that
> > created the path, but then again in the spirit of PMS portage isn't
> > the only PM.
> > 
> > So:
> > /var/lib/repos/gentoo ?
> > 
> 
> Subdirectories of /var/lib should be named after the tool/package name. 
> There's no tool or package called 'repos'.

Technically mgorny is correct here. FHS requires that everything under
/var/lib be under a directory for the package or for the distro [1].
Note the comment about packaging support in 5.8.1.

Based on that this is my thought:

* /var/lib/portage is for portage specific stuff -- maybe even /var/db/pkg
in the future should go to /var/lib/portage/pkg.

* /var/lib/gentoo, on the other hand, could be where repos, distfiles
and binpkgs go.
William

[1]
http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation


signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
napisał:
> On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec  wrote:
> > 
> > So, "portage" should not be a directory name in the new default path.
> > 
> 
> Well, in my examples I proposed it as that is the software that
> created the path, but then again in the spirit of PMS portage isn't
> the only PM.
> 
> So:
> /var/lib/repos/gentoo ?
> 

Subdirectories of /var/lib should be named after the tool/package name. 
There's no tool or package called 'repos'.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec  wrote:
>
> So, "portage" should not be a directory name in the new default path.
>

Well, in my examples I proposed it as that is the software that
created the path, but then again in the spirit of PMS portage isn't
the only PM.

So:
/var/lib/repos/gentoo ?

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

On Thu, 12 Jul 2018 11:49:37 -0700
Raymond Jennings  wrote:

> In that case, I vote for /var/cache/portage, since that's literally
> what purpose it serves.  Namely, the cache of the gentoo infra's
> current copy of teh portage tree.
> 
> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner 
> wrote:
> >
> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> >  wrote:  
> >>
> >> Just for the record, but would putting a setting inside
> >> /etc/portage/make.conf be the appropriate way to handle this?
> >>  
> >
> > The settings already exist (and have existed for 10 years.) This
> > bikeshed discussion is literally trying to decide what the default
> > should be.
> >
> > -A
> >  
> 

This is not a personal attack against you.  Just picked this one to say
something again...


PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
"gentoo".  "portage is the default package manager, but not the only
choice.  Far too often, it takes awhile to figure out what someone is
trying to say because of that confusion between the tree and the
package manager.

PLUS, it has been decided already long ago that the directory name
should reflect the repository name.  We have been enforcing that rule
for overlays for a long time.  It has just been taking a long time to
get our tooling in order so that we can change our own to follow that
rule.

So, "portage" should not be a directory name in the new default path.

-- 
Brian Dolbec 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Thu, Jul 12, 2018 at 3:34 PM konsolebox  wrote:
>
> I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf.
>

Regardless of the base directory location, I might suggest a path
dedicated to repositories, of which the main gentoo repo is just an
initial one, and overlays could be placed as additional directories
inside.

/var/lib/portage/repositories/main
/var/lib/gentoo/repositories/main
/var/cache/gentoo/repos/main
/var/cache/portage/repositories/gentoo

(Something along those lines.)

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf.

On Fri, Jul 13, 2018, 2:50 AM Raymond Jennings  wrote:

> In that case, I vote for /var/cache/portage, since that's literally
> what purpose it serves.  Namely, the cache of the gentoo infra's
> current copy of teh portage tree.
>
> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner  wrote:
> >
> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings 
> wrote:
> >>
> >> Just for the record, but would putting a setting inside
> >> /etc/portage/make.conf be the appropriate way to handle this?
> >>
> >
> > The settings already exist (and have existed for 10 years.) This
> bikeshed discussion is literally trying to decide what the default should
> be.
> >
> > -A
> >
>
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

In that case, I vote for /var/cache/portage, since that's literally
what purpose it serves.  Namely, the cache of the gentoo infra's
current copy of teh portage tree.

On Thu, Jul 12, 2018 at 11:00 AM Alec Warner  wrote:
>
> On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings  wrote:
>>
>> Just for the record, but would putting a setting inside
>> /etc/portage/make.conf be the appropriate way to handle this?
>>
>
> The settings already exist (and have existed for 10 years.) This bikeshed 
> discussion is literally trying to decide what the default should be.
>
> -A
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings 
wrote:

> Just for the record, but would putting a setting inside
> /etc/portage/make.conf be the appropriate way to handle this?
>
>
The settings already exist (and have existed for 10 years.) This bikeshed
discussion is literally trying to decide what the default should be.

-A

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

Just for the record, but would putting a setting inside
/etc/portage/make.conf be the appropriate way to handle this?

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 11:16 PM William Hubbs  wrote:
>
> That is the other part of this debate, some are saying /var/lib, and
> others are saying /var/db.
>
>  It turns out that /var/db is much more common than I thought it was
>  (it exists in all *bsd variants at least), so that could be an argument
>  for putting the repos in there.
>

FreeBSD does not put the package repository in /var/db.

Portage cloned many of the file paths from FreeBSD along with the
name/concept.  FreeBSD puts their package repository in /usr/ports, so
I'm not sure I'd hold them out as a great example of FHS.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Nils Freydank]

Am Mittwoch, 11. Juli 2018, 18:19:39 CEST schrieb Alec Warner:
> [...]
> 
> +1 to this. The challenge (in moving it) is that its been "/usr/portage"
> for a long time so many tools
> may have hard coded this location; as opposed to querying portage for where
> the tree is, e.g.:
> 
> PORTDIR=$(portageq get_repo_path / gentoo)
> 
> -A
Some people including myself moved the tree to /var by variable definitions 
(and not wild mounting) a while ago. This configuration *is* supported for a 
while now but not the default and if tools break they have to be fixed anyway. 

(Side note: At least most of the common tools like gentoolkit parts or repoman 
work on my machines with the moved tree.)

- Nils

-- 
GPG fingerprint: '766B 8122 1342 6912 3401 492A 8B54 D7A3 FF3C DB17'
Holgersson

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Dennis Schridde]

On Thursday, 12 July 2018 07:21:20 CEST Ulrich Mueller wrote:
> > On Wed, 11 Jul 2018, Richard Yao wrote:
> > This does not answer my question. Is it really a FHS violation? The
> > contents of /usr changes when doing updates using the system package
> > manager. When not doing updates, it really is readonly and the FHS
> > says that /usr is for readonly things. I do not see how it is
> > different from anything else in /usr.
> 
> What about a system that is building binpkgs? Or an rsync mirror?

There are systems building software for other systems using $ROOT, too.  They 
need to download into distfiles and sometimes they build binpkgs, too.

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[Please fix your mailer. Your message has a broken "References" header.]

> On Wed, 11 Jul 2018, Richard Yao wrote:

> This does not answer my question. Is it really a FHS violation? The
> contents of /usr changes when doing updates using the system package
> manager. When not doing updates, it really is readonly and the FHS
> says that /usr is for readonly things. I do not see how it is
> different from anything else in /usr.

What about a system that is building binpkgs? Or an rsync mirror?

Ulrich


pgpYwcvVrI0Bi.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Wed, Jul 11, 2018 at 06:24:20PM -0400, Rich Freeman wrote:
> On Wed, Jul 11, 2018 at 6:11 PM Richard Yao  wrote:
> >
> > Is it a violation of the FHS? /usr is for readonly data and the portage 
> > tree is generally readonly, except when being updated. The same is true of 
> > everything else in /usr.
> >
> 
> It is application metadata.  It belongs in /var.  No other packages
> write to /usr when they're doing internal updates.  Obviously you need
> a writable /usr to actually install package changes, but that
> shouldn't be necessary just to sync the repository.
> 
> I was asking around and it seems like most distros stick their
> repositories in /var/lib.  I can't imagine that too many would have
> even considered sticking them in /usr.

That is the other part of this debate, some are saying /var/lib, and
others are saying /var/db.

 It turns out that /var/db is much more common than I thought it was
 (it exists in all *bsd variants at least), so that could be an argument
 for putting the repos in there.

> > I am confused as to how we only now realized it was a FHS violation when it 
> > has been there for ~15 years. I was under the impression that /usr was the 
> > correct place for it.
> 
> It has certainly been pointed out in the past.  Nothing was changed
> for the same reason that nothing will probably be changed this time -
> people don't like change and the people who know better just slowly
> patch around Gentoo's oddities.  Somebody was just posting a manifesto
> about deploying more experimental technologies, and here we can't move
> a repository out of /usr.

Another reason this couldn't be changed in the past was catalyst had a
lot of hard coded references to /usr/portage. This has been fixe. in
catalyst-3 and I understand that releng is now using catalyst-3.

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

W dniu śro, 11.07.2018 o godzinie 18∶26 -0400, użytkownik Richard Yao
napisał:
> > On Jul 11, 2018, at 6:23 PM, Michał Górny  wrote:
> > 
> > W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
> > napisał:
> > > > > On Jul 11, 2018, at 4:43 PM, Rich Freeman  wrote:
> > > > > 
> > > > > On Wed, Jul 11, 2018 at 4:34 PM Richard Yao  wrote:
> > > > > 
> > > > > On my system, /usr/portage is a separate mountpoint. There is no need 
> > > > > to have on,h top level directories be separate mountpoints.
> > > > 
> > > > It makes sense to follow FHS.  Sure, I can work around poor designs by
> > > > sticking mount points all over the place, or manually setting my
> > > > config to put stuff in sane locations.  It makes more sense to put all
> > > > the volatile stuff in /var, than to mix it up all over the place and
> > > > get users to set up separate mountpoints to make up for it.
> > > 
> > > Is it a violation of the FHS? /usr is for readonly data and the portage 
> > > tree is generally readonly, except when being updated. The same is true 
> > > of everything else in /usr.
> > > 
> > > I am confused as to how we only now realized it was a FHS violation when 
> > > it has been there for ~15 years. I was under the impression that /usr was 
> > > the correct place for it.
> > > > 
> > 
> > And we're back to the usual Gentoo argument of 'it was like this for
> > N years'.  So FYI, something 'being there for ~15 years' doesn't make it
> > right.  It only means that:
> > 
> > a. Gentoo devs were wrong 15 years ago.
> > 
> > b. Gentoo devs are still wrong today.
> > 
> > c. Gentoo devs can't manage to make such a simple change because they're
> > too concerned about hurting somebody's feelings about a path.
> 
> This does not answer my question. Is it really a FHS violation? The contents 
> of /usr changes when doing updates using the system package manager. When not 
> doing updates, it really is readonly and the FHS says that /usr is for 
> readonly things. I do not see how it is different from anything else in /usr.
> 

You are bending the definition to the limit.

1. Repository updates can be done as unprivileged user (and it's
generally insane to --sync as root when you can do it unprivileged!).

2. Package managers can update repository cache while *not* performing
system updates.  This is writing.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 6:24 PM, Rich Freeman  wrote:
> 
>> On Wed, Jul 11, 2018 at 6:11 PM Richard Yao  wrote:
>> 
>> Is it a violation of the FHS? /usr is for readonly data and the portage tree 
>> is generally readonly, except when being updated. The same is true of 
>> everything else in /usr.
>> 
> 
> It is application metadata.  It belongs in /var.  No other packages
> write to /usr when they're doing internal updates.  Obviously you need
> a writable /usr to actually install package changes, but that
> shouldn't be necessary just to sync the repository.
> 
> I was asking around and it seems like most distros stick their
> repositories in /var/lib.  I can't imagine that too many would have
> even considered sticking them in /usr.
I would consider the package manager to be special in that it is a step of the 
system update process, but I agree that it could be nicer to have in /var.

We have a problem using /var/lib because /var/lib/portage is already in use. I 
guess /var/portage is not  a terrible choice.
> 
>> I am confused as to how we only now realized it was a FHS violation when it 
>> has been there for ~15 years. I was under the impression that /usr was the 
>> correct place for it.
> 
> It has certainly been pointed out in the past.  Nothing was changed
> for the same reason that nothing will probably be changed this time -
> people don't like change and the people who know better just slowly
> patch around Gentoo's oddities.  Somebody was just posting a manifesto
> about deploying more experimental technologies, and here we can't move
> a repository out of /usr.
> 
> And if nothing else, can we at least move /usr/portage/distfiles
> someplace else?  Surely you have to agree that this doesn't belong in
> usr, or nested in the middle of a repository?
Well, if it is changed during system updates, then the same logic applies, but 
quite honestly, I never liked having the distfiles or packages directories 
there. You can have unnecessary headaches when mounting inside a mount point 
other than /.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 6:23 PM, Michał Górny  wrote:
> 
> W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
> napisał:
 On Jul 11, 2018, at 4:43 PM, Rich Freeman  wrote:
 
 On Wed, Jul 11, 2018 at 4:34 PM Richard Yao  wrote:
 
 On my system, /usr/portage is a separate mountpoint. There is no need to 
 have on,h top level directories be separate mountpoints.
>>> 
>>> It makes sense to follow FHS.  Sure, I can work around poor designs by
>>> sticking mount points all over the place, or manually setting my
>>> config to put stuff in sane locations.  It makes more sense to put all
>>> the volatile stuff in /var, than to mix it up all over the place and
>>> get users to set up separate mountpoints to make up for it.
>> 
>> Is it a violation of the FHS? /usr is for readonly data and the portage tree 
>> is generally readonly, except when being updated. The same is true of 
>> everything else in /usr.
>> 
>> I am confused as to how we only now realized it was a FHS violation when it 
>> has been there for ~15 years. I was under the impression that /usr was the 
>> correct place for it.
>>> 
> 
> And we're back to the usual Gentoo argument of 'it was like this for
> N years'.  So FYI, something 'being there for ~15 years' doesn't make it
> right.  It only means that:
> 
> a. Gentoo devs were wrong 15 years ago.
> 
> b. Gentoo devs are still wrong today.
> 
> c. Gentoo devs can't manage to make such a simple change because they're
> too concerned about hurting somebody's feelings about a path.
This does not answer my question. Is it really a FHS violation? The contents of 
/usr changes when doing updates using the system package manager. When not 
doing updates, it really is readonly and the FHS says that /usr is for readonly 
things. I do not see how it is different from anything else in /usr.

I have been thinking that having it there was compliant for years and honestly, 
I don’t see how it is not complaint. Saying it is not compliant is not an 
explanation.
> 
> -- 
> Best regards,
> Michał Górny

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 6:11 PM Richard Yao  wrote:
>
> Is it a violation of the FHS? /usr is for readonly data and the portage tree 
> is generally readonly, except when being updated. The same is true of 
> everything else in /usr.
>

It is application metadata.  It belongs in /var.  No other packages
write to /usr when they're doing internal updates.  Obviously you need
a writable /usr to actually install package changes, but that
shouldn't be necessary just to sync the repository.

I was asking around and it seems like most distros stick their
repositories in /var/lib.  I can't imagine that too many would have
even considered sticking them in /usr.

> I am confused as to how we only now realized it was a FHS violation when it 
> has been there for ~15 years. I was under the impression that /usr was the 
> correct place for it.

It has certainly been pointed out in the past.  Nothing was changed
for the same reason that nothing will probably be changed this time -
people don't like change and the people who know better just slowly
patch around Gentoo's oddities.  Somebody was just posting a manifesto
about deploying more experimental technologies, and here we can't move
a repository out of /usr.

And if nothing else, can we at least move /usr/portage/distfiles
someplace else?  Surely you have to agree that this doesn't belong in
usr, or nested in the middle of a repository?

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
napisał:
> > On Jul 11, 2018, at 4:43 PM, Rich Freeman  wrote:
> > 
> > > On Wed, Jul 11, 2018 at 4:34 PM Richard Yao  wrote:
> > > 
> > > On my system, /usr/portage is a separate mountpoint. There is no need to 
> > > have on,h top level directories be separate mountpoints.
> > 
> > It makes sense to follow FHS.  Sure, I can work around poor designs by
> > sticking mount points all over the place, or manually setting my
> > config to put stuff in sane locations.  It makes more sense to put all
> > the volatile stuff in /var, than to mix it up all over the place and
> > get users to set up separate mountpoints to make up for it.
> 
> Is it a violation of the FHS? /usr is for readonly data and the portage tree 
> is generally readonly, except when being updated. The same is true of 
> everything else in /usr.
> 
> I am confused as to how we only now realized it was a FHS violation when it 
> has been there for ~15 years. I was under the impression that /usr was the 
> correct place for it.
> > 

And we're back to the usual Gentoo argument of 'it was like this for
N years'.  So FYI, something 'being there for ~15 years' doesn't make it
right.  It only means that:

a. Gentoo devs were wrong 15 years ago.

b. Gentoo devs are still wrong today.

c. Gentoo devs can't manage to make such a simple change because they're
too concerned about hurting somebody's feelings about a path.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 4:43 PM, Rich Freeman  wrote:
> 
>> On Wed, Jul 11, 2018 at 4:34 PM Richard Yao  wrote:
>> 
>> On my system, /usr/portage is a separate mountpoint. There is no need to 
>> have on,h top level directories be separate mountpoints.
> 
> It makes sense to follow FHS.  Sure, I can work around poor designs by
> sticking mount points all over the place, or manually setting my
> config to put stuff in sane locations.  It makes more sense to put all
> the volatile stuff in /var, than to mix it up all over the place and
> get users to set up separate mountpoints to make up for it.

Is it a violation of the FHS? /usr is for readonly data and the portage tree is 
generally readonly, except when being updated. The same is true of everything 
else in /usr.

I am confused as to how we only now realized it was a FHS violation when it has 
been there for ~15 years. I was under the impression that /usr was the correct 
place for it.
> 
> If somebody is doing a new Gentoo install, why would they want to put
> the repository in /usr, and nest a few GB of distfiles inside of the
> repo?  Why should that be the place we direct them?  There is no
> history for them.  A brand new install should put things in the most
> logical place.
> 
> By all means let existing users decide whether to move stuff.  I'm
> sure we have plenty of users with make.conf in /etc/.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 4:42 PM, William Hubbs  wrote:
> 
>> On Wed, Jul 11, 2018 at 04:25:20PM -0400, Richard Yao wrote:
>>> On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
 On 07/10/18 16:35, M. J. Everitt wrote:
> On 10/07/18 21:09, William Hubbs wrote:
>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
 On 09/07/18 23:12, Zac Medico wrote:
> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> I'd mostly argue any such change should only affect new systems
> 
 Yes, changing defaults for existing systems would be annoying.
 
 My recommendation is to have catalyst set the new defaults in the stage
 tarballs.
 
 When sys-apps/portage changes its internal defaults, I'd like for the
 upgrade process to call a tool that generates configuration files when
 necessary to ensure that the existing paths remain constant.
>>> I think it should be possible for RelEng to make a start on catalyst
>>> updates - is there anything that would inhibit going ahead with this,
>>> potentially?
>> No, nothing. Whatever catalyst puts it the default config will become
>> our new default.
> I would still like to see notice about what the new defaults are and how
> to migrate current systems to them.
> 
> 
> Thanks,
> 
> William
> 
>> -- 
>> Thanks,
>> Zac
>> 
> 
> 
 I'd like to propose that further to the discussion here on the -dev
 mailing list, the Council discuss and make a firm proposal on the new
 default paths, and then RelEng can make the appropriate updates to the
 catalyst builds. A news item can be compiled, with an appropriate wiki
 article perhaps on migration strategy (I may volunteer to format such a
 page with some appropriate guidance).
 Regards,
 Michael / veremitz.
 
>>> This is a mess, many systems are setup with portage already on a
>>> seperate partition for reasons. What advantage does it provide to move
>>> the tree now after all these years? I have seen nothing more then lets
>>> do this cause I like the ideal lately and it is getting old, there is no
>>> benefit that would justify moving the tree or many other changes that
>>> are being made in Gentoo lately.
>> 
>> People who want to move it could just set PORTDIR in make.conf. I don't
>> see any reason to move it either.
> 
> Actually, I believe that PORTDIR is becoming a thing of the past.
I used to use it 5 years ago. If it does not work due to regressions, we should 
fix that.
> 
> Also, the default definitely should not be on /usr per fhs. This would
> allow /usr to be mounted read only.
> This doesn't affect things like the example above where /usr/portage is
> a mount point.
> 
>>> 
>>> 
>>> 
>> 
>> 
> 
> 
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 4:34 PM Richard Yao  wrote:
>
> On my system, /usr/portage is a separate mountpoint. There is no need to have 
> on,h top level directories be separate mountpoints.

It makes sense to follow FHS.  Sure, I can work around poor designs by
sticking mount points all over the place, or manually setting my
config to put stuff in sane locations.  It makes more sense to put all
the volatile stuff in /var, than to mix it up all over the place and
get users to set up separate mountpoints to make up for it.

If somebody is doing a new Gentoo install, why would they want to put
the repository in /usr, and nest a few GB of distfiles inside of the
repo?  Why should that be the place we direct them?  There is no
history for them.  A brand new install should put things in the most
logical place.

By all means let existing users decide whether to move stuff.  I'm
sure we have plenty of users with make.conf in /etc/.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Wed, Jul 11, 2018 at 04:25:20PM -0400, Richard Yao wrote:
> On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
> > On 07/10/18 16:35, M. J. Everitt wrote:
> >> On 10/07/18 21:09, William Hubbs wrote:
> >>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>  On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> > On 09/07/18 23:12, Zac Medico wrote:
> >> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> >>> I'd mostly argue any such change should only affect new systems
> >>>
> >> Yes, changing defaults for existing systems would be annoying.
> >>
> >> My recommendation is to have catalyst set the new defaults in the stage
> >> tarballs.
> >>
> >> When sys-apps/portage changes its internal defaults, I'd like for the
> >> upgrade process to call a tool that generates configuration files when
> >> necessary to ensure that the existing paths remain constant.
> > I think it should be possible for RelEng to make a start on catalyst
> > updates - is there anything that would inhibit going ahead with this,
> > potentially?
>  No, nothing. Whatever catalyst puts it the default config will become
>  our new default.
> >>> I would still like to see notice about what the new defaults are and how
> >>> to migrate current systems to them.
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> William
> >>>
>  -- 
>  Thanks,
>  Zac
> 
> >>>
> >>>
> >> I'd like to propose that further to the discussion here on the -dev
> >> mailing list, the Council discuss and make a firm proposal on the new
> >> default paths, and then RelEng can make the appropriate updates to the
> >> catalyst builds. A news item can be compiled, with an appropriate wiki
> >> article perhaps on migration strategy (I may volunteer to format such a
> >> page with some appropriate guidance).
> >> Regards,
> >> Michael / veremitz.
> >>
> > This is a mess, many systems are setup with portage already on a
> > seperate partition for reasons. What advantage does it provide to move
> > the tree now after all these years? I have seen nothing more then lets
> > do this cause I like the ideal lately and it is getting old, there is no
> > benefit that would justify moving the tree or many other changes that
> > are being made in Gentoo lately.
> 
> People who want to move it could just set PORTDIR in make.conf. I don't
> see any reason to move it either.

Actually, I believe that PORTDIR is becoming a thing of the past.

Also, the default definitely should not be on /usr per fhs. This would
allow /usr to be mounted read only.
This doesn't affect things like the example above where /usr/portage is
a mount point.

> > 
> > 
> > 
> 
> 





signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 11:56 AM, Rich Freeman  wrote:
> 
>> On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings  wrote:
>> 
>> I do think it would be a wise idea to "grandfather" the current layout
>> for awhile.
>> 
> 
> I don't see why we would ever stop supporting it, at least in general.
> Maybe if some day somebody had a solution for a read-only /usr with
> signature checking that might require portage to be mounted elsewhere,
> but I don't ever see that becoming the default.
> 
> Portage just looks for the repository where you tell it to.  If you
> tell it that the repository is in /var, it will use it.  If you put it
> in /tmp, that's fine too.
> 
> This is just about the default, which should follow FHS.  The case of
> separate mounts is exactly why /usr is a bad spot - the access
> patterns for something like the repository have far more in common
> with /var than /usr.

On my system, /usr/portage is a separate mountpoint. There is no need to have 
on,h top level directories be separate mountpoints.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
> On 07/10/18 16:35, M. J. Everitt wrote:
>> On 10/07/18 21:09, William Hubbs wrote:
>>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
 On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> On 09/07/18 23:12, Zac Medico wrote:
>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>> I'd mostly argue any such change should only affect new systems
>>>
>> Yes, changing defaults for existing systems would be annoying.
>>
>> My recommendation is to have catalyst set the new defaults in the stage
>> tarballs.
>>
>> When sys-apps/portage changes its internal defaults, I'd like for the
>> upgrade process to call a tool that generates configuration files when
>> necessary to ensure that the existing paths remain constant.
> I think it should be possible for RelEng to make a start on catalyst
> updates - is there anything that would inhibit going ahead with this,
> potentially?
 No, nothing. Whatever catalyst puts it the default config will become
 our new default.
>>> I would still like to see notice about what the new defaults are and how
>>> to migrate current systems to them.
>>>
>>>
>>> Thanks,
>>>
>>> William
>>>
 -- 
 Thanks,
 Zac

>>>
>>>
>> I'd like to propose that further to the discussion here on the -dev
>> mailing list, the Council discuss and make a firm proposal on the new
>> default paths, and then RelEng can make the appropriate updates to the
>> catalyst builds. A news item can be compiled, with an appropriate wiki
>> article perhaps on migration strategy (I may volunteer to format such a
>> page with some appropriate guidance).
>> Regards,
>> Michael / veremitz.
>>
> This is a mess, many systems are setup with portage already on a
> seperate partition for reasons. What advantage does it provide to move
> the tree now after all these years? I have seen nothing more then lets
> do this cause I like the ideal lately and it is getting old, there is no
> benefit that would justify moving the tree or many other changes that
> are being made in Gentoo lately.

People who want to move it could just set PORTDIR in make.conf. I don't
see any reason to move it either.
> 
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

On Wed, Jul 11, 2018 at 11:56 AM, Rich Freeman  wrote:

> On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings 
> wrote:
> >
> > I do think it would be a wise idea to "grandfather" the current layout
> > for awhile.
> >
>
> I don't see why we would ever stop supporting it, at least in general.
> Maybe if some day somebody had a solution for a read-only /usr with
> signature checking that might require portage to be mounted elsewhere,
> but I don't ever see that becoming the default.
>
> Portage just looks for the repository where you tell it to.  If you
> tell it that the repository is in /var, it will use it.  If you put it
> in /tmp, that's fine too.
>

+1 to this. The challenge (in moving it) is that its been "/usr/portage"
for a long time so many tools
may have hard coded this location; as opposed to querying portage for where
the tree is, e.g.:

PORTDIR=$(portageq get_repo_path / gentoo)

-A


> This is just about the default, which should follow FHS.  The case of
> separate mounts is exactly why /usr is a bad spot - the access
> patterns for something like the repository have far more in common
> with /var than /usr.
>
-- 
> Rich
>
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings  wrote:
>
> I do think it would be a wise idea to "grandfather" the current layout
> for awhile.
>

I don't see why we would ever stop supporting it, at least in general.
Maybe if some day somebody had a solution for a read-only /usr with
signature checking that might require portage to be mounted elsewhere,
but I don't ever see that becoming the default.

Portage just looks for the repository where you tell it to.  If you
tell it that the repository is in /var, it will use it.  If you put it
in /tmp, that's fine too.

This is just about the default, which should follow FHS.  The case of
separate mounts is exactly why /usr is a bad spot - the access
patterns for something like the repository have far more in common
with /var than /usr.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

As long as an announcement is made in advance (perhaps as a NEWS item)
and portage itself is prepared to do an in-place migration if
necessary, I think things will be fine.

I do think it would be a wise idea to "grandfather" the current layout
for awhile.

On Wed, Jul 11, 2018 at 6:24 AM Gordon Pettey  wrote:
> On Wed, Jul 11, 2018 at 2:29 AM, Jory A. Pratt  wrote:
> > This is a mess, many systems are setup with portage already on a
> > seperate partition for reasons. What advantage does it provide to move
> > the tree now after all these years? I have seen nothing more then lets
> > do this cause I like the ideal lately and it is getting old, there is no
> > benefit that would justify moving the tree or many other changes that
> > are being made in Gentoo lately.
>
> 1. If you're able to mount /usr/portage from another filesystem, why
> would you think it wouldn't work in with /var/cache/portage?

> 1a. If your system is already installed, why do you think this even
> affects you? Did you read?

> 2. Pretty sure following FHS more closely is something most people
> would see as a benefit.

I agree on this point, and I always found /usr/portage to be...well, strange.



For me, though, the most important issue is giving end users advanced
notice and making sure nothing breaks.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Gordon Pettey]

On Wed, Jul 11, 2018 at 2:29 AM, Jory A. Pratt  wrote:
> This is a mess, many systems are setup with portage already on a
> seperate partition for reasons. What advantage does it provide to move
> the tree now after all these years? I have seen nothing more then lets
> do this cause I like the ideal lately and it is getting old, there is no
> benefit that would justify moving the tree or many other changes that
> are being made in Gentoo lately.

1. If you're able to mount /usr/portage from another filesystem, why
would you think it wouldn't work in with /var/cache/portage?
1a. If your system is already installed, why do you think this even
affects you? Did you read?
2. Pretty sure following FHS more closely is something most people
would see as a benefit.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Jory A. Pratt]

On 07/10/18 16:35, M. J. Everitt wrote:
> On 10/07/18 21:09, William Hubbs wrote:
>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
 On 09/07/18 23:12, Zac Medico wrote:
> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>> I'd mostly argue any such change should only affect new systems
>>
> Yes, changing defaults for existing systems would be annoying.
>
> My recommendation is to have catalyst set the new defaults in the stage
> tarballs.
>
> When sys-apps/portage changes its internal defaults, I'd like for the
> upgrade process to call a tool that generates configuration files when
> necessary to ensure that the existing paths remain constant.
 I think it should be possible for RelEng to make a start on catalyst
 updates - is there anything that would inhibit going ahead with this,
 potentially?
>>> No, nothing. Whatever catalyst puts it the default config will become
>>> our new default.
>> I would still like to see notice about what the new defaults are and how
>> to migrate current systems to them.
>>
>>
>> Thanks,
>>
>> William
>>
>>> -- 
>>> Thanks,
>>> Zac
>>>
>>
>>
> I'd like to propose that further to the discussion here on the -dev
> mailing list, the Council discuss and make a firm proposal on the new
> default paths, and then RelEng can make the appropriate updates to the
> catalyst builds. A news item can be compiled, with an appropriate wiki
> article perhaps on migration strategy (I may volunteer to format such a
> page with some appropriate guidance).
> Regards,
> Michael / veremitz.
> 
This is a mess, many systems are setup with portage already on a
seperate partition for reasons. What advantage does it provide to move
the tree now after all these years? I have seen nothing more then lets
do this cause I like the ideal lately and it is getting old, there is no
benefit that would justify moving the tree or many other changes that
are being made in Gentoo lately.



-- 
===
Jory A. Pratt
Gentoo Linux Developer [Mozilla Lead]
E-Mail: anar...@gentoo.org
GnuPG FP  : D4AC 8D63 0B16 F7C9 08E9  B909 A0CC C3BA B4D0 88B4
GnuPG ID  : B4D088B4
===



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[M. J. Everitt]

On 10/07/18 21:09, William Hubbs wrote:
> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
>>> On 09/07/18 23:12, Zac Medico wrote:
 On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> I'd mostly argue any such change should only affect new systems
>
 Yes, changing defaults for existing systems would be annoying.

 My recommendation is to have catalyst set the new defaults in the stage
 tarballs.

 When sys-apps/portage changes its internal defaults, I'd like for the
 upgrade process to call a tool that generates configuration files when
 necessary to ensure that the existing paths remain constant.
>>> I think it should be possible for RelEng to make a start on catalyst
>>> updates - is there anything that would inhibit going ahead with this,
>>> potentially?
>> No, nothing. Whatever catalyst puts it the default config will become
>> our new default.
> I would still like to see notice about what the new defaults are and how
> to migrate current systems to them.
>
>
> Thanks,
>
> William
>
>> -- 
>> Thanks,
>> Zac
>>
>
>
I'd like to propose that further to the discussion here on the -dev
mailing list, the Council discuss and make a firm proposal on the new
default paths, and then RelEng can make the appropriate updates to the
catalyst builds. A news item can be compiled, with an appropriate wiki
article perhaps on migration strategy (I may volunteer to format such a
page with some appropriate guidance).
Regards,
Michael / veremitz.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> > On 09/07/18 23:12, Zac Medico wrote:
> >> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> >>> I'd mostly argue any such change should only affect new systems
> >>>
> >> Yes, changing defaults for existing systems would be annoying.
> >>
> >> My recommendation is to have catalyst set the new defaults in the stage
> >> tarballs.
> >>
> >> When sys-apps/portage changes its internal defaults, I'd like for the
> >> upgrade process to call a tool that generates configuration files when
> >> necessary to ensure that the existing paths remain constant.
> > I think it should be possible for RelEng to make a start on catalyst
> > updates - is there anything that would inhibit going ahead with this,
> > potentially?
> 
> No, nothing. Whatever catalyst puts it the default config will become
> our new default.

I would still like to see notice about what the new defaults are and how
to migrate current systems to them.


Thanks,

William

> -- 
> Thanks,
> Zac
> 





signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Tue, 10 Jul 2018, Ulrich Mueller wrote:

> On Mon, 9 Jul 2018, William Hubbs wrote:
>> Agreed, /var/db I guess is a Gentoo invention of some kind?

> No, it exists in FreeBSD too.

As was pointed out to me, it exists in all three BSD variants [1,2,3].
Its purpose there is for "miscellaneous automatically generated
system-specific database files".

Ulrich

[1] https://www.freebsd.org/cgi/man.cgi?hier%287%29
[2] http://netbsd.gw.com/cgi-bin/man-cgi?hier+7+NetBSD-current
[3] https://man.openbsd.org/hier


pgpb0oOLPJ7DM.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> On 09/07/18 23:12, Zac Medico wrote:
>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>> I'd mostly argue any such change should only affect new systems
>>>
>> Yes, changing defaults for existing systems would be annoying.
>>
>> My recommendation is to have catalyst set the new defaults in the stage
>> tarballs.
>>
>> When sys-apps/portage changes its internal defaults, I'd like for the
>> upgrade process to call a tool that generates configuration files when
>> necessary to ensure that the existing paths remain constant.
> I think it should be possible for RelEng to make a start on catalyst
> updates - is there anything that would inhibit going ahead with this,
> potentially?

No, nothing. Whatever catalyst puts it the default config will become
our new default.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[M. J. Everitt]

On 09/07/18 23:12, Zac Medico wrote:
> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>> I'd mostly argue any such change should only affect new systems
>>
> Yes, changing defaults for existing systems would be annoying.
>
> My recommendation is to have catalyst set the new defaults in the stage
> tarballs.
>
> When sys-apps/portage changes its internal defaults, I'd like for the
> upgrade process to call a tool that generates configuration files when
> necessary to ensure that the existing paths remain constant.
I think it should be possible for RelEng to make a start on catalyst
updates - is there anything that would inhibit going ahead with this,
potentially?

MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> 
> I'd mostly argue any such change should only affect new systems
>

Yes, changing defaults for existing systems would be annoying.

My recommendation is to have catalyst set the new defaults in the stage
tarballs.

When sys-apps/portage changes its internal defaults, I'd like for the
upgrade process to call a tool that generates configuration files when
necessary to ensure that the existing paths remain constant.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Mon, 9 Jul 2018, William Hubbs wrote:

> On Mon, Jul 09, 2018 at 04:53:43PM -0400, Rich Freeman wrote:
>> Though I do prefer /var/lib or /var/cache over /var/db, simply
>> because /var/lib is actually in FHS.

> Agreed, /var/db I guess is a Gentoo invention of some kind?

No, it exists in FreeBSD too.


pgpTQoqGY9KyG.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 5:34 PM Kristian Fiskerstrand  wrote:
>
> I'd mostly argue any such change should only affect new systems
>

++

If a user wants to migrate it is pretty easy to do.  Update the
setting and do an mv, or don't do an mv in which case it will just
regenerate.  I think /var/db/pkg is the only thing that is
particularly sensitive there (if users lose that then they have a mess
- probably recoverable if they do an emerge -e world and then go
hunting for orphans).

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Kristian Fiskerstrand]

On 07/09/2018 11:14 PM, William Hubbs wrote:
>> Though I do prefer /var/lib or /var/cache over /var/db, simply because
>> /var/lib is actually in FHS.
> Agreed, /var/db I guess is a Gentoo invention of some kind?

well, for a gentoo-based PMS that might not be a bad thing.. but I'd say
cache is out of the question, whether it is /var/lib or /var/db doesn't
matter too much to me, but it needs to be announced properly ahead of
time to adjust LVM2 volumes etc etc if impacting existing systems... I'd
mostly argue any such change should only affect new systems

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Mon, Jul 09, 2018 at 04:53:43PM -0400, Rich Freeman wrote:
> On Mon, Jul 9, 2018 at 4:13 PM Michał Górny  wrote:
> >
> > W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
> > napisał:
> > > On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > > > sys-apps/portage-mgorny has already done that.  The defaults locations
> > > > have been changed to:
> > > >
> > > >   DISTDIR="/var/cache/portage/distfiles"
> > > >   PKGDIR="/var/cache/portage/packages"
> > > >   RPMDIR="/var/cache/portage/rpm"
> > > >
> > > > Plus repositories are in /var/db/repos/.  This is also the layout
> > > > used by eselect-repository.
> > >
> > > I like this idea, but slightly different; I think we should stay out of
> > > /var/db. We don't want folks to go messing around in there and nuke
> > > /var/db/pkg by mistake.
> > >
> >
> > Following that reasoning, we shouldn't use /var at all because people
> > might 'go messing around in there and nuke /var/* by mistake'.  Or any
> > directory.  Our only hope is Windows where we can create P:\ and not
> > worry that people might 'go messing around in there and nuke the system
> > by mistake'.
> >
> 
> ++
> 
> Though I do prefer /var/lib or /var/cache over /var/db, simply because
> /var/lib is actually in FHS.

Agreed, /var/db I guess is a Gentoo invention of some kind?

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 4:13 PM Michał Górny  wrote:
>
> W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
> napisał:
> > On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > > sys-apps/portage-mgorny has already done that.  The defaults locations
> > > have been changed to:
> > >
> > >   DISTDIR="/var/cache/portage/distfiles"
> > >   PKGDIR="/var/cache/portage/packages"
> > >   RPMDIR="/var/cache/portage/rpm"
> > >
> > > Plus repositories are in /var/db/repos/.  This is also the layout
> > > used by eselect-repository.
> >
> > I like this idea, but slightly different; I think we should stay out of
> > /var/db. We don't want folks to go messing around in there and nuke
> > /var/db/pkg by mistake.
> >
>
> Following that reasoning, we shouldn't use /var at all because people
> might 'go messing around in there and nuke /var/* by mistake'.  Or any
> directory.  Our only hope is Windows where we can create P:\ and not
> worry that people might 'go messing around in there and nuke the system
> by mistake'.
>

++

Though I do prefer /var/lib or /var/cache over /var/db, simply because
/var/lib is actually in FHS.

That said, all my comments should be taken as suggestions.  I don't
really have a huge concern with most of these proposals.  Any of them
are better than /usr, with distfiles being stacked inside the repo
(ugh!).


-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
napisał:
> On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > sys-apps/portage-mgorny has already done that.  The defaults locations
> > have been changed to:
> > 
> >   DISTDIR="/var/cache/portage/distfiles"
> >   PKGDIR="/var/cache/portage/packages"
> >   RPMDIR="/var/cache/portage/rpm"
> > 
> > Plus repositories are in /var/db/repos/.  This is also the layout
> > used by eselect-repository.
> 
> I like this idea, but slightly different; I think we should stay out of
> /var/db. We don't want folks to go messing around in there and nuke
> /var/db/pkg by mistake.
> 

Following that reasoning, we shouldn't use /var at all because people
might 'go messing around in there and nuke /var/* by mistake'.  Or any
directory.  Our only hope is Windows where we can create P:\ and not
worry that people might 'go messing around in there and nuke the system
by mistake'.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

On 07/09/2018 01:07 PM, Zac Medico wrote:
> On 07/09/2018 01:00 PM, William Hubbs wrote:
>> On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
 On Mon, 9 Jul 2018, Rich Freeman wrote:
>>>
 I'd also consider /var/cache here as well.  FHS specifically suggests
 using it for web caches and the like (let's set aside the issue with
 making that global), though for the most part it is more metadata
 caching.  A key principle is that it can be wiped without loss of
 data, and I think that is generally true for the repository since it
 can be synced.
>>>
>>> I don't think that criterium is fulfilled, because you cannot easily
>>> restore the previous state after it's been wiped. At least not when
>>> syncing from a rsync mirror (which may have been updated in the mean
>>> time).
>>  
>> The criteria for /var/cache do not require being able to restore the
>> exact previous state; they just require that the application be able to
>> regenerate or restore the data , so they are definitely fulfilled.[1].
> 
> If it's an rsync tree then we cannot restore the precise state, for
> example you might not be able to rebuild one of your installed packages
> if the corresponding ebuild has been removed upstream.

Whoops I didn't mean to simply repeat what Ulrich said. My point is that
the spirit of the FHS might be that "nothing is lost", but you certainly
can lose some valuable state if it's an rsync tree.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> sys-apps/portage-mgorny has already done that.  The defaults locations
> have been changed to:
> 
>   DISTDIR="/var/cache/portage/distfiles"
>   PKGDIR="/var/cache/portage/packages"
>   RPMDIR="/var/cache/portage/rpm"
> 
> Plus repositories are in /var/db/repos/.  This is also the layout
> used by eselect-repository.

I like this idea, but slightly different; I think we should stay out of
/var/db. We don't want folks to go messing around in there and nuke
/var/db/pkg by mistake.

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

On 07/09/2018 01:00 PM, William Hubbs wrote:
> On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
>>> On Mon, 9 Jul 2018, Rich Freeman wrote:
>>
>>> I'd also consider /var/cache here as well.  FHS specifically suggests
>>> using it for web caches and the like (let's set aside the issue with
>>> making that global), though for the most part it is more metadata
>>> caching.  A key principle is that it can be wiped without loss of
>>> data, and I think that is generally true for the repository since it
>>> can be synced.
>>
>> I don't think that criterium is fulfilled, because you cannot easily
>> restore the previous state after it's been wiped. At least not when
>> syncing from a rsync mirror (which may have been updated in the mean
>> time).
>  
> The criteria for /var/cache do not require being able to restore the
> exact previous state; they just require that the application be able to
> regenerate or restore the data , so they are definitely fulfilled.[1].

If it's an rsync tree then we cannot restore the precise state, for
example you might not be able to rebuild one of your installed packages
if the corresponding ebuild has been removed upstream.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
> > On Mon, 9 Jul 2018, Rich Freeman wrote:
> 
> > I'd also consider /var/cache here as well.  FHS specifically suggests
> > using it for web caches and the like (let's set aside the issue with
> > making that global), though for the most part it is more metadata
> > caching.  A key principle is that it can be wiped without loss of
> > data, and I think that is generally true for the repository since it
> > can be synced.
> 
> I don't think that criterium is fulfilled, because you cannot easily
> restore the previous state after it's been wiped. At least not when
> syncing from a rsync mirror (which may have been updated in the mean
> time).
 
The criteria for /var/cache do not require being able to restore the
exact previous state; they just require that the application be able to
regenerate or restore the data , so they are definitely fulfilled.[1].

> Also Portage doesn't treat it likea a cache, i.e. it doesn't start to
> fetch ebuilds from remote if it doesn't find them in the local tree.

There is no definition of how a cache should be treated in fhs, so I
don't see this as an argument against /var/cache either.

William

[1] 
https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData


signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

On Mon, 9 Jul 2018 12:21:36 -0500
William Hubbs  wrote:

> All,
> 
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?
> 
> If not, what is the status of us being able to do this?
> 
> Thanks,
> 
> William
> 

I don't recall a tracker bug ever being created.

It required the stages be generated by catalyst-3 which can be
configured to any location for the tree defaults.  catalyst-2 had paths
hard-coded all over the place, plus used those paths as keys in
python dictionaries...

I believe all stages are now built with catalyst-3 for all arches, but
I don't know about some of the lesser used arches as some of those are
older dates.

That and a portage release with the new default location set in it's
backup configs.

So, it should be ready to convert if the minor arches stage are being
generated with catalyst-3

-- 
Brian Dolbec 



pgpJWo1O2DkxL.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

W dniu pon, 09.07.2018 o godzinie 12∶21 -0500, użytkownik William Hubbs
napisał:
> All,
> 
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?
> 
> If not, what is the status of us being able to do this?

sys-apps/portage-mgorny has already done that.  The defaults locations
have been changed to:

  DISTDIR="/var/cache/portage/distfiles"
  PKGDIR="/var/cache/portage/packages"
  RPMDIR="/var/cache/portage/rpm"

Plus repositories are in /var/db/repos/.  This is also the layout
used by eselect-repository.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Mon, 9 Jul 2018, Rich Freeman wrote:

> I'd also consider /var/cache here as well.  FHS specifically suggests
> using it for web caches and the like (let's set aside the issue with
> making that global), though for the most part it is more metadata
> caching.  A key principle is that it can be wiped without loss of
> data, and I think that is generally true for the repository since it
> can be synced.

I don't think that criterium is fulfilled, because you cannot easily
restore the previous state after it's been wiped. At least not when
syncing from a rsync mirror (which may have been updated in the mean
time).

Also Portage doesn't treat it likea a cache, i.e. it doesn't start to
fetch ebuilds from remote if it doesn't find them in the local tree.

Ulrich


pgpWGN18tTNne.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 2:11 PM Johannes Huber  wrote:
>
> Am 09.07.2018 um 20:05 schrieb Rich Freeman:
> > On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller  wrote:
> >>
> >>> On Mon, 9 Jul 2018, William Hubbs wrote:
> >>
> >>> is there a tracker for when the portage tree can be moved out of
> >>> /usr/portage by default?
> >>
> >>> If not, what is the status of us being able to do this?
> >>
> >> Please remind me, what was the plan for the new location?
> >> Somewhere under /var/db or /var/lib, IIRC?
> >>
> >
> > I'd also consider /var/cache here as well.  FHS specifically suggests
> > using it for web caches and the like (let's set aside the issue with
> > making that global), though for the most part it is more metadata
> > caching.  A key principle is that it can be wiped without loss of
> > data, and I think that is generally true for the repository since it
> > can be synced.
> >
> > Stuff in /var/lib can't be deleted without some kind of loss of
> > application state.  /var/db isn't in FHS, and I note that even mysql
> > sticks its stuff in /var/lib.
> >
>
> Imho it would make sense to split up portage files with this change.
> Move the tree (ebuilds, profiles etc) to /var/lib/... and the metadata
> cache to /var/db as it can be regenerated out of the tree.
>

Are you talking about the metadata that gets synced as part of the
repository?  Conceptually I like the idea of splitting it out, but IMO
the whole repository is really just one big cache, so keeping it
together since it always has to be consistent isn't a huge problem.

If you're talking about the stuff in /var/cache/edb, then that should
be separate from the repository, but should still be in cache.

I'd probably create /var/cache/portage, with subdirectories for
repositories (with a subdir for each one synced by portage), edb,
distfiles, and binary packages.
/var/cache/portage/repos/main
/var/cache/portage/repos/my-favorite-overlay
/var/cache/portage/distfiles
/var/cache/portage/edb

The stuff in /var/db/pkg should probably go in /var/lib/portage/pkg or
something like that, at least long-term.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Johannes Huber]

Am 09.07.2018 um 20:05 schrieb Rich Freeman:
> On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller  wrote:
>>
>>> On Mon, 9 Jul 2018, William Hubbs wrote:
>>
>>> is there a tracker for when the portage tree can be moved out of
>>> /usr/portage by default?
>>
>>> If not, what is the status of us being able to do this?
>>
>> Please remind me, what was the plan for the new location?
>> Somewhere under /var/db or /var/lib, IIRC?
>>
> 
> I'd also consider /var/cache here as well.  FHS specifically suggests
> using it for web caches and the like (let's set aside the issue with
> making that global), though for the most part it is more metadata
> caching.  A key principle is that it can be wiped without loss of
> data, and I think that is generally true for the repository since it
> can be synced.
> 
> Stuff in /var/lib can't be deleted without some kind of loss of
> application state.  /var/db isn't in FHS, and I note that even mysql
> sticks its stuff in /var/lib.
> 

Imho it would make sense to split up portage files with this change.
Move the tree (ebuilds, profiles etc) to /var/lib/... and the metadata
cache to /var/db as it can be regenerated out of the tree.

Best regards,
Johannes



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller  wrote:
>
> > On Mon, 9 Jul 2018, William Hubbs wrote:
>
> > is there a tracker for when the portage tree can be moved out of
> > /usr/portage by default?
>
> > If not, what is the status of us being able to do this?
>
> Please remind me, what was the plan for the new location?
> Somewhere under /var/db or /var/lib, IIRC?
>

I'd also consider /var/cache here as well.  FHS specifically suggests
using it for web caches and the like (let's set aside the issue with
making that global), though for the most part it is more metadata
caching.  A key principle is that it can be wiped without loss of
data, and I think that is generally true for the repository since it
can be synced.

Stuff in /var/lib can't be deleted without some kind of loss of
application state.  /var/db isn't in FHS, and I note that even mysql
sticks its stuff in /var/lib.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 1:26 PM Alec Warner  wrote:
>
> The former is probably 3 times easier than the latter.
>  - Get testers to move their tree and report issues[0].
>  - Change the stage3 defaults to be the new location.
>  - Explicitly do nothing else.
>
> New installs will get the new location, old installs will get the old 
> location.
>
> [0] A number of people already point PORTDIR at some other location and 
> appear to operate without major issues.
>

IMO it would be best to just fix this for new installs, and post a
news item about it with some instructions for users to migrate.  There
really isn't much to it though.  I'd think you could just move it at
the filesystem level and then change the pointers.  Or you can just
change the pointers and do a sync to pull down a fresh copy and clean
up later.

I'm not sure where we default distfiles to these days but that should
also go outside of /usr.  It should also not be a subdirectory of
PORTDIR.  Ideally you should just be able to rm -r $PORTDIR and then
do a sync and get it right back.

(Also, your email got former/latter swapped around.)

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

> On Mon, 9 Jul 2018, William Hubbs wrote:

> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?

> If not, what is the status of us being able to do this?

Please remind me, what was the plan for the new location?
Somewhere under /var/db or /var/lib, IIRC?

Ulrich


pgpRbE4DJzMgU.pgp
Description: PGP signature

Re: [gentoo-dev] rfc: moving default location of portage tree

[Dennis Schridde]

On Monday, 9 July 2018 19:26:54 CEST Alec Warner wrote:
> [0] A number of people already point PORTDIR at some other location and
> appear to operate without major issues.

I do have it in /var/cache/portage/gentoo (alongside /var/cache/portage/
{distfiles,packages,local} and that works quite well.

--Dennis

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

On Mon, Jul 9, 2018 at 1:21 PM, William Hubbs  wrote:

> All,
>
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?


I suspect the answer is 'whenever' but that mostly depends on
implementation and what you want to accomplish.

Do you want:

 - All hosts everywhere to move from $CURRENT to $NEW?
 - Only new installs to move from $CURRENT to $NEW?


>
> If not, what is the status of us being able to do this?


The former is probably 3 times easier than the latter.
 - Get testers to move their tree and report issues[0].
 - Change the stage3 defaults to be the new location.
 - Explicitly do nothing else.

New installs will get the new location, old installs will get the old
location.

The latter is harder (one must design and execute a migration for existing
installs) and I suspect the value of such a migration is low and the risk
high.
Is there any advantage to migrating existing installs?

[0] A number of people already point PORTDIR at some other location and
appear to operate without major issues.


>
> Thanks,
>
> William
>
>

[gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

All,

is there a tracker for when the portage tree can be moved out of
/usr/portage by default?

If not, what is the status of us being able to do this?

Thanks,

William



signature.asc
Description: Digital signature