Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-21 Thread Michał Górny
On Wed, 2021-05-19 at 11:22 -0400, Wolfgang E. Sanyer wrote:
> On Wed, May 19, 2021 at 11:02 AM Michał Górny  wrote:
> > 
> > On Wed, 2021-05-19 at 10:37 -0400, Wolfgang E. Sanyer wrote:
> > > See some comments in-line below.
> > > 
> > > 
> > > On Wed, May 19, 2021 at 8:33 AM Michał Górny  wrote:
> > > > 
> > > > Hi,
> > > > 
> > > > Please review the pre-GLEP inlined below.  Its purpose is to formally
> > > > define the format of layout.conf.  It's pretty much inevitable these
> > > > days, so we should specify it.  However, it doesn't really fit into PMS,
> > > > and other formats (Manifests, metadata.xml) are already defined
> > > > in GLEPs, so this just follows suit.
> > > > 
> > > > Pre-GLEP follows.
> > > > 
> > > > ---
> > > > GLEP: 
> > > > Title: Repository configuration file (layout.conf)
> > > > Author: Michał Górny 
> > > > Type: Standards Track
> > > > Status: Draft
> > > > Version: 1.0
> > > > Created: 2021-05-19
> > > > Last-Modified: 2021-05-19
> > > > Post-History: 2021-05-19
> > > > Content-Type: text/x-rst
> > > > ---
> > > > 
> > > > Abstract
> > > > 
> > > > 
> > > > The ``metadata/layout.conf`` file format is specified as used by Portage
> > > > and PkgCore.  A standard set of configuration keys is described
> > > > including the keys currently used in the Gentoo repository.
> > > > 
> > > > 
> > > > Motivation
> > > > ==
> > > > 
> > > > The ``metadata/layout.conf`` file was first added to the Gentoo
> > > > repository in Oct 2011, to facilitate setting of hashes used
> > > > in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> > > > to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> > > > used to indicate the repository's masters and effectively became
> > > > obligatory for all repositories.
> > > > 
> > > > Today, ``layout.conf`` is used for various repository configuration
> > > > knobs that can be expressed as simple values and therefore
> > > > do not justify adding new files to the repository.  This primarily
> > > > involves the configuration of development tools but also includes a few
> > > > keys relevant to the behavior of the package manager.
> > > > 
> > > > However, ``layout.conf`` is currently not covered by any formal
> > > > specification.  The PMS neglects its existence entirely, and the keys
> > > > used are roughly defined by their first use of Portage or PkgCore.
> > > > This GLEP aims to overcome this by providing a formal specification
> > > > for the file, as well as an up-to-date list of permitted configuration
> > > > keys.
> > > > 
> > > > 
> > > > Specification
> > > > =
> > > > 
> > > > layout.conf file format
> > > > ---
> > > > 
> > > > Every ebuild repository must contain a ``metadata/layout.conf`` file.
> > > > The file uses a line-oriented text format.  Lines starting with ``#``
> > > > represent comments and are ignored, as are lines consisting entirely
> > > > of whitespace.  The remaining lines must contain a key followed
> > > > by equals sign (``=``), followed by a (possibly empty) value.  Each of
> > > > these elements can be surrounded by additional whitespace that
> > > > is stripped.
> > > 
> > > Is a space around the "=" optional? I see it used below, but here it 
> > > sounds
> > > like there should be no space.
> > 
> > The last sentence says you can add extra space and it will be stripped.
> > If you can think of a better way of wording that (I really wanted to
> > avoid 'optional whitespace, followed by key, followed by optional
> > whitespace... ' ;-).
> 
> 臘‍♂️ ofc I would miss that.
> 
> I always thought that python did a pretty good job
> [explaining their format mini-language][1]. That's obviously overkill for what
> you're describing here, but maybe something like this would be helpful?
> 
> # comments are allowed
>  = [ [...]]
> 
> Any lines starting with a ``#`` represents a comment and are ignored, as are
> lines consisting entirely of whitespace.
> 
> A valid line consists of key and an equal sign. (A valid key is a
> string containing
> any character except for a space)[^1]. The list of space-separated
> values is optional.
> 
> Note: any additional spaces are ultimately stripped
> 
> [1]: 
> https://docs.python.org/3/library/string.html#format-specification-mini-language
> [^1]: Should we include this or nah?

Thanks, this looks like a nice idea.  I'm going to send a v2 shortly
and we'll see how it worked out.

> 
> > > 
> > > > Configuration keys
> > > > --
> > > > 
> > > > The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> > > > listed in this specification are entirely optional.  The package
> > > > managers may choose to implement a subset of listed keys.  Unknown keys
> > > > must be ignored.
> > > > 
> > > > The following keys are currently defined:
> > > 
> > > Is the intent for the GLEP to define and specify every possible key-value
> > > combination? If not, then perhaps this 

Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-21 Thread Michał Górny
On Thu, 2021-05-20 at 21:33 -0500, John Helmert III wrote:
> On Wed, May 19, 2021 at 02:32:27PM +0200, Michał Górny wrote:
> > Hi,
> > 
> > Please review the pre-GLEP inlined below.  Its purpose is to formally
> > define the format of layout.conf.  It's pretty much inevitable these
> > days, so we should specify it.  However, it doesn't really fit into PMS,
> > and other formats (Manifests, metadata.xml) are already defined
> > in GLEPs, so this just follows suit.
> > 
> > Pre-GLEP follows.
> > 
> > ---
> > GLEP: 
> > Title: Repository configuration file (layout.conf)
> > Author: Michał Górny 
> > Type: Standards Track
> > Status: Draft
> > Version: 1.0
> > Created: 2021-05-19
> > Last-Modified: 2021-05-19
> > Post-History: 2021-05-19
> > Content-Type: text/x-rst
> > ---
> > 
> > Abstract
> > 
> > 
> > The ``metadata/layout.conf`` file format is specified as used by Portage
> > and PkgCore.  A standard set of configuration keys is described
> 
> I can't speak for pkgcore but I can't find anywhere this capitalization
> scheme is used. Internally and in its docs it seems 'Pkgcore' is used at
> the beginning of sentences, but generally 'pkgcore' is used.

Ok, fixed.

> 
> > including the keys currently used in the Gentoo repository.
> > 
> > 
> > Motivation
> > ==
> > 
> > The ``metadata/layout.conf`` file was first added to the Gentoo
> > repository in Oct 2011, to facilitate setting of hashes used
> > in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> > to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> > used to indicate the repository's masters and effectively became
> > obligatory for all repositories.
> > 
> > Today, ``layout.conf`` is used for various repository configuration
> > knobs that can be expressed as simple values and therefore
> > do not justify adding new files to the repository.  This primarily
> > involves the configuration of development tools but also includes a few
> > keys relevant to the behavior of the package manager.
> > 
> > However, ``layout.conf`` is currently not covered by any formal
> > specification.  The PMS neglects its existence entirely, and the keys
> > used are roughly defined by their first use of Portage or PkgCore.
> > This GLEP aims to overcome this by providing a formal specification
> > for the file, as well as an up-to-date list of permitted configuration
> > keys.
> > 
> > 
> > Specification
> > =
> > 
> > layout.conf file format
> > ---
> > 
> > Every ebuild repository must contain a ``metadata/layout.conf`` file.
> > The file uses a line-oriented text format.  Lines starting with ``#``
> > represent comments and are ignored, as are lines consisting entirely
> > of whitespace.  The remaining lines must contain a key followed
> > by equals sign (``=``), followed by a (possibly empty) value.  Each of
> 
> "...followed by zero or more space separated values" would be better I
> think. Currently it reads like only one value is allowed.

Well, I think I messed it up below.  A space-separated list of tokens is
still one value, just being a list.  We definitely have single-value
variables, so I'll just clarify.

> 
> > these elements can be surrounded by additional whitespace that
> > is stripped.
> > 
> > 
> > Configuration keys
> > --
> > 
> > The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> > listed in this specification are entirely optional.  The package
> > managers may choose to implement a subset of listed keys.  Unknown keys
> > must be ignored.
> > 
> > The following keys are currently defined:
> > 
> > masters = 
> >   Specifies the master repositories of this repository.  For stand-alone
> >   repositories, this must be set to an empty value.  Otherwise, it can
> >   list one or more repositories, separated by spaces.  This key must
> >   be specified.
> > 
> > manifest-hashes = 
> >   Specifies the list of hashes that should be used for new distfiles
> >   in the Manifest files.  The development tools may create a subset
> >   of the specified hashes if it is not updating the checksums for
> >   the specified distfile, or does not support the hash in question.
> >   The hash names are specified in GLEP 74.  [#GLEP74]_  The default
> >   set of hashes is implementation-defined.
> > 
> > manifest-required-hashes = 
> >   Specifies the list of hashes that must be used in Manifest files.
> >   The development tools must support all the hashes listed there,
> >   and update distfile checksums to use these hashes (refetching
> >   if necessary).  This must be a subset of manifest-hashes.  If not
> >   specified, all hashes from manifest-hashes (or the default set)
> >   are considered required.
> > 
> > use-manifests = ``strict``, ``true`` or ``false``
> >   Indicates the policy for creating and using Manifest files.  If set
> >   to ``strict``, Manifest files are created and files are required to
> >   match digests found 

Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-20 Thread John Helmert III
On Wed, May 19, 2021 at 02:32:27PM +0200, Michał Górny wrote:
> Hi,
> 
> Please review the pre-GLEP inlined below.  Its purpose is to formally
> define the format of layout.conf.  It's pretty much inevitable these
> days, so we should specify it.  However, it doesn't really fit into PMS,
> and other formats (Manifests, metadata.xml) are already defined
> in GLEPs, so this just follows suit.
> 
> Pre-GLEP follows.
> 
> ---
> GLEP: 
> Title: Repository configuration file (layout.conf)
> Author: Michał Górny 
> Type: Standards Track
> Status: Draft
> Version: 1.0
> Created: 2021-05-19
> Last-Modified: 2021-05-19
> Post-History: 2021-05-19
> Content-Type: text/x-rst
> ---
> 
> Abstract
> 
> 
> The ``metadata/layout.conf`` file format is specified as used by Portage
> and PkgCore.  A standard set of configuration keys is described

I can't speak for pkgcore but I can't find anywhere this capitalization
scheme is used. Internally and in its docs it seems 'Pkgcore' is used at
the beginning of sentences, but generally 'pkgcore' is used.

> including the keys currently used in the Gentoo repository.
> 
> 
> Motivation
> ==
> 
> The ``metadata/layout.conf`` file was first added to the Gentoo
> repository in Oct 2011, to facilitate setting of hashes used
> in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> used to indicate the repository's masters and effectively became
> obligatory for all repositories.
> 
> Today, ``layout.conf`` is used for various repository configuration
> knobs that can be expressed as simple values and therefore
> do not justify adding new files to the repository.  This primarily
> involves the configuration of development tools but also includes a few
> keys relevant to the behavior of the package manager.
> 
> However, ``layout.conf`` is currently not covered by any formal
> specification.  The PMS neglects its existence entirely, and the keys
> used are roughly defined by their first use of Portage or PkgCore.
> This GLEP aims to overcome this by providing a formal specification
> for the file, as well as an up-to-date list of permitted configuration
> keys.
> 
> 
> Specification
> =
> 
> layout.conf file format
> ---
> 
> Every ebuild repository must contain a ``metadata/layout.conf`` file.
> The file uses a line-oriented text format.  Lines starting with ``#``
> represent comments and are ignored, as are lines consisting entirely
> of whitespace.  The remaining lines must contain a key followed
> by equals sign (``=``), followed by a (possibly empty) value.  Each of

"...followed by zero or more space separated values" would be better I
think. Currently it reads like only one value is allowed.

> these elements can be surrounded by additional whitespace that
> is stripped.
> 
> 
> Configuration keys
> --
> 
> The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> listed in this specification are entirely optional.  The package
> managers may choose to implement a subset of listed keys.  Unknown keys
> must be ignored.
> 
> The following keys are currently defined:
> 
> masters = 
>   Specifies the master repositories of this repository.  For stand-alone
>   repositories, this must be set to an empty value.  Otherwise, it can
>   list one or more repositories, separated by spaces.  This key must
>   be specified.
> 
> manifest-hashes = 
>   Specifies the list of hashes that should be used for new distfiles
>   in the Manifest files.  The development tools may create a subset
>   of the specified hashes if it is not updating the checksums for
>   the specified distfile, or does not support the hash in question.
>   The hash names are specified in GLEP 74.  [#GLEP74]_  The default
>   set of hashes is implementation-defined.
> 
> manifest-required-hashes = 
>   Specifies the list of hashes that must be used in Manifest files.
>   The development tools must support all the hashes listed there,
>   and update distfile checksums to use these hashes (refetching
>   if necessary).  This must be a subset of manifest-hashes.  If not
>   specified, all hashes from manifest-hashes (or the default set)
>   are considered required.
> 
> use-manifests = ``strict``, ``true`` or ``false``
>   Indicates the policy for creating and using Manifest files.  If set
>   to ``strict``, Manifest files are created and files are required to
>   match digests found in Manifests.  If set to ``true``, Manifests
>   are created but digest mismatches are ignored.  If set to ``false``,
>   Manifests are not used at all.  The default is ``strict``.
> 
> update-changelog = ``true`` or ``false``
>   Indicates whether the development tools should write ChangeLog files.
>   The default is ``false``.
> 
> cache-formats = 
>   Specifies one or more cache formats used by the repository.
>   The currently defined values are ``pms`` for the original 

Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-19 Thread Wolfgang E. Sanyer
On Wed, May 19, 2021 at 11:02 AM Michał Górny  wrote:
>
> On Wed, 2021-05-19 at 10:37 -0400, Wolfgang E. Sanyer wrote:
> > See some comments in-line below.
> >
> >
> > On Wed, May 19, 2021 at 8:33 AM Michał Górny  wrote:
> > >
> > > Hi,
> > >
> > > Please review the pre-GLEP inlined below.  Its purpose is to formally
> > > define the format of layout.conf.  It's pretty much inevitable these
> > > days, so we should specify it.  However, it doesn't really fit into PMS,
> > > and other formats (Manifests, metadata.xml) are already defined
> > > in GLEPs, so this just follows suit.
> > >
> > > Pre-GLEP follows.
> > >
> > > ---
> > > GLEP: 
> > > Title: Repository configuration file (layout.conf)
> > > Author: Michał Górny 
> > > Type: Standards Track
> > > Status: Draft
> > > Version: 1.0
> > > Created: 2021-05-19
> > > Last-Modified: 2021-05-19
> > > Post-History: 2021-05-19
> > > Content-Type: text/x-rst
> > > ---
> > >
> > > Abstract
> > > 
> > >
> > > The ``metadata/layout.conf`` file format is specified as used by Portage
> > > and PkgCore.  A standard set of configuration keys is described
> > > including the keys currently used in the Gentoo repository.
> > >
> > >
> > > Motivation
> > > ==
> > >
> > > The ``metadata/layout.conf`` file was first added to the Gentoo
> > > repository in Oct 2011, to facilitate setting of hashes used
> > > in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> > > to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> > > used to indicate the repository's masters and effectively became
> > > obligatory for all repositories.
> > >
> > > Today, ``layout.conf`` is used for various repository configuration
> > > knobs that can be expressed as simple values and therefore
> > > do not justify adding new files to the repository.  This primarily
> > > involves the configuration of development tools but also includes a few
> > > keys relevant to the behavior of the package manager.
> > >
> > > However, ``layout.conf`` is currently not covered by any formal
> > > specification.  The PMS neglects its existence entirely, and the keys
> > > used are roughly defined by their first use of Portage or PkgCore.
> > > This GLEP aims to overcome this by providing a formal specification
> > > for the file, as well as an up-to-date list of permitted configuration
> > > keys.
> > >
> > >
> > > Specification
> > > =
> > >
> > > layout.conf file format
> > > ---
> > >
> > > Every ebuild repository must contain a ``metadata/layout.conf`` file.
> > > The file uses a line-oriented text format.  Lines starting with ``#``
> > > represent comments and are ignored, as are lines consisting entirely
> > > of whitespace.  The remaining lines must contain a key followed
> > > by equals sign (``=``), followed by a (possibly empty) value.  Each of
> > > these elements can be surrounded by additional whitespace that
> > > is stripped.
> >
> > Is a space around the "=" optional? I see it used below, but here it sounds
> > like there should be no space.
>
> The last sentence says you can add extra space and it will be stripped.
> If you can think of a better way of wording that (I really wanted to
> avoid 'optional whitespace, followed by key, followed by optional
> whitespace... ' ;-).

臘‍♂️ ofc I would miss that.

I always thought that python did a pretty good job
[explaining their format mini-language][1]. That's obviously overkill for what
you're describing here, but maybe something like this would be helpful?

# comments are allowed
 = [ [...]]

Any lines starting with a ``#`` represents a comment and are ignored, as are
lines consisting entirely of whitespace.

A valid line consists of key and an equal sign. (A valid key is a
string containing
any character except for a space)[^1]. The list of space-separated
values is optional.

Note: any additional spaces are ultimately stripped

[1]: 
https://docs.python.org/3/library/string.html#format-specification-mini-language
[^1]: Should we include this or nah?

> >
> > > Configuration keys
> > > --
> > >
> > > The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> > > listed in this specification are entirely optional.  The package
> > > managers may choose to implement a subset of listed keys.  Unknown keys
> > > must be ignored.
> > >
> > > The following keys are currently defined:
> >
> > Is the intent for the GLEP to define and specify every possible key-value
> > combination? If not, then perhaps this section should be truncated a bit? 
> > i.e.
> > "here are some examples of currently defined keys" and then list a handful.
> >
> > Additionally, maybe the spec should include some sort of requirement for
> > documentation of variables, similar to what you've done below.
> >
> > Edit: ok, nevermind I see your recommendation at the bottom. But still,
> > I would probably either (1) insist that new keys be added to this GLEP, or
> > (2) 

Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-19 Thread Michał Górny
On Wed, 2021-05-19 at 10:37 -0400, Wolfgang E. Sanyer wrote:
> See some comments in-line below.
> 
> 
> On Wed, May 19, 2021 at 8:33 AM Michał Górny  wrote:
> > 
> > Hi,
> > 
> > Please review the pre-GLEP inlined below.  Its purpose is to formally
> > define the format of layout.conf.  It's pretty much inevitable these
> > days, so we should specify it.  However, it doesn't really fit into PMS,
> > and other formats (Manifests, metadata.xml) are already defined
> > in GLEPs, so this just follows suit.
> > 
> > Pre-GLEP follows.
> > 
> > ---
> > GLEP: 
> > Title: Repository configuration file (layout.conf)
> > Author: Michał Górny 
> > Type: Standards Track
> > Status: Draft
> > Version: 1.0
> > Created: 2021-05-19
> > Last-Modified: 2021-05-19
> > Post-History: 2021-05-19
> > Content-Type: text/x-rst
> > ---
> > 
> > Abstract
> > 
> > 
> > The ``metadata/layout.conf`` file format is specified as used by Portage
> > and PkgCore.  A standard set of configuration keys is described
> > including the keys currently used in the Gentoo repository.
> > 
> > 
> > Motivation
> > ==
> > 
> > The ``metadata/layout.conf`` file was first added to the Gentoo
> > repository in Oct 2011, to facilitate setting of hashes used
> > in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> > to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> > used to indicate the repository's masters and effectively became
> > obligatory for all repositories.
> > 
> > Today, ``layout.conf`` is used for various repository configuration
> > knobs that can be expressed as simple values and therefore
> > do not justify adding new files to the repository.  This primarily
> > involves the configuration of development tools but also includes a few
> > keys relevant to the behavior of the package manager.
> > 
> > However, ``layout.conf`` is currently not covered by any formal
> > specification.  The PMS neglects its existence entirely, and the keys
> > used are roughly defined by their first use of Portage or PkgCore.
> > This GLEP aims to overcome this by providing a formal specification
> > for the file, as well as an up-to-date list of permitted configuration
> > keys.
> > 
> > 
> > Specification
> > =
> > 
> > layout.conf file format
> > ---
> > 
> > Every ebuild repository must contain a ``metadata/layout.conf`` file.
> > The file uses a line-oriented text format.  Lines starting with ``#``
> > represent comments and are ignored, as are lines consisting entirely
> > of whitespace.  The remaining lines must contain a key followed
> > by equals sign (``=``), followed by a (possibly empty) value.  Each of
> > these elements can be surrounded by additional whitespace that
> > is stripped.
> 
> Is a space around the "=" optional? I see it used below, but here it sounds
> like there should be no space.

The last sentence says you can add extra space and it will be stripped.
If you can think of a better way of wording that (I really wanted to
avoid 'optional whitespace, followed by key, followed by optional
whitespace... ' ;-).

> 
> > Configuration keys
> > --
> > 
> > The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> > listed in this specification are entirely optional.  The package
> > managers may choose to implement a subset of listed keys.  Unknown keys
> > must be ignored.
> > 
> > The following keys are currently defined:
> 
> Is the intent for the GLEP to define and specify every possible key-value
> combination? If not, then perhaps this section should be truncated a bit? i.e.
> "here are some examples of currently defined keys" and then list a handful.
> 
> Additionally, maybe the spec should include some sort of requirement for
> documentation of variables, similar to what you've done below.
> 
> Edit: ok, nevermind I see your recommendation at the bottom. But still,
> I would probably either (1) insist that new keys be added to this GLEP, or
> (2) insist that new keys are documented properly.

Let's put it like this: the goal is to document all the useful keys.
I can live with some ancient-not-really-useful keys not being documented
(e.g. Portage has some stale/redundant Manifest-related keys, I think).

> 
> > masters = 
> >   Specifies the master repositories of this repository.  For stand-alone
> >   repositories, this must be set to an empty value.  Otherwise, it can
> >   list one or more repositories, separated by spaces.  This key must
> >   be specified.
> 
> I realize this is the only compulsory key, but perhaps it should still go in
> a separate section in order to ensure that it stands out? i.e. "The following
> keys are mandory..." and then later "the following optional keys are
> currently defined"

I'm not sure, to be honest.  I didn't really want to add a section for
one key ;-).

> 
> > manifest-hashes = 
> >   Specifies the list of hashes that should be used for new distfiles
> >   in the 

Re: [gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-19 Thread Wolfgang E. Sanyer
See some comments in-line below.


On Wed, May 19, 2021 at 8:33 AM Michał Górny  wrote:
>
> Hi,
>
> Please review the pre-GLEP inlined below.  Its purpose is to formally
> define the format of layout.conf.  It's pretty much inevitable these
> days, so we should specify it.  However, it doesn't really fit into PMS,
> and other formats (Manifests, metadata.xml) are already defined
> in GLEPs, so this just follows suit.
>
> Pre-GLEP follows.
>
> ---
> GLEP: 
> Title: Repository configuration file (layout.conf)
> Author: Michał Górny 
> Type: Standards Track
> Status: Draft
> Version: 1.0
> Created: 2021-05-19
> Last-Modified: 2021-05-19
> Post-History: 2021-05-19
> Content-Type: text/x-rst
> ---
>
> Abstract
> 
>
> The ``metadata/layout.conf`` file format is specified as used by Portage
> and PkgCore.  A standard set of configuration keys is described
> including the keys currently used in the Gentoo repository.
>
>
> Motivation
> ==
>
> The ``metadata/layout.conf`` file was first added to the Gentoo
> repository in Oct 2011, to facilitate setting of hashes used
> in Manifest2 files.  In Mar 2012, it was used to indicate the transition
> to the new ``md5-dict`` cache format.  In Jul 2013, it started being
> used to indicate the repository's masters and effectively became
> obligatory for all repositories.
>
> Today, ``layout.conf`` is used for various repository configuration
> knobs that can be expressed as simple values and therefore
> do not justify adding new files to the repository.  This primarily
> involves the configuration of development tools but also includes a few
> keys relevant to the behavior of the package manager.
>
> However, ``layout.conf`` is currently not covered by any formal
> specification.  The PMS neglects its existence entirely, and the keys
> used are roughly defined by their first use of Portage or PkgCore.
> This GLEP aims to overcome this by providing a formal specification
> for the file, as well as an up-to-date list of permitted configuration
> keys.
>
>
> Specification
> =
>
> layout.conf file format
> ---
>
> Every ebuild repository must contain a ``metadata/layout.conf`` file.
> The file uses a line-oriented text format.  Lines starting with ``#``
> represent comments and are ignored, as are lines consisting entirely
> of whitespace.  The remaining lines must contain a key followed
> by equals sign (``=``), followed by a (possibly empty) value.  Each of
> these elements can be surrounded by additional whitespace that
> is stripped.

Is a space around the "=" optional? I see it used below, but here it sounds
like there should be no space.

> Configuration keys
> --
>
> The ``layout.conf`` file must contain the ``masters`` key.  Other keys
> listed in this specification are entirely optional.  The package
> managers may choose to implement a subset of listed keys.  Unknown keys
> must be ignored.
>
> The following keys are currently defined:

Is the intent for the GLEP to define and specify every possible key-value
combination? If not, then perhaps this section should be truncated a bit? i.e.
"here are some examples of currently defined keys" and then list a handful.

Additionally, maybe the spec should include some sort of requirement for
documentation of variables, similar to what you've done below.

Edit: ok, nevermind I see your recommendation at the bottom. But still,
I would probably either (1) insist that new keys be added to this GLEP, or
(2) insist that new keys are documented properly.

> masters = 
>   Specifies the master repositories of this repository.  For stand-alone
>   repositories, this must be set to an empty value.  Otherwise, it can
>   list one or more repositories, separated by spaces.  This key must
>   be specified.

I realize this is the only compulsory key, but perhaps it should still go in
a separate section in order to ensure that it stands out? i.e. "The following
keys are mandory..." and then later "the following optional keys are
currently defined"

> manifest-hashes = 
>   Specifies the list of hashes that should be used for new distfiles
>   in the Manifest files.  The development tools may create a subset
>   of the specified hashes if it is not updating the checksums for
>   the specified distfile, or does not support the hash in question.
>   The hash names are specified in GLEP 74.  [#GLEP74]_  The default
>   set of hashes is implementation-defined.
>
> manifest-required-hashes = 
>   Specifies the list of hashes that must be used in Manifest files.
>   The development tools must support all the hashes listed there,
>   and update distfile checksums to use these hashes (refetching
>   if necessary).  This must be a subset of manifest-hashes.  If not
>   specified, all hashes from manifest-hashes (or the default set)
>   are considered required.
>
> use-manifests = ``strict``, ``true`` or ``false``
>   Indicates the policy for creating and using Manifest files.  If set

[gentoo-dev] [pre-GLEP] Repository configuration file (layout.conf)

2021-05-19 Thread Michał Górny
Hi,

Please review the pre-GLEP inlined below.  Its purpose is to formally
define the format of layout.conf.  It's pretty much inevitable these
days, so we should specify it.  However, it doesn't really fit into PMS,
and other formats (Manifests, metadata.xml) are already defined
in GLEPs, so this just follows suit.

Pre-GLEP follows.

---
GLEP: 
Title: Repository configuration file (layout.conf)
Author: Michał Górny 
Type: Standards Track
Status: Draft
Version: 1.0
Created: 2021-05-19
Last-Modified: 2021-05-19
Post-History: 2021-05-19
Content-Type: text/x-rst
---

Abstract


The ``metadata/layout.conf`` file format is specified as used by Portage
and PkgCore.  A standard set of configuration keys is described
including the keys currently used in the Gentoo repository.


Motivation
==

The ``metadata/layout.conf`` file was first added to the Gentoo
repository in Oct 2011, to facilitate setting of hashes used
in Manifest2 files.  In Mar 2012, it was used to indicate the transition
to the new ``md5-dict`` cache format.  In Jul 2013, it started being
used to indicate the repository's masters and effectively became
obligatory for all repositories.

Today, ``layout.conf`` is used for various repository configuration
knobs that can be expressed as simple values and therefore
do not justify adding new files to the repository.  This primarily
involves the configuration of development tools but also includes a few
keys relevant to the behavior of the package manager.

However, ``layout.conf`` is currently not covered by any formal
specification.  The PMS neglects its existence entirely, and the keys
used are roughly defined by their first use of Portage or PkgCore.
This GLEP aims to overcome this by providing a formal specification
for the file, as well as an up-to-date list of permitted configuration
keys.


Specification
=

layout.conf file format
---

Every ebuild repository must contain a ``metadata/layout.conf`` file.
The file uses a line-oriented text format.  Lines starting with ``#``
represent comments and are ignored, as are lines consisting entirely
of whitespace.  The remaining lines must contain a key followed
by equals sign (``=``), followed by a (possibly empty) value.  Each of
these elements can be surrounded by additional whitespace that
is stripped.


Configuration keys
--

The ``layout.conf`` file must contain the ``masters`` key.  Other keys
listed in this specification are entirely optional.  The package
managers may choose to implement a subset of listed keys.  Unknown keys
must be ignored.

The following keys are currently defined:

masters = 
  Specifies the master repositories of this repository.  For stand-alone
  repositories, this must be set to an empty value.  Otherwise, it can
  list one or more repositories, separated by spaces.  This key must
  be specified.

manifest-hashes = 
  Specifies the list of hashes that should be used for new distfiles
  in the Manifest files.  The development tools may create a subset
  of the specified hashes if it is not updating the checksums for
  the specified distfile, or does not support the hash in question.
  The hash names are specified in GLEP 74.  [#GLEP74]_  The default
  set of hashes is implementation-defined.

manifest-required-hashes = 
  Specifies the list of hashes that must be used in Manifest files.
  The development tools must support all the hashes listed there,
  and update distfile checksums to use these hashes (refetching
  if necessary).  This must be a subset of manifest-hashes.  If not
  specified, all hashes from manifest-hashes (or the default set)
  are considered required.

use-manifests = ``strict``, ``true`` or ``false``
  Indicates the policy for creating and using Manifest files.  If set
  to ``strict``, Manifest files are created and files are required to
  match digests found in Manifests.  If set to ``true``, Manifests
  are created but digest mismatches are ignored.  If set to ``false``,
  Manifests are not used at all.  The default is ``strict``.

update-changelog = ``true`` or ``false``
  Indicates whether the development tools should write ChangeLog files.
  The default is ``false``.

cache-formats = 
  Specifies one or more cache formats used by the repository.
  The currently defined values are ``pms`` for the original format
  specified in the PMS and ``md5-dict`` for the md5-dict format
  introduced in Portage 2.2.0_alpha68.  The default is
  implementation-defined.

eapis-deprecated = 
  Specifies one or more EAPIs that are to be considered deprecated
  by the development tools for use in ebuilds, i.e. their use should
  trigger a warning.  If not specified, no EAPIs are deprecated.

eapis-banned = 
  Specifies one or more EAPIs that are to be considered banned
  by the development tools for use in ebuilds, i.e. their use should
  be blocked.  If not specified, no EAPIs are banned.

repo-name = 
  Specifies the repository name.  If