Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Nick Coghlan]

On 4 September 2017 at 23:33, Nick Coghlan  wrote:
> However, I'm also wondering if it may still be worthwhile writing a
> metadata 1.3 PEP that does the following things:
>
> 1. Explicitly notes the addition of the two new fields
> 2. Describes the process change for packaging interoperability specifications
> 3. Defines a canonical transformation between the human-readable
> key:value format and a more automation friendly JSON format

Discussing this idea further with Dustin, I now think it's conflating
two different activities (one related to process management, one
related to actually updating the metadata specification).

So I'd suggest that any metadata 1.3 PEP restrict itself to only
covering the first point (more formally documenting the two fields
added since metadata 1.2) and the last point (defining a standard
translation from the Key:Value format to JSON and back).

For the process management question around improving traceability in
our spec management processes, I'm wondering if it may make sense to
write a meta-PEP called just "The Python Packaging Authority" that
attempts to better articulate what that term actually encompasses:

* The published interoperability specifications at
https://packaging.python.org/specifications/
* The process for updating those specifications at
https://www.pypa.io/en/latest/specifications/
* Ecosystem & specification level design discussions on distutils-sig
* Funding and resource allocation authority delegated from the PSF
Board to the Python Packaging Working Group (overseen by the PSF's
Director of Operations & Infrastructure Manager)
* PyPI operating authority delegated from the PSF Board to the PSF
Infrastructure team (overseen by the PSF's Director of Operations &
Infrastructure Manager)

Such a meta-PEP could also clearly document the limits of Guido's
original design delegation back at the 2013 Python Language Summit
(i.e. as soon as we need or want to change anything in CPython or the
standard library, then that's a change that needs to go through the
regular PEP process, not the PyPA/distutils-sig specific variant of
it)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Nick Coghlan]

On 4 September 2017 at 23:33, Nick Coghlan  wrote:
> Some time ago, I started the process [1] of adjusting how
> distutils-sig uses the PEP process so that the reference
> specifications will live on packaging.python.org, and we use the PEP
> process to manage *changes* to those specifications, rather than
> serving as the specifications themselves (that is, adopting a process
> closer to the URL-centric way the Python language reference is
> managed, rather than using the RFCstyle PEP-number-centric model the
> way we do now).

I'm happy to report that Dustin Ingram has started making some
progress on this change, with
https://packaging.python.org/specifications/ now broken out into a set
of distinct subpages with appropriate URLs, and
https://packaging.python.org/specifications/core-metadata/ now
coversing *all* the fields that can appear in PKG-INFO and METADATA
files, not just the ones that are post-1.2 additions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Daniel Holth]

Well, none of the metadata generated by bdist wheel conforms to an accepted
pep. But if you rely on the json file then you won't be interoperable with
wheels from any other generator.

On Mon, Sep 4, 2017, 10:06 Alex Grönholm  wrote:

> Yes, I see the inclusion of a metadata file which conforms to an
> unaccepted PEP as potentially dangerous.
>
> Perhaps I should disable it in the next release?
>
> Daniel Holth kirjoitti 04.09.2017 klo 17:03:
>
> Some people enjoy using metadata.json which tracked pep 426 but I have
> been meaning to take it out, and perhaps keep the key/value to json
> converter as a command.
>
> On Mon, Sep 4, 2017, 09:33 Nick Coghlan  wrote:
>
>> Some time ago, I started the process [1] of adjusting how
>> distutils-sig uses the PEP process so that the reference
>> specifications will live on packaging.python.org, and we use the PEP
>> process to manage *changes* to those specifications, rather than
>> serving as the specifications themselves (that is, adopting a process
>> closer to the URL-centric way the Python language reference is
>> managed, rather than using the RFCstyle PEP-number-centric model the
>> way we do now).
>>
>> I never actually finished that work, and as a result, it's currently
>> thoroughly unclear [2] that Description-Content-Type and
>> Provides-Extra are defined at
>> https://packaging.python.org/specifications/#core-metadata rather than
>> in a PEP.
>>
>> I'm currently at the CPython core development sprint in San Francisco,
>> and I'm thinking that finalising that migration [3] and updating the
>> affected PEPs accordingly (most notably, PEP 345) is likely to be a
>> good use of my time.
>>
>> However, I'm also wondering if it may still be worthwhile writing a
>> metadata 1.3 PEP that does the following things:
>>
>> 1. Explicitly notes the addition of the two new fields
>> 2. Describes the process change for packaging interoperability
>> specifications
>> 3. Defines a canonical transformation between the human-readable
>> key:value format and a more automation friendly JSON format
>>
>> That PEP would then essentially be the first one to use the new
>> process: it would supersede PEP 345 as the latest metadata
>> specification, but it would *also* redirect readers to the relevant
>> URL on packaging.python.org as the canonical source of the
>> specification, rather than being the reference documentation in its
>> own right.
>>
>> Cheers,
>> Nick.
>>
>> [1] https://github.com/pypa/pypa.io/issues/11#issuecomment-173833061
>> [2] https://github.com/python/peps/issues/388
>>
>> P.S. Daniel, if you're currently thinking "I proposed defining an
>> incremental metadata 1.3 tweak years ago!", aye, you did. My
>> subsequent additions to PEP 426 were a classic case of second-system
>> syndrome: https://en.wikipedia.org/wiki/Second-system_effect (which we
>> suspected long ago, hence that PEP's original deferral)
>>
>> Fortunately, the disciplining effect of working with a primarily
>> volunteer contributor base has prevented my over-engineered
>> change-for-change's-sake-rather-than-to-solve-actual-user-problems
>> version from becoming reality ;)
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
> ___
> Distutils-SIG maillist  -  
> Distutils-SIG@python.orghttps://mail.python.org/mailman/listinfo/distutils-sig
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Alex Grönholm]

Yes, I see the inclusion of a metadata file which conforms to an 
unaccepted PEP as potentially dangerous.


Perhaps I should disable it in the next release?


Daniel Holth kirjoitti 04.09.2017 klo 17:03:


Some people enjoy using metadata.json which tracked pep 426 but I have 
been meaning to take it out, and perhaps keep the key/value to json 
converter as a command.



On Mon, Sep 4, 2017, 09:33 Nick Coghlan > wrote:


Some time ago, I started the process [1] of adjusting how
distutils-sig uses the PEP process so that the reference
specifications will live on packaging.python.org
, and we use the PEP
process to manage *changes* to those specifications, rather than
serving as the specifications themselves (that is, adopting a process
closer to the URL-centric way the Python language reference is
managed, rather than using the RFCstyle PEP-number-centric model the
way we do now).

I never actually finished that work, and as a result, it's currently
thoroughly unclear [2] that Description-Content-Type and
Provides-Extra are defined at
https://packaging.python.org/specifications/#core-metadata rather than
in a PEP.

I'm currently at the CPython core development sprint in San Francisco,
and I'm thinking that finalising that migration [3] and updating the
affected PEPs accordingly (most notably, PEP 345) is likely to be a
good use of my time.

However, I'm also wondering if it may still be worthwhile writing a
metadata 1.3 PEP that does the following things:

1. Explicitly notes the addition of the two new fields
2. Describes the process change for packaging interoperability
specifications
3. Defines a canonical transformation between the human-readable
key:value format and a more automation friendly JSON format

That PEP would then essentially be the first one to use the new
process: it would supersede PEP 345 as the latest metadata
specification, but it would *also* redirect readers to the relevant
URL on packaging.python.org  as the
canonical source of the
specification, rather than being the reference documentation in its
own right.

Cheers,
Nick.

[1] https://github.com/pypa/pypa.io/issues/11#issuecomment-173833061
[2] https://github.com/python/peps/issues/388

P.S. Daniel, if you're currently thinking "I proposed defining an
incremental metadata 1.3 tweak years ago!", aye, you did. My
subsequent additions to PEP 426 were a classic case of second-system
syndrome: https://en.wikipedia.org/wiki/Second-system_effect (which we
suspected long ago, hence that PEP's original deferral)

Fortunately, the disciplining effect of working with a primarily
volunteer contributor base has prevented my over-engineered
change-for-change's-sake-rather-than-to-solve-actual-user-problems
version from becoming reality ;)

--
Nick Coghlan   | ncogh...@gmail.com 
 |   Brisbane, Australia
___
Distutils-SIG maillist  - Distutils-SIG@python.org

https://mail.python.org/mailman/listinfo/distutils-sig



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Migrating interoperability specs to packaging.python.org

[Daniel Holth]

Some people enjoy using metadata.json which tracked pep 426 but I have been
meaning to take it out, and perhaps keep the key/value to json converter as
a command.

On Mon, Sep 4, 2017, 09:33 Nick Coghlan  wrote:

> Some time ago, I started the process [1] of adjusting how
> distutils-sig uses the PEP process so that the reference
> specifications will live on packaging.python.org, and we use the PEP
> process to manage *changes* to those specifications, rather than
> serving as the specifications themselves (that is, adopting a process
> closer to the URL-centric way the Python language reference is
> managed, rather than using the RFCstyle PEP-number-centric model the
> way we do now).
>
> I never actually finished that work, and as a result, it's currently
> thoroughly unclear [2] that Description-Content-Type and
> Provides-Extra are defined at
> https://packaging.python.org/specifications/#core-metadata rather than
> in a PEP.
>
> I'm currently at the CPython core development sprint in San Francisco,
> and I'm thinking that finalising that migration [3] and updating the
> affected PEPs accordingly (most notably, PEP 345) is likely to be a
> good use of my time.
>
> However, I'm also wondering if it may still be worthwhile writing a
> metadata 1.3 PEP that does the following things:
>
> 1. Explicitly notes the addition of the two new fields
> 2. Describes the process change for packaging interoperability
> specifications
> 3. Defines a canonical transformation between the human-readable
> key:value format and a more automation friendly JSON format
>
> That PEP would then essentially be the first one to use the new
> process: it would supersede PEP 345 as the latest metadata
> specification, but it would *also* redirect readers to the relevant
> URL on packaging.python.org as the canonical source of the
> specification, rather than being the reference documentation in its
> own right.
>
> Cheers,
> Nick.
>
> [1] https://github.com/pypa/pypa.io/issues/11#issuecomment-173833061
> [2] https://github.com/python/peps/issues/388
>
> P.S. Daniel, if you're currently thinking "I proposed defining an
> incremental metadata 1.3 tweak years ago!", aye, you did. My
> subsequent additions to PEP 426 were a classic case of second-system
> syndrome: https://en.wikipedia.org/wiki/Second-system_effect (which we
> suspected long ago, hence that PEP's original deferral)
>
> Fortunately, the disciplining effect of working with a primarily
> volunteer contributor base has prevented my over-engineered
> change-for-change's-sake-rather-than-to-solve-actual-user-problems
> version from becoming reality ;)
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig