Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steve Holden]

What would _really_ help is getting the groups that maintain each dead
parrot to collaborate on a "Python Legacy release" that adds back anything
with a maintainer to the stdlib of the current Python version.

Even that will demand large resources, and if it's to be organised in a way
that doesn't involve lots of dev time then perhaps the PSF could be
involved? I presume the Steering Committee are the people to consider
directions like this.

Steve Holden


On Thu, May 23, 2019 at 7:03 PM Barry Warsaw  wrote:

> On May 20, 2019, at 13:15, Christian Heimes  wrote:
> >
> > here is the first version of my PEP 594 to deprecate and eventually
> remove modules from the standard library. The PEP started last year with
> talk during the Python Language Summit 2018,
> https://lwn.net/Articles/755229/.
> >
> > The PEP can be confirmed in two stages. I'm not planning any code
> changes for 3.8. Instead I only like to document a bunch of modules as
> deprecated. Active deprecation is planned for 3.9 and removal for 3.10. The
> long deprecation phase gives us 3 years to change our minds or handle edge
> cases, too.
>
> Hi Christian,
>
> First, I appreciate the work you’ve done on this, and the general
> sentiment.  But I do feel like this is a partial solution to a problem the
> Python distribution has had for a very long time.  Maybe it’s a good step
> forward, but in a sense it is incomplete and only chips away at the problem.
>
> We have two competing pressures, one to provide a rich standard library
> with lots of useful features that come right out of the box.  Let’s not
> underestimate the value that this has for our users, and the contribution
> such a stdlib has made to making Python as popular as it is.
>
> But it’s also true that lots of the stdlib don’t get the love they need to
> stay relevant, and a curated path to keeping only the most useful and
> modern libraries.  I wonder how much the long development cycle and
> relatively big overhead for contributing to stdlib maintenance causes a big
> part of our headaches with the stdlib.  Current stdlib development
> processes also incur burden for alternative implementations.
>
> We’ve had many ideas over the years, such as stripping the CPython repo
> stdlib to its bare minimum and providing some way of *distributing* a sumo
> tarball.  But none have made it far enough to be adopted.  I don’t have any
> new bright ideas for how to make this work, but I think finding a holistic
> approach to these competing pressures is in the best long term interest of
> Python.
>
> That all said, if PEP 594 can be viewed as part of the bigger picture,
> then maybe it’s a good idea.  Perhaps the approaches for maintaining such
> deprecated libraries can be used as an experiment for finding a more
> modern, useful, and vibrant approach to stdlib maintenance.
>
> Cheers,
> -Barry
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steve Holden]

It might also serve to identify those with an interest in maintaining the
non-core packages, which might even be given some special status on PyPI.


On Thu, May 23, 2019 at 9:01 AM Alex Walters 
wrote:

> I've watched the entire thread and its taken me a few days to put a finger
> on what bothers me about it.
>
> In my opinion, this shouldn't be a pep describing the list of modules that
> need to go as "dead batteries", but should be a process pep describing how
> dead batteries should be removed, and the individual modules should be
> given
> their own pep.  I think reactions to individual module peps will give a
> better indication of if it's a used module or not.
>
> > -Original Message-
> > From: Python-Dev  > list=sdamon@python.org> On Behalf Of Christian Heimes
> > Sent: Monday, May 20, 2019 4:15 PM
> > To: Python Dev 
> > Subject: [Python-Dev] PEP 594: Removing dead batteries from the standard
> > library
> >
> > Hi,
> >
> > here is the first version of my PEP 594 to deprecate and eventually
> remove
> > modules from the standard library. The PEP started last year with talk
> during
> > the Python Language Summit 2018, https://lwn.net/Articles/755229/.
> >
> > The PEP can be confirmed in two stages. I'm not planning any code changes
> > for 3.8. Instead I only like to document a bunch of modules as
> deprecated.
> > Active deprecation is planned for 3.9 and removal for 3.10. The long
> > deprecation phase gives us 3 years to change our minds or handle edge
> > cases, too.
> >
> > Regards,
> > Christian
> >
> >
> >
>
> 
> > PEP: 594
> > Title: Removing dead batteries from the standard library
> > Author: Christian Heimes 
> > Status: Active
> > Type: Process
> > Content-Type: text/x-rst
> > Created: 20-May-2019
> > Post-History:
> >
> >
> > Abstract
> > 
> >
> > This PEP proposed a list of standard library modules to be removed from
> the
> > standard library. The modules are mostly historic data formats and APIs
> that
> > have been superseded a long time ago, e.g. Mac OS 9 and Commodore.
> >
> >
> > Rationale
> > =
> >
> > Back in the early days of Python, the interpreter came with a large set
> of
> > useful modules. This was often refrained to as "batteries included"
> > philosophy and was one of the corner stones to Python's success story.
> > Users didn't have to figure out how to download and install separate
> > packages in order to write a simple web server or parse email.
> >
> > Times have changed. The introduction of the cheese shop (PyPI),
> setuptools,
> > and later pip, it became simple and straight forward to download and
> install
> > packages. Nowadays Python has a rich and vibrant ecosystem of third party
> > packages. It's pretty much standard to either install packages from PyPI
> or
> > use one of the many Python or Linux distributions.
> >
> > On the other hand, Python's standard library is piling up cruft,
> unnecessary
> > duplication of functionality, and dispensable features. This is
> undesirable
> > for several reasons.
> >
> > * Any additional module increases the maintenance cost for the Python
> core
> >   development team. The team has limited resources, reduced maintenance
> > cost
> >   frees development time for other improvements.
> > * Modules in the standard library are generally favored and seen as the
> >   de-facto solution for a problem. A majority of users only pick 3rd
> party
> >   modules to replace a stdlib module, when they have a compelling reason,
> > e.g.
> >   lxml instead of `xml`. The removal of an unmaintained stdlib module
> >   increases the chances of a community contributed module to become
> > widely
> >   used.
> > * A lean and mean standard library benefits platforms with limited
> resources
> >   like devices with just a few hundred kilobyte of storage (e.g. BBC
> >   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
> >   (e.g. pyodide) also benefit from reduced download size.
> >
> > The modules in the PEP have been selected for deprecation because their
> > removal is either least controversial or most beneficial. For example
> > least controversial are 30 years old multimedia formats like ``sunau``
> > audio format, which was used on SPARC and NeXT workstations in the late
> > 1980t

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Barry Warsaw]

On May 20, 2019, at 13:15, Christian Heimes  wrote:
> 
> here is the first version of my PEP 594 to deprecate and eventually remove 
> modules from the standard library. The PEP started last year with talk during 
> the Python Language Summit 2018, https://lwn.net/Articles/755229/.
> 
> The PEP can be confirmed in two stages. I'm not planning any code changes for 
> 3.8. Instead I only like to document a bunch of modules as deprecated. Active 
> deprecation is planned for 3.9 and removal for 3.10. The long deprecation 
> phase gives us 3 years to change our minds or handle edge cases, too.

Hi Christian,

First, I appreciate the work you’ve done on this, and the general sentiment.  
But I do feel like this is a partial solution to a problem the Python 
distribution has had for a very long time.  Maybe it’s a good step forward, but 
in a sense it is incomplete and only chips away at the problem.

We have two competing pressures, one to provide a rich standard library with 
lots of useful features that come right out of the box.  Let’s not 
underestimate the value that this has for our users, and the contribution such 
a stdlib has made to making Python as popular as it is.

But it’s also true that lots of the stdlib don’t get the love they need to stay 
relevant, and a curated path to keeping only the most useful and modern 
libraries.  I wonder how much the long development cycle and relatively big 
overhead for contributing to stdlib maintenance causes a big part of our 
headaches with the stdlib.  Current stdlib development processes also incur 
burden for alternative implementations.

We’ve had many ideas over the years, such as stripping the CPython repo stdlib 
to its bare minimum and providing some way of *distributing* a sumo tarball.  
But none have made it far enough to be adopted.  I don’t have any new bright 
ideas for how to make this work, but I think finding a holistic approach to 
these competing pressures is in the best long term interest of Python.

That all said, if PEP 594 can be viewed as part of the bigger picture, then 
maybe it’s a good idea.  Perhaps the approaches for maintaining such deprecated 
libraries can be used as an experiment for finding a more modern, useful, and 
vibrant approach to stdlib maintenance.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Jeff Kintscher]

+1

On 5/23/19 1:00 AM, Alex Walters wrote:

I've watched the entire thread and its taken me a few days to put a finger
on what bothers me about it.

In my opinion, this shouldn't be a pep describing the list of modules that
need to go as "dead batteries", but should be a process pep describing how
dead batteries should be removed, and the individual modules should be given
their own pep.  I think reactions to individual module peps will give a
better indication of if it's a used module or not.


-Original Message-
From: Python-Dev  On Behalf Of Christian Heimes
Sent: Monday, May 20, 2019 4:15 PM
To: Python Dev 
Subject: [Python-Dev] PEP 594: Removing dead batteries from the standard
library

Hi,

here is the first version of my PEP 594 to deprecate and eventually remove
modules from the standard library. The PEP started last year with talk

during

the Python Language Summit 2018, https://lwn.net/Articles/755229/.

The PEP can be confirmed in two stages. I'm not planning any code changes
for 3.8. Instead I only like to document a bunch of modules as deprecated.
Active deprecation is planned for 3.9 and removal for 3.10. The long
deprecation phase gives us 3 years to change our minds or handle edge
cases, too.

Regards,
Christian






PEP: 594
Title: Removing dead batteries from the standard library
Author: Christian Heimes 
Status: Active
Type: Process
Content-Type: text/x-rst
Created: 20-May-2019
Post-History:


Abstract


This PEP proposed a list of standard library modules to be removed from

the

standard library. The modules are mostly historic data formats and APIs

that

have been superseded a long time ago, e.g. Mac OS 9 and Commodore.


Rationale
=

Back in the early days of Python, the interpreter came with a large set of
useful modules. This was often refrained to as "batteries included"
philosophy and was one of the corner stones to Python's success story.
Users didn't have to figure out how to download and install separate
packages in order to write a simple web server or parse email.

Times have changed. The introduction of the cheese shop (PyPI),

setuptools,

and later pip, it became simple and straight forward to download and

install

packages. Nowadays Python has a rich and vibrant ecosystem of third party
packages. It's pretty much standard to either install packages from PyPI

or

use one of the many Python or Linux distributions.

On the other hand, Python's standard library is piling up cruft,

unnecessary

duplication of functionality, and dispensable features. This is

undesirable

for several reasons.

* Any additional module increases the maintenance cost for the Python core
   development team. The team has limited resources, reduced maintenance
cost
   frees development time for other improvements.
* Modules in the standard library are generally favored and seen as the
   de-facto solution for a problem. A majority of users only pick 3rd party
   modules to replace a stdlib module, when they have a compelling reason,
e.g.
   lxml instead of `xml`. The removal of an unmaintained stdlib module
   increases the chances of a community contributed module to become
widely
   used.
* A lean and mean standard library benefits platforms with limited

resources

   like devices with just a few hundred kilobyte of storage (e.g. BBC
   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
   (e.g. pyodide) also benefit from reduced download size.

The modules in the PEP have been selected for deprecation because their
removal is either least controversial or most beneficial. For example
least controversial are 30 years old multimedia formats like ``sunau``
audio format, which was used on SPARC and NeXT workstations in the late
1980ties. The ``crypt`` module has fundamental flaws that are better

solved

outside the standard library.

This PEP also designates some modules as not scheduled for removal. Some
modules have been deprecated for several releases or seem unnecessary at
first glance. However it is beneficial to keep the modules in the standard
library, mostly for environments where installing a package from PyPI is

not

an option. This can be cooperate environments or class rooms where
external
code is not permitted without legal approval.

* The usage of FTP is declining, but some files are still provided over
   the FTP protocol or hosters offer FTP to upload content. Therefore
   ``ftplib`` is going to stay.
* The ``optparse`` and ``getopt`` module are widely used. They are mature
   modules with very low maintenance overhead.
* According to David Beazley [5]_ the ``wave`` module is easy to teach to
   kids and can make crazy sounds. Making a computer generate crazy sounds
is
   powerful and highly motivating exercise for a 9yo aspiring developer.

It's

   a fun battery to keep.


Deprecation sc

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Alex Walters]

I've watched the entire thread and its taken me a few days to put a finger
on what bothers me about it.

In my opinion, this shouldn't be a pep describing the list of modules that
need to go as "dead batteries", but should be a process pep describing how
dead batteries should be removed, and the individual modules should be given
their own pep.  I think reactions to individual module peps will give a
better indication of if it's a used module or not.

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Christian Heimes
> Sent: Monday, May 20, 2019 4:15 PM
> To: Python Dev 
> Subject: [Python-Dev] PEP 594: Removing dead batteries from the standard
> library
> 
> Hi,
> 
> here is the first version of my PEP 594 to deprecate and eventually remove
> modules from the standard library. The PEP started last year with talk
during
> the Python Language Summit 2018, https://lwn.net/Articles/755229/.
> 
> The PEP can be confirmed in two stages. I'm not planning any code changes
> for 3.8. Instead I only like to document a bunch of modules as deprecated.
> Active deprecation is planned for 3.9 and removal for 3.10. The long
> deprecation phase gives us 3 years to change our minds or handle edge
> cases, too.
> 
> Regards,
> Christian
> 
> 
>

> PEP: 594
> Title: Removing dead batteries from the standard library
> Author: Christian Heimes 
> Status: Active
> Type: Process
> Content-Type: text/x-rst
> Created: 20-May-2019
> Post-History:
> 
> 
> Abstract
> 
> 
> This PEP proposed a list of standard library modules to be removed from
the
> standard library. The modules are mostly historic data formats and APIs
that
> have been superseded a long time ago, e.g. Mac OS 9 and Commodore.
> 
> 
> Rationale
> =
> 
> Back in the early days of Python, the interpreter came with a large set of
> useful modules. This was often refrained to as "batteries included"
> philosophy and was one of the corner stones to Python's success story.
> Users didn't have to figure out how to download and install separate
> packages in order to write a simple web server or parse email.
> 
> Times have changed. The introduction of the cheese shop (PyPI),
setuptools,
> and later pip, it became simple and straight forward to download and
install
> packages. Nowadays Python has a rich and vibrant ecosystem of third party
> packages. It's pretty much standard to either install packages from PyPI
or
> use one of the many Python or Linux distributions.
> 
> On the other hand, Python's standard library is piling up cruft,
unnecessary
> duplication of functionality, and dispensable features. This is
undesirable
> for several reasons.
> 
> * Any additional module increases the maintenance cost for the Python core
>   development team. The team has limited resources, reduced maintenance
> cost
>   frees development time for other improvements.
> * Modules in the standard library are generally favored and seen as the
>   de-facto solution for a problem. A majority of users only pick 3rd party
>   modules to replace a stdlib module, when they have a compelling reason,
> e.g.
>   lxml instead of `xml`. The removal of an unmaintained stdlib module
>   increases the chances of a community contributed module to become
> widely
>   used.
> * A lean and mean standard library benefits platforms with limited
resources
>   like devices with just a few hundred kilobyte of storage (e.g. BBC
>   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
>   (e.g. pyodide) also benefit from reduced download size.
> 
> The modules in the PEP have been selected for deprecation because their
> removal is either least controversial or most beneficial. For example
> least controversial are 30 years old multimedia formats like ``sunau``
> audio format, which was used on SPARC and NeXT workstations in the late
> 1980ties. The ``crypt`` module has fundamental flaws that are better
solved
> outside the standard library.
> 
> This PEP also designates some modules as not scheduled for removal. Some
> modules have been deprecated for several releases or seem unnecessary at
> first glance. However it is beneficial to keep the modules in the standard
> library, mostly for environments where installing a package from PyPI is
not
> an option. This can be cooperate environments or class rooms where
> external
> code is not permitted without legal approval.
> 
> * The usage of FTP is declining, but some files are still provided over
>   the FTP protocol or hosters offer FTP to upload content. Therefore
>   ``ftpl

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 23/05/2019 02.58, Steven D'Aprano wrote:
> On Wed, May 22, 2019 at 01:31:18PM +0200, Christian Heimes wrote:
>> On 22/05/2019 12.19, Steven D'Aprano wrote:
>>> I don't think this PEP should become a document about "Why you should 
>>> use PAM". I appreciate that from your perspective as a Red Hat security 
>>> guy, you want everyone to use best practices as you see them, but it 
>>> isn't Python's position to convince Linux distros or users to use PAM.
>>
> 
>> I think the PEP should make clear why spwd is bad and pining for The 
>> Fjords. The document should point users to correct alternatives. There 
>> is no correct and secure way to use the spwd module to verify user 
>> accounts. Any use of spwd for logins introduces critical security 
>> bugs.
> 
> When you use absolute language about security without considering 
> threat models, like "there is no ... way" and "Any use", you lose 
> credibility in my eyes.
> 
> I have a Linux desktop where I am the only user but not the only user 
> account. If I use spwd, what vulnerabilty am I introducing? That's not a 
> rhetorical question. If spwd does introduce a threat that isn't already 
> there, then please educate me, I genuinely want to know.

I can give you more details once I have resolved some CVEs. The problem
can result into full system compromise by a local or remote attacker
without any trace in the system audit and security logs. Depending on
other circumstances, the issue is CVSS HIGH to CRITICAL, perhaps up to
CVSS score 9.9.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/22/2019 4:09 AM, Christian Heimes wrote:

On 22/05/2019 01.11, Glenn Linderman wrote:

On 5/21/2019 2:00 PM, Nathaniel Smith wrote:

On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  wrote:

After maintaining my own version of http.server to fix or workaround some of 
its deficiencies for some years, I discovered bottle.py. It has far more 
capability, is far better documented, and is just as quick to deploy. While I 
haven't yet converted all past projects to use bottle.py, it will likely happen 
in time, unless something even simpler to use is discovered, although I can 
hardly imagine that happening.

bottle.py uses http.server for its local development mode (the one you
see in their quickstart example at the top of their README). Same with
flask, django, and probably a bunch of other frameworks. It's *very*
widely used.

-n


The source for bottle.py version 0.13-dev has an import for http.client, but not http.server. I 
hadn't tracked down every indirect dependency in the bottle.py source code, but it seems that if 
one uses the "default server" for bottle, that it is "wsgiref", imported from 
wsgiref.simple_server, and that in turn does import BaseHTTPRequestHandler and HTTPServer from 
http.server.

It is the higher-level code in http.server that has significant deficiencies that have caused me 
problems over the years... a "SimpleHTTPRequestHandler" that is so simple it doesn't do 
POST, PUT or PASTE, a "CGIHTTPRequestHandler" that only implements part of the CGI 
protocol, only CGI support in POST, no support for PUT or PASTE, and no support for https, and not 
much bug fix activity in those areas.

Maybe http.server should be split into the "basic parts" (used by bottle.py, and other 
frameworks), and the "higher-level parts", which could then be discarded by this PEP! At 
this point, though, I'd have to agree that the whole should not be discarded. Thanks for making me 
dig deeper.

The idea has merrit. However I feel its out of scope for the PEP. The 
http.server module and socketserver module are still widely used for debug and 
toy examples.

Could you please open a bug to track your proposal? We may pursue it in a 
couple of years from now.

issue 37018

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

On Wed, May 22, 2019 at 01:31:18PM +0200, Christian Heimes wrote:
> On 22/05/2019 12.19, Steven D'Aprano wrote:
> > I don't think this PEP should become a document about "Why you should 
> > use PAM". I appreciate that from your perspective as a Red Hat security 
> > guy, you want everyone to use best practices as you see them, but it 
> > isn't Python's position to convince Linux distros or users to use PAM.
> 

> I think the PEP should make clear why spwd is bad and pining for The 
> Fjords. The document should point users to correct alternatives. There 
> is no correct and secure way to use the spwd module to verify user 
> accounts. Any use of spwd for logins introduces critical security 
> bugs.

When you use absolute language about security without considering 
threat models, like "there is no ... way" and "Any use", you lose 
credibility in my eyes.

I have a Linux desktop where I am the only user but not the only user 
account. If I use spwd, what vulnerabilty am I introducing? That's not a 
rhetorical question. If spwd does introduce a threat that isn't already 
there, then please educate me, I genuinely want to know.

But if it doesn't, then you ought to tone down the absolutist language 
about "no way" and "any use" because that is factually untrue.


Later, you wrote:

> Steven, I feel like you are turning my own words and arguments against me.

Yes? Is that supposed to be a bad thing?

If you make an argument which is intended to support the PEP, but it 
actually undercuts the PEP, then you should fix the argument or 
acknowledge that the case for the PEP is not as strong as you hoped.

We do you no favours by ignoring weak or incoherent arguments.

PEPs are supposed to be fair, not partisan. Of course you have a 
position on this matter: you want to see spwd removed, and you want to 
put the best possible case for it. But to be a fair PEP, you also need 
to acknowledge counter-arguments and weaknesses in the argument, not 
just hope people won't notice them and then protest when they do.


> By the way, all relevant BSD, Linux, and Darwin (macOS) distributions 
> come with PAM support. Almost all use PAM by default. AFAIK only the 
> minimal Alpine container does not have PAM installed by default. This 
> is not Red Hat trying to evangelize the world. PAM is *the* industry 
> standards on Unix-like OS.

I appreciate that PAM is the standard, but Unix-like users are often an 
individualistic lot and "PAM is the default" doesn't mean "everyone uses 
PAM". See, for example, Arfrever's earlier post.

This PEP isn't about using PAM. It's about removing spwd and crypt. It's 
okay to say that PAM is the industry standard, you don't have to justify 
that in depth on the PEP. Nor do you need to justify why most people 
should be using PAM, but you ought to acknowledge that there may be some 
people who aren't.

For those who (rightly or wrongly) won't, can't or simply don't know how 
to use PAM, removing spwd is a functional regression. As PEP author, 
your job is to justify that functional regression and offer an 
alternative. We have a number of options:

- tell them to re-implement spwd (but that undercuts your argument 
  for removal);

- tell them to use a third party module (but which?);

- add a replacement module that Does The Right Thing;

- or at least a recipe somewhere (but once the spwd module is 
  removed, where will the recipe live?);

- anything else?


As PEP author, you get to choose which option the PEP suggests, but if 
you pick the one that undercuts the argument for removal, don't be 
surprised if people notice and consider the case for removal to be weak.



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Nathaniel Smith]

On Wed, May 22, 2019, 04:32 Christian Heimes  wrote:

> On 22/05/2019 12.19, Steven D'Aprano wrote:
> > I don't think this PEP should become a document about "Why you should
> > use PAM". I appreciate that from your perspective as a Red Hat security
> > guy, you want everyone to use best practices as you see them, but it
> > isn't Python's position to convince Linux distros or users to use PAM.
>
> I think the PEP should make clear why spwd is bad and pining for The
> Fjords. The document should point users to correct alternatives. There is
> no correct and secure way to use the spwd module to verify user accounts.
> Any use of spwd for logins introduces critical security bugs.
>
> By the way, all relevant BSD, Linux, and Darwin (macOS) distributions come
> with PAM support. Almost all use PAM by default. AFAIK only the minimal
> Alpine container does not have PAM installed by default. This is not Red
> Hat trying to evangelize the world. PAM is *the* industry standards on
> Unix-like OS.
>

The removal of spwd seems reasonable to me, and I don't think you need to
write 20 seperate PEPs for each module, but I do think you should split the
spwd/crypt modules off into their own PEP. The discussion about these
modules is qualitatively different than some of the others (the security
implications etc.), and trying to mix qualitatively different discussions
always makes people frustrated.

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

Between this discussion and Steve Dower's recently referenced blog post 
at 
https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/from 
which I quote below:


It’s been widely known for many years that Windows is the only 
mainstream operating system that does not include a Python interpreter 
out of the box.


and Victor's and Eric's work to make embedding Python easier, it makes 
me wonder when emacs will include a Python interpreter as an extension 
language.



On 5/22/2019 2:26 AM, Stephen J. Turnbull wrote:

Neil Schemenauer writes:

  > Here is an alternative, straw man, proposal.  Split the CPython repo
  > into two parts:
  >
  > - core Python: minimal possible stdlib
  > - everything else

I take issue with the characterization of "straw man," it's a
practical idea that turns out to be not so easy to implement.

We tried to do this with XEmacs, and found it just didn't work for us.
It requires a lot of effort to pare a distribution down to a minimal
core.  (In fact, we never got close.)  We gave up on that idea after
only a year, and this was an age and an environment that did not have
pip-like support for installation of 3rd-party packages (which you
deprecate as "not really a great solution"), and a lot of people were
still downloading at ISDN speeds or worse.  The demand for a minimal
core was correspondingly greater than today, I think.

We did bundle most applications such as MUAs and even C/C++
development support into sumo packages (one for general use, one for
packages that didn't function at all without the "MULE" multilingual
environment which was optional in XEmacs).  But we still ended up with
many packages staying in the core distribution, because other parts of
the core or the installation process used them, and as you say "If I
have a script that uses one of these modules and it gets removed, my
script breaks."  That annoyed core developers as much as anyone, and
they generally had priorities that did not emphasize minimalism (to
say the least).

  > When Python is released, provide installers for a Python that only
  > includes the "core" part and a second installer that includes
  > everything else.  I realize this is more work for the release team.
  > Hopefully with some scripting, it will not be too labour intensive.

Creating the distributions (core and sumo) was not particularly
labor-intensive for us because we mostly didn't distribute binaries.
A few moderately baroque scripts did the trick.  Automation has
advanced, but due to the amount of it it's still fragile, especially
when you're producing turn-key binary installers.

As I mentioned above, it wasn't the release process that was the
bottleneck, it was other factors that made it hard to reduce the
core.  We had an "xemacs-base" package that provided a set of "extra
tools" that some packages required, and libraries moved in and out of
it pretty much every release as some developer got tired of telling
people to install xemacs-base (back to core) or working around missing
features (move it to xemacs-base where it could be updated easily).

These are the same things that Python maintainers mention as pros and
cons of having things in stdlib or out on PyPI.  We learned there is a
sweet spot.  I know we didn't hit it most of the time, but it's very
clear that both extremes are hard, if not infeasible, to maintain.
(At the maximalist extreme, even GNU Emacs doesn't really try to
include everything written in Emacs Lisp any more.)

  > To help the people who need 100s or 1000s of extra PyPI packages,
  > we could develop a tool that creates a "sumo" Python installer,
  > grabbing packages from PyPI and building a installer package.

This could be done by any third party (it wouldn't even need to be a
public distribution, it could easily be an enterprise that open
sources an internal tool or a GSoC project).  And it could be hosted
on PyPI itself, since it's not the kind of thing that enterprises need
to install on 100,000 workstations.

  > To install that package, you would not need network access.

AIUI, it's not the network access that is usually the problem these
days, although it can be annoying if net access is flaky or if you
have to sneakernet USBs around.  Still, "sneakernet" is basically a
one-off cost.  The problem is that collections of random software from
untrusted sources are verboten in the enterprise.  (Ah, those Walnut
Creek CDs were wonderful!  I wonder what a modern AV tool would find
in them?!)  I suppose that in those situations people already have the
necessary tooling to roll the sumo or provide an intranet repository;
the problem is navigating the red tape.

  > That doesn't need to happen right away.  Also, maybe other Python
  > distributions can fill that need if core Python devs don't want to
  > build it.

I don't think this is about what core devs do or don't want to build.
I also don't feel a strong sense among the core developers that they
don't want to maint

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 22/05/2019 06.59, Stephen J. Turnbull wrote:
> Christian Heimes writes:
> 
>  > It's all open source. It's up to the Python community to adopt
>  > packages and provide them on PyPI.
>  > 
>  > Python core will not maintain and distribute the packages. I'll
>  > merely provide a repository with packages to help kick-starting the
>  > process.
> 
> This looks to me like an opening to a special class of supply chain
> attacks.  I realize that PyPI is not yet particularly robust to such
> attacks, and we have seen "similar name" attacks (malware uploaded
> under a name similar to a popular package).  ISTM that this approach
> to implementing the PEP will enable "identical name" attacks.  (By
> download count, stdlib packages are as popular as Python. :-)

I don't consider this an argument against my proposal, but an argument in favor 
of improving PyPI.


I propose a deal: If you get PEP 453 (ensurepip) revoked, ensurepip removed 
from the standard library, and the recommendation for the requests package on 
urllib.request replaced with a big, fat security warning, then I'll reconsider 
my proposal to recommend PyPI.


:)

My PEP acts in good faith. As long as CPython's stdlib ships pip and embraces 
PyPI, I don't see any reason to distrust PyPI. Yes, PyPI is not Fort Knox. In 
my humble opinion it's more than secure enough for my proposal.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 22/05/2019 12.19, Steven D'Aprano wrote:
> I don't think this PEP should become a document about "Why you should 
> use PAM". I appreciate that from your perspective as a Red Hat security 
> guy, you want everyone to use best practices as you see them, but it 
> isn't Python's position to convince Linux distros or users to use PAM.

I think the PEP should make clear why spwd is bad and pining for The Fjords. 
The document should point users to correct alternatives. There is no correct 
and secure way to use the spwd module to verify user accounts. Any use of spwd 
for logins introduces critical security bugs.

By the way, all relevant BSD, Linux, and Darwin (macOS) distributions come with 
PAM support. Almost all use PAM by default. AFAIK only the minimal Alpine 
container does not have PAM installed by default. This is not Red Hat trying to 
evangelize the world. PAM is *the* industry standards on Unix-like OS.

> To put it another way... I think that if you want to make the case for 
> PAM, put it on the web (a blog?) and link to it.
> 
> As far as the spwd module is concerned, on the one hand you're saying 
> "we should remove it because nobody should ever read from /etc/shadow", 
> and then on the other hand you're all "but go ahead and read /etc/shadow 
> if you like, it is perfectly trivial to do":
> 
>> By the way, the /etc/shadow shadow(5) format is trivial and can be 
>> parsed with a few lines of code. There is no need to use spwd.
> 
> so I think you're undercutting your own argument. If reading from 
> /etc/shadow is such a bad thing that we must remove it, why tell people 
> that they can parse it themselves?
> 
> Not that we could stop them, even if we wanted to.

Steven, I feel like you are turning my own words and arguments against me.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 22/05/2019 01.11, Glenn Linderman wrote:
> On 5/21/2019 2:00 PM, Nathaniel Smith wrote:
>> On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  
>> wrote:
>>> After maintaining my own version of http.server to fix or workaround some 
>>> of its deficiencies for some years, I discovered bottle.py. It has far more 
>>> capability, is far better documented, and is just as quick to deploy. While 
>>> I haven't yet converted all past projects to use bottle.py, it will likely 
>>> happen in time, unless something even simpler to use is discovered, 
>>> although I can hardly imagine that happening.
>> bottle.py uses http.server for its local development mode (the one you
>> see in their quickstart example at the top of their README). Same with
>> flask, django, and probably a bunch of other frameworks. It's *very*
>> widely used.
>>
>> -n
>>
> The source for bottle.py version 0.13-dev has an import for http.client, but 
> not http.server. I hadn't tracked down every indirect dependency in the 
> bottle.py source code, but it seems that if one uses the "default server" for 
> bottle, that it is "wsgiref", imported from wsgiref.simple_server, and that 
> in turn does import BaseHTTPRequestHandler and HTTPServer from http.server.
> 
> It is the higher-level code in http.server that has significant deficiencies 
> that have caused me problems over the years... a "SimpleHTTPRequestHandler" 
> that is so simple it doesn't do POST, PUT or PASTE, a "CGIHTTPRequestHandler" 
> that only implements part of the CGI protocol, only CGI support in POST, no 
> support for PUT or PASTE, and no support for https, and not much bug fix 
> activity in those areas.
> 
> Maybe http.server should be split into the "basic parts" (used by bottle.py, 
> and other frameworks), and the "higher-level parts", which could then be 
> discarded by this PEP! At this point, though, I'd have to agree that the 
> whole should not be discarded. Thanks for making me dig deeper.

The idea has merrit. However I feel its out of scope for the PEP. The 
http.server module and socketserver module are still widely used for debug and 
toy examples.

Could you please open a bug to track your proposal? We may pursue it in a 
couple of years from now.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

On Tue, May 21, 2019 at 01:59:56PM -0400, Terry Reedy wrote:
> On 5/21/2019 9:01 AM, Steven D'Aprano wrote:
> ...
> >Many Python users don't have the privilege of being able to install
> >arbitrary, unvetted packages from PyPI. They get to use only packages
> >from approved vendors, including the stdlib, what they write themselves,
> >and nothing else. Please don't dismiss this part of the Python community
> >just because they don't typically hang around in the same forums we do.
> ...
> 
> The problem with this argument, taken by itself, it that it would argue 
> for adding to the stdlib 100s or 1000s of modules or packages that would 
> be useful to many more people than the modules proposed to be dropped. 

No -- taken *by itself* it is only an argument against removing what 
already exists (or at least to be conservative in what we remove). That 
is all I'm saying.

It requires an *additional* argument that we add anything new, and I'm 
not making that argument here.

The additional argument is valid as a counter to "just put it on PyPI". 
The argument goes like this:

- if we agree that the aardvark module is useful and appropriate for 
  the std lib (not every useful module is, for many reasons!) we could 
  still decide not to add it;

- if we add it to the std lib, it will be available to 100% of users
  of the std lib;

- but if we say "put it on PyPI", it will only be available to (let's
  say) 85% of users.

But I'm not making that argument here because I'm not asking to add 
anything.



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

On Wed, May 22, 2019 at 01:59:59PM +0900, Stephen J. Turnbull wrote:

> This looks to me like an opening to a special class of supply chain
> attacks.
[...]

> One thing we *could* do that would require moderate effort would be to
> put them up on PyPI ourselves, and require that would-be maintainers
> be given a (light) vetting before handing over the keys.  (Maybe just
> require that they be subscribers to the Dead Parrot SIG? :-)

Because black hat hackers don't know how to subscribe to a SIG? *wink*

I'm just gonna leave this here... 

https://www.ietf.org/rfc/rfc3514.txt

Python is open source, anyone can fork any module from the std lib for
any purposes. We can't stop that. But your earlier point about supply 
chain attacks is very valid. Modules we shift out of the stdlib become 
more vulnerable to supply chain attacks, because more people will have 
to download them, giving more opportunity for typo-squatter attacks.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

On Wed, May 22, 2019 at 10:07:31AM +0200, Christian Heimes wrote:
> On 22/05/2019 06.20, Arfrever Frehtes Taifersar Arahesis wrote:

> > It is possible to have a modern Linux desktop system with PAM not
> > installed at all, and therefore not used.
[...]

Christian wrote:

> Thanks for bringing this up. I don't think we need to care about this care.
> 
> A PAM-free Linux system is an IMHO very special and exotic case. It's 
> certainly not a setup anybody should run on a server. 

I've heard of rare cases of people running Python on Linux desktops... 
*wink*


> There are a lot 
> of good reasons to use PAM. I'll update the BPO with reasons soonish.

I don't think this PEP should become a document about "Why you should 
use PAM". I appreciate that from your perspective as a Red Hat security 
guy, you want everyone to use best practices as you see them, but it 
isn't Python's position to convince Linux distros or users to use PAM.

To put it another way... I think that if you want to make the case for 
PAM, put it on the web (a blog?) and link to it.

As far as the spwd module is concerned, on the one hand you're saying 
"we should remove it because nobody should ever read from /etc/shadow", 
and then on the other hand you're all "but go ahead and read /etc/shadow 
if you like, it is perfectly trivial to do":

> By the way, the /etc/shadow shadow(5) format is trivial and can be 
> parsed with a few lines of code. There is no need to use spwd.

so I think you're undercutting your own argument. If reading from 
/etc/shadow is such a bad thing that we must remove it, why tell people 
that they can parse it themselves?

Not that we could stop them, even if we wanted to.

-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Inada Naoki]

2019年5月22日(水) 18:57 Steven D'Aprano :
>
> > All deprecated modules will also undergo a feature freeze. No additional
> > features should be added. Bug should still be fixed.
>
> I disagree with this. If people are requesting or contributing features
> to a module, that's a good sign that it *is* in use and isn't dead
> and we shouldn't remove it.
>

If people are requesting or contributing for a module but we can not
handle them for long time, I think it's sign we can not maintain the
module well.
It is not dead, but leaking battery.

Maybe, the module requires some domain specific knowledge and
it is very far from Python core development.
Domain expert will maintain it well than Python core developers.

So I don't think "it is still used" is not always equal to "Python
core developers
should maintain it" or "it should be shipped with Python".

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

On Tue, May 21, 2019 at 01:13:30PM -0400, Edwin Zimmerman wrote:
[...]
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Encoding: base64, Size: 8.4K --]


Wow! I haven't see one of those for *years* -- I haven't noticed one for 
about 15 years, and I thought it was an obsolete format that even 
Windows email stopped using.

I think the presence of that is somewhat ironic on a thread arguing that 
we ought to remove working modules for obsolete formats.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

Let me be clear: I do not oppose the removal of modules where necessary, 
but I do not like this PEP as it stands. But full credit to Christian 
for graciously accepting feedback; I also acknowledge that if this PEP 
is accepted, we still have at least two releases to change our minds 
about individual modules.

My largest issue with the PEP as a whole is that I don't like lumping a 
whole lot of unrelated modules together in one PEP. I believe it 
encourages a risky sense of "we ought to clean house and throw out all 
this junk!" rather than a measured consideration of pros and cons of 
each module individually.

I would rather see a serious of much smaller mini-PEPs, one per module. 
They need not be large: we can take as a given that we can and should 
remove "dead batteries" (but not merely unloved batteries in good 
working order, or slightly leaking batteries that just need a bit of 
care.) Each mini-PEP need only lay out the case for and against removal 
of one module (or perhaps a small number of closely related modules, not 
defend the principle in the abstract.

As I see it, there is a problem with this approach of a single omnibus 
PEP. A single PEP has reduced visibility for users of any specific 
module. This can easily lead to users of a module being under- 
represented in the discussion even if they are subscribed to Python-Dev.

Let's say Alice is a heavy user of the "aardvark" module. If Alice sees 
a PEP "Deprecate and remove aardvark", she is likely to speak up and air 
her objections.

But if Alice sees a single PEP "Remove dead batteries", she's likely to 
think "of course the aardvark library is not a dead battery, people like 
me us it all the time" and not realise she needs to make her objection 
known. Especially if the discussion thread is large. This thread has 
already reached 70 messages.

"To meaningfully contribute, I will need to read all 70 messages
in the Dead Battery thread, most of which are about modules I 
don't care about."

versus:

"I need to read 5 messages in the Remove Aardvark thread."

So I think that, relative to having one mini-PEP per module, a single 
PEP may be unintentionally biased towards the "Yes remove the lot" case 
and under-representing those who use some of these unloved batteries.

The problem is compounded by the prejudicial language used: nobody 
thinks of the modules they use as "dead batteries". How can they be dead 
when they work fine?

E.g. the binhex module *just works* -- if there are any bugs in it, I 
haven't come across them. binhex is not an obsolete file format for 
people like me who sometimes needs to read and write old Mac files.
It might be an old and unloved battery, but its not dead.


> On the other hand, Python's standard library is piling up cruft, unnecessary
> duplication of functionality, and dispensable features. This is undesirable
> for several reasons.

What you are calling cruft and dispensable, others call useful and 
indispensable. We've already seen at least three modules removed from 
the list of "cruft": colorsys, nttplib, fileinput. There may have been 
more by now -- I haven't caught up on all 70 posts yet.

I also think the PEP doesn't make the case for some of the things being 
asserted. For example:

> * Any additional module increases the maintenance cost for the Python core
>   development team. The team has limited resources, reduced maintenance cost
>   frees development time for other improvements.

This may often be the case, but not always.

If a module is stable, there's little maintenance cost to speak of. It 
works, there are (hopefully!) no known bugs, or at least no *serious* 
bugs, so why touch it? A module that isn't being changed could mean two 
things:

1. It is essentially complete. It does what it does, more or less 
correctly, and doesn't need new features or bug fixes.

2. It is suffering from bitrot.

I think the PEP fails to make the case for 2 over 1.


> * Modules in the standard library are generally favored and seen as the
>   de-facto solution for a problem. A majority of users only pick 3rd party
>   modules to replace a stdlib module, when they have a compelling reason, e.g.
>   lxml instead of `xml`.

Is this a problem that we need to fix?


>   The removal of an unmaintained stdlib module
>   increases the chances of a community contributed module to become widely
>   used.

I don't think the conclusion is justified. It could just as easily lead 
to a lack of any such library (the would-be users lack the ability or 
time to maintain one), or a plethora of alternatives:

"Should I use binhex 1.2, or MacBinHex 0.5, or BinnyMacHexFace 5.7, or 
xehnib 0.8?, or ... "



> * A lean and mean standard library benefits platforms with limited resources
>   like devices with just a few hundred kilobyte of storage (e.g. BBC
>   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
>   (e.g. pyodide) also benefit from reduced download size.

And I think this a

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Stephen J. Turnbull]

Neil Schemenauer writes:

 > Here is an alternative, straw man, proposal.  Split the CPython repo
 > into two parts:
 > 
 > - core Python: minimal possible stdlib
 > - everything else

I take issue with the characterization of "straw man," it's a
practical idea that turns out to be not so easy to implement.

We tried to do this with XEmacs, and found it just didn't work for us.
It requires a lot of effort to pare a distribution down to a minimal
core.  (In fact, we never got close.)  We gave up on that idea after
only a year, and this was an age and an environment that did not have
pip-like support for installation of 3rd-party packages (which you
deprecate as "not really a great solution"), and a lot of people were
still downloading at ISDN speeds or worse.  The demand for a minimal
core was correspondingly greater than today, I think.

We did bundle most applications such as MUAs and even C/C++
development support into sumo packages (one for general use, one for
packages that didn't function at all without the "MULE" multilingual
environment which was optional in XEmacs).  But we still ended up with
many packages staying in the core distribution, because other parts of
the core or the installation process used them, and as you say "If I
have a script that uses one of these modules and it gets removed, my
script breaks."  That annoyed core developers as much as anyone, and
they generally had priorities that did not emphasize minimalism (to
say the least).

 > When Python is released, provide installers for a Python that only
 > includes the "core" part and a second installer that includes
 > everything else.  I realize this is more work for the release team.
 > Hopefully with some scripting, it will not be too labour intensive.

Creating the distributions (core and sumo) was not particularly
labor-intensive for us because we mostly didn't distribute binaries.
A few moderately baroque scripts did the trick.  Automation has
advanced, but due to the amount of it it's still fragile, especially
when you're producing turn-key binary installers.

As I mentioned above, it wasn't the release process that was the
bottleneck, it was other factors that made it hard to reduce the
core.  We had an "xemacs-base" package that provided a set of "extra
tools" that some packages required, and libraries moved in and out of
it pretty much every release as some developer got tired of telling
people to install xemacs-base (back to core) or working around missing
features (move it to xemacs-base where it could be updated easily).

These are the same things that Python maintainers mention as pros and
cons of having things in stdlib or out on PyPI.  We learned there is a
sweet spot.  I know we didn't hit it most of the time, but it's very
clear that both extremes are hard, if not infeasible, to maintain.
(At the maximalist extreme, even GNU Emacs doesn't really try to
include everything written in Emacs Lisp any more.)

 > To help the people who need 100s or 1000s of extra PyPI packages,
 > we could develop a tool that creates a "sumo" Python installer,
 > grabbing packages from PyPI and building a installer package.

This could be done by any third party (it wouldn't even need to be a
public distribution, it could easily be an enterprise that open
sources an internal tool or a GSoC project).  And it could be hosted
on PyPI itself, since it's not the kind of thing that enterprises need
to install on 100,000 workstations.

 > To install that package, you would not need network access.

AIUI, it's not the network access that is usually the problem these
days, although it can be annoying if net access is flaky or if you
have to sneakernet USBs around.  Still, "sneakernet" is basically a
one-off cost.  The problem is that collections of random software from
untrusted sources are verboten in the enterprise.  (Ah, those Walnut
Creek CDs were wonderful!  I wonder what a modern AV tool would find
in them?!)  I suppose that in those situations people already have the
necessary tooling to roll the sumo or provide an intranet repository;
the problem is navigating the red tape.

 > That doesn't need to happen right away.  Also, maybe other Python
 > distributions can fill that need if core Python devs don't want to
 > build it.

I don't think this is about what core devs do or don't want to build.
I also don't feel a strong sense among the core developers that they
don't want to maintain a "batteries included" standard library.  Sure,
there's Christian's PEP, but there's also deliberate activity to
promote more participation at the core (ie, committer) level.  AIUI,
Christian's PEP is about transparency of what is already happening
(WONTFIX-ing or long periods of inactivity on issues with modules of
"low importance" to the active core developers), and a first step
toward making a managed process for the deprecation of modules we
can't afford to maintain given others we have committed to maintain.

Steve
___

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 22/05/2019 08.30, Giampaolo Rodola' wrote:
> 
> 
> On Tue, 21 May 2019 at 04:30, Antoine Pitrou  > wrote:
> 
> 
> NNTP is still quite used (often through GMane, but probably not only) so
> I'd question the removal of nntplib.
> 
> 
> I concur nntplib should be left alone. There are possibly even less used 
> network protocols such as telnet (tenetlib) which are not targeted by this 
> PEP but could have been by following the same logic. FTP is another one that 
> despite no longer popular, still has a constant user base (you’d be surprised 
> by the amount of traffic we got over FTP in the last company I worked for). 
> Overall, I think the bar for a module removal should be set very high, 
> especially for “standard” things such as these network protocols, that 
> despite being old are not likely to change. That means that also the 
> maintenance burden for python-dev will be low or close to none after all. 

* To be honest, I missed telnetlib. https://github.com/python/peps/pull/1073
* The maintenance burden for nntplib is actually high, because tests are flaky.
* nntplib has no maintainer and is missing features like COMPRESS extension.

> It seems to me also spwd/crypt modules fall into this category (all UNIX 
> platforms have the shadow password db, which is nice to have in python out of 
> the box). In that case the removal appears to be more justified by the 
> security implications than them being not widely used though, so I would use 
> more caution and treat them differently (eg. opt for doc warning + 
> investigate a possible replacement). Also note that spwd could be used to 
> parse /etc/passwd, despite I admit its primary use case is password checking. 
> Certain users may even not care about the additional security provided by PAM 
> (eg. internal devop scripts).

spwd and crypt are dead batteries, because their usefulness has been surpassed 
about two decades (!) ago. They are also very dangerous batteries because they 
leak hydrofluoric acid at scale. It's as least as bad as the acid + bathtub 
scene from the first season of Breaking Bad [1]. HF is nasty [2].

I can reveal more details in a week or two.

Christian

[1] https://breakingbad.fandom.com/wiki/Hydrofluoric_acid
[2] https://en.wikipedia.org/wiki/Hydrofluoric_acid#Health_and_safety

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 22/05/2019 06.20, Arfrever Frehtes Taifersar Arahesis wrote:
> 2019-05-21 00:06 UTC+02:00, Christian Heimes  wrote:
>> On 20/05/2019 23.27, Antoine Pitrou wrote:
>>> Removing the crypt module would remove support for system-standard
>>> password files.  I don't understand the rationale.
>>
>> Applications *must* not access system-standard password files directly. On
>> any sanely and securely configured systems, application cannot even access
>> system password files like /etc/shadow. Access restrictions and system
>> security policies will prevent read access. Also applications cannot assume
>> that users are present in any user file. They may come from LDAP, SSSD,
>> ActiveDirectory, or other sources.
>>
>> The correct way to interact with system users is to use the proper APIs,
>> that are NSS (name service switch) and PAM (pluggable authentication
>> modules). NSS looks up and enumerate users and groups. PAM performs password
>> validation and much, much, much more. The pwd and grp modules use the
>> correct APIs to interact with NSS. If you need to check or change passwords,
>> you must go through PAM.
> 
> It is possible to have a modern Linux desktop system with PAM not
> installed at all, and therefore not used.
> 
> Examples of packages in Gentoo Linux which have OPTIONAL dependency on PAM:
> shadow, sudo, openssh, libcap, systemd, util-linux, screen, cronie,
> polkit, cups, sddm, kscreenlocker, xscreensaver
> (So a KDE Plasma desktop environment and its direct and indirect
> dependencies can be installed without PAM.)
> 
> The suggested substitutes for spwd module, i.e. python-pam and
> simpleplam, look like they would not work on a PAM-free system.
 
Thanks for bringing this up. I don't think we need to care about this care.

A PAM-free Linux system is an IMHO very special and exotic case. It's certainly 
not a setup anybody should run on a server. There are a lot of good reasons to 
use PAM. I'll update the BPO with reasons soonish.

By the way, the /etc/shadow shadow(5) format is trivial and can be parsed with 
a few lines of code. There is no need to use spwd.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Neil Schemenauer]

On 2019-05-21, Brett Cannon wrote:
> On Tue., May 21, 2019, 13:07 Neil Schemenauer, 
> wrote:

> > Here is an alternative, straw man, proposal.  Split the CPython repo
> > into two parts:
> >
> > - core Python: minimal possible stdlib
> > - everything else
> 
> How to this lighten the burden of maintaining those modules which aren't in
> the core install? Are you suggesting modules in the core install get
> serious focus and the rest is more of a legacy, unsupported release for
> some time to give people an extended period to shift to the core install?
> Or do you have something else in mind?

It would give us the freedom to choose how we want to do it.  It
would give a lightweight Python install for the people who don't need
all the batteries, much lighter than what the PEP 594 strategy could
provide.

For CI, we can decide what should be tested.  Obviously the core
part is always tested.  Initially, we could continue testing all
parts of non-core.  Later, we could decide that certain parts of
non-core get tested less often (e.g. full nntplib tests only
nightly).  BTW, I think it would be great if automated nightly jobs
could also also run tests for PyPI modules like NumPy, requests,
Pandas, etc.

The installers could offer options as to which parts of the non-core
library to install.  Modules that no longer receive the same quality
of development and testing could get moved to a "deprecated"
section.  Users who want the best backwards compatibility would
install everything.  If we want to remove something from the
"deprecated" section, I think we should give a lot of notice.  A
couple of years is not enough.

Here is a sketch for a Linux-like package system:

python3-stdlib-base
all recommended stdlib packages (e.g. same as stdlib after
PEP 594 removals)

python3-stdlib-deprecated
packages suggested for removal by PEP 594

python3-stdlib-broken
packages that have bugs or are really not recommended to
be used.  I'm not sure if we need this but stuff like crypt
could go here.

python3-stdlib-all
depends on the above three packages


Ideally these packages don't contain the module files themselves.
Instead, they depend on individual packages for each module.  E.g.
python3-stdlib-deprecated would depend on python3-nntplib.  So,
someone could install python3-stdlib-base and python3-nntplib if
that's all they need.

I'm not familiar with the internals of 'pip' but I would guess we
could do a similar thing by creating meta PyPI packages that
correspond to these sets of packages.  So, someone could download
the small "core" Python installer and then run:

pip install stdlib-base

or something like that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Giampaolo Rodola']

On Tue, 21 May 2019 at 04:30, Antoine Pitrou  wrote:

>
> NNTP is still quite used (often through GMane, but probably not only) so
> I'd question the removal of nntplib.


I concur nntplib should be left alone. There are possibly even less used
network protocols such as telnet (tenetlib) which are not targeted by this
PEP but could have been by following the same logic. FTP is another one
that despite no longer popular, still has a constant user base (you’d be
surprised by the amount of traffic we got over FTP in the last company I
worked for). Overall, I think the bar for a module removal should be set
very high, especially for “standard” things such as these network
protocols, that despite being old are not likely to change. That means that
also the maintenance burden for python-dev will be low or close to none
after all.

It seems to me also spwd/crypt modules fall into this category (all UNIX
platforms have the shadow password db, which is nice to have in python out
of the box). In that case the removal appears to be more justified by the
security implications than them being not widely used though, so I would
use more caution and treat them differently (eg. opt for doc warning +
investigate a possible replacement). Also note that spwd could be used to
parse /etc/passwd, despite I admit its primary use case is password
checking. Certain users may even not care about the additional security
provided by PAM (eg. internal devop scripts).
-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Robert Collins]

This vector exists today for all new stdlib modules: once added, any
existing dependency could include that name to cater it to be imported on
prior python versions.

Rob

On Wed, 22 May 2019, 17:03 Stephen J. Turnbull, <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Christian Heimes writes:
>
>  > It's all open source. It's up to the Python community to adopt
>  > packages and provide them on PyPI.
>  >
>  > Python core will not maintain and distribute the packages. I'll
>  > merely provide a repository with packages to help kick-starting the
>  > process.
>
> This looks to me like an opening to a special class of supply chain
> attacks.  I realize that PyPI is not yet particularly robust to such
> attacks, and we have seen "similar name" attacks (malware uploaded
> under a name similar to a popular package).  ISTM that this approach
> to implementing the PEP will enable "identical name" attacks.  (By
> download count, stdlib packages are as popular as Python. :-)
>
> It now appears that there's been substantial pushback against removing
> packages that could be characterized as "obsolete and superseded but
> still in use", so this may not be a sufficient great risk to be worth
> addressing.  I guess this post is already a warning to those who are
> taking care of the "similar name" malware that this class of attacks
> will be opened up.
>
> One thing we *could* do that would require moderate effort would be to
> put them up on PyPI ourselves, and require that would-be maintainers
> be given a (light) vetting before handing over the keys.  (Maybe just
> require that they be subscribers to the Dead Parrot SIG? :-)
>
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/robertc%40robertcollins.net
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Random832]

On Tue, May 21, 2019, at 14:17, Christian Heimes wrote:
> thanks for bringing this topic up. Initially I considered http.server, 
> too. But as Guido put it, it's both used and useful for local testing 
> and quick hacks. I'm already facing opposition for modules that are 
> less controversial and useful than http.server, too. 

Could the high-level functionality of http.server (and perhaps socketserver as 
a whole) be rebuilt on top of asyncio, with serve_forever implicitly starting 
an asyncio loop with the server as the only task? Or are the paradigms too 
different?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Stephen J. Turnbull]

Christian Heimes writes:

 > It's all open source. It's up to the Python community to adopt
 > packages and provide them on PyPI.
 > 
 > Python core will not maintain and distribute the packages. I'll
 > merely provide a repository with packages to help kick-starting the
 > process.

This looks to me like an opening to a special class of supply chain
attacks.  I realize that PyPI is not yet particularly robust to such
attacks, and we have seen "similar name" attacks (malware uploaded
under a name similar to a popular package).  ISTM that this approach
to implementing the PEP will enable "identical name" attacks.  (By
download count, stdlib packages are as popular as Python. :-)

It now appears that there's been substantial pushback against removing
packages that could be characterized as "obsolete and superseded but
still in use", so this may not be a sufficient great risk to be worth
addressing.  I guess this post is already a warning to those who are
taking care of the "similar name" malware that this class of attacks
will be opened up.

One thing we *could* do that would require moderate effort would be to
put them up on PyPI ourselves, and require that would-be maintainers
be given a (light) vetting before handing over the keys.  (Maybe just
require that they be subscribers to the Dead Parrot SIG? :-)

Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Arfrever Frehtes Taifersar Arahesis]

2019-05-21 00:06 UTC+02:00, Christian Heimes  wrote:
> On 20/05/2019 23.27, Antoine Pitrou wrote:
>> Removing the crypt module would remove support for system-standard
>> password files.  I don't understand the rationale.
>
> Applications *must* not access system-standard password files directly. On
> any sanely and securely configured systems, application cannot even access
> system password files like /etc/shadow. Access restrictions and system
> security policies will prevent read access. Also applications cannot assume
> that users are present in any user file. They may come from LDAP, SSSD,
> ActiveDirectory, or other sources.
>
> The correct way to interact with system users is to use the proper APIs,
> that are NSS (name service switch) and PAM (pluggable authentication
> modules). NSS looks up and enumerate users and groups. PAM performs password
> validation and much, much, much more. The pwd and grp modules use the
> correct APIs to interact with NSS. If you need to check or change passwords,
> you must go through PAM.

It is possible to have a modern Linux desktop system with PAM not
installed at all, and therefore not used.

Examples of packages in Gentoo Linux which have OPTIONAL dependency on PAM:
shadow, sudo, openssh, libcap, systemd, util-linux, screen, cronie,
polkit, cups, sddm, kscreenlocker, xscreensaver
(So a KDE Plasma desktop environment and its direct and indirect
dependencies can be installed without PAM.)

The suggested substitutes for spwd module, i.e. python-pam and
simpleplam, look like they would not work on a PAM-free system.

--
Arfrever Frehtes Taifersar Arahesis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Brett Cannon]

On Tue., May 21, 2019, 13:07 Neil Schemenauer, 
wrote:

> On 2019-05-21, Terry Reedy wrote:
> > The problem with this argument, taken by itself, it that it would argue
> for
> > adding to the stdlib 100s or 1000s of modules or packages that would be
> > useful to many more people than the modules proposed to be dropped.
>
> I don't think it does.  We are not talking about 100s or 1000s of
> modules.  We are talking about modules which have been in Python's
> stdlib for years or decades.  If I have a script that uses one of
> these modules and it gets removed, my script breaks.
>
> Installing it from PyPI is not really a great solution.  We are
> going to be breaking working scripts just like if we add new
> language keywords, etc.  I think we need to be extremely careful
> with trying to maintain backwards compatibility, at least as far as
> we reasonably can.
>
> The problem I have with this PEP is that I think it both too
> aggressive and too conservative at the same time.  For almost all
> modules on the list, I'm sure there will be many people who are
> harmed by its removal.  OTOH, having to maintain all of the modules
> in the stdlib is a heavy burden.  Also, new users can be lured into
> using a module that is not really the best solution anymore.
>
> Here is an alternative, straw man, proposal.  Split the CPython repo
> into two parts:
>
> - core Python: minimal possible stdlib
> - everything else
>
> When Python is released, provide installers for a Python that only
> includes the "core" part and a second installer that includes
> everything else.  I realize this is more work for the release team.
> Hopefully with some scripting, it will not be too labour intensive.
>
> The core Python installer should become the recommended installer.
> People who need backwards compability with older versions of Python
> can download the big installer package.
>

How to this lighten the burden of maintaining those modules which aren't in
the core install? Are you suggesting modules in the core install get
serious focus and the rest is more of a legacy, unsupported release for
some time to give people an extended period to shift to the core install?
Or do you have something else in mind?

-Brett



> To help the people who need 100s or 1000s of extra PyPI packages, we
> could develop a tool that creates a "sumo" Python installer,
> grabbing packages from PyPI and building a installer package.  To
> install that package, you would not need network access.  That
> doesn't need to happen right away.  Also, maybe other Python
> distributions can fill that need if core Python devs don't want to
> build it.
>
> Regards,
>
>   Neil
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 2:00 PM, Nathaniel Smith wrote:

On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  wrote:

After maintaining my own version of http.server to fix or workaround some of 
its deficiencies for some years, I discovered bottle.py. It has far more 
capability, is far better documented, and is just as quick to deploy. While I 
haven't yet converted all past projects to use bottle.py, it will likely happen 
in time, unless something even simpler to use is discovered, although I can 
hardly imagine that happening.

bottle.py uses http.server for its local development mode (the one you
see in their quickstart example at the top of their README). Same with
flask, django, and probably a bunch of other frameworks. It's *very*
widely used.

-n

The source for bottle.py version 0.13-dev has an import for http.client, 
but not http.server. I hadn't tracked down every indirect dependency in 
the bottle.py source code, but it seems that if one uses the "default 
server" for bottle, that it is "wsgiref", imported from 
wsgiref.simple_server, and that in turn does import 
BaseHTTPRequestHandler and HTTPServer from http.server.


It is the higher-level code in http.server that has significant 
deficiencies that have caused me problems over the years... a 
"SimpleHTTPRequestHandler" that is so simple it doesn't do POST, PUT or 
PASTE, a "CGIHTTPRequestHandler" that only implements part of the CGI 
protocol, only CGI support in POST, no support for PUT or PASTE, and no 
support for https, and not much bug fix activity in those areas.


Maybe http.server should be split into the "basic parts" (used by 
bottle.py, and other frameworks), and the "higher-level parts", which 
could then be discarded by this PEP! At this point, though, I'd have to 
agree that the whole should not be discarded. Thanks for making me dig 
deeper.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Victor Stinner]

Le mar. 21 mai 2019 à 23:05, Nathaniel Smith  a écrit :
> If the tests don't work and the module is unmaintained, then maybe we
> should disable the tests and put a prominent notice in the docs saying
> that it's unmaintained and any use is at your own risk. It's not a
> pleasant thing to do, but if that's the reality of the situation then
> it's probably better to be up front about it than to stick our fingers
> in our ears and waste folks time with spurious test failures. And
> perhaps someone would actually step up to maintain it.

Well, I already did that in the past.

Lib/test/test_ftplib.py:@skipUnless(False, "FIXME: bpo-32706")
=> skipped since 2018-01-29

Lib/test/test_nntplib.py:@unittest.skipIf(True, "FIXME: see bpo-32128")
=> skipped since 2017-11-25.

Oh, I even closed https://bugs.python.org/issue32128. Well, I'm kind
of tired of waiting for fixes in nntplib, so I decided to immediately
close the issue. I'm quite sure that nobody will fix it "soon" :-)
(hint: there FIXME is still there)

I'm not sure about documenting the module as "unmaintained".

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Nathaniel Smith]

On Tue, May 21, 2019 at 4:25 AM Victor Stinner  wrote:
>
> Le mar. 21 mai 2019 à 13:18, André Malo  a écrit :
> > There's software in production using both. (It doesn't mean it's on pypi or
> > even free software).
> >
> > What would be the maintenance burden of those modules anyway? (at least for
> > nntp, I guess it's not gonna change).
>
> The maintenance burden is real even if it's not visible. For example,
> test_nntplib is causing frequently issues on our CI:
>
> https://bugs.python.org/issue19756
> https://bugs.python.org/issue19613
> https://bugs.python.org/issue31850
>
> It's failing frequently since 2013, and nobody managed to come with a
> fix.. in 6 years.

If the tests don't work and the module is unmaintained, then maybe we
should disable the tests and put a prominent notice in the docs saying
that it's unmaintained and any use is at your own risk. It's not a
pleasant thing to do, but if that's the reality of the situation then
it's probably better to be up front about it than to stick our fingers
in our ears and waste folks time with spurious test failures. And
perhaps someone would actually step up to maintain it.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Nathaniel Smith]

On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  wrote:
> After maintaining my own version of http.server to fix or workaround some of 
> its deficiencies for some years, I discovered bottle.py. It has far more 
> capability, is far better documented, and is just as quick to deploy. While I 
> haven't yet converted all past projects to use bottle.py, it will likely 
> happen in time, unless something even simpler to use is discovered, although 
> I can hardly imagine that happening.

bottle.py uses http.server for its local development mode (the one you
see in their quickstart example at the top of their README). Same with
flask, django, and probably a bunch of other frameworks. It's *very*
widely used.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 11:15 AM, Christian Heimes wrote:

On 21/05/2019 18.29, Glenn Linderman wrote:

On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.

But http.server could be on the remove list too... it gets mighty little 
support, has very little functionality, and implements a CGI interface 
(although that also has very little functionality), and you have the CGI tools 
on the remove list, rendering the CGI interface implemented by http.server less 
easily usable.

Further, it doesn't directly support https:, and browsers are removing/reducing 
support for http:.

I can't speak to xmlrpc or logging configuration.

Hi,

thanks for bringing this topic up. Initially I considered http.server, too. But 
as Guido put it, it's both used and useful for local testing and quick hacks. 
I'm already facing opposition for modules that are less controversial and 
useful than http.server, too.


Indeed. That's why they were created in the first place (http.server, 
and those other modules). But these days there are better alternatives. 
That's the point of the PEP, if I understand correctly... get rid of the 
junkware, point to the betterware, and avoid the need to learn the 
betterware later, because you wasted time and effort using the junkware 
at first, because it was "included".



I have two remarks:

1) The http.server does not act as a CGI server by default. In CGI server mode, 
it does not depend on the cgi module.


CGIHTTPRequestHandler is part of the package. While it does not depend 
on the cgi module, its existence encourages the users to search for, 
find, and use the cgi and cgitb modules when using 
CGIHTTPRequestHandler. I certainly did in past projects, prior to 
finding bottle.py.



2) The lack of HTTPS support is not a major problem for connections on localhost. There 
is a RFC draft to consider any connection to "localhost" as secure, 
https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-06


Which certainly didn't exist when Chrome (noting the author's 
organization) and other browsers first started implementing feature 
restrictions for http, and while I haven't tested this since the RFC you 
mention was published, it was certainly a difficulty I faced when 
attempting to use features with security restrictions for local testing 
a couple years ago.



Christian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Neil Schemenauer]

On 2019-05-21, Terry Reedy wrote:
> The problem with this argument, taken by itself, it that it would argue for
> adding to the stdlib 100s or 1000s of modules or packages that would be
> useful to many more people than the modules proposed to be dropped.

I don't think it does.  We are not talking about 100s or 1000s of
modules.  We are talking about modules which have been in Python's
stdlib for years or decades.  If I have a script that uses one of
these modules and it gets removed, my script breaks.

Installing it from PyPI is not really a great solution.  We are
going to be breaking working scripts just like if we add new
language keywords, etc.  I think we need to be extremely careful
with trying to maintain backwards compatibility, at least as far as
we reasonably can.

The problem I have with this PEP is that I think it both too
aggressive and too conservative at the same time.  For almost all
modules on the list, I'm sure there will be many people who are
harmed by its removal.  OTOH, having to maintain all of the modules
in the stdlib is a heavy burden.  Also, new users can be lured into
using a module that is not really the best solution anymore.

Here is an alternative, straw man, proposal.  Split the CPython repo
into two parts:

- core Python: minimal possible stdlib
- everything else

When Python is released, provide installers for a Python that only
includes the "core" part and a second installer that includes
everything else.  I realize this is more work for the release team.
Hopefully with some scripting, it will not be too labour intensive.

The core Python installer should become the recommended installer.
People who need backwards compability with older versions of Python
can download the big installer package.

To help the people who need 100s or 1000s of extra PyPI packages, we
could develop a tool that creates a "sumo" Python installer,
grabbing packages from PyPI and building a installer package.  To
install that package, you would not need network access.  That
doesn't need to happen right away.  Also, maybe other Python
distributions can fill that need if core Python devs don't want to
build it.

Regards,

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 20.35, Guido van Rossum wrote:
> On Tue, May 21, 2019 at 11:17 AM Christian Heimes  > wrote:
> 
> I'm already facing opposition for modules that are less controversial and 
> useful than http.server, too.
> 
> 
> There's another argument here. This is an "omnibus" PEP, meaning it proposes 
> many unrelated changes. In order to get a consensus to pass the PEP, it may 
> be necessary to compromise. IOW I would recommend removing modules from the 
> PEP that bring up strong opposition, *even* if you yourself feel strongly 
> that those modules should be removed.
> 
> The vast majority of modules on the list hasn't elicited any kind of feedback 
> at all -- those are clearly safe to remove (many people are probably, like 
> myself, hard-pressed to remember what they do). I'm not saying drop anything 
> from the list that elicits any pushback, but once the debate has gone back 
> and forth twice, it may be a hint that a module still has fans. Threatening 
> to open a CVE is more likely to reduce support for the PEP than it is to 
> convince anyone.

It was not a threat, but an illustration how critical the flaw with spwd + 
crypt is. The approach performs only authentication and completely bypasses any 
authorization. It does not take any login restrictions into account like 
account enabled flag, host/service based access control, IP restriction, 
credential strength, and so on. I would give the issue a CVSS rating between 
8.3 (high) to 9.6 (critical), perhaps 
CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:L

By the way, Giampaolo and I have known each other for many years. I know that 
he'll address the issue and file a CVE himself.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Guido van Rossum]

On Tue, May 21, 2019 at 11:17 AM Christian Heimes 
wrote:

> I'm already facing opposition for modules that are less controversial and
> useful than http.server, too.


There's another argument here. This is an "omnibus" PEP, meaning it
proposes many unrelated changes. In order to get a consensus to pass the
PEP, it may be necessary to compromise. IOW I would recommend removing
modules from the PEP that bring up strong opposition, *even* if you
yourself feel strongly that those modules should be removed.

The vast majority of modules on the list hasn't elicited any kind of
feedback at all -- those are clearly safe to remove (many people are
probably, like myself, hard-pressed to remember what they do). I'm not
saying drop anything from the list that elicits any pushback, but once the
debate has gone back and forth twice, it may be a hint that a module still
has fans. Threatening to open a CVE is more likely to reduce support for
the PEP than it is to convince anyone.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 16.46, Guido van Rossum wrote:
> +1. Let's keep colorsys then.

I let colorsys off the hock, https://github.com/python/peps/pull/1070

Thanks for your feedback, Walter and Petr!

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Antoine Pitrou]

On Tue, 21 May 2019 13:59:56 -0400
Terry Reedy  wrote:
> On 5/21/2019 9:01 AM, Steven D'Aprano wrote:
> ...
> > Many Python users don't have the privilege of being able to install
> > arbitrary, unvetted packages from PyPI. They get to use only packages
> > from approved vendors, including the stdlib, what they write themselves,
> > and nothing else. Please don't dismiss this part of the Python community
> > just because they don't typically hang around in the same forums we do.  
> ...
> 
> The problem with this argument, taken by itself, it that it would argue 
> for adding to the stdlib 100s or 1000s of modules or packages that would 
> be useful to many more people than the modules proposed to be dropped. 
> That would require multiplying the number of core developers by perhaps 
> 10.  We have thus far decided to not do this.
> 
> We have instead left sumo distributions with vetted (?) additions to 
> other groups.  But as it is, the stdlib is, as far as I last knew, 
> richer than, for instance, the standard C stdlib.
> 
> Given that presence in the stdlib is restricted, what should be 
> included.  Is mere inertia a sufficient reason?  I don't think it should be.

One fundamental thing the stdlib provides is stability.  If you use an
API from the stdlib today, chances are it'll still work in 10 years,
even on the most recent Python version. Good luck relying on that with
most third-party packages.

But if stdlib modules can disappear when we decide they don't get
enough usage, the stability promise disappears as well.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 18.29, Glenn Linderman wrote:
> On 5/20/2019 2:20 PM, Christian Heimes wrote:
>> On 20/05/2019 23.12, Andrew Svetlov wrote:
>>> socketserver.py is also questionable
>> I briefly though about the module, but didn't consider it for removal. The 
>> http.server, xmlrpc.server, and logging configuration server are implemented 
>> on top of the socketserver. I don't want to remove the socketserver module 
>> without a suitable replacement for http.server in the standard library.
> 
> But http.server could be on the remove list too... it gets mighty little 
> support, has very little functionality, and implements a CGI interface 
> (although that also has very little functionality), and you have the CGI 
> tools on the remove list, rendering the CGI interface implemented by 
> http.server less easily usable.
> 
> Further, it doesn't directly support https:, and browsers are 
> removing/reducing support for http:.
> 
> I can't speak to xmlrpc or logging configuration.

Hi,

thanks for bringing this topic up. Initially I considered http.server, too. But 
as Guido put it, it's both used and useful for local testing and quick hacks. 
I'm already facing opposition for modules that are less controversial and 
useful than http.server, too. 

I have two remarks:

1) The http.server does not act as a CGI server by default. In CGI server mode, 
it does not depend on the cgi module.
2) The lack of HTTPS support is not a major problem for connections on 
localhost. There is a RFC draft to consider any connection to "localhost" as 
secure, https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-06

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Brett Cannon]

On Tue., May 21, 2019, 04:25 Victor Stinner,  wrote:

> Le mar. 21 mai 2019 à 13:18, André Malo  a écrit :
> > There's software in production using both. (It doesn't mean it's on pypi
> or
> > even free software).
> >
> > What would be the maintenance burden of those modules anyway? (at least
> for
> > nntp, I guess it's not gonna change).
>
> The maintenance burden is real even if it's not visible. For example,
> test_nntplib is causing frequently issues on our CI:
>
> https://bugs.python.org/issue19756
> https://bugs.python.org/issue19613
> https://bugs.python.org/issue31850
>
> It's failing frequently since 2013, and nobody managed to come with a
> fix.. in 6 years.
>
> There are 11 open issues with "nntp" in their title (8 with exactly
> "nntplib" in their title).
>

I would be curious to know how many feature PRs there might exist as that
also is a burden for any of these modules beyond bugs and any general
upkeep we do (e.g. someone on Twitter argued that because someone merged a
new feature that means that a module should stay, but to me that just says
one person cared enough to write a PR and one core dev viewed it as
harmless to merge which isn't necessarily representative of usage).

-Brett

P.S. I have an idea about under-supported modules in the stdlib and feature
requests, but it's too early to share (nor the appropriate thread of
discussion).


> test_nntplib uses the public server news.trigofacile.com which is
> operated by Julien ÉLIE. Two years ago, Julien asked me if there is
> any plan to support the NNTP "COMPRESS" command.
>
> Victor
>
>
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Terry Reedy]

On 5/21/2019 9:01 AM, Steven D'Aprano wrote:
...

Many Python users don't have the privilege of being able to install
arbitrary, unvetted packages from PyPI. They get to use only packages
from approved vendors, including the stdlib, what they write themselves,
and nothing else. Please don't dismiss this part of the Python community
just because they don't typically hang around in the same forums we do.

...

The problem with this argument, taken by itself, it that it would argue 
for adding to the stdlib 100s or 1000s of modules or packages that would 
be useful to many more people than the modules proposed to be dropped. 
That would require multiplying the number of core developers by perhaps 
10.  We have thus far decided to not do this.


We have instead left sumo distributions with vetted (?) additions to 
other groups.  But as it is, the stdlib is, as far as I last knew, 
richer than, for instance, the standard C stdlib.


Given that presence in the stdlib is restricted, what should be 
included.  Is mere inertia a sufficient reason?  I don't think it should be.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Oleg Broytman]

``http.server`` is used in ``pydoc``:

https://github.com/python/cpython/blob/ccb7ca728e09b307f9e9fd36ec40353137e68a3b/Lib/pydoc.py#L2236

On Tue, May 21, 2019 at 10:12:50AM -0700, Guido van Rossum  
wrote:
> I still like http.server for quick temporary hacks where I want to be able
> to point a browser at some code I wrote 5 minutes ago and that I plan to
> discard in an hour. Usually it's running at localhost:8000. Remembering how
> to use Django, flask or Tornado seems overkill for that purpose.
> 
> On Tue, May 21, 2019 at 9:39 AM Glenn Linderman 
> wrote:
> 
> > On 5/20/2019 2:20 PM, Christian Heimes wrote:
> >
> > On 20/05/2019 23.12, Andrew Svetlov wrote:
> >
> > socketserver.py is also questionable
> >
> > I briefly though about the module, but didn't consider it for removal. The 
> > http.server, xmlrpc.server, and logging configuration server are 
> > implemented on top of the socketserver. I don't want to remove the 
> > socketserver module without a suitable replacement for http.server in the 
> > standard library.
> >
> >
> > But http.server could be on the remove list too... it gets mighty little
> > support, has very little functionality, and implements a CGI interface
> > (although that also has very little functionality), and you have the CGI
> > tools on the remove list, rendering the CGI interface implemented by
> > http.server less easily usable.
> >
> > Further, it doesn't directly support https:, and browsers are
> > removing/reducing support for http:.
> >
> > I can't speak to xmlrpc or logging configuration.
> -- 
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him/his **(why is my pronoun here?)*
> 

Oleg.
-- 
Oleg Broytmanhttps://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/21/2019 10:12 AM, Guido van Rossum wrote:
I still like http.server for quick temporary hacks where I want to be 
able to point a browser at some code I wrote 5 minutes ago and that I 
plan to discard in an hour. Usually it's running at localhost:8000. 
Remembering how to use Django, flask or Tornado seems overkill for 
that purpose.


After maintaining my own version of http.server to fix or workaround 
some of its deficiencies for some years, I discovered bottle.py. It has 
far more capability, is far better documented, and is just as quick to 
deploy. While I haven't yet converted all past projects to use 
bottle.py, it will likely happen in time, unless something even simpler 
to use is discovered, although I can hardly imagine that happening.


I agree that Django, flask, and Tornado have bigger learning curves, due 
to having their own philosophies on how things should be done... I 
looked at each on my way to finding bottle.py, and in each case decided 
their philosophies, while probably useful to people that like or need 
the capabilities offered, were excessive to learn if the capabilities 
were not needed.


Providing http.server as an included battery is a disservice to novice 
users, because they might try to use it for something real.


On Tue, May 21, 2019 at 9:39 AM Glenn Linderman > wrote:


On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.


But http.server could be on the remove list too... it gets mighty
little support, has very little functionality, and implements a
CGI interface (although that also has very little functionality),
and you have the CGI tools on the remove list, rendering the CGI
interface implemented by http.server less easily usable.

Further, it doesn't directly support https:, and browsers are
removing/reducing support for http:.

I can't speak to xmlrpc or logging configuration.




___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him/his //(why is my pronoun here?)/ 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Edwin Zimmerman]

On Tuesday, May 21, 2019 12:30 PM Glenn Linderman wrote

On 5/20/2019 2:20 PM, Christian Heimes wrote:
On 20/05/2019 23.12, Andrew Svetlov wrote:
socketserver.py is also questionable
I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration
server are implemented on top of the socketserver. I don't want to remove the 
socketserver module without a suitable replacement for
http.server in the standard library.

But http.server could be on the remove list too... it gets mighty little 
support, has very little functionality, and implements a
CGI interface (although that also has very little functionality), and you have 
the CGI tools on the remove list, rendering the CGI
interface implemented by http.server less easily usable.

Further, it doesn't directly support https:, and browsers are removing/reducing 
support for http:.
All the same, I’d hate to see http.server leave.  It’s doubtful that anyone 
uses the thing for a production server, but I use it
all the time for low-volume development stuff on Windows.
<>___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Guido van Rossum]

I still like http.server for quick temporary hacks where I want to be able
to point a browser at some code I wrote 5 minutes ago and that I plan to
discard in an hour. Usually it's running at localhost:8000. Remembering how
to use Django, flask or Tornado seems overkill for that purpose.

On Tue, May 21, 2019 at 9:39 AM Glenn Linderman 
wrote:

> On 5/20/2019 2:20 PM, Christian Heimes wrote:
>
> On 20/05/2019 23.12, Andrew Svetlov wrote:
>
> socketserver.py is also questionable
>
> I briefly though about the module, but didn't consider it for removal. The 
> http.server, xmlrpc.server, and logging configuration server are implemented 
> on top of the socketserver. I don't want to remove the socketserver module 
> without a suitable replacement for http.server in the standard library.
>
>
> But http.server could be on the remove list too... it gets mighty little
> support, has very little functionality, and implements a CGI interface
> (although that also has very little functionality), and you have the CGI
> tools on the remove list, rendering the CGI interface implemented by
> http.server less easily usable.
>
> Further, it doesn't directly support https:, and browsers are
> removing/reducing support for http:.
>
> I can't speak to xmlrpc or logging configuration.
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Glenn Linderman]

On 5/20/2019 2:20 PM, Christian Heimes wrote:

On 20/05/2019 23.12, Andrew Svetlov wrote:

socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.


But http.server could be on the remove list too... it gets mighty little 
support, has very little functionality, and implements a CGI interface 
(although that also has very little functionality), and you have the CGI 
tools on the remove list, rendering the CGI interface implemented by 
http.server less easily usable.


Further, it doesn't directly support https:, and browsers are 
removing/reducing support for http:.


I can't speak to xmlrpc or logging configuration.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steve Holden]

Good point!

On Tue, May 21, 2019 at 2:01 PM Paul Moore  wrote:

> On Tue, 21 May 2019 at 13:50, Steve Holden  wrote:
> >
> > That seems entirely reasonable. I wonder if the larger community could
> somehow form an organization (the Dead Parrot SIG?) that would at least
> curate and monitor efforts to ensure their continued utility?
>
> I have no idea whether there is enough community support to make this
> work, but regardless, I strongly support a SIG by that name. It may
> actually be even *more* amusing if it doesn't have enough members to
> make it viable :-)
>
> Paul
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Guido van Rossum]

+1. Let's keep colorsys then.

On Tue, May 21, 2019 at 7:41 AM Petr Viktorin  wrote:

> On 5/21/19 12:06 PM, Christian Heimes wrote:
> > On 21/05/2019 11.49, Nathaniel Smith wrote:
> >> On Tue, May 21, 2019 at 2:40 AM Walter Dörwald 
> wrote:
> >>>
> >>> On 20 May 2019, at 22:15, Christian Heimes wrote:
> >>>
>  Hi,
> 
>  here is the first version of my PEP 594 to deprecate and eventually
>  remove modules from the standard library. The PEP started last year
>  with talk during the Python Language Summit 2018,
>  https://lwn.net/Articles/755229/.
> 
>  [...]
> 
>  colorsys
>  
> 
>  The `colorsys `_
>  module
>  defines color conversion functions between RGB, YIQ, HSL, and HSV
>  coordinate
>  systems. The Pillow library provides much faster conversation between
>  color systems.
> 
>  Module type
> pure Python
>  Deprecated in
> 3.8
>  To be removed in
> 3.10
>  Substitute
> `Pillow `_,
> `colorspacious `_
> >>>
> >>> I'm using colorsys constantly as the basis for a tool that converts CSS
> >>> colors between different coordinate systems. I don't see how that could
> >>> be done via Pillow (which AFAICT only converts complete images).
> >>> RGB<->HSV<->HLS conversion seems to be not available (or not obvious)
> in
> >>> colorspacious.
> >>
> >> Correct, colorspacious doesn't support HSV or HLS. I suppose it would
> >> be trivial enough to add...
> >>
> >> The 'colour' package has them (along with everything else you can
> >> dream of): https://colour.readthedocs.io/en/latest/colour.models.html
> >
> > Nice catch, I just added
> https://python-colormath.readthedocs.io/en/latest/ to my update PR. I'll
> add colour to the list, too.
> >
> > (It didn't pop up on my radar because I wasn't checking for British
> spelling)
> >
> > Christian
>
>
> Please, don't remove colorsys. HSL and HSV aren't outdated, and AFAIK,
> the module is not a maintenance burden.
>
> The packages on PyPI offer more advanced models. Perhaps they could be
> linked in the documentation, if we want to endorse them. But just as
> we're not removing `array` in favor of `numpy`, we shouldn't remove
> colorsys either. Colorsys is not dead, it's just small.
>
> I assume one of the reasons colorspacious doesn't support HSV or HLS is
> that HSV/HLS tends to be used for quite different purposes than the more
> fancy color spaces. You'd use HSL/HSV for a simple color picker or a
> saturation shift on a web page, GUI or game (where you don't have
> *exact* colors anyway); things like sRGB or CIELab are for color
> management, photos, printing, etc.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Guido van Rossum]

Quick note on the pip discussion: if the likely remaining users of a module
slated for deprecation are professionals maintaining some legacy code, pip
is a fine solution. OTOH if the likely users are beginners, maybe pip is
not great.

In general I think it's fine to err on the side of caution -- as someone
already said, we're only removing *dead* batteries, not *leaky* ones. If
there's too much pushback for a particular module, maybe it's not dead yet.
We can make another list in a few years.

On Tue, May 21, 2019 at 6:28 AM Paul Moore  wrote:

> On Tue, 21 May 2019 at 14:03, Steven D'Aprano  wrote:
>
> > I know that saying anything against pip and virtual environments is
> > heresy, but honestly, "just install it from PyPI" is not friendly to
> > beginners or those who just want something that works without a load of
> > extra complexity.
>
> Speaking as a pip developer, I 100% support this comment. For a major
> segment of Python's user base, if something isn't in the stdlib (or
> their preferred distribution - Anaconda or a Linux distro, for
> example) then it's a significant step upward in complexity.
>
> Having said this, I'm not as sure I agree with the rest of Steven's
> posting. I think that in general, this PEP is sensible and strikes a
> good balance. It's all very well saying "the Python core devs aren't
> willing to support these libraries any more" - but being quite that
> blunt invites a backlash. The PEP demonstrates that we're doing due
> dilligence over the process, and even if some people disagree with the
> decisions made, the arguments have been made in public. "You can get
> equivalents off PyPI" is only a very minor part of the argument here,
> the main thrust being "... and in general we don't think many people
> need to". It's a judgement call, and it needs review, but "rarely
> used" captures the sense well enough (given that we want a short
> phrase, not an explanatory paragraph...)
>
> Regarding the title, It strikes me as fine, it's a slightly light
> hearted play on the "batteries included" idea, and even if the
> batteries in question aren't entirely dead, they are definitely pinin'
> for the fjords. Discussions like this need a bit of levity to keep us
> all grounded, IMO.
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Skip Montanaro]

> If this were my PEP, I'd call it "Removing unloved batteries from the 
> standard library".

Or maybe, "Removing obsolete and (potentially) dangerous batteries
from the standard library."

I can certainly understand why either class of module would be
removed. When bsddb185 was tossed out, I put it up on PyPI
(https://pypi.org/project/bsddb185/) so the source wouldn't be lost. I
never expected to actually need it, but I got pinged for the first
time a few months ago. Someone needed help building it.

So, no matter the reason for removal, I think it would be reasonable
to toss all these modules into GitHub or PyPI. Someone will want them.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Petr Viktorin]

On 5/21/19 12:06 PM, Christian Heimes wrote:

On 21/05/2019 11.49, Nathaniel Smith wrote:

On Tue, May 21, 2019 at 2:40 AM Walter Dörwald  wrote:


On 20 May 2019, at 22:15, Christian Heimes wrote:


Hi,

here is the first version of my PEP 594 to deprecate and eventually
remove modules from the standard library. The PEP started last year
with talk during the Python Language Summit 2018,
https://lwn.net/Articles/755229/.

[...]

colorsys


The `colorsys `_
module
defines color conversion functions between RGB, YIQ, HSL, and HSV
coordinate
systems. The Pillow library provides much faster conversation between
color systems.

Module type
   pure Python
Deprecated in
   3.8
To be removed in
   3.10
Substitute
   `Pillow `_,
   `colorspacious `_


I'm using colorsys constantly as the basis for a tool that converts CSS
colors between different coordinate systems. I don't see how that could
be done via Pillow (which AFAICT only converts complete images).
RGB<->HSV<->HLS conversion seems to be not available (or not obvious) in
colorspacious.


Correct, colorspacious doesn't support HSV or HLS. I suppose it would
be trivial enough to add...

The 'colour' package has them (along with everything else you can
dream of): https://colour.readthedocs.io/en/latest/colour.models.html


Nice catch, I just added https://python-colormath.readthedocs.io/en/latest/ to 
my update PR. I'll add colour to the list, too.

(It didn't pop up on my radar because I wasn't checking for British spelling)

Christian



Please, don't remove colorsys. HSL and HSV aren't outdated, and AFAIK, 
the module is not a maintenance burden.


The packages on PyPI offer more advanced models. Perhaps they could be 
linked in the documentation, if we want to endorse them. But just as 
we're not removing `array` in favor of `numpy`, we shouldn't remove 
colorsys either. Colorsys is not dead, it's just small.


I assume one of the reasons colorspacious doesn't support HSV or HLS is 
that HSV/HLS tends to be used for quite different purposes than the more 
fancy color spaces. You'd use HSL/HSV for a simple color picker or a 
saturation shift on a web page, GUI or game (where you don't have 
*exact* colors anyway); things like sRGB or CIELab are for color 
management, photos, printing, etc.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 15.54, Victor Stinner wrote:
> IMHO we should simply acknowledge this fact by mentioning it in the
> PEP. We are aware that "pip install" is not always a valid option, but
> we decided anyway to deprecate/remove modules because <...>.

I like the idea. Could you create an issue or PR, please?

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Victor Stinner]

IMHO we should simply acknowledge this fact by mentioning it in the
PEP. We are aware that "pip install" is not always a valid option, but
we decided anyway to deprecate/remove modules because <...>.

Victor

Le mar. 21 mai 2019 à 15:29, Paul Moore  a écrit :
>
> On Tue, 21 May 2019 at 14:03, Steven D'Aprano  wrote:
>
> > I know that saying anything against pip and virtual environments is
> > heresy, but honestly, "just install it from PyPI" is not friendly to
> > beginners or those who just want something that works without a load of
> > extra complexity.
>
> Speaking as a pip developer, I 100% support this comment. For a major
> segment of Python's user base, if something isn't in the stdlib (or
> their preferred distribution - Anaconda or a Linux distro, for
> example) then it's a significant step upward in complexity.
>
> Having said this, I'm not as sure I agree with the rest of Steven's
> posting. I think that in general, this PEP is sensible and strikes a
> good balance. It's all very well saying "the Python core devs aren't
> willing to support these libraries any more" - but being quite that
> blunt invites a backlash. The PEP demonstrates that we're doing due
> dilligence over the process, and even if some people disagree with the
> decisions made, the arguments have been made in public. "You can get
> equivalents off PyPI" is only a very minor part of the argument here,
> the main thrust being "... and in general we don't think many people
> need to". It's a judgement call, and it needs review, but "rarely
> used" captures the sense well enough (given that we want a short
> phrase, not an explanatory paragraph...)
>
> Regarding the title, It strikes me as fine, it's a slightly light
> hearted play on the "batteries included" idea, and even if the
> batteries in question aren't entirely dead, they are definitely pinin'
> for the fjords. Discussions like this need a bit of levity to keep us
> all grounded, IMO.
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Paul Moore]

On Tue, 21 May 2019 at 14:03, Steven D'Aprano  wrote:

> I know that saying anything against pip and virtual environments is
> heresy, but honestly, "just install it from PyPI" is not friendly to
> beginners or those who just want something that works without a load of
> extra complexity.

Speaking as a pip developer, I 100% support this comment. For a major
segment of Python's user base, if something isn't in the stdlib (or
their preferred distribution - Anaconda or a Linux distro, for
example) then it's a significant step upward in complexity.

Having said this, I'm not as sure I agree with the rest of Steven's
posting. I think that in general, this PEP is sensible and strikes a
good balance. It's all very well saying "the Python core devs aren't
willing to support these libraries any more" - but being quite that
blunt invites a backlash. The PEP demonstrates that we're doing due
dilligence over the process, and even if some people disagree with the
decisions made, the arguments have been made in public. "You can get
equivalents off PyPI" is only a very minor part of the argument here,
the main thrust being "... and in general we don't think many people
need to". It's a judgement call, and it needs review, but "rarely
used" captures the sense well enough (given that we want a short
phrase, not an explanatory paragraph...)

Regarding the title, It strikes me as fine, it's a slightly light
hearted play on the "batteries included" idea, and even if the
batteries in question aren't entirely dead, they are definitely pinin'
for the fjords. Discussions like this need a bit of levity to keep us
all grounded, IMO.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 15.01, Steven D'Aprano wrote:
> Christian, I'm glad that you are privileged enough to find it simple and 
> straight forward to download and install, but for many Python users, it 
> is not so simple or straight forward.
> 
> Many Python users don't have the privilege of being able to install 
> arbitrary, unvetted packages from PyPI. They get to use only packages 
> from approved vendors, including the stdlib, what they write themselves, 
> and nothing else. Please don't dismiss this part of the Python community 
> just because they don't typically hang around in the same forums we do.
> 
> I've worked with organisations where downloading and installing software 
> from the internet was grounds for instant dismissal. I've worked with 
> organisations with regulatory requirements to do due-dilegance on their 
> entire software stack, and getting permission to install an unapproved 
> library could take 3-6 months elapsed time and dozens of person-hours, 
> including a full review of the licencing and copyright by lawyers.
> 
> I've also worked with kids using school computers who don't have either 
> the legal permission or the knowledge to use pip install.
> 
> Sometimes their school administrators are ... how shall I put this 
> kindly? ... over zealous in their efforts to protect the students from 
> malware and spyware (apart from the school's own spyware, of course...) 
> and rather lacking in their understanding of the difference between 
> piracy and Open Source software. Getting Python installed by the school 
> admiinistrator is one thing, but allowing the kids to run pip and 
> install software themselves is unthinkable.
> 
> And remember, in some juristictions, installing software from the 
> internet can put you in breach of some draconian laws. At the very 
> least, kids may face expulsion.

This argument has bring brought up several times. For that reason the PEP lists 
mostly modules that deal with historic and irrelevant data formats. The wave 
module stays because it is useful in education contexts. The use case is 
explicitly mentioned in the latest version of the PEP.

Do you have a concrete and tangible case example, in which a deprecated module 
is absolutely necessary, a user gets permission to update Python, but does not 
get permission to get a PyPI package installed?

You could argue that cgi, spwd, and crypt are not history data formats. My 
counter argument: *If* you have to work in a severely restricted environment, 
*then* you are surely not allowed to run your own CGI web server or access the 
system password database.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Paul Moore]

On Tue, 21 May 2019 at 13:50, Steve Holden  wrote:
>
> That seems entirely reasonable. I wonder if the larger community could 
> somehow form an organization (the Dead Parrot SIG?) that would at least 
> curate and monitor efforts to ensure their continued utility?

I have no idea whether there is enough community support to make this
work, but regardless, I strongly support a SIG by that name. It may
actually be even *more* amusing if it doesn't have enough members to
make it viable :-)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steven D'Aprano]

The PEP title is prejudicial and inaccurate. These aren't "dead 
batteries", these are *working batteries* that you want to remove.

If you want a fair and open debate on this, please change the title to 
something less prejudicial. If this were my PEP, I'd call it "Removing 
unloved batteries from the standard library".


On Mon, May 20, 2019 at 10:15:22PM +0200, Christian Heimes wrote:

> Times have changed. The introduction of the cheese shop (PyPI), setuptools,
> and later pip, it became simple and straight forward to download and install
> packages. Nowadays Python has a rich and vibrant ecosystem of third party
> packages. It's pretty much standard to either install packages from PyPI or
> use one of the many Python or Linux distributions.

Christian, I'm glad that you are privileged enough to find it simple and 
straight forward to download and install, but for many Python users, it 
is not so simple or straight forward.

Many Python users don't have the privilege of being able to install 
arbitrary, unvetted packages from PyPI. They get to use only packages 
from approved vendors, including the stdlib, what they write themselves, 
and nothing else. Please don't dismiss this part of the Python community 
just because they don't typically hang around in the same forums we do.

I've worked with organisations where downloading and installing software 
from the internet was grounds for instant dismissal. I've worked with 
organisations with regulatory requirements to do due-dilegance on their 
entire software stack, and getting permission to install an unapproved 
library could take 3-6 months elapsed time and dozens of person-hours, 
including a full review of the licencing and copyright by lawyers.

I've also worked with kids using school computers who don't have either 
the legal permission or the knowledge to use pip install.

Sometimes their school administrators are ... how shall I put this 
kindly? ... over zealous in their efforts to protect the students from 
malware and spyware (apart from the school's own spyware, of course...) 
and rather lacking in their understanding of the difference between 
piracy and Open Source software. Getting Python installed by the school 
admiinistrator is one thing, but allowing the kids to run pip and 
install software themselves is unthinkable.

And remember, in some juristictions, installing software from the 
internet can put you in breach of some draconian laws. At the very 
least, kids may face expulsion.

Despite all the advances in packaging and installation, many people 
still do have trouble installing third-party software. Internet forums 
are full of people asking for help troubleshooting pip install.

"Simple and straight forward" it might be for an expert, but many Python 
users are not experts. The Internet is full of difficult, conflicting 
and/or bad advice about installing software ("just use sudo pip 
install"). Let's look at this advice from Red Hat:

If you want to use third-party packages, create a virtual 
environment using python3 -m venv --system-site-packages myenv 
(or for Python 2, install python2-virtualenv and run python2 
-m virtualenv --system-site-packages myenv). Then, activate 
the environment using source myenv/bin/activate, and install 
packages into it using pip install. The packages will then be 
available as long as the environment is activated.

https://developers.redhat.com/blog/2018/11/14/python-in-rhel-8/

That's some serious barrier to entry right there. We're going from:

batteries included

to 

what's a virtual enviroment? what's activation? why isn't pip 
working for me?

I know that saying anything against pip and virtual environments is 
heresy, but honestly, "just install it from PyPI" is not friendly to 
beginners or those who just want something that works without a load of 
extra complexity.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Ned Batchelder]

On 5/21/19 8:37 AM, Christian Heimes wrote:

On 21/05/2019 14.06, Anders Munch wrote:

Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org] På vegne 
af Christian Heimes

* The removed modules will be available through PyPI.

Will they?  That's not the impression I got from the PEP.

It's all open source. It's up to the Python community to adopt packages and 
provide them on PyPI.

Python core will not maintain and distribute the packages. I'll merely provide 
a repository with packages to help kick-starting the process.



This statement should be in the PEP in some form. For example: "The 
modules could be made available on PyPI. The Python core team will not 
publish or maintain the packages."


--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steve Holden]

That seems entirely reasonable. I wonder if the larger community could
somehow form an organization (the Dead Parrot SIG?) that would at least
curate and monitor efforts to ensure their continued utility?


On Tue, May 21, 2019 at 1:40 PM Christian Heimes 
wrote:

> On 21/05/2019 14.06, Anders Munch wrote:
> > Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org]
> På vegne af Christian Heimes
> >> * The removed modules will be available through PyPI.
> >
> > Will they?  That's not the impression I got from the PEP.
>
> It's all open source. It's up to the Python community to adopt packages
> and provide them on PyPI.
>
> Python core will not maintain and distribute the packages. I'll merely
> provide a repository with packages to help kick-starting the process.
>
> Christian
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 14.06, Anders Munch wrote:
> Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org] På 
> vegne af Christian Heimes
>> * The removed modules will be available through PyPI.
> 
> Will they?  That's not the impression I got from the PEP. 

It's all open source. It's up to the Python community to adopt packages and 
provide them on PyPI.

Python core will not maintain and distribute the packages. I'll merely provide 
a repository with packages to help kick-starting the process.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[André Malo]

On Dienstag, 21. Mai 2019 13:46:34 CEST Christian Heimes wrote:
> On 21/05/2019 13.08, André Malo wrote:
> > On Montag, 20. Mai 2019 23:27:49 CEST Antoine Pitrou wrote:
> >> NNTP is still quite used (often through GMane, but probably not only)
> >> so
> >> I'd question the removal of nntplib.
> >> 
> >> cgitb used to be used by some Web frameworks in order to format
> >> exceptions.  Perhaps one should check if that's still the case.
> > 
> > I concur with both of those.
> > There's software in production using both. (It doesn't mean it's on pypi
> > or even free software).
> 
> There is always somebody who uses a feature. This argument blocks any
> innovation or cleanup. Victor just reminded me of https://xkcd.com/1172/
> .
> 
> * The removed modules will be available through PyPI.
> * You don't have to start to worry until Python 3.10 is released in over 3
> years from now. * The modules are fully supported in Python 3.8 and 3.9.
> Python 3.9 will reach EOL late 2026 or early 2027.

Correct. However that's a valid argument for the whole stdlib.

I agree with Victor  (in the other branch), that we should call the PEP how 
it's meant.
It effectively boils down to: "We don't want to maintain these modules 
anymore, volunteers step forward within the next 3 years". It would 
definitely draw a clear line and cut short a lot of discussions (like this 
one).
And it would be perfectly fine, for me at least.

nd
-- 
package Hacker::Perl::Another::Just;print
qq~@{[reverse split/::/ =>__PACKAGE__]}~;

#  André Malo  #  http://www.perlig.de  #


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 13.50, Victor Stinner wrote:
> Hi Christian,
> 
> I dislike the PEP 594 title: "Removing dead batteries from the
> standard library". A module is never "dead", there are always users,
> even if there are less than 5 of them.

I'm open for suggestions for a better title.

> Extract of the Rationale: "The modules are mostly historic data
> formats and APIs that have been superseded a long time ago"
> 
> If we take the example of NNTP: even if it's old and "superseded",
> people still uses NTTP. Python doesn't have to be opinionated about
> formats.

NNTP was added late in the writing of the PEP. I can rephrase the rational

> Wait, I like the overall PEP. I'm just talking about how it's explained.
> 
> IMHO the question here is if the core developers want to continue to
> pay the price of the maintenance burden for modules which have "few"
> users (define few...). The other question is if it would be acceptable
> to move these modules to PyPI. "import wave" would continue to work as
> previously, but the maintenance would be moved *outside* Python.
> 
> Who will maintain these modules on PyPI? Do we have to define this right now?

Whoever steps up and is interested to maintain the module. But we don't have to 
define this now. We have more than a year to come up with a plan. Except for 
the parser module, all modules will be maintained by Python core until 2026.

> If a module is pure Python, well, the easy solution is to embed it
> into your code: cp /path/to/cpython/Lib/.py
> /path/to/yourapp/.py. If it's a C extension, it's more
> complicated.

This only affects nis, ossaudiodev, spwd, and msi. For crypt I already have a 
ctypes wrapper. ossaudiodev is dead and has been replaced by ALSA about two 
decades ago. spwd is bad and users should use PAM instead.

> The PEP is full of "Substitute: none". I'm not comfortable with that.
> I would prefer to require that there is always a solution, like
> putting the code on PyPI and let it die there. The old mojo "the
> stdlib is where modules die" would become "PyPI is where old stdlib
> modules die" :-)

The latest version of the PEP lists many additional substitutes. I'll post a 
new version soonish.

 
> Python itself is bad at fixing DeprecationWarning: imp and asyncore
> are deprecated for years, but they are still widely used inside
> Python...

We are keeping the old packages to make transition from 2.7 to 3.8 easier. This 
is going to change with 3.9.

> I dislike DeprecationWarning *but* ... well, does it really hurt so
> much to keep these modules around? asyncore has know bugs, *but* many
> tests of our test suite are based on that and we managed to make these
> tests very reliable only with minor work.

I'm struggeling with asyncore in SSL tests, mostly because I don't grasp how 
asyncore works and I have no intention to dig into a deprecated technology.

> Even if it's not said this way, in fact this PEP reopens the question
> of splitting the stdlib. It would be helpful to have some references
> into the PEP about previous discussions on this topic. Recent
> discussion:
> 
> "Amber Brown: Batteries Included, But They're Leaking "
> http://pyfound.blogspot.com/2019/05/amber-brown-batteries-included-but.html
> 
> By the way, even if it's not directly related, the PEP doesn't address
> one of the common question: be able to override a stdlib module with a
> more recent version. For example, if the Python provided by your OS
> contains a bug in asyncio, you are stuck and cannot easily upgrade it
> to get a new version without the fix.

As you said, it's not directly related. These topics are not within scope for 
the PEP. Please open a new thread. I like to keep this thread on topic.


> For me, "getopt" is a dead battery. The PEP says "Although users are
> encouraged to use argparse instead, the getopt module is still widely
> used."
> 
> Well, that makes sense. But so, what is the metric to decide if a
> module is "widely used" or not?
> 
> Should we keep a module at soon as at least one user says "I'm using
> it!". In other replies, users asked to keep spwd, fileinput, nntplib,
> etc.

getopt is very short, has no maintainance overhead, and useful for quick hacks. 
fileinput stays as well for the same reason.

> Sorry, I mostly opened questions. I don't have answers. I just wanted
> to share that this PEP scares me a little bit. At least, the current
> version of the PEP.

The PEP will be implemented over a course of two releases and three years. It 
gives us enough time to address concern or change our minds.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[André Malo]

On Dienstag, 21. Mai 2019 13:50:30 CEST Victor Stinner wrote:

> Well, that makes sense. But so, what is the metric to decide if a
> module is "widely used" or not?

Yes. Exactly the question that pops up right now. I think, that's the main 
issue when including batteries in general (that's not news though :-). And 
the problem I see there is: There *is* no valid answer.

(Sorry if I seem to be just annoying. That's not intended, I'm just not 
carrying the good news.)

nd
-- 
package Hacker::Perl::Another::Just;print
qq~@{[reverse split/::/ =>__PACKAGE__]}~;

#  André Malo  #  http://www.perlig.de  #


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Anders Munch]

Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org] På vegne 
af Christian Heimes
> * The removed modules will be available through PyPI.

Will they?  That's not the impression I got from the PEP. 

regards, Anders

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Victor Stinner]

Hi Christian,

I dislike the PEP 594 title: "Removing dead batteries from the
standard library". A module is never "dead", there are always users,
even if there are less than 5 of them.

Extract of the Rationale: "The modules are mostly historic data
formats and APIs that have been superseded a long time ago"

If we take the example of NNTP: even if it's old and "superseded",
people still uses NTTP. Python doesn't have to be opinionated about
formats.


Wait, I like the overall PEP. I'm just talking about how it's explained.

IMHO the question here is if the core developers want to continue to
pay the price of the maintenance burden for modules which have "few"
users (define few...). The other question is if it would be acceptable
to move these modules to PyPI. "import wave" would continue to work as
previously, but the maintenance would be moved *outside* Python.

Who will maintain these modules on PyPI? Do we have to define this right now?

If a module is pure Python, well, the easy solution is to embed it
into your code: cp /path/to/cpython/Lib/.py
/path/to/yourapp/.py. If it's a C extension, it's more
complicated.

For me the risk of the PEP 594 is to create a new Python 3 mess.
Pedantic core devs who explain to users that their code is wrong and
they must fix it... But the code was running fine for 20 years!

The PEP is full of "Substitute: none". I'm not comfortable with that.
I would prefer to require that there is always a solution, like
putting the code on PyPI and let it die there. The old mojo "the
stdlib is where modules die" would become "PyPI is where old stdlib
modules die" :-)

Python itself is bad at fixing DeprecationWarning: imp and asyncore
are deprecated for years, but they are still widely used inside
Python...

"Stop using deprecated imp module; imp should now emit a real
DeprecationWarning" is open since 2015:
https://bugs.python.org/issue25160

"Replace asyncore" open since 2016
https://bugs.python.org/issue28533

"test_poplib replace asyncore" open since 2017
https://bugs.python.org/issue30514

I dislike DeprecationWarning *but* ... well, does it really hurt so
much to keep these modules around? asyncore has know bugs, *but* many
tests of our test suite are based on that and we managed to make these
tests very reliable only with minor work.

---

Even if it's not said this way, in fact this PEP reopens the question
of splitting the stdlib. It would be helpful to have some references
into the PEP about previous discussions on this topic. Recent
discussion:

"Amber Brown: Batteries Included, But They're Leaking "
http://pyfound.blogspot.com/2019/05/amber-brown-batteries-included-but.html

By the way, even if it's not directly related, the PEP doesn't address
one of the common question: be able to override a stdlib module with a
more recent version. For example, if the Python provided by your OS
contains a bug in asyncio, you are stuck and cannot easily upgrade it
to get a new version without the fix.

---

For me, "getopt" is a dead battery. The PEP says "Although users are
encouraged to use argparse instead, the getopt module is still widely
used."

Well, that makes sense. But so, what is the metric to decide if a
module is "widely used" or not?

Should we keep a module at soon as at least one user says "I'm using
it!". In other replies, users asked to keep spwd, fileinput, nntplib,
etc.


---

Sorry, I mostly opened questions. I don't have answers. I just wanted
to share that this PEP scares me a little bit. At least, the current
version of the PEP.

Victor


Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Sebastian Rittau]

Am 21.05.19 um 13:39 schrieb André Malo:


So what I hear is, this battery is definitely not dead, which is what the
PEP is all about.
it's just half charged (or discharged, depending on your POV), so to speak.

Substitute: "none" should read pypi then?


Every single module on this list will have some users. To me the 
question is: Are there enough users to justify the extra burden on the 
maintainers, but also users not using that package? (The latter will 
have to download the package, have it installed on their systems, it 
makes it harder to find other things in the documentation etc.) If there 
not many users left, I believe these should either vendor the last 
official package or to band together to maintain the package themselves 
on pypi.


 - Sebastian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 13.08, André Malo wrote:
> On Montag, 20. Mai 2019 23:27:49 CEST Antoine Pitrou wrote:
>> NNTP is still quite used (often through GMane, but probably not only) so
>> I'd question the removal of nntplib.
>>
>> cgitb used to be used by some Web frameworks in order to format
>> exceptions.  Perhaps one should check if that's still the case.
> 
> I concur with both of those.
> There's software in production using both. (It doesn't mean it's on pypi or 
> even free software).

There is always somebody who uses a feature. This argument blocks any 
innovation or cleanup. Victor just reminded me of https://xkcd.com/1172/ .

* The removed modules will be available through PyPI.
* You don't have to start to worry until Python 3.10 is released in over 3 
years from now.
* The modules are fully supported in Python 3.8 and 3.9. Python 3.9 will reach 
EOL late 2026 or early 2027.

That's plenty of time.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[André Malo]

On Dienstag, 21. Mai 2019 13:24:34 CEST Victor Stinner wrote:
> Le mar. 21 mai 2019 à 13:18, André Malo  a écrit :
> > There's software in production using both. (It doesn't mean it's on pypi
> > or even free software).
> > 
> > What would be the maintenance burden of those modules anyway? (at least
> > for nntp, I guess it's not gonna change).
> 
> The maintenance burden is real even if it's not visible. For example,
> test_nntplib is causing frequently issues on our CI:
> 
> https://bugs.python.org/issue19756
> https://bugs.python.org/issue19613
> https://bugs.python.org/issue31850
> 
> It's failing frequently since 2013, and nobody managed to come with a
> fix.. in 6 years.
> 
> There are 11 open issues with "nntp" in their title (8 with exactly
> "nntplib" in their title).
> 
> test_nntplib uses the public server news.trigofacile.com which is
> operated by Julien ÉLIE. Two years ago, Julien asked me if there is
> any plan to support the NNTP "COMPRESS" command.

So what I hear is, this battery is definitely not dead, which is what the 
PEP is all about.
it's just half charged (or discharged, depending on your POV), so to speak.

Substitute: "none" should read pypi then?

nd
-- 
"Solides und umfangreiches Buch"
  -- aus einer Rezension




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Victor Stinner]

Le mar. 21 mai 2019 à 13:18, André Malo  a écrit :
> There's software in production using both. (It doesn't mean it's on pypi or
> even free software).
>
> What would be the maintenance burden of those modules anyway? (at least for
> nntp, I guess it's not gonna change).

The maintenance burden is real even if it's not visible. For example,
test_nntplib is causing frequently issues on our CI:

https://bugs.python.org/issue19756
https://bugs.python.org/issue19613
https://bugs.python.org/issue31850

It's failing frequently since 2013, and nobody managed to come with a
fix.. in 6 years.

There are 11 open issues with "nntp" in their title (8 with exactly
"nntplib" in their title).

test_nntplib uses the public server news.trigofacile.com which is
operated by Julien ÉLIE. Two years ago, Julien asked me if there is
any plan to support the NNTP "COMPRESS" command.

Victor





--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[André Malo]

On Montag, 20. Mai 2019 23:27:49 CEST Antoine Pitrou wrote:
> NNTP is still quite used (often through GMane, but probably not only) so
> I'd question the removal of nntplib.
> 
> cgitb used to be used by some Web frameworks in order to format
> exceptions.  Perhaps one should check if that's still the case.

I concur with both of those.
There's software in production using both. (It doesn't mean it's on pypi or 
even free software).

What would be the maintenance burden of those modules anyway? (at least for 
nntp, I guess it's not gonna change).

nd
-- 
package Hacker::Perl::Another::Just;print
qq~@{[reverse split/::/ =>__PACKAGE__]}~;

#  André Malo  #  http://pub.perlig.de  #


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 12.19, Giampaolo Rodola' wrote:
> I find this one useful and would be a bit sad to see it go. FWIW I use it in 
> pyftpdlib and I suppose there are other apps out there relying on UNIX 
> password db for authentication. The fact that it’s a C module is also an 
> incentive to leave it in the stdlib IMO (pure python modules can easily be 
> copied in the project instead of retrieving them from PYPI as a third party 
> dep - e.g. this is how I am likely going to replace asyncore/asynchat).

If you use the spwd module for authentication, then you have a major security 
problem in your application. You must use the PAM stack to authenticate access 
to a service.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Giampaolo Rodola']

On Tue, 21 May 2019 at 03:17, Christian Heimes  wrote:

spwd
> 
>
> The `spwd `_ module provides
> direct access to Unix shadow password database using non-standard APIs.
> In general it's a bad idea to use the spwd. The spwd circumvents system
> security policies, it does not use the PAM stack, and is
> only compatible with local user accounts.
>
> Module type
>   C extension
> Deprecated in
>   3.8
> To be removed in
>   3.10
> Substitute
>   **none**


I find this one useful and would be a bit sad to see it go. FWIW I use it
in pyftpdlib and I suppose there are other apps out there relying on UNIX
password db for authentication. The fact that it’s a C module is also an
incentive to leave it in the stdlib IMO (pure python modules can easily be
copied in the project instead of retrieving them from PYPI as a third party
dep - e.g. this is how I am likely going to replace asyncore/asynchat).

A big +1 to this PEP by the way.
-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 11.49, Nathaniel Smith wrote:
> On Tue, May 21, 2019 at 2:40 AM Walter Dörwald  wrote:
>>
>> On 20 May 2019, at 22:15, Christian Heimes wrote:
>>
>>> Hi,
>>>
>>> here is the first version of my PEP 594 to deprecate and eventually
>>> remove modules from the standard library. The PEP started last year
>>> with talk during the Python Language Summit 2018,
>>> https://lwn.net/Articles/755229/.
>>>
>>> [...]
>>>
>>> colorsys
>>> 
>>>
>>> The `colorsys `_
>>> module
>>> defines color conversion functions between RGB, YIQ, HSL, and HSV
>>> coordinate
>>> systems. The Pillow library provides much faster conversation between
>>> color systems.
>>>
>>> Module type
>>>   pure Python
>>> Deprecated in
>>>   3.8
>>> To be removed in
>>>   3.10
>>> Substitute
>>>   `Pillow `_,
>>>   `colorspacious `_
>>
>> I'm using colorsys constantly as the basis for a tool that converts CSS
>> colors between different coordinate systems. I don't see how that could
>> be done via Pillow (which AFAICT only converts complete images).
>> RGB<->HSV<->HLS conversion seems to be not available (or not obvious) in
>> colorspacious.
> 
> Correct, colorspacious doesn't support HSV or HLS. I suppose it would
> be trivial enough to add...
> 
> The 'colour' package has them (along with everything else you can
> dream of): https://colour.readthedocs.io/en/latest/colour.models.html

Nice catch, I just added https://python-colormath.readthedocs.io/en/latest/ to 
my update PR. I'll add colour to the list, too.

(It didn't pop up on my radar because I wasn't checking for British spelling)

Christian
 


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Steve Holden]

It's covered in "Python in a Nutshell," Alex Martelli having been a
promoter of its ability simplify many utility programs for a long time.

Not that that's any guide as to what should be in 3.10, by which time we'll
be four minor releases out of date anyway.


On Tue, May 21, 2019 at 9:16 AM Christian Heimes 
wrote:

> On 21/05/2019 02.16, Inada Naoki wrote:
> > I use fileinput for several times per year.
> >
> > fileinput is handy tool to write single script file to analyze log files.
> >
> > * In such tools, I don't need real argument parser.
> > * Some log files are compressed and some are not.
> >   It seems argparse doesn't support transparent decompression.
> > * I don't want to use 3rd party library for such single script files.
>
> OK, let's keep it. I was under the impression that it's not used.
>
> Christian
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Antoine Pitrou]

On Tue, 21 May 2019 00:55:20 +0200
Christian Heimes  wrote:

> On 21/05/2019 00.13, Antoine Pitrou wrote:
> > On Tue, 21 May 2019 00:06:35 +0200
> > Christian Heimes  wrote:  
> >> On 20/05/2019 23.27, Antoine Pitrou wrote:  
> >>> NNTP is still quite used (often through GMane, but probably not only) so
> >>> I'd question the removal of nntplib.
> >>
> >> Is NNTP support important enough to keep the module in the standard 
> >> library?  
> > 
> > I'd phrase the question differently: is NNTP dead enough, or nntplib
> > painful enough to maintain, that it's worth removing it from the stdlib?  
> 
> The module itself does not create much work. But its tests are a regular 
> source of pain and instabilities. The tests for nntplib depend on external 
> NNTP servers. These servers are sometimes down, very slow, or don't work over 
> IPv6. I'm sure that Pablo and Victor can tell you some war stories. I briefly 
> mentioned the issues in the PEP, too.

I plead guilty for adding those tests :-/  Back then, it seemed
reasonable to test the nntplib module against real-world servers, since
like all old Internet text protocols NNTP is not always implemented
rigorously.

If the NNTP networked tests are an annoyance, they should probably be
skipped on CI.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Nathaniel Smith]

On Tue, May 21, 2019 at 2:40 AM Walter Dörwald  wrote:
>
> On 20 May 2019, at 22:15, Christian Heimes wrote:
>
> > Hi,
> >
> > here is the first version of my PEP 594 to deprecate and eventually
> > remove modules from the standard library. The PEP started last year
> > with talk during the Python Language Summit 2018,
> > https://lwn.net/Articles/755229/.
> >
> > [...]
> >
> > colorsys
> > 
> >
> > The `colorsys `_
> > module
> > defines color conversion functions between RGB, YIQ, HSL, and HSV
> > coordinate
> > systems. The Pillow library provides much faster conversation between
> > color systems.
> >
> > Module type
> >   pure Python
> > Deprecated in
> >   3.8
> > To be removed in
> >   3.10
> > Substitute
> >   `Pillow `_,
> >   `colorspacious `_
>
> I'm using colorsys constantly as the basis for a tool that converts CSS
> colors between different coordinate systems. I don't see how that could
> be done via Pillow (which AFAICT only converts complete images).
> RGB<->HSV<->HLS conversion seems to be not available (or not obvious) in
> colorspacious.

Correct, colorspacious doesn't support HSV or HLS. I suppose it would
be trivial enough to add...

The 'colour' package has them (along with everything else you can
dream of): https://colour.readthedocs.io/en/latest/colour.models.html

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Walter Dörwald]

On 20 May 2019, at 22:15, Christian Heimes wrote:


Hi,

here is the first version of my PEP 594 to deprecate and eventually 
remove modules from the standard library. The PEP started last year 
with talk during the Python Language Summit 2018, 
https://lwn.net/Articles/755229/.


[...]

colorsys


The `colorsys `_ 
module
defines color conversion functions between RGB, YIQ, HSL, and HSV 
coordinate

systems. The Pillow library provides much faster conversation between
color systems.

Module type
  pure Python
Deprecated in
  3.8
To be removed in
  3.10
Substitute
  `Pillow `_,
  `colorspacious `_


I'm using colorsys constantly as the basis for a tool that converts CSS 
colors between different coordinate systems. I don't see how that could 
be done via Pillow (which AFAICT only converts complete images). 
RGB<->HSV<->HLS conversion seems to be not available (or not obvious) in 
colorspacious.


colorsys is a module where we can be pretty sure that it has zero bugs, 
and doesn't require any maintenance or security updates, so I don't see 
any reason to deprecate it.



[...]


Servus,
   Walter
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 02.16, Inada Naoki wrote:
> I use fileinput for several times per year.
> 
> fileinput is handy tool to write single script file to analyze log files.
> 
> * In such tools, I don't need real argument parser.
> * Some log files are compressed and some are not.
>   It seems argparse doesn't support transparent decompression.
> * I don't want to use 3rd party library for such single script files.

OK, let's keep it. I was under the impression that it's not used.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Guido van Rossum]

Yeah, I was surprised to see fileinput on the list. It's indeed a handy
little tool.

On Mon, May 20, 2019 at 5:19 PM Inada Naoki  wrote:

> I use fileinput for several times per year.
>
> fileinput is handy tool to write single script file to analyze log files.
>
> * In such tools, I don't need real argument parser.
> * Some log files are compressed and some are not.
>   It seems argparse doesn't support transparent decompression.
> * I don't want to use 3rd party library for such single script files.
>
> I'd like to add transparent decompression to argparse if fileinput
> is deprecated.
>
> Regards,
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Inada Naoki]

I use fileinput for several times per year.

fileinput is handy tool to write single script file to analyze log files.

* In such tools, I don't need real argument parser.
* Some log files are compressed and some are not.
  It seems argparse doesn't support transparent decompression.
* I don't want to use 3rd party library for such single script files.

I'd like to add transparent decompression to argparse if fileinput
is deprecated.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 01.06, Terry Reedy wrote:
> On 5/20/2019 6:06 PM, Christian Heimes wrote:
> 
>>> Removing the crypt module would remove support for system-standard
>>> password files.  I don't understand the rationale.
>>
>> Applications *must* not access system-standard password files directly. On 
>> any sanely and securely configured systems, application cannot even access 
>> system password files like /etc/shadow. Access restrictions and system 
>> security policies will prevent read access. Also applications cannot assume 
>> that users are present in any user file. They may come from LDAP, SSSD, 
>> ActiveDirectory, or other sources.
>>
>> The correct way to interact with system users is to use the proper APIs, 
>> that are NSS (name service switch) and PAM (pluggable authentication 
>> modules). NSS looks up and enumerate users and groups. PAM performs password 
>> validation and much, much, much more. The pwd and grp modules use the 
>> correct APIs to interact with NSS. If you need to check or change passwords, 
>> you must go through PAM.
> 
> Add this to the PEP?  It might suggest that crypt should go away sooner.

Yes, I'll do that. I'm currently collecting updates from feedback in PR 
https://github.com/python/peps/pull/1063

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Terry Reedy]

On 5/20/2019 6:06 PM, Christian Heimes wrote:


Removing the crypt module would remove support for system-standard
password files.  I don't understand the rationale.


Applications *must* not access system-standard password files directly. On any 
sanely and securely configured systems, application cannot even access system 
password files like /etc/shadow. Access restrictions and system security 
policies will prevent read access. Also applications cannot assume that users 
are present in any user file. They may come from LDAP, SSSD, ActiveDirectory, 
or other sources.

The correct way to interact with system users is to use the proper APIs, that 
are NSS (name service switch) and PAM (pluggable authentication modules). NSS 
looks up and enumerate users and groups. PAM performs password validation and 
much, much, much more. The pwd and grp modules use the correct APIs to interact 
with NSS. If you need to check or change passwords, you must go through PAM.


Add this to the PEP?  It might suggest that crypt should go away sooner.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 21/05/2019 00.13, Antoine Pitrou wrote:
> On Tue, 21 May 2019 00:06:35 +0200
> Christian Heimes  wrote:
>> On 20/05/2019 23.27, Antoine Pitrou wrote:
>>> NNTP is still quite used (often through GMane, but probably not only) so
>>> I'd question the removal of nntplib.  
>>
>> Is NNTP support important enough to keep the module in the standard library?
> 
> I'd phrase the question differently: is NNTP dead enough, or nntplib
> painful enough to maintain, that it's worth removing it from the stdlib?

The module itself does not create much work. But its tests are a regular source 
of pain and instabilities. The tests for nntplib depend on external NNTP 
servers. These servers are sometimes down, very slow, or don't work over IPv6. 
I'm sure that Pablo and Victor can tell you some war stories. I briefly 
mentioned the issues in the PEP, too.

> If the stdlib didn't have NNTP support, obviously nobody would suggest
> adding it nowadays.  But it has that support, and there are certainly
> uses of it in the wild, so we must take that into account.
> 
>>> If the wave module depends on the audioop module, and if the wave
>>> module is kept in the stdlib, then the audioop module can't be removed.  
>>
>> No, it can be removed. I explained the situation in the "wave" section of 
>> the PEP.
> 
> My bad.  I had skipped that.

The section in audioop was confusing. I have updated the audioop paragraph and 
the crypt paragraph with your feedback. I'll keep the PR 
https://github.com/python/peps/pull/1063 option for a couple of days to collect 
more feedback.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Antoine Pitrou]

On Tue, 21 May 2019 00:41:19 +0200
Simon Cross  wrote:
> Woot. +100 on this PEP.
> 
> > If the stdlib didn't have NNTP support, obviously nobody would suggest
> > adding it nowadays.  
> 
> Perhaps this is a good reason to keep nntplib in the deprecation list?

No, because the same applies to getopt, optparse and others, which are
not in the deprecation list.

> Another question is "are there any places using nntplib where `pip install
> nntplib`" is not an reasonable option?

You could ask pretty much the same question about most stdlib packages.
Isn't "pip install ftplib" a reasonable option?  How about "pip
install email" or "pip install xmlrpc"?

The PEP is about "dead batteries".  Not "batteries that some people
think would be reasonable to install from PyPI".

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Simon Cross]

Woot. +100 on this PEP.

> If the stdlib didn't have NNTP support, obviously nobody would suggest
> adding it nowadays.

Perhaps this is a good reason to keep nntplib in the deprecation list?
Another question is "are there any places using nntplib where `pip install
nntplib`" is not an reasonable option? There's quite a lot of time before
3.10 to move nntplib into an outside package.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Antoine Pitrou]

On Tue, 21 May 2019 00:06:35 +0200
Christian Heimes  wrote:
> On 20/05/2019 23.27, Antoine Pitrou wrote:
> > NNTP is still quite used (often through GMane, but probably not only) so
> > I'd question the removal of nntplib.  
> 
> Is NNTP support important enough to keep the module in the standard library?

I'd phrase the question differently: is NNTP dead enough, or nntplib
painful enough to maintain, that it's worth removing it from the stdlib?

If the stdlib didn't have NNTP support, obviously nobody would suggest
adding it nowadays.  But it has that support, and there are certainly
uses of it in the wild, so we must take that into account.

> > If the wave module depends on the audioop module, and if the wave
> > module is kept in the stdlib, then the audioop module can't be removed.  
> 
> No, it can be removed. I explained the situation in the "wave" section of the 
> PEP.

My bad.  I had skipped that.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Jeff Kintscher]

What replacements are available for NNTP? All I could find was pynntp, 
which had a single release 6 years ago.


https://github.com/greenbender/pynntp

//Jeff

On 5/20/19 2:27 PM, Antoine Pitrou wrote:

NNTP is still quite used (often through GMane, but probably not only) so
I'd question the removal of nntplib.

cgitb used to be used by some Web frameworks in order to format
exceptions.  Perhaps one should check if that's still the case.

If the wave module depends on the audioop module, and if the wave
module is kept in the stdlib, then the audioop module can't be removed.

Removing the crypt module would remove support for system-standard
password files.  I don't understand the rationale.

Regards

Antoine.


On Mon, 20 May 2019 22:15:22 +0200
Christian Heimes  wrote:

Hi,

here is the first version of my PEP 594 to deprecate and eventually remove 
modules from the standard library. The PEP started last year with talk during 
the Python Language Summit 2018, https://lwn.net/Articles/755229/.

The PEP can be confirmed in two stages. I'm not planning any code changes for 
3.8. Instead I only like to document a bunch of modules as deprecated. Active 
deprecation is planned for 3.9 and removal for 3.10. The long deprecation phase 
gives us 3 years to change our minds or handle edge cases, too.

Regards,
Christian



PEP: 594
Title: Removing dead batteries from the standard library
Author: Christian Heimes 
Status: Active
Type: Process
Content-Type: text/x-rst
Created: 20-May-2019
Post-History:


Abstract


This PEP proposed a list of standard library modules to be removed from the
standard library. The modules are mostly historic data formats and APIs that
have been superseded a long time ago, e.g. Mac OS 9 and Commodore.


Rationale
=

Back in the early days of Python, the interpreter came with a large set of
useful modules. This was often refrained to as "batteries included"
philosophy and was one of the corner stones to Python's success story.
Users didn't have to figure out how to download and install separate
packages in order to write a simple web server or parse email.

Times have changed. The introduction of the cheese shop (PyPI), setuptools,
and later pip, it became simple and straight forward to download and install
packages. Nowadays Python has a rich and vibrant ecosystem of third party
packages. It's pretty much standard to either install packages from PyPI or
use one of the many Python or Linux distributions.

On the other hand, Python's standard library is piling up cruft, unnecessary
duplication of functionality, and dispensable features. This is undesirable
for several reasons.

* Any additional module increases the maintenance cost for the Python core
   development team. The team has limited resources, reduced maintenance cost
   frees development time for other improvements.
* Modules in the standard library are generally favored and seen as the
   de-facto solution for a problem. A majority of users only pick 3rd party
   modules to replace a stdlib module, when they have a compelling reason, e.g.
   lxml instead of `xml`. The removal of an unmaintained stdlib module
   increases the chances of a community contributed module to become widely
   used.
* A lean and mean standard library benefits platforms with limited resources
   like devices with just a few hundred kilobyte of storage (e.g. BBC
   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
   (e.g. pyodide) also benefit from reduced download size.

The modules in the PEP have been selected for deprecation because their
removal is either least controversial or most beneficial. For example
least controversial are 30 years old multimedia formats like ``sunau``
audio format, which was used on SPARC and NeXT workstations in the late
1980ties. The ``crypt`` module has fundamental flaws that are better solved
outside the standard library.

This PEP also designates some modules as not scheduled for removal. Some
modules have been deprecated for several releases or seem unnecessary at
first glance. However it is beneficial to keep the modules in the standard
library, mostly for environments where installing a package from PyPI is not
an option. This can be cooperate environments or class rooms where external
code is not permitted without legal approval.

* The usage of FTP is declining, but some files are still provided over
   the FTP protocol or hosters offer FTP to upload content. Therefore
   ``ftplib`` is going to stay.
* The ``optparse`` and ``getopt`` module are widely used. They are mature
   modules with very low maintenance overhead.
* According to David Beazley [5]_ the ``wave`` module is easy to teach to
   kids and can make crazy sounds. Making a computer generate crazy sounds is
   powerful and highly motivating exercise for a 9yo aspiring developer. It's
   a fun battery to keep.


Deprecation schedule
=

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 20/05/2019 23.27, Antoine Pitrou wrote:
> NNTP is still quite used (often through GMane, but probably not only) so
> I'd question the removal of nntplib.

Is NNTP support important enough to keep the module in the standard library?

> cgitb used to be used by some Web frameworks in order to format
> exceptions.  Perhaps one should check if that's still the case.

A search on github did not reveal any relevant use of cgitb besides tons of 
copies of test_cgitb and an optional debugging middleware in Paste. I checked 
Django, Plone, CherryPy, flask, and bottle. None uses cgitb.

> If the wave module depends on the audioop module, and if the wave
> module is kept in the stdlib, then the audioop module can't be removed.

No, it can be removed. I explained the situation in the "wave" section of the 
PEP.

> Removing the crypt module would remove support for system-standard
> password files.  I don't understand the rationale.

Applications *must* not access system-standard password files directly. On any 
sanely and securely configured systems, application cannot even access system 
password files like /etc/shadow. Access restrictions and system security 
policies will prevent read access. Also applications cannot assume that users 
are present in any user file. They may come from LDAP, SSSD, ActiveDirectory, 
or other sources.

The correct way to interact with system users is to use the proper APIs, that 
are NSS (name service switch) and PAM (pluggable authentication modules). NSS 
looks up and enumerate users and groups. PAM performs password validation and 
much, much, much more. The pwd and grp modules use the correct APIs to interact 
with NSS. If you need to check or change passwords, you must go through PAM.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Antoine Pitrou]

NNTP is still quite used (often through GMane, but probably not only) so
I'd question the removal of nntplib.

cgitb used to be used by some Web frameworks in order to format
exceptions.  Perhaps one should check if that's still the case.

If the wave module depends on the audioop module, and if the wave
module is kept in the stdlib, then the audioop module can't be removed.

Removing the crypt module would remove support for system-standard
password files.  I don't understand the rationale.

Regards

Antoine.


On Mon, 20 May 2019 22:15:22 +0200
Christian Heimes  wrote:
> Hi,
> 
> here is the first version of my PEP 594 to deprecate and eventually remove 
> modules from the standard library. The PEP started last year with talk during 
> the Python Language Summit 2018, https://lwn.net/Articles/755229/.
> 
> The PEP can be confirmed in two stages. I'm not planning any code changes for 
> 3.8. Instead I only like to document a bunch of modules as deprecated. Active 
> deprecation is planned for 3.9 and removal for 3.10. The long deprecation 
> phase gives us 3 years to change our minds or handle edge cases, too.
> 
> Regards,
> Christian
> 
> 
> 
> PEP: 594
> Title: Removing dead batteries from the standard library
> Author: Christian Heimes 
> Status: Active
> Type: Process
> Content-Type: text/x-rst
> Created: 20-May-2019
> Post-History:
> 
> 
> Abstract
> 
> 
> This PEP proposed a list of standard library modules to be removed from the
> standard library. The modules are mostly historic data formats and APIs that
> have been superseded a long time ago, e.g. Mac OS 9 and Commodore.
> 
> 
> Rationale
> =
> 
> Back in the early days of Python, the interpreter came with a large set of
> useful modules. This was often refrained to as "batteries included"
> philosophy and was one of the corner stones to Python's success story.
> Users didn't have to figure out how to download and install separate
> packages in order to write a simple web server or parse email.
> 
> Times have changed. The introduction of the cheese shop (PyPI), setuptools,
> and later pip, it became simple and straight forward to download and install
> packages. Nowadays Python has a rich and vibrant ecosystem of third party
> packages. It's pretty much standard to either install packages from PyPI or
> use one of the many Python or Linux distributions.
> 
> On the other hand, Python's standard library is piling up cruft, unnecessary
> duplication of functionality, and dispensable features. This is undesirable
> for several reasons.
> 
> * Any additional module increases the maintenance cost for the Python core
>   development team. The team has limited resources, reduced maintenance cost
>   frees development time for other improvements.
> * Modules in the standard library are generally favored and seen as the
>   de-facto solution for a problem. A majority of users only pick 3rd party
>   modules to replace a stdlib module, when they have a compelling reason, e.g.
>   lxml instead of `xml`. The removal of an unmaintained stdlib module
>   increases the chances of a community contributed module to become widely
>   used.
> * A lean and mean standard library benefits platforms with limited resources
>   like devices with just a few hundred kilobyte of storage (e.g. BBC
>   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
>   (e.g. pyodide) also benefit from reduced download size.
> 
> The modules in the PEP have been selected for deprecation because their
> removal is either least controversial or most beneficial. For example
> least controversial are 30 years old multimedia formats like ``sunau``
> audio format, which was used on SPARC and NeXT workstations in the late
> 1980ties. The ``crypt`` module has fundamental flaws that are better solved
> outside the standard library.
> 
> This PEP also designates some modules as not scheduled for removal. Some
> modules have been deprecated for several releases or seem unnecessary at
> first glance. However it is beneficial to keep the modules in the standard
> library, mostly for environments where installing a package from PyPI is not
> an option. This can be cooperate environments or class rooms where external
> code is not permitted without legal approval.
> 
> * The usage of FTP is declining, but some files are still provided over
>   the FTP protocol or hosters offer FTP to upload content. Therefore
>   ``ftplib`` is going to stay.
> * The ``optparse`` and ``getopt`` module are widely used. They are mature
>   modules with very low maintenance overhead.
> * According to David Beazley [5]_ the ``wave`` module is easy to teach to
>   kids and can make crazy sounds. Making a computer generate crazy sounds is
>   powerful and highly motivating exercise for a 9yo aspiring developer. It's
>   a fun battery to keep.
> 
> 
> Deprecation schedule
> 
> 
> 3.8
> ---
> 
> Thi

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

On 20/05/2019 23.12, Andrew Svetlov wrote:
> socketserver.py is also questionable

I briefly though about the module, but didn't consider it for removal. The 
http.server, xmlrpc.server, and logging configuration server are implemented on 
top of the socketserver. I don't want to remove the socketserver module without 
a suitable replacement for http.server in the standard library.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Andrew Svetlov]

socketserver.py is also questionable

On Mon, May 20, 2019 at 11:15 PM Christian Heimes  wrote:
>
> Hi,
>
> here is the first version of my PEP 594 to deprecate and eventually remove 
> modules from the standard library. The PEP started last year with talk during 
> the Python Language Summit 2018, https://lwn.net/Articles/755229/.
>
> The PEP can be confirmed in two stages. I'm not planning any code changes for 
> 3.8. Instead I only like to document a bunch of modules as deprecated. Active 
> deprecation is planned for 3.9 and removal for 3.10. The long deprecation 
> phase gives us 3 years to change our minds or handle edge cases, too.
>
> Regards,
> Christian
>
>
> 
> PEP: 594
> Title: Removing dead batteries from the standard library
> Author: Christian Heimes 
> Status: Active
> Type: Process
> Content-Type: text/x-rst
> Created: 20-May-2019
> Post-History:
>
>
> Abstract
> 
>
> This PEP proposed a list of standard library modules to be removed from the
> standard library. The modules are mostly historic data formats and APIs that
> have been superseded a long time ago, e.g. Mac OS 9 and Commodore.
>
>
> Rationale
> =
>
> Back in the early days of Python, the interpreter came with a large set of
> useful modules. This was often refrained to as "batteries included"
> philosophy and was one of the corner stones to Python's success story.
> Users didn't have to figure out how to download and install separate
> packages in order to write a simple web server or parse email.
>
> Times have changed. The introduction of the cheese shop (PyPI), setuptools,
> and later pip, it became simple and straight forward to download and install
> packages. Nowadays Python has a rich and vibrant ecosystem of third party
> packages. It's pretty much standard to either install packages from PyPI or
> use one of the many Python or Linux distributions.
>
> On the other hand, Python's standard library is piling up cruft, unnecessary
> duplication of functionality, and dispensable features. This is undesirable
> for several reasons.
>
> * Any additional module increases the maintenance cost for the Python core
>   development team. The team has limited resources, reduced maintenance cost
>   frees development time for other improvements.
> * Modules in the standard library are generally favored and seen as the
>   de-facto solution for a problem. A majority of users only pick 3rd party
>   modules to replace a stdlib module, when they have a compelling reason, e.g.
>   lxml instead of `xml`. The removal of an unmaintained stdlib module
>   increases the chances of a community contributed module to become widely
>   used.
> * A lean and mean standard library benefits platforms with limited resources
>   like devices with just a few hundred kilobyte of storage (e.g. BBC
>   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
>   (e.g. pyodide) also benefit from reduced download size.
>
> The modules in the PEP have been selected for deprecation because their
> removal is either least controversial or most beneficial. For example
> least controversial are 30 years old multimedia formats like ``sunau``
> audio format, which was used on SPARC and NeXT workstations in the late
> 1980ties. The ``crypt`` module has fundamental flaws that are better solved
> outside the standard library.
>
> This PEP also designates some modules as not scheduled for removal. Some
> modules have been deprecated for several releases or seem unnecessary at
> first glance. However it is beneficial to keep the modules in the standard
> library, mostly for environments where installing a package from PyPI is not
> an option. This can be cooperate environments or class rooms where external
> code is not permitted without legal approval.
>
> * The usage of FTP is declining, but some files are still provided over
>   the FTP protocol or hosters offer FTP to upload content. Therefore
>   ``ftplib`` is going to stay.
> * The ``optparse`` and ``getopt`` module are widely used. They are mature
>   modules with very low maintenance overhead.
> * According to David Beazley [5]_ the ``wave`` module is easy to teach to
>   kids and can make crazy sounds. Making a computer generate crazy sounds is
>   powerful and highly motivating exercise for a 9yo aspiring developer. It's
>   a fun battery to keep.
>
>
> Deprecation schedule
> 
>
> 3.8
> ---
>
> This PEP targets Python 3.8. Version 3.8.0 final is scheduled to be released
> a few months before Python 2.7 will reach its end of lifetime. We expect that
> Python 3.8 will be targeted by users that migrate to Python 3 in 2019 and
> 2020. To reduce churn and to allow a smooth transition from Python 2,
> Python 3.8 will neither raise `DeprecationWarning` nor remove any
> modules that have been scheduled for removal. Instead deprecated modules will
> just be *documented* as deprecated. Optionally modu

[Python-Dev] PEP 594: Removing dead batteries from the standard library

[Christian Heimes]

Hi,

here is the first version of my PEP 594 to deprecate and eventually remove 
modules from the standard library. The PEP started last year with talk during 
the Python Language Summit 2018, https://lwn.net/Articles/755229/.

The PEP can be confirmed in two stages. I'm not planning any code changes for 
3.8. Instead I only like to document a bunch of modules as deprecated. Active 
deprecation is planned for 3.9 and removal for 3.10. The long deprecation phase 
gives us 3 years to change our minds or handle edge cases, too.

Regards,
Christian



PEP: 594
Title: Removing dead batteries from the standard library
Author: Christian Heimes 
Status: Active
Type: Process
Content-Type: text/x-rst
Created: 20-May-2019
Post-History:


Abstract


This PEP proposed a list of standard library modules to be removed from the
standard library. The modules are mostly historic data formats and APIs that
have been superseded a long time ago, e.g. Mac OS 9 and Commodore.


Rationale
=

Back in the early days of Python, the interpreter came with a large set of
useful modules. This was often refrained to as "batteries included"
philosophy and was one of the corner stones to Python's success story.
Users didn't have to figure out how to download and install separate
packages in order to write a simple web server or parse email.

Times have changed. The introduction of the cheese shop (PyPI), setuptools,
and later pip, it became simple and straight forward to download and install
packages. Nowadays Python has a rich and vibrant ecosystem of third party
packages. It's pretty much standard to either install packages from PyPI or
use one of the many Python or Linux distributions.

On the other hand, Python's standard library is piling up cruft, unnecessary
duplication of functionality, and dispensable features. This is undesirable
for several reasons.

* Any additional module increases the maintenance cost for the Python core
  development team. The team has limited resources, reduced maintenance cost
  frees development time for other improvements.
* Modules in the standard library are generally favored and seen as the
  de-facto solution for a problem. A majority of users only pick 3rd party
  modules to replace a stdlib module, when they have a compelling reason, e.g.
  lxml instead of `xml`. The removal of an unmaintained stdlib module
  increases the chances of a community contributed module to become widely
  used.
* A lean and mean standard library benefits platforms with limited resources
  like devices with just a few hundred kilobyte of storage (e.g. BBC
  Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
  (e.g. pyodide) also benefit from reduced download size.

The modules in the PEP have been selected for deprecation because their
removal is either least controversial or most beneficial. For example
least controversial are 30 years old multimedia formats like ``sunau``
audio format, which was used on SPARC and NeXT workstations in the late
1980ties. The ``crypt`` module has fundamental flaws that are better solved
outside the standard library.

This PEP also designates some modules as not scheduled for removal. Some
modules have been deprecated for several releases or seem unnecessary at
first glance. However it is beneficial to keep the modules in the standard
library, mostly for environments where installing a package from PyPI is not
an option. This can be cooperate environments or class rooms where external
code is not permitted without legal approval.

* The usage of FTP is declining, but some files are still provided over
  the FTP protocol or hosters offer FTP to upload content. Therefore
  ``ftplib`` is going to stay.
* The ``optparse`` and ``getopt`` module are widely used. They are mature
  modules with very low maintenance overhead.
* According to David Beazley [5]_ the ``wave`` module is easy to teach to
  kids and can make crazy sounds. Making a computer generate crazy sounds is
  powerful and highly motivating exercise for a 9yo aspiring developer. It's
  a fun battery to keep.


Deprecation schedule


3.8
---

This PEP targets Python 3.8. Version 3.8.0 final is scheduled to be released
a few months before Python 2.7 will reach its end of lifetime. We expect that
Python 3.8 will be targeted by users that migrate to Python 3 in 2019 and
2020. To reduce churn and to allow a smooth transition from Python 2,
Python 3.8 will neither raise `DeprecationWarning` nor remove any
modules that have been scheduled for removal. Instead deprecated modules will
just be *documented* as deprecated. Optionally modules may emit a
`PendingDeprecationWarning`.

All deprecated modules will also undergo a feature freeze. No additional
features should be added. Bug should still be fixed.

3.9
---

Starting with Python 3.9, deprecated modules will start issuing
`DeprecationWarning`.


3.10


In 3.10