Re: [Python-Dev] Process to remove a Python feature

[Chris Angelico]

On Wed, May 9, 2018 at 2:53 AM, Serhiy Storchaka  wrote:
> 08.05.18 19:10, Ethan Furman пише:
>>>
>>> X.Y+1: added a deprecation warning. Many users need to support only two
>>> recent versions and can move to using the
>>> replacement now.
>>
>>
>> I'm curious how you arrived at this conclusion?  I know I've only worked
>> at two different Python-using companies, but neither aggressively tracks the
>> latest Python minor version, and as a library author I support more than the
>> two most recent versions.
>
>
> Maybe I was too optimistic. ;-)  Libraries need to support more Python
> versions of course. But two versions is a minimum, and I thing that for many
> applications (if they are targeted to the specific OS version or shipped
> with an own Python) this is enough. Even if their number is not large, they
> will get a benefit from introducing a replacement before adding a warning.
>
> If you support versions X.Y-1 and X.Y, you just use the old feature. If you
> support versions X.Y and X.Y+1, you replace it with the new feature. If you
> support versions X.Y-1, X.Y and X.Y+1 you need either to ignore varnings, or
> to add a runtime check for switching between using the old and the new
> feature. This complicates the code. But at least the code is clear in the
> first two cases.

Here in this house, we have:

* 3.8, with or without various patches as are being proposed upstream
* 3.7, a slightly old alpha build, as a secondary on the laptop
* 3.6 on the latest Ubuntu
* 3.5 on the Raspberry Pi
* 3.4 as shipped by Debian, as the laptop's primary Python 3
* And I'm not even counting the various different 2.7s.

My brother and I built a TCP-managed doorbell involving the rPi. At an
absolute minimum, it has to support 3.4, 3.5, and 3.6; and supporting
3.7 is important, given how close it is to release. Supporting 3.8 is
no harder than supporting 3.7, but anything that did actual version
number checks would need to be aware of it. So that's potentially five
different versions. (Fortunately, I have all five installed on one
computer, so testing isn't hard.)

Supporting just two versions seems a bit too hopeful. Supporting three
would be a minimum for an in-house app; and if your users run
different Linux distros with different release cadences, four wouldn't
be unlikely, even among current releases.

As a general rule, I prefer to avoid writing "before" and "after"
code. For a lot of deprecations, that's easy - when "await" started
becoming a keyword, I renamed a function to "wait" [1], and it was
100% compatible with all versions. If there absolutely has to be code
doing two different things, I'd rather catch ImportError than do
actual version checks. But if it has to check versions, it'll need to
be aware of quite a few.

ChrisA
[1] 
https://github.com/Rosuav/LetMeKnow/commit/2ecbbdcc3588139932525140ceb8c2cb66930284
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Serhiy Storchaka]

08.05.18 19:10, Ethan Furman пише:
X.Y+1: added a deprecation warning. Many users need to support only 
two recent versions and can move to using the

replacement now.


I'm curious how you arrived at this conclusion?  I know I've only worked 
at two different Python-using companies, but neither aggressively tracks 
the latest Python minor version, and as a library author I support more 
than the two most recent versions.


Maybe I was too optimistic. ;-)  Libraries need to support more Python 
versions of course. But two versions is a minimum, and I thing that for 
many applications (if they are targeted to the specific OS version or 
shipped with an own Python) this is enough. Even if their number is not 
large, they will get a benefit from introducing a replacement before 
adding a warning.


If you support versions X.Y-1 and X.Y, you just use the old feature. If 
you support versions X.Y and X.Y+1, you replace it with the new feature. 
If you support versions X.Y-1, X.Y and X.Y+1 you need either to ignore 
varnings, or to add a runtime check for switching between using the old 
and the new feature. This complicates the code. But at least the code is 
clear in the first two cases.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Serhiy Storchaka]

04.05.18 22:08, Matthias Bussonnier пише:
Maybe to push people forward, but from experience it is hard to predict 
future, so saying when

it _will_ be remove is hard.


Right. But the data of removing is usually specified when the code for 
removing already is written, or even merged in the next branch. This is 
true for most warnings introduced in 3.7 that warns about the removal in 
3.8. This is true for Py3k warnings backported to 2.7.


In short-term deprecations you have the only one release in which the 
warning is emitted.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Steven D'Aprano]

On Tue, May 08, 2018 at 03:20:50PM +0200, Stéphane Blondon wrote:

> I wonder if it would be interesting to have a package available by
> pypi.org which would provide the removed features.
[...]
> Cons:
>  - it pushes the question to how many times the previous behavior should
> be maintained from python language to 'obsolete' package. So it's not
> completely solved.

Its not solved at all.

Who is responsible for moving code to this third-party package?

Who is responsible for maintaining that third-party package?

If it is the core developers, then it isn't really removed at all.

If nobody is maintaining it, then its going to suffer bitrot and become 
unusable.

Python is open source, so anyone can fork a module or library from the 
std lib and move it to PyPI, they don't need the core dev's blessing, 
they just need to follow the licence requirements. 


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Serhiy Storchaka]

08.05.18 18:51, Joni Orponen пише:

If one can import from the future can one can also import from the past?


One can move removed feature to a third-party module and import them 
from it.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Ethan Furman]

On 05/04/2018 11:48 AM, Serhiy Storchaka wrote:

04.05.18 20:57, Matthias Bussonnier пише:



But when I hit a DeprecationWarning message there is one crucial piece of
information missing most of the time: Since which version number it's deprecated
(and sometime since when the replacement is available could be good if overlap
between functionality there was).


I think the information about since which version number it will be removed is 
more useful. Different cases need
different deprecation periods. The more common the case, the longer deprecation 
period should be. Some recently added
warnings contain this information.


If we are going to provide extra metadata, we may as well supply it all.  Some folks will need the start date, some the 
end date, and probably some both dates.




X.Y+1: added a deprecation warning. Many users need to support only two recent 
versions and can move to using the
replacement now.


I'm curious how you arrived at this conclusion?  I know I've only worked at two different Python-using companies, but 
neither aggressively tracks the latest Python minor version, and as a library author I support more than the two most 
recent versions.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Joni Orponen]

On Tue, May 8, 2018 at 3:20 PM, Stéphane Blondon  wrote:

> Le 02/05/2018 à 11:11, Victor Stinner a écrit :
> > * Communicate on python-dev, Twitter, Python Insider blog, etc.
> > * Collaborate with major Python projects to help them to migrate the
> alternative
>
> I wonder if it would be interesting to have a package available by
> pypi.org which would provide the removed features. In your example, the
> developers would have to update their source code:
>
> # giving 'obsolete' as name for this package
> platform.linux_distribution() -> obsolete.platform.linux_distribution()


If one can import from the future can one can also import from the past?

-- 
Joni Orponen
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Stéphane Blondon]

Le 02/05/2018 à 11:11, Victor Stinner a écrit :
> * Communicate on python-dev, Twitter, Python Insider blog, etc.
> * Collaborate with major Python projects to help them to migrate the 
> alternative

I wonder if it would be interesting to have a package available by
pypi.org which would provide the removed features. In your example, the
developers would have to update their source code:

# giving 'obsolete' as name for this package
platform.linux_distribution() -> obsolete.platform.linux_distribution()


The code in 'obsolete' package could come from the removed code from
cpython or a wrapper around a third-party package ('distro' package in
the example).

Plus:
 - quick temporary fix for users -> the removal is less painful
 - the name of the import is a hint that something has to be fixed ->
useful for new comers on the user source code

Cons:
 - it pushes the question to how many times the previous behavior should
be maintained from python language to 'obsolete' package. So it's not
completely solved.
 - it adds a step to removal procedure.
 - I guess there are some features not movable into a package.

-- 
Stéphane



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Nick Coghlan]

On 5 May 2018 at 07:59, Terry Reedy  wrote:

> On 5/2/2018 5:11 AM, Victor Stinner wrote:
>
> As a follow-up to the "[Python-Dev] (Looking for) A Retrospective on
>> the Move to Python 3" thread, I will like to clarify how a feature
>> should be removed from Python.
>>
>
> Would it be possible (and sensible) to use the 2to3 machinery to produce
> 36to37.py, etc., to do mechanical replacements when possible and flag other
> things when necessary?
>

That capability actually exists in the python-future project (in the form
of their "pasteurize" script, which takes idiomatic Python 3 code and turns
it into Python 2.7 compatible code).

One major issue though is that without a JIT compiler to speed things back
up, the polyfills needed to target old versions in the general case (rather
than when restricting yourself to a "backport friendly Python subset") are
often either going to be unacceptably slow, or else unacceptably irritating
to maintain, so folks have invested their time into figuring out how to
ship their own runtimes in order to avoid being restricted by
redistributors with slow runtime update cycles, and in being able to rely
on PyPI packages rather than standard library packages. (That's very
different from the incentives for browser vendors, who have users that
expect them to be able to make arbitrary websites work, and blame the
browser rather than the site when particular sites they want to use don't
work properly)

The other thing worth keeping in mind is that one of the core assumptions
of JavaScript tooling is that debugging tools will be able to reach out to
the internet and download additional files (most notably source maps) to
help make sense of the currently running code. Without that ability to
trace from compiled code back to the preferred source form for editing,
making sense of an otherwise straightforward traceback can become a complex
debugging exercise.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Steve Dower]

To be fair, if they hadn’t already been deprecated we would have had to 
deprecate the old behaviour for a couple of releases before changing it. It 
just so happened that we did the deprecation first without knowing what the fix 
would be :)

Top-posted from my Windows phone

From: Victor Stinner
Sent: Friday, May 4, 2018 15:24
To: Serhiy Storchaka
Cc: python-dev
Subject: Re: [Python-Dev] Process to remove a Python feature

2018-05-04 20:48 GMT+02:00 Serhiy Storchaka <storch...@gmail.com>:
> I think the information about since which version number it will be removed
> is more useful.

About deprecation, there is the funny story of bytes filenames on
Windows. I deprecated this feature in Windows 3.3 since it was broken.
I really wanted hard to remove support for bytes filenames on Windows.

One day, Steve Dower showed up with the PEP 529 "Change Windows
filesystem encoding to UTF-8" and he just *removed the deprecation
warning* from Python 3.6, since bytes filenames are now working as
intended :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Terry Reedy]

On 5/4/2018 6:18 PM, Victor Stinner wrote:

2018-05-04 23:59 GMT+02:00 Terry Reedy :

Would it be possible (and sensible) to use the 2to3 machinery to produce
36to37.py, etc., to do mechanical replacements when possible and flag other
things when necessary?


I suggest you to watch Daniele Esposti's talk "Evolution or stagnation
programming languages". He explains that Javascript is more successful
than Python to introduce *language* evolutions thanks to transpiling
(things like babel and polyfill):
https://www.pycon.it/conference/talks/evolution-or-stagnation-programming-languages


I ran through the slides and found the babelsite.  What I found: Babel 
translates new code back to a sufficiently powerful and presumably 
ubiquitous older version.  It does so on a selectable feature basis 
rather than a language version basis.  (In other words, define your own 
'new' version.) Polyfill supplies the backported new objects needed to 
make the back translations run with new semantics.


This would be equivalent of defining, for instance, 2.7 as a base 
version and having 3xbytesto27, 35asyncto27, etc for every new 3.x 
feature.  Some people wanted this, but, of course, 2.7 is *not* 
installed everywhere.  If Microsoft were to treat Python like it once 
did Basic, and install it on all Windows machines, it would start with 
recent 3.x.


Neither the slides nor site said anything about bug fixes and about the 
need to have multiple versions of every function touched.


Because of the unique features of how Javascript is distributed and 
used, I don't see how the Babel example would apply very well to Python.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Victor Stinner]

2018-05-04 20:48 GMT+02:00 Serhiy Storchaka :
> I think the information about since which version number it will be removed
> is more useful.

About deprecation, there is the funny story of bytes filenames on
Windows. I deprecated this feature in Windows 3.3 since it was broken.
I really wanted hard to remove support for bytes filenames on Windows.

One day, Steve Dower showed up with the PEP 529 "Change Windows
filesystem encoding to UTF-8" and he just *removed the deprecation
warning* from Python 3.6, since bytes filenames are now working as
intended :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Victor Stinner]

2018-05-04 23:59 GMT+02:00 Terry Reedy :
> Would it be possible (and sensible) to use the 2to3 machinery to produce
> 36to37.py, etc., to do mechanical replacements when possible and flag other
> things when necessary?

I suggest you to watch Daniele Esposti's talk "Evolution or stagnation
programming languages". He explains that Javascript is more successful
than Python to introduce *language* evolutions thanks to transpiling
(things like babel and polyfill):
https://www.pycon.it/conference/talks/evolution-or-stagnation-programming-languages

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Terry Reedy]

On 5/2/2018 5:11 AM, Victor Stinner wrote:


As a follow-up to the "[Python-Dev] (Looking for) A Retrospective on
the Move to Python 3" thread, I will like to clarify how a feature
should be removed from Python.


Would it be possible (and sensible) to use the 2to3 machinery to produce 
36to37.py, etc., to do mechanical replacements when possible and flag 
other things when necessary?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Brett Cannon]

On Fri, 4 May 2018 at 12:09 Matthias Bussonnier <
bussonniermatth...@gmail.com> wrote:

> On Fri, 4 May 2018 at 11:49, Serhiy Storchaka  wrote:
>
>> 04.05.18 20:57, Matthias Bussonnier пише:
>> > But when I hit a DeprecationWarning message there is one crucial piece
>> of
>> > information missing most of the time: Since which version number it's
>> > deprecated
>> > (and sometime since when the replacement is available could be good if
>> > overlap
>> > between functionality there was).
>>
>> I think the information about since which version number it will be
>> removed is more useful. Different cases need different deprecation
>> periods. The more common the case, the longer deprecation period should
>> be. Some recently added warnings contain this information.
>>
>
> Maybe to push people forward, but from experience it is hard to predict
> future, so saying when
> it _will_ be remove is hard.  When you _want_ to remove it probably.
> Victor's first mail in
> this thread is a good example. The functionality was marked to be removed
> from 3.7, but I think
> it is likely too late now. You can always update, but I hate giving
> differing information between software version.
>

So there is actually an opportunity here to programmatically help prevent
missing a removal. If we attach a version number for when a
DeprecationWarning is expected to go away due to removal we can then have
the warning itself raise a warning that while the DeprecationWarning was
supposed to be gone by e.g. 3.7, it's still being raised. The drawback to
this is it will make the first cut-over to a newer version of Python a bit
more painful, but as long as warnings don't raise exceptions it should just
be a noisy.


>
> I'm curious about your use case for the version of removal,
> I usually don't care when it's going to be removed, I prefer since
> when the functionality is deprecated.
>
> if pyversion < deprecated_version:
>   old_stuff
> else:
>   new_stuff
>
> And I soon as my project drop deprecated_version, I remove the
> conditional.
> I do not try/except on purpose to be able to grep for when to remove the
> code.
>

The other way to look at that is:

if pyversion < replacement:
  old_stuff
else:
  new_stuff

In which case you don't care about when the deprecation happened, just when
it's replacement was introduced (which Serhiy suggested being a version
before the deprecation is added).


> Could you share you use case ? Of be ore detailed ?
>
> We can also be more generic and say that if DeprecationWarning messages
> could contain
> timeline informations it would likely encourage the migration.
>
> One related question is how much are DeprecationWarning messages stables
> between versions ?
>

Same as any other exception; typically left alone unless there's a better
way to phrase things (IOW don't parse the message ;) .

So at this point there are basically two threads going on here.

One is an official deprecation policy. E.g. X.Y something happens, X.Y+1
something else happens, etc.

The other one is how to make what gets deprecated more discoverable. E.g.
an informational PEP that tracks what's planned, warnings having more
metadata, etc.

Both of these things seem independent, although one of them will require a
PEP while the other just takes some work (I leave it as an exercise for the
reader to figure out which one is which ;) .

-Brett


> Would any update to many of these be accepted of refused because users
> might be filtering them ?
> --
> Matthias
>
>
>>
>> Ideally any deprecated feature should have a replacement, and this
>> replacement should be available in at least one version before adding
>> the deprecation warning.
>>
>> X.Y: added a replacement
>>
>> X.Y+1: added a deprecation warning. Many users need to support only two
>> recent versions and can move to using the replacement now.
>>
>> X.Y+3 (or X.Y+2): removed the deprecated feature. Versions older than
>> X.Y should grew out of use at that moment.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>>
> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/bussonniermatthias%40gmail.com
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Matthias Bussonnier]

On Fri, 4 May 2018 at 11:49, Serhiy Storchaka  wrote:

> 04.05.18 20:57, Matthias Bussonnier пише:
> > But when I hit a DeprecationWarning message there is one crucial piece of
> > information missing most of the time: Since which version number it's
> > deprecated
> > (and sometime since when the replacement is available could be good if
> > overlap
> > between functionality there was).
>
> I think the information about since which version number it will be
> removed is more useful. Different cases need different deprecation
> periods. The more common the case, the longer deprecation period should
> be. Some recently added warnings contain this information.
>

Maybe to push people forward, but from experience it is hard to predict
future, so saying when
it _will_ be remove is hard.  When you _want_ to remove it probably.
Victor's first mail in
this thread is a good example. The functionality was marked to be removed
from 3.7, but I think
it is likely too late now. You can always update, but I hate giving
differing information between software version.

I'm curious about your use case for the version of removal,
I usually don't care when it's going to be removed, I prefer since
when the functionality is deprecated.

if pyversion < deprecated_version:
  old_stuff
else:
  new_stuff

And I soon as my project drop deprecated_version, I remove the conditional.
I do not try/except on purpose to be able to grep for when to remove the
code.

Could you share you use case ? Of be ore detailed ?

We can also be more generic and say that if DeprecationWarning messages
could contain
timeline informations it would likely encourage the migration.

One related question is how much are DeprecationWarning messages stables
between versions ?
Would any update to many of these be accepted of refused because users
might be filtering them ?
-- 
Matthias


>
> Ideally any deprecated feature should have a replacement, and this
> replacement should be available in at least one version before adding
> the deprecation warning.
>
> X.Y: added a replacement
>
> X.Y+1: added a deprecation warning. Many users need to support only two
> recent versions and can move to using the replacement now.
>
> X.Y+3 (or X.Y+2): removed the deprecated feature. Versions older than
> X.Y should grew out of use at that moment.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/bussonniermatthias%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Guido van Rossum]

No, the reason they're hidden by default is that for most users they're not
actionable most of the time.

On Fri, May 4, 2018 at 11:55 AM, Nathaniel Smith  wrote:

> On Fri, May 4, 2018, 11:50 Serhiy Storchaka  wrote:
>
>>
>> Ideally any deprecated feature should have a replacement, and this
>> replacement should be available in at least one version before adding
>> the deprecation warning.
>>
>> X.Y: added a replacement
>>
>> X.Y+1: added a deprecation warning. Many users need to support only two
>> recent versions and can move to using the replacement now.
>>
>
> Isn't the whole point of making DeprecationWarnings hidden by default that
> it lets us add the new thing and deprecate the old thing in the same
> release, without creating too much disruption?
>
> -n
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Nathaniel Smith]

On Fri, May 4, 2018, 11:50 Serhiy Storchaka  wrote:

>
> Ideally any deprecated feature should have a replacement, and this
> replacement should be available in at least one version before adding
> the deprecation warning.
>
> X.Y: added a replacement
>
> X.Y+1: added a deprecation warning. Many users need to support only two
> recent versions and can move to using the replacement now.
>

Isn't the whole point of making DeprecationWarnings hidden by default that
it lets us add the new thing and deprecate the old thing in the same
release, without creating too much disruption?

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Serhiy Storchaka]

04.05.18 20:57, Matthias Bussonnier пише:

But when I hit a DeprecationWarning message there is one crucial piece of
information missing most of the time: Since which version number it's 
deprecated
(and sometime since when the replacement is available could be good if 
overlap

between functionality there was).


I think the information about since which version number it will be 
removed is more useful. Different cases need different deprecation 
periods. The more common the case, the longer deprecation period should 
be. Some recently added warnings contain this information.


Ideally any deprecated feature should have a replacement, and this 
replacement should be available in at least one version before adding 
the deprecation warning.


X.Y: added a replacement

X.Y+1: added a deprecation warning. Many users need to support only two 
recent versions and can move to using the replacement now.


X.Y+3 (or X.Y+2): removed the deprecated feature. Versions older than 
X.Y should grew out of use at that moment.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Matthias Bussonnier]

I would like to take a step back,

I think that for removal of feature you need to make sure that the "old
way" is
not common enough. I think that in many context, users of Python see
DeprecationWarnings as a stick. A deprecation warning means you'll have to
do
some work. A pep or a document that list all the removal may be useful, but
I'm
unsure if in reading what's knew you'll catch all the removal of
functionality
you use and convince yourself (or your management) to thouroughly go through
your code base and fix these things. Maybe to help transition you need to
soften
the stick by being more explicit as since when things are deprecated and
what can
be done.

Personally I find deprecations when I run test-suites and use libraries
interactively, not by reading ahead of time documents. Python has made huge
progress in the deprecation messages. And the usage of "X Is is deprecated,
use
Y" has become common, this is already tremendously more helpful than just
"X is
deprecated".

But when I hit a DeprecationWarning message there is one crucial piece of
information missing most of the time: Since which version number it's
deprecated
(and sometime since when the replacement is available could be good if
overlap
between functionality there was). Yes it is available in the docs (or in the
docstring), but if I have to switch context to search in the documentation I
might not bother. In particular when the version is in the deprecation
message,
as I know the python requirement of my project, I can immediately know if
it's a
search and replace away, or if I need to have conditional branches.

That is at least to me the biggest change that would push me to update the
deprecated features in my own project, and make me do drive by
contributions on
Open-Source project I use where I spotted one of those.

There are other improvement possible to deprecation messages, but I think
this
one would have the largest impact for many developers who do not closely
follow
python development, and for whom DeprecationWarnings are just an extra
hassle.

Also if there is a standard, or conventional way _in warnings_ to say since
when
something is deprecated, then it's relatively easy to search for it when
version N+p is
released and remove what you believe is old enough.

Thanks,
-- 
Matthias


On Fri, 4 May 2018 at 09:03, Serhiy Storchaka  wrote:

> 04.05.18 16:14, Steven D'Aprano пише:
> > Have there been many features removed since 3.1? I know there were some
> > features removed in 3.0, like callable(), which were later put back in,
> > but I can't think of anythin removed since then. If there were, the pace
> > of it is pretty slow.
>
> Read "What's New" documents, sections "Removed" and "Porting to Python
> X.Y". There is a long list in every release.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/bussonniermatthias%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Serhiy Storchaka]

04.05.18 16:14, Steven D'Aprano пише:

Have there been many features removed since 3.1? I know there were some
features removed in 3.0, like callable(), which were later put back in,
but I can't think of anythin removed since then. If there were, the pace
of it is pretty slow.


Read "What's New" documents, sections "Removed" and "Porting to Python 
X.Y". There is a long list in every release.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Steven D'Aprano]

On Fri, May 04, 2018 at 12:56:57PM +0200, Victor Stinner wrote:

> Sorry, when I wrote "Python 4" I mean "the new Python release which
> introduces a lot of backward incompatible changes and will annoy
> everyone". It can be Python 3.9 or 3.10, or whatever version
> (including 4.3 if you want) :-)

I call that "Python 4000", in analogy with Python 3000 which became 
Python 3, and to further emphasise how far away it is, I've started 
calling it "Python 5000".

As I understand it, Guido has said that we won't be doing a repeat of 
the 2 to 3 break-a-lot-of-stuff-at-once transition unless there is some 
unforeseen necessity.


> My point is that deprecating a feature is one thing, removing it is
> something else.
> 
> We should slow down feature removal, or more generally reduce the
> number of backward incompatible changes per release.

Have there been many features removed since 3.1? I know there were some 
features removed in 3.0, like callable(), which were later put back in, 
but I can't think of anythin removed since then. If there were, the pace 
of it is pretty slow.


> Maybe keep a deprecating warning for 10 years is just fine.
> 
> Extract of the Zen of Python: "Although practicality beats purity." ;-)

Aside from things that need to be removed for security reasons, this 
seems good to me.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Victor Stinner]

2018-05-02 19:24 GMT+02:00 Brett Cannon :
> On Wed, 2 May 2018 at 02:12 Victor Stinner  wrote:
>> Does it mean that the Python 3 release following Python 2 end-of-life
>> (2020) will be our next feared "Python 4"? Are we going to remove all
>> deprecated features at once, to maximize incompatibilities and make
>> users unhappy?
>
> I don't see why removing features that already raise a DeprecationWarning
> would require bumping the major version number. Personally, I assumed either
> Python 3.9 or 3.10 would be the version where we were okay clearing out the
> stuff that had been raising DeprecationWarning for years.

Sorry, when I wrote "Python 4" I mean "the new Python release which
introduces a lot of backward incompatible changes and will annoy
everyone". It can be Python 3.9 or 3.10, or whatever version
(including 4.3 if you want) :-)

My point is that deprecating a feature is one thing, removing it is
something else.

We should slow down feature removal, or more generally reduce the
number of backward incompatible changes per release.

Maybe keep a deprecating warning for 10 years is just fine.

Extract of the Zen of Python: "Although practicality beats purity." ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Brett Cannon]

On Thu, 3 May 2018 at 12:01 Facundo Batista 
wrote:

> 2018-05-02 14:24 GMT-03:00 Brett Cannon :
>
> >> Maybe we should create a tool to list features scheduled for removal,
> >> and open a discussion to check each removal?
> >
> > I don't know if a tool is necessary. We could have a meta issue or text
> file
> > somewhere to track what's to be removed in a certain version.
>
> Maybe a specific PEP that list all removals that happened, those that
> will happen next version, and next ones after that?
>

I don't know if we need a history section as the What's New docs cover
that. But having a PEP or some other file/doc that acts like a future
What's New but for removals seems like a reasonable idea.

-Brett


>
> IOW, a single point where one can see which features/details will fly
> away and when.
>
> Probably it's a good idea to see everything at once and evaluate if
> "we're removing too many things at once", and have a better
> deprecation/removal cadence.
>
> --
> .Facundo
>
> Blog: http://www.taniquetil.com.ar/plog/
> PyAr: http://www.python.org/ar/
> Twitter: @facundobatista
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Facundo Batista]

2018-05-02 14:24 GMT-03:00 Brett Cannon :

>> Maybe we should create a tool to list features scheduled for removal,
>> and open a discussion to check each removal?
>
> I don't know if a tool is necessary. We could have a meta issue or text file
> somewhere to track what's to be removed in a certain version.

Maybe a specific PEP that list all removals that happened, those that
will happen next version, and next ones after that?

IOW, a single point where one can see which features/details will fly
away and when.

Probably it's a good idea to see everything at once and evaluate if
"we're removing too many things at once", and have a better
deprecation/removal cadence.

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Twitter: @facundobatista
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Brett Cannon]

On Wed, 2 May 2018 at 02:12 Victor Stinner  wrote:

> Hi,
>
> As a follow-up to the "[Python-Dev] (Looking for) A Retrospective on
> the Move to Python 3" thread, I will like to clarify how a feature
> should be removed from Python.
>
> We have multiple tools:
>
> * Emit a PendingDeprecationWarning warning at runtime for *slow*
> deprecation (remove a feature in at least 3 cycles)
> * Emit a DeprecationWarning warning at runtime for fast deprecation
> (remove a feature in 2 cycles)
> * Document that a feature is deprecated in the module documentation
> * "What's New in Python X.Y?" documents: especially Deprecated,
> Removed and Porting to Python X.Y sections.
> * Communicate on python-dev, Twitter, Python Insider blog, etc.
> * Collaborate with major Python projects to help them to migrate the
> alternative
>
> IMHO a feature should not be removed if there is no alternative, or if
> the alternative is too new.
>
> The usual process is:
>
> * Document the deprecation and emit DeprecationWarning in version N
> * Document the removal and remove the feature in version N+1
>
> Slow deprecation:
>
> * Document the deprecation and emit PendingDeprecationWarning in version N
> * Emit a DeprecationWarning in version N+1
> * Document the removal and remove the feature in version N+2
>
> The hidden ghost is the old rule:
>
>"keep old APIs around to make porting from Python 2 easier"
>
> Is this rule still applicable?
>

Python 2 is still supported, so I assume so.


>
> Does it mean that the Python 3 release following Python 2 end-of-life
> (2020) will be our next feared "Python 4"? Are we going to remove all
> deprecated features at once, to maximize incompatibilities and make
> users unhappy?
>

I don't see why removing features that already raise a DeprecationWarning
would require bumping the major version number. Personally, I assumed
either Python 3.9 or 3.10 would be the version where we were okay clearing
out the stuff that had been raising DeprecationWarning for years.


>
> Should we always document in which version a feature will be removed?
>

We should at least open an issue to track when the removal is scheduled.
But if we know ahead of time then I so no reason not to document it.


> Some features are deprecated since many versions, and the deprecated
> features are still there. In most cases, it's because of the Python 2
> rule.
>
> Sometimes, we forget to remove features which has been scheduled for
> removal in a specific version.
>

Right, which is why we should open an issue immediately. I had a "remove
pyvenv" issue open and assigned to myself for years while I waited for
Python 3.8 development to open. Staring at that issue for so long made sure
I didn't forget. ;)


>
> Maybe we should create a tool to list features scheduled for removal,
> and open a discussion to check each removal?
>

I don't know if a tool is necessary. We could have a meta issue or text
file somewhere to track what's to be removed in a certain version.


>
> Ten years ago, I wanted to remove most modules and many functions from
> the standard library. Now my experience showed me that *each* removal
> is very painful, hurt more projects than expected, and takes longer
> than 3 years to be fully effective (no longer used in most common 3rd
> party modules).
>
> The usual issue is to write a single code base working on all Python
> versions. For example, the new alternative is only available on recent
> Python versions, which requires to have at least two code paths to
> support all versions. Sometimes, there are 3 code paths...
>
> For a recent example, see "remove platform.linux_distribution()":
> https://bugs.python.org/issue28167
>
> Removing a feature from the C API is more complex, since there is no
> portable way to emit a deprecation warning at compilation. There is
> Py_DEPRECATED() which seems to only be available on GCC (3.1 and
> newer).
>
> Maybe we should stop to remove features, except if there is really a
> good reason to do that?
>

I thought that already was the policy. ;) I think the real question is what
is people's definition of a "good reason". For instance, I'm all for
removing unsupported code so we don't have to maintain it, even if it's
just from code modernization and such (i.e. pruning down the stdlib). But I
also know others disagree with me and are fine just having modules sit
there as long as the issue rate is low enough to not notice. It's just one
of those things we don't have an official policy on (yet?).

-Brett


>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 

Re: [Python-Dev] Process to remove a Python feature

[Victor Stinner]

2018-05-02 11:17 GMT+02:00 Antoine Pitrou :
> It's at least possible with gcc, clang and MSVC:
> https://stackoverflow.com/questions/295120/c-mark-as-deprecated/295229#295229
>
> You can even add a deprecation message.

Aha, I opened an issue: https://bugs.python.org/issue33407 "Implement
Py_DEPRECATED() macro for Visual Studio"

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Process to remove a Python feature

[Antoine Pitrou]

On Wed, 2 May 2018 11:11:16 +0200
Victor Stinner  wrote:
> 
> Removing a feature from the C API is more complex, since there is no
> portable way to emit a deprecation warning at compilation. There is
> Py_DEPRECATED() which seems to only be available on GCC (3.1 and
> newer).

It's at least possible with gcc, clang and MSVC:
https://stackoverflow.com/questions/295120/c-mark-as-deprecated/295229#295229

You can even add a deprecation message.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Process to remove a Python feature

[Victor Stinner]

Hi,

As a follow-up to the "[Python-Dev] (Looking for) A Retrospective on
the Move to Python 3" thread, I will like to clarify how a feature
should be removed from Python.

We have multiple tools:

* Emit a PendingDeprecationWarning warning at runtime for *slow*
deprecation (remove a feature in at least 3 cycles)
* Emit a DeprecationWarning warning at runtime for fast deprecation
(remove a feature in 2 cycles)
* Document that a feature is deprecated in the module documentation
* "What's New in Python X.Y?" documents: especially Deprecated,
Removed and Porting to Python X.Y sections.
* Communicate on python-dev, Twitter, Python Insider blog, etc.
* Collaborate with major Python projects to help them to migrate the alternative

IMHO a feature should not be removed if there is no alternative, or if
the alternative is too new.

The usual process is:

* Document the deprecation and emit DeprecationWarning in version N
* Document the removal and remove the feature in version N+1

Slow deprecation:

* Document the deprecation and emit PendingDeprecationWarning in version N
* Emit a DeprecationWarning in version N+1
* Document the removal and remove the feature in version N+2

The hidden ghost is the old rule:

   "keep old APIs around to make porting from Python 2 easier"

Is this rule still applicable?

Does it mean that the Python 3 release following Python 2 end-of-life
(2020) will be our next feared "Python 4"? Are we going to remove all
deprecated features at once, to maximize incompatibilities and make
users unhappy?

Should we always document in which version a feature will be removed?
Some features are deprecated since many versions, and the deprecated
features are still there. In most cases, it's because of the Python 2
rule.

Sometimes, we forget to remove features which has been scheduled for
removal in a specific version.

Maybe we should create a tool to list features scheduled for removal,
and open a discussion to check each removal?

Ten years ago, I wanted to remove most modules and many functions from
the standard library. Now my experience showed me that *each* removal
is very painful, hurt more projects than expected, and takes longer
than 3 years to be fully effective (no longer used in most common 3rd
party modules).

The usual issue is to write a single code base working on all Python
versions. For example, the new alternative is only available on recent
Python versions, which requires to have at least two code paths to
support all versions. Sometimes, there are 3 code paths...

For a recent example, see "remove platform.linux_distribution()":
https://bugs.python.org/issue28167

Removing a feature from the C API is more complex, since there is no
portable way to emit a deprecation warning at compilation. There is
Py_DEPRECATED() which seems to only be available on GCC (3.1 and
newer).

Maybe we should stop to remove features, except if there is really a
good reason to do that?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com