Re: [python-committers] Deprecation Policy PEP

[Ezio Melotti]

On Sat, Jan 30, 2016 at 1:39 AM, Barry Warsaw  wrote:
> On Jan 29, 2016, at 06:11 PM, Brett Cannon wrote:
>
>>+1 from me as well, especially once Serhiy's comments are addressed.
>
> Me too, but only if you add a PendingDeprecationWarning to
> PendingDeprecationWarning .
>

The original plan actually was:
"""
We could also introduce a MetaDeprecationWarning and make
PendingDeprecationWarning inherit from it so that it can be used to
pending-deprecate itself.  Once PendingDeprecationWarning is gone, the
MetaDeprecationWarning will become useless and can then be used to
meta-deprecate itself.
"""

> depreception-ly y'rs,
> -Barry
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Ezio Melotti]

On Sat, Jan 30, 2016 at 5:36 AM, Guido van Rossum  wrote:
> Following the lead of 2.7.10 and 2.7.11 we could continue with 3.10, 3.11, 
> etc.
>

I think we should continue with 3.10, 3.11, etc.
Changing the major version should be done for incompatible changes,
and just doing it after 3.9 will probably just create confusion for
both users that will wonder if it's incompatible with Python 3 and for
things like the executable name.
Hopefully we won't need to jump to Python 4 for a long time.

> I also want the 3->4 transition to feel like a non-event for most
> users. How we'll do that I don't know yet, but I want it to be a lot
> smoother than 2->3.
>
> --
> --Guido van Rossum (python.org/~guido)
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Ezio Melotti]

On Fri, Jan 29, 2016 at 11:59 PM, Serhiy Storchaka  wrote:
> On 29.01.16 21:56, Ezio Melotti wrote:
>>
>> On Fri, Jan 29, 2016 at 8:00 PM, Serhiy Storchaka 
>> wrote:
>>>
>>> What about adding deprecations in bugfix releases? If current behavior is
>>> obviously incorrect and should be fixed in development branch, but this
>>> can
>>> break existing code that implicitly depends on current behavior. Can we
>>> add
>>> documentation-only warnings or use PendingDeprecationWarning if possible?
>>
>>
>> Usually deprecations are added in minor releases -- I don't know if we
>> added DeprecationWarnings in bugfix releases before.  Adding
>> documentation-only warnings in previous releases shouldn't be a
>> problem.  We already did this in the 2.7 docs for APIs that have been
>> deprecated/renamed/removed in 3.x.
>> If for example we deprecate something in 3.6, I see no harm in adding
>> a doc warning to the 2.7/3.5 docs stating that the feature is
>> deprecated starting from 3.6 and removed in 3.8.
>> I don't think we need to add DWs/PWDs though.
>
>
> We already added DeprecationWarnings in bugfix releases of 2.7 (but only
> enabled using the ``-3`` flag). There is no such mechanism for backporting
> warnings in 3.x, but there is a need. I'm interesting wherever using
> PendingDeprecationWarning instead of DeprecationWarnings is good enough for
> this as well as they are less intrusive.
>

If both are silenced by default, what makes PDW less intrusive?
For 2.7 the situation is a bit different because we have the -3 flag,
but for 3.x I think doc warnings are enough.

>>> Some deprecation can be documentation-only.
>>
>>
>> Do you have examples where this has been done?
>> I don't see the point of telling doc readers that a feature is
>> deprecated but keeping the same information hidden to developers.  If
>> the actual warnings cause some issue, then the issue should be
>> addresses (the issue of being noisy has already been addressed by
>> silencing them by default), but having doc-only deprecation warnings
>> seems inconsistent and potentially confusing.
>
>
> An attribute of a module. While in theory we can add a warning using a hack
> with replacing a module with module subclass instance, nobody does this in
> the stdlib.
>

Good point.  For this case having a doc-only warning should be ok, as
it's not worth the trouble required to raise an actual warning.
Perhaps this also requires a longer deprecation period, to give the
desv more time to notice.

> Removing some special behavior can be considered as needed a deprecation
> period. For example the change of datetime.time.__bool__ (made without a
> deprecation in issue13936) or ElementTree.Element.__bool__ (added indefinite
> FutureWarning).
>
>> If I understand correctly, this only affects pickleable APIs that have
>> been moved/renamed.
>
>
> Since most Python classes are pickleable by default, this affects modules,
> Python classes with stable structure that doesn't have unpickleable
> attributes, classes specially designed for pickling, simple subclasses of
> pickleable classes, and factory functions or methods that are used for
> pickle representation.
>

What I meant is that it shouldn't affect API that get removed, but
only the ones that have been moved/renamed (e.g. assertEquals ->
assertEqual).
If an API is gone, keeping the name around wouldn't help for pickle, would it?

>> If it can be done in a _compat_pickle.py it might be ok.
>
>
> Currently _compat_pickle.py is used only for pickles created in 2.x. We need
> parallel mechanism for 3.x pickles. I think this mechanism should be
> documented in the same document that specifies API removal, i.e. in this
> PEP.
>
>>> When use ``deprecated`` and when use ``deprecated-removed``?
>>
>>
>> If we agree to always specify when the API will be removed, then we
>> won't need to use "deprecated" anymore.
>> If we want to keep using indefinite deprecations we will use it for them.
>>
>> This should be specified in the PEP once we reach a consensus.
>
>
> May be define that the "deprecated" directive has term to next change of
> mayor version number (Python 4 currently)? It can be prolonged further, but
> the existing of the API beyond this term shouldn't be expected.
>

So make deprecated:: 3.6 a shorthand for deprecated-removed:: 3.6 4.0?
It could be done, but I'm not sure it's worth it.

Best Regards,
Ezio Melotti
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Ezio Melotti]

On Sat, Jan 30, 2016 at 4:08 AM, Martin Panter  wrote:
>> What and when to deprecate
>> ==
>>
>> * The number of releases before an API is removed is decided
>>   on a case-by-case basis depending on widely used the API is
>
> depending on [how] widely used
>
>> * In general it's better to be conservative, and if the API is
>>   deprecated in ``3.X``, it shouldn't be removed before ``3.X+2``.
>>   This should also take into account which Python versions are
>>   currently .
>
> which Python versions are [current].
>
> More explicitly, I would say as a guideline, if the proposed
> alternative is unavailable in 3.Y, consider waiting until 3.Y becomes
> unsupported or unused before removing (or even fully deprecating) an
> API.
>
>> Porting from 2.x to 3.x
>> ---
>>
>> In order to make porting code to 3.X easier:
>>
>> * nothing that is available and not deprecated in 2.7 should be
>>   removed from Python 3 as long as 2.7 is officially supported;
>
> What about an API not documented in Python 3, like
> http.client.HTTPMessage.getallmatchingheaders()
> ? In this case I think the method
> was rendered useless in 3.0 and it is not worth fixing.
>

The goal is to avoid unnecessary breakage.  In this case the code will
need to be fixed regardless, so the best we can do is to add a -3
warning to 2.7 .

> Also I presume you mean not originally deprecated in 2.7.0. In other
> words adding a “python2 -3” warning in the next 2.7 bug fix doesn’t
> give me a license to remove an API from 3.6.
>

Yes, even if you could argue that even removing things that were
already deprecated will make porting more difficult.  Perhaps I should
rephrase the section focusing the working on making the porting
easier.

>> Deprecation Progression
>> ===
>>
>> 1. in release ``3.X`` a ``DeprecationWarning`` is added
>> 2. in release ``3.X+N`` the API is removed
>>
>> ``N`` can be ``>=1``, should be decided beforehand, and should be
>> documented explicitly.
>
> How do we decide on the value of N for something that has to wait
> until 2.7 is dead? Just estimate based on anticipated future release
> dates? E.g. inspect.getargspec(). In some cases I think indefinite
> deprecation is better.
>

An estimation is fine.
I would use 4.0 for indefinite deprecations.  If we use the rst
directives to collect all the deprecated APIs in a single page, we can
go through them once 2.7 is dead.

>> Deprecation Process
>> ===
>
>> 2. attach to the issue a patch to deprecate the API that:
>>
>>   * adds a ``DeprecationWarning`` to the code
>>   * adds the deprecation to the documentation
>>   * adds a test to verify that the warning is raised
>
> Often people propose a test that will detect when the version changes
> to >= X+N and complains if the API has not been removed. Should this
> be encouraged or discouraged?
>

I did it once and it ended up breaking tests in the middle of the next release.
For the sake of the release managers, I don't think it should be done,
especially if we come up with a better way to track deprecations.

>>   * possibly updates code/examples that uses the deprecated API
>
> Also adjust any tests to suppress the new warning. Code to do this
> typically looks like
>
> with warnings.catch_warnings():
> warnings.simplefilter("ignore", DeprecationWarning)
> deprecated.api()
>

For tests there should be a convenience function in test.support.

>> 3. after review, apply the patch to the current in-development
>>branch and close the issue.
>
> Also apply similar changes to 2.7 if applicable?
>
>> Documenting the deprecations
>> 
>>
>> * All deprecations should be documented in the docs, using the
>>   ``deprecated-removed`` directive.
>
> If an undocumented API is being deprecated, you may not have to touch
> the main documentation (but still consider a notice in What’s New).
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Ezio Melotti]

On Tue, Feb 2, 2016 at 4:54 PM, R. David Murray  wrote:
> On Tue, 02 Feb 2016 15:33:58 +0200, Ezio Melotti  
> wrote:
>> On Sat, Jan 30, 2016 at 5:36 AM, Guido van Rossum  wrote:
>> > Following the lead of 2.7.10 and 2.7.11 we could continue with 3.10, 3.11, 
>> > etc.
>> >
>>
>> I think we should continue with 3.10, 3.11, etc.
>> Changing the major version should be done for incompatible changes,
>> and just doing it after 3.9 will probably just create confusion for
>> both users that will wonder if it's incompatible with Python 3 and for
>> things like the executable name.
>> Hopefully we won't need to jump to Python 4 for a long time.
>>
>> > I also want the 3->4 transition to feel like a non-event for most
>> > users. How we'll do that I don't know yet, but I want it to be a lot
>> > smoother than 2->3.
>
> I think Guido's point is that we shouldn't *make* incompatible changes,
> and that the 4.0 transition should be smooth so that people learn we are
> committed to that.  This is potentially analogous to the linux
> transition from 2.x to 3.x, despite the fact that it goes against the
> rules of semantic versioning.
>
> That said, I don't view removing deprecated things as a incompatible
> change, since code that has dealt with the deprecations will run on both
> the version before (usually versions plural) and the version after the
> removal.  Whether we should do a mass removal in 4.0 (or the first post
> 2.7 release) is a question, and so far the sense I get of the community
> is that there is not even close to a consensus on that.  But it would
> give a semantic versioning meaning to the change from 3.x to 4.x,
> without actually being all that disruptive.  On the other hand, a mass
> removal would be more disruption than removals spaced over several
> releases, so FUD might well arise as an issue, as you say.
>

There are two main types of changes that can make a new release
incompatible with the previous:
* removing/renaming (deprecated) APIs;
* semantic changes (e.g. int/int division, print function, lazy
builtins, nested scopes, absolute imports, unicode strings by default,
and adding new keywords);

For the former, we do it regularly in minor releases.  For the latter
the situation is more complicated.  We did most of those in 3.0, but
we changed semantics with nested scopes in 2.2 [0] and introduced new
keywords like "yield" and "with" in 2.3 [1] and 2.6 [2] (using
__future__ imports) first.
I believe that all these things can be done in a minor release, and
could have been done for Python 3.0 too, but it would have been really
difficult considering the magnitude of some of the changes.

Some amount of breakage happens with any new release, but we can
choose two different approaches:
1) spread the breakage evenly on several minor releases;
2) holding off until the next major release and do it at once;

If we choose the former, we might never need to bump the major release
to 4.0.  If we choose the latter (like we did for Python 3) we will
have an incompatible Python 4 (even though the incompatibilities might
be minor).
There's also a third option of bumping to 4.0 once some major (but
backward compatible) change is done, such as removing the GIL or
adding a JIT (or both).
I'm not sure making a mass-removal of deprecated APIs for 4.0 makes
too much sense, especially if we want to minimize breakage.

Also note that by choosing the first option we might increase the
chances that a program written for 3.6 won't run on e.g. 3.9, thus
forcing developers to keep up with the changes that each minor release
introduces.  Providing 3to3 fixers might make the transition easier.

What we decide will likely affect the deprecation policy we want to follow.

Best Regards,
Ezio Melotti

> Regardless though, the name is an issue :)
>
> So, I guess I see the arguments between going 3.9->4.0 vs 3.9->3.10
> as fairly balanced and don't have a strong preference.
>
> --David
> ___
> python-committers mailing list
> python-committers@python.org
> https://mail.python.org/mailman/listinfo/python-committers
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Barry Warsaw]

On Feb 02, 2016, at 03:33 PM, Ezio Melotti wrote:

>Changing the major version should be done for incompatible changes, and just
>doing it after 3.9 will probably just create confusion for both users that
>will wonder if it's incompatible with Python 3 and for things like the
>executable name.  Hopefully we won't need to jump to Python 4 for a long
>time.

Changing the major version number is a much bigger political and psychological
step than just bumping 3 to 4.  I say let's put it off as long as possible,
and there's nothing wrong with two digit minor versions.

Cheers,
-Barry
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[R. David Murray]

On Tue, 02 Feb 2016 15:33:58 +0200, Ezio Melotti  wrote:
> On Sat, Jan 30, 2016 at 5:36 AM, Guido van Rossum  wrote:
> > Following the lead of 2.7.10 and 2.7.11 we could continue with 3.10, 3.11, 
> > etc.
> >
> 
> I think we should continue with 3.10, 3.11, etc.
> Changing the major version should be done for incompatible changes,
> and just doing it after 3.9 will probably just create confusion for
> both users that will wonder if it's incompatible with Python 3 and for
> things like the executable name.
> Hopefully we won't need to jump to Python 4 for a long time.
> 
> > I also want the 3->4 transition to feel like a non-event for most
> > users. How we'll do that I don't know yet, but I want it to be a lot
> > smoother than 2->3.

I think Guido's point is that we shouldn't *make* incompatible changes,
and that the 4.0 transition should be smooth so that people learn we are
committed to that.  This is potentially analogous to the linux
transition from 2.x to 3.x, despite the fact that it goes against the
rules of semantic versioning.

That said, I don't view removing deprecated things as a incompatible
change, since code that has dealt with the deprecations will run on both
the version before (usually versions plural) and the version after the
removal.  Whether we should do a mass removal in 4.0 (or the first post
2.7 release) is a question, and so far the sense I get of the community
is that there is not even close to a consensus on that.  But it would
give a semantic versioning meaning to the change from 3.x to 4.x,
without actually being all that disruptive.  On the other hand, a mass
removal would be more disruption than removals spaced over several
releases, so FUD might well arise as an issue, as you say.

Regardless though, the name is an issue :)

So, I guess I see the arguments between going 3.9->4.0 vs 3.9->3.10
as fairly balanced and don't have a strong preference.

--David
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Terry Reedy]

On 2/2/2016 9:54 AM, R. David Murray wrote:

On Tue, 02 Feb 2016 15:33:58 +0200, Ezio Melotti  wrote:

On Sat, Jan 30, 2016 at 5:36 AM, Guido van Rossum  wrote:

Following the lead of 2.7.10 and 2.7.11 we could continue with 3.10, 3.11, etc.



I think we should continue with 3.10, 3.11, etc.
Changing the major version should be done for incompatible changes,
and just doing it after 3.9 will probably just create confusion for
both users that will wonder if it's incompatible with Python 3 and for
things like the executable name.
Hopefully we won't need to jump to Python 4 for a long time.


I also want the 3->4 transition to feel like a non-event for most
users. How we'll do that I don't know yet, but I want it to be a lot
smoother than 2->3.


I think Guido's point is that we shouldn't *make* incompatible changes,


A change change, as opposed to a removal change, is something like 1/2 
changing from 0 to .5, {}.keys() changing from a list to a dict_keys, 
and (the most disruptive) 'abc' and str() changing from bytes to 
unicode.  I think these were all needed, but now they are done.


The only comparable proposal I know of is changing .5 from a float to a 
decimal.  I think it would be much better to add new syntax, such as 
.5d, instead of (disruptively) changing the meaning of current syntax.



and that the 4.0 transition should be smooth so that people learn we are
committed to that.  This is potentially analogous to the linux
transition from 2.x to 3.x, despite the fact that it goes against the
rules of semantic versioning.

That said, I don't view removing deprecated things as a incompatible
change, since code that has dealt with the deprecations will run on both
the version before (usually versions plural) and the version after the
removal.


A major difference between change and removal is that using something 
removed immediately fails instead doing something not expected and 
likely wrong and maybe eventually failing.  Also, change in meaning 
usually makes cross version code impossible (without adding 'if old: do 
one thing; else: do another' conditionals).  Whereas, as you say above, 
replacements can be backported at least a couple of versions.


> Whether we should do a mass removal in 4.0 (or the first post

2.7 release) is a question, and so far the sense I get of the community
is that there is not even close to a consensus on that.  But it would
give a semantic versioning meaning to the change from 3.x to 4.x,
without actually being all that disruptive.


By my calculation, 3.9 will be the version after 2.7 is discontinued. 
If we do release a '3.9', make turning on DeprecationWarnings a bit 
easier so users can check if any packages they use might fail in 4.0 (or 
later)  After studying the -W doc entry move closely than many users are 
likely to, , I conclude that "-W default::DeprecationWarning" is the 
right command-line option.  Correct?  This example could be added to the 
doc.  Installers could offer to set PYTHONWARNINGS to 
'default::DeprecationWarning'.


For 4.0, I propose this: remove deprecated features from the main part 
of the doc, so they are not part of the Python 4.0 language.  Features 
not removed (yet) from the code would be relegated to an Appendix, '3.x 
remnants', which would start by declaring that the items therein are not 
in '4.0' and should not be used in new 4.0 code.  (The check for 4.0 
conformance would be to run with deprecation warnings on.)


Timing of actual removal could depend on ease of replacement.  For 
instance, things that can be deterministically replaced by a 'dep_fixer' 
could be removed in 4.0.  The unittest synonym removals are an examples. 
 Things with a replacement that can be suggested but might need to be 
checked might go in 4.1.  Things without a suggest replacement, that 
need a human rewrite, to be kept to 4.2.  (I am not sure how many things 
fit in the latter two categories.)


The deterministic fixes could be further categorized by when the 
replacement is available.  The fixer could take a parameter to specify 
the earliest version the code must work with.  Someone using the fixer 
today might specify that the result must work with 3.3 and later.  Five 
years from now, minimum versions would usually be higher.  The first 
version of a fixer could be included with 3.6.


-- tjr

PS.  Never removing obsolete items can eventually lead to problems.  See 
https://motherboard.vice.com/read/cern-engineers-have-to-identify-and-disconnect-9000-obsolete-cables 
for a non-programming example.


___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Steven D'Aprano]

On Tue, Feb 02, 2016 at 09:40:52AM -0500, Barry Warsaw wrote:
> On Feb 02, 2016, at 03:33 PM, Ezio Melotti wrote:
> 
> >Changing the major version should be done for incompatible changes, and just
> >doing it after 3.9 will probably just create confusion for both users that
> >will wonder if it's incompatible with Python 3 and for things like the
> >executable name.  Hopefully we won't need to jump to Python 4 for a long
> >time.
> 
> Changing the major version number is a much bigger political and psychological
> step than just bumping 3 to 4.  I say let's put it off as long as possible,
> and there's nothing wrong with two digit minor versions.

It will break any code that does

if sys.version < "3.3": ...

for example. Sure, people shouldn't do that, but they will.

Are there any other languages or systems which bumped the major version 
number while keeping language changes relatively minor? I can think of a 
couple of examples of languages which bumped the major version number 
without making big disruptive changes:

* Python 1.5 -> 2.0 (technically there was a 1.6 in there as well, but 
that wasn't used by many people).

Admittedly, fewer people used Python back then, and the version bump 
wasn't following a similiar, but backwards incompatible, version bump.

* PHP 5.x -> 7.0. I count nine feature removals, but none of them look 
big. http://php.net/ChangeLog-7.php

* Java major version bumps are rarely backwards incompatible, as far as 
I can tell. I had a Java developer explain to me that he treats the N 
in "Java N" as effectively a minor release number.


The point being, I'm not entirely sure I agree that a major version bump 
would *necessarily* be considered a big deal, let alone a barrier to 
adoption. It might be nice to have a really big feature added in 4.0 to 
justify the version bump, but I don't think it's a "Must Do". Having 4.0 
being a lightweight evolutionary step up from 3.9 might even reassure 
people that we're back to normal operating practice, without the 
extraordinary 2->3 transition.



-- 
Steve
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers

Re: [python-committers] Deprecation Policy PEP

[Barry Warsaw]

On Feb 03, 2016, at 11:03 AM, Steven D'Aprano wrote:

>The point being, I'm not entirely sure I agree that a major version bump
>would *necessarily* be considered a big deal, let alone a barrier to
>adoption.

The problem isn't so much the major version bump, but what to do about the
command name on *nix.  I'm admittedly biased about that being "the"
problem. ;)

I think you'd get a lot of resistance if you named it 'python'.  You can't
name it 'python3'.  Maybe 'python4' would be okay, but please let's not do
that any time soon.  We're still in the midst of a transition to Python 3 and
I think anything that muddies those waters will just mean more FUD that some
of us front-liners have to contend with.

OTOH, I can see it being worth it for something really big like getting rid of
the GIL.

Cheers,
-Barry
___
python-committers mailing list
python-committers@python.org
https://mail.python.org/mailman/listinfo/python-committers