Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Nick Coghlan]

On 9 May 2017 at 05:08, Chris Barker  wrote:
> On Fri, May 5, 2017 at 9:30 PM, Nick Coghlan  wrote:
>>
>> By contrast, 3.6 users can just unconditionally call os.fspath(), and
>> know the result will work with all stdlib APIs, without allowing
>> nonsense like attempting to use "str(str)" as a filesystem path.
>>
>> Doing that implicitly in various APIs is certainly a nice UX
>> enhancement, but it's the replacement of str with os.fspath as the
>> recommended coercion function that removes the major barrier to
>> pathlib adoption.
>
>
> honestly, I'm not sure about that -- burying a str() call in a lib is
> clearly risky, but using one in your user code, not so much -- you usually
> know you have a path-like object (if not a Path) itself. the extra need for
> wrapping str() around things was a barrier to pathlib adoption -- and it
> doesn't take much of a barrier to reduce the adoption of something new.

Exactly - the key barrier to adoption was the lack of "os.fspath()",
since it meant that neither the standard library *nor* third party
helper libraries could reliably deal with the conversion task on
behalf of their users. Instead, people had to carefully manage their
own code, ensuring both that they used str() in the appropriate
places, *and* that they didn't accidentally call str() on the wrong
thing.

os.fspath() changes that to be much simpler and friendlier to an
iterative development process:

1. By default, expect things to just work
2. If they blow up with TypeError, wrap the offending argument in a
call to os.fspath()
3. (optional) file an RFE with the maintainers of the affected API to
call os.fspath() implicitly on that argument

The number of APIs that fall into the second category is going to be
non-zero for quite a while (both in the standard library and
elsewhere), but enough folks proceeding to step 3 will bring it down
over time.

Outside customers complaining to their commercial support vendors
though, it's rarely helpful to insist to maintainers (whether of
standard library modules or third party ones) that their failure to
adopt a feature only added in the latest version of Python now counts
as a bug in their software.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Chris Barker]

On Fri, May 5, 2017 at 9:30 PM, Nick Coghlan  wrote:

> By contrast, 3.6 users can just unconditionally call os.fspath(), and
> know the result will work with all stdlib APIs, without allowing
> nonsense like attempting to use "str(str)" as a filesystem path.
>


> Doing that implicitly in various APIs is certainly a nice UX
> enhancement, but it's the replacement of str with os.fspath as the
> recommended coercion function that removes the major barrier to
> pathlib adoption.


honestly, I'm not sure about that -- burying a str() call in a lib is
clearly risky, but using one in your user code, not so much -- you usually
know you have a path-like object (if not a Path) itself. the extra need for
wrapping str() around things was a barrier to pathlib adoption -- and it
doesn't take much of a barrier to reduce the adoption of something new.

The goal is clearly that ALL stdlib function take a path_like -- but if
we're not there yet, we're not there yet :-(

-CHB






>
> > Is there any chance of it breaking already working code? That would be
> the
> > one compelling reason to not back-port.
>
> "Don't make the third party compatibility testing matrix impossibly
> complicated" is our other major reason not to backport new features.
>
> We already have platform providers supporting 3.6.0 in the wild
> (Heroku and AWS Lambda), and we know 3.6.1 is going to be in at least
> Fedora 26, and probably Ubuntu 17.10.
>
> We want CI providers like Travis/GitLab/Circle CI/etc to be able to
> provide a *single* 3.6.x release (typically the most recent one), and
> have the guarantee be "code that passes on this maintenance release
> will only fail on earlier maintenance releases if you hit a genuine
> bug in those releases".
>
> Changing a function signature from accepting "Union[str, bytes]" (or
> potentially even just "str") to instead accepting "os.PathLike"
> doesn't count as fixing a genuine bug in the earlier releases - it
> just means we made the API more permissive in a maintenance release.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Chris Barker]

On Fri, May 5, 2017 at 2:34 PM, Terry Reedy  wrote:

> So it would be really great if any updates to shutils (and other stdlib
>> packages) to support the new protocol be back-ported.
>>
>
> Not if they change the language.


we're not talking about language changes here -- we are talking about
updates to the stdlib.

Eric wrote:

The problem is that when someone writes code a year from now and tests it
> under Python 3.6.2, then a customer of theirs finds it doesn't work in
> 3.6.1. This will happen if 3.6.2 supports Path parameters to functions that
> 3.6.1 does not.
>

This does, of course apply to the stdlib -- not just the language.

We've burned ourselves before with this, most famously with True and False
> in 2.2.1: http://python-history.blogspot.com/2013/11/the-history-of-
> bool-true-and-false.html


I'd argue that adding three built-ins is a much bigger deal than enhancing
a subset of one standard package -- not that the same issues wouldn't arise.

I think it's still worth, but I can certainly see why it might not be.

Too bad we didn't do a bit more testing for completely when it was beta

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Nick Coghlan]

On 6 May 2017 at 05:36, Chris Barker  wrote:
> It doesn't actually say "everywhere possible in the stdlib", but if the goal
> is to facilitate migration, as stated, then the any but truly obscure
> functions should be covered -- and shutil is certainly not obscure.
>
> So it would be really great if any updates to shutils (and other stdlib
> packages) to support the new protocol be back-ported.

This argument would apply to all the other protocols we've added over
time, and would lead to significant language churn in maintenance
releases if we regularly relied on it.

As another example from 3.6, we found that the non-type metaclasses in
the standard library (most notably abc.ABCMeta) don't play nice with
the arbitrary keyword support in __init_subclass__, since they don't
pass the additional keywords from the class definition up to
type.__new__ (which in turn would pass them along to
__init_subclass__).

That's an oversight we're definitely going to fix, but we'll fix it in
3.7, not in 3.6.x.

> Yes, it's "just" adding an extra call, but in my experience, low barrier to
> entry are enough to discourage adoption -- and handful of shutil function
> failing will certainly be enough to keep some folks from adopting the new
> approach.

The critical problem prior to 3.6 wasn't the extra call, it was that
the recommended extra call had the undesirable side effect of allowing
*any* object to be used as a "file system path". As a result, you
couldn't easily separate the check out into a helper function without
putting a lot of annoying "isinstance" checks into it (or else relying
on functools.singledispatch).

By contrast, 3.6 users can just unconditionally call os.fspath(), and
know the result will work with all stdlib APIs, without allowing
nonsense like attempting to use "str(str)" as a filesystem path.

Doing that implicitly in various APIs is certainly a nice UX
enhancement, but it's the replacement of str with os.fspath as the
recommended coercion function that removes the major barrier to
pathlib adoption.

> Is there any chance of it breaking already working code? That would be the
> one compelling reason to not back-port.

"Don't make the third party compatibility testing matrix impossibly
complicated" is our other major reason not to backport new features.

We already have platform providers supporting 3.6.0 in the wild
(Heroku and AWS Lambda), and we know 3.6.1 is going to be in at least
Fedora 26, and probably Ubuntu 17.10.

We want CI providers like Travis/GitLab/Circle CI/etc to be able to
provide a *single* 3.6.x release (typically the most recent one), and
have the guarantee be "code that passes on this maintenance release
will only fail on earlier maintenance releases if you hit a genuine
bug in those releases".

Changing a function signature from accepting "Union[str, bytes]" (or
potentially even just "str") to instead accepting "os.PathLike"
doesn't count as fixing a genuine bug in the earlier releases - it
just means we made the API more permissive in a maintenance release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Koos Zevenhoven]

On May 5, 2017 10:39 PM, "Chris Barker"  wrote:

Sorry to come late to the game, It wasn't immediately clear to me what the
implications were of the "enhancement or bugfix" distinction...

On Thu, May 4, 2017 at 9:46 PM, Nick Coghlan  wrote:

> That improved casting mechanism and the implicit support in the low
> level APIs is the main benefit I see in PEP 519, and if we were
> talking about an os module API that was missing os.path support, I'd
> be more likely to be on the "it's a bug" side of things.
>

absolutely.


> It's only higher level APIs like shutil that I put into the same
> category as third party libraries for now: Python 3.6 users shouldn't
> expect implicit use of the fspath protocol to be universal yet,


I think stdlib packages, like shutil are very much distinct from third
party  libs, and users certainly expect the entire stdlib to be updated to
support new language features.

We very often make the distinction between third party libs and stdlibs --
in fact, that is one reason we are reluctant to add new packages to the
stdlib...

Indeed, IIRC, that was the entire motivation for PEP 519 -- it started as a
"let's make Path objects work with the stdlib", but when we looked at how
to do that, it became clear that new protocol was teh best way to do that
and also provide flexibility to do that.

And the PEP states:

"""
Changes to Python's standard library are also proposed to utilize this
protocol where appropriate to facilitate the use of path objects where
historically only str and/or bytes file system paths are accepted. The goal
is to facilitate the migration of users towards rich path objects while
providing an easy way to work with code expecting str or bytes .
"""

It doesn't actually say "everywhere possible in the stdlib", but if the
goal is to facilitate migration, as stated, then the any but truly obscure
functions should be covered -- and shutil is certainly not obscure.


Indeed.

So it would be really great if any updates to shutils (and other stdlib
packages) to support the new protocol be back-ported.

Yes, it's "just" adding an extra call, but in my experience, low barrier to
entry are enough to discourage adoption -- and handful of shutil function
failing will certainly be enough to keep some folks from adopting the new
approach.


I suppose the worst thing to happen is what Eric describes. But technically
speaking, passing os.PathLike to shutil functions might currently be
undefined behavior, so the change is 'legal' if it's not documented ;).

-- Koos (mobile)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Terry Reedy]

On 5/5/2017 3:36 PM, Chris Barker wrote:
Sorry to come late to the game, It wasn't immediately clear to me what 
the implications were of the "enhancement or bugfix" distinction...


An enhancement changes the definition of the Python language, a bugfix 
does not.  Enhancements can only be part of a new version of Python.  A 
bugfix can potentially go in any 'active' implementation that has the bug.


'Python 3.6' is a version of Python.  The Python 3.6 docs more-or-less 
define the Python 3.6.  Subsequent doc patches refine and clarify the 
definition, but should not change it.  New doc patches are released 
daily on docs.python.org.


CPython 3.6.0 was the first full implementation of Python 3.6, whose 
definition was mostly fixed at the first beta.  CPython 3.6.1 was the 
next officially released implementation *of the same Python 3.6 
language*, but with some implementation bugs fixed. The same will be 
true of 3.6.2, etcetera.


If someone privately compiles cpython as it is at some time today, it 
will be an implementation of 3.6 that is hopefully better than 3.6.1 and 
worse than the future 3.6.2, but will not be either.  Call it 
3.6.(datetime).  3rd party distributions based on CPython can muddy 
things even further by selectively incorporating upstream patches, so 
that XYZ Python 3.6(.whatever) does not correspond to any date-stamped 
or released version of repository CPython.


So it would be really great if any updates to shutils (and other stdlib 
packages) to support the new protocol be back-ported.


Not if they change the language.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Eric V. Smith]

On 5/5/2017 3:36 PM, Chris Barker wrote:
Sorry to come late to the game, It wasn't immediately clear to me what 
the implications were of the "enhancement or bugfix" distinction...


On Thu, May 4, 2017 at 9:46 PM, Nick Coghlan > wrote:


That improved casting mechanism and the implicit support in the low
level APIs is the main benefit I see in PEP 519, and if we were
talking about an os module API that was missing os.path support, I'd
be more likely to be on the "it's a bug" side of things.


absolutely.

It's only higher level APIs like shutil that I put into the same
category as third party libraries for now: Python 3.6 users shouldn't
expect implicit use of the fspath protocol to be universal yet,


I think stdlib packages, like shutil are very much distinct from third 
party  libs, and users certainly expect the entire stdlib to be updated 
to support new language features.


We very often make the distinction between third party libs and stdlibs 
-- in fact, that is one reason we are reluctant to add new packages to 
the stdlib...


Indeed, IIRC, that was the entire motivation for PEP 519 -- it started 
as a "let's make Path objects work with the stdlib", but when we looked 
at how to do that, it became clear that new protocol was teh best way to 
do that and also provide flexibility to do that.


And the PEP states:

"""
Changes to Python's standard library are also proposed to utilize this 
protocol where appropriate to facilitate the use of path objects where 
historically only str and/or bytes file system paths are accepted. The 
goal is to facilitate the migration of users towards rich path objects 
while providing an easy way to work with code expecting str or bytes .

"""

It doesn't actually say "everywhere possible in the stdlib", but if the 
goal is to facilitate migration, as stated, then the any but truly 
obscure functions should be covered -- and shutil is certainly not obscure.


So it would be really great if any updates to shutils (and other stdlib 
packages) to support the new protocol be back-ported.


Yes, it's "just" adding an extra call, but in my experience, low barrier 
to entry are enough to discourage adoption -- and handful of shutil 
function failing will certainly be enough to keep some folks from 
adopting the new approach.


Is there any chance of it breaking already working code? That would be 
the one compelling reason to not back-port.


The problem is not breaking existing code. The problem is that when 
someone writes code a year from now and tests it under Python 3.6.2, 
then a customer of theirs finds it doesn't work in 3.6.1. This will 
happen if 3.6.2 supports Path parameters to functions that 3.6.1 does not.


We've burned ourselves before with this, most famously with True and 
False in 2.2.1: 
http://python-history.blogspot.com/2013/11/the-history-of-bool-true-and-false.html


--
Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Chris Barker]

Sorry to come late to the game, It wasn't immediately clear to me what the
implications were of the "enhancement or bugfix" distinction...

On Thu, May 4, 2017 at 9:46 PM, Nick Coghlan  wrote:

> That improved casting mechanism and the implicit support in the low
> level APIs is the main benefit I see in PEP 519, and if we were
> talking about an os module API that was missing os.path support, I'd
> be more likely to be on the "it's a bug" side of things.
>

absolutely.


> It's only higher level APIs like shutil that I put into the same
> category as third party libraries for now: Python 3.6 users shouldn't
> expect implicit use of the fspath protocol to be universal yet,


I think stdlib packages, like shutil are very much distinct from third
party  libs, and users certainly expect the entire stdlib to be updated to
support new language features.

We very often make the distinction between third party libs and stdlibs --
in fact, that is one reason we are reluctant to add new packages to the
stdlib...

Indeed, IIRC, that was the entire motivation for PEP 519 -- it started as a
"let's make Path objects work with the stdlib", but when we looked at how
to do that, it became clear that new protocol was teh best way to do that
and also provide flexibility to do that.

And the PEP states:

"""
Changes to Python's standard library are also proposed to utilize this
protocol where appropriate to facilitate the use of path objects where
historically only str and/or bytes file system paths are accepted. The goal
is to facilitate the migration of users towards rich path objects while
providing an easy way to work with code expecting str or bytes .
"""

It doesn't actually say "everywhere possible in the stdlib", but if the
goal is to facilitate migration, as stated, then the any but truly obscure
functions should be covered -- and shutil is certainly not obscure.

So it would be really great if any updates to shutils (and other stdlib
packages) to support the new protocol be back-ported.

Yes, it's "just" adding an extra call, but in my experience, low barrier to
entry are enough to discourage adoption -- and handful of shutil function
failing will certainly be enough to keep some folks from adopting the new
approach.

Is there any chance of it breaking already working code? That would be the
one compelling reason to not back-port.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Nick Coghlan]

On 4 May 2017 at 08:25, Victor Stinner  wrote:
> If you start to backport support for the fspath protocol, be prepared
> to have to backport it *many* places. I expect that slowly in the near
> future, many functions will be patched to support the fspath protocol.
>
> I suggest to only do that in master. It's not that hard to cast
> manually to a string: it's just str(path), no?

For 3.6+, "os.fspath(path)" would be more robust - unlike str(), it
will still error out immediately if you pass it something completely
inappropriate.

That improved casting mechanism and the implicit support in the low
level APIs is the main benefit I see in PEP 519, and if we were
talking about an os module API that was missing os.path support, I'd
be more likely to be on the "it's a bug" side of things.

It's only higher level APIs like shutil that I put into the same
category as third party libraries for now: Python 3.6 users shouldn't
expect implicit use of the fspath protocol to be universal yet, so
they may still need some explicit calls to os.fspath() in their own
code to get different APIs to work well together.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Koos Zevenhoven]

On Thu, May 4, 2017 at 8:30 PM, Terry Reedy  wrote:
> On 5/4/2017 10:43 AM, Koos Zevenhoven wrote:
>> On Thu, May 4, 2017 at 4:19 AM, Terry Reedy  wrote:
>>> Enhancing public APIs in normal (non-provisional) modules in bugfix
>>> releases
>>> has turned out to be a bad thing to do.  Hence the policy to not do that.
>>> The few exceptions have been necessary to fix a bug that needed to be
>>> fixed,
>>> and could not reasonably be fixed otherwise.
>>
>> Such exceptions can of course more easily be made when the adoption of
>> a version is still small, and almost all users will never see X.Y.0 or
>> X.Y.1.
>
> This is not an allowed excuse for breaking the policy.  The x.y language is
> defined when x.y.0 is released.  Please stop.
>

Don't worry, I didn't even start :)

I do think it can cause problems if most of a stdlib module supports
PathLike and some parts do not. People will write code in 3.7
believing it works on 3.6, while it doesn't. Anyway, I'm completely
happy if the policy outweighs this issue, and I have absolutely no
need to argue about the decision.

—Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Serhiy Storchaka]

On 04.05.17 21:01, Berker Peksağ wrote:

We've already backported a few patches that improves the PEP 519
support in the stdlib with the permission from the release manager of
3.6. I'd ask Ned whether bpo-30218 qualifies for backporting to 3.6.


AFAIK it was before releasing 3.6.1. Some users avoid using x.0 versions 
of software. For 3.6.2 we can have stronger rule.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Berker Peksağ]

On Wed, May 3, 2017 at 9:15 PM, Brett Cannon  wrote:
> My allergies have hit me hard so I'm not thinking at full capacity, but did
> we ever decide if supporting os.PathLike in the stdlib was viewed as an
> enhancement or bugfix? Specifically I'm thinking of
> https://bugs.python.org/issue30218 for adding support to
> shutil.unpack_archive() and whether it should be backported to 3.6.

We've already backported a few patches that improves the PEP 519
support in the stdlib with the permission from the release manager of
3.6. I'd ask Ned whether bpo-30218 qualifies for backporting to 3.6.

--Berker
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Terry Reedy]

On 5/4/2017 10:43 AM, Koos Zevenhoven wrote:


On Thu, May 4, 2017 at 4:19 AM, Terry Reedy  wrote:

What did not get done for 3.6 should be proposed for 3.7.



Anyone, feel free. The nightmare part is done, so this could be a case
where a PR actually pays off in terms of being able to use the
feature. There's no need for any unnecessary masochism (should there
ever be?).


I have no idea what this means, but don't need to.


Enhancing public APIs in normal (non-provisional) modules in bugfix releases
has turned out to be a bad thing to do.  Hence the policy to not do that.
The few exceptions have been necessary to fix a bug that needed to be fixed,
and could not reasonably be fixed otherwise.


Such exceptions can of course more easily be made when the adoption of
a version is still small, and almost all users will never see X.Y.0 or
X.Y.1.


This is not an allowed excuse for breaking the policy.  The x.y language 
is defined when x.y.0 is released.  Please stop.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Brett Cannon]

Thanks for the feedback, everyone. I'll just consider it an enhancement
then.

On Wed, 3 May 2017 at 15:07 Terry Reedy  wrote:

> On 5/3/2017 2:15 PM, Brett Cannon wrote:
> > My allergies have hit me hard so I'm not thinking at full capacity, but
> > did we ever decide if supporting os.PathLike in the stdlib was viewed as
> > an enhancement or bugfix? Specifically I'm thinking of
> > https://bugs.python.org/issue30218 for adding support to
> > shutil.unpack_archive() and whether it should be backported to 3.6.
>
> On the face of it, that particular issue looks like an enhancement that
> should have gone into 3.6 (if ever), but did not.  I notice that
> https://www.python.org/dev/peps/pep-0519/#implementation
> did not include "Update shutil", so it was not done, at least not
> completely.
>
> Was shutil updated at all?


Probably not explicitly. A lot of support just fell through thanks to
updating os and os.path to support path-like objects.


>   Is unpack_archive the only shutil function
> not updated?


Not sure. I opened https://bugs.python.org/issue30235 to track finding out.

-Brett

  If so, I could see the omission as a bug.


> If the patch for 30218 were applied in 3.6, would the doc
> https://docs.python.org/3/library/shutil.html#shutil.unpack_archive
> need to be changed, with a note "Added in 3.6.2: filename can be any
> pathlike object"?  If so, it is an enhancement.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Serhiy Storchaka]

On 03.05.17 21:15, Brett Cannon wrote:

My allergies have hit me hard so I'm not thinking at full capacity, but
did we ever decide if supporting os.PathLike in the stdlib was viewed as
an enhancement or bugfix? Specifically I'm thinking
of https://bugs.python.org/issue30218 for adding support to
shutil.unpack_archive() and whether it should be backported to 3.6.


I concur with Terry, Victor and Nick.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Nick Coghlan]

On 4 May 2017 at 04:15, Brett Cannon  wrote:
> My allergies have hit me hard so I'm not thinking at full capacity, but did
> we ever decide if supporting os.PathLike in the stdlib was viewed as an
> enhancement or bugfix? Specifically I'm thinking of
> https://bugs.python.org/issue30218 for adding support to
> shutil.unpack_archive() and whether it should be backported to 3.6.

I'd treat it as an enhancement, similar to adding native context
management support to various APIs. Otherwise, as Terry pointed out,
we're setting folks up for cases where testing on the latest
maintenance release seems fine, but their app breaks when deployed on
a slightly older version.

It's one thing for that to happen when the app is encountering a
genuine standard library bug, but something else entirely when they're
instead inadvertently relying on an enhancement that we could have
deferred to the next feature release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Koos Zevenhoven]

On Thu, May 4, 2017 at 4:19 AM, Terry Reedy  wrote:
> On 5/3/2017 7:13 PM, Koos Zevenhoven wrote:
>>
[...]

>> Shutil was among the most important to be updated, IMO.
>>
>> I had made some sort of list of affected modules elsewhere [1]:
>> ntpath, posixpath, os.scandir, os.[other stuff], DirEntry (tempted to
>> say os.DirEntry, but that is
>> not true), shutil.[stuff], (io.)open, fileinput, filecmp, zipfile,
>> tarfile, tempfile (for the 'dir' keyword arguments), maybe even glob
>> and fnmatch (are the patterns paths?)
>
>
> What did not get done for 3.6 should be proposed for 3.7.
>

Anyone, feel free. The nightmare part is done, so this could be a case
where a PR actually pays off in terms of being able to use the
feature. There's no need for any unnecessary masochism (should there
ever be?).

[...]
>
> Enhancing public APIs in normal (non-provisional) modules in bugfix releases
> has turned out to be a bad thing to do.  Hence the policy to not do that.
> The few exceptions have been necessary to fix a bug that needed to be fixed,
> and could not reasonably be fixed otherwise.

Such exceptions can of course more easily be made when the adoption of
a version is still small, and almost all users will never see X.Y.0 or
X.Y.1. The fraction of 3.6 users is probably super tiny right now, and
even those users are likely to eagerly update to bugfix releases. For
instance, are there any major (LTS?) linux distros that already come
with 3.6.0 or 3.6.1? Well OK, 3.6.2 may be too late for some.

—Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Terry Reedy]

On 5/3/2017 7:13 PM, Koos Zevenhoven wrote:

On Thu, May 4, 2017 at 1:07 AM, Terry Reedy  wrote:

On 5/3/2017 2:15 PM, Brett Cannon wrote:


My allergies have hit me hard so I'm not thinking at full capacity, but
did we ever decide if supporting os.PathLike in the stdlib was viewed as an
enhancement or bugfix? Specifically I'm thinking of
https://bugs.python.org/issue30218 for adding support to
shutil.unpack_archive() and whether it should be backported to 3.6.



On the face of it, that particular issue looks like an enhancement that
should have gone into 3.6


Agreed.


(if ever), but did not.  I notice that
https://www.python.org/dev/peps/pep-0519/#implementation
did not include "Update shutil", so it was not done, at least not
completely.


Shutil was among the most important to be updated, IMO.

I had made some sort of list of affected modules elsewhere [1]:
ntpath, posixpath, os.scandir, os.[other stuff], DirEntry (tempted to
say os.DirEntry, but that is
not true), shutil.[stuff], (io.)open, fileinput, filecmp, zipfile,
tarfile, tempfile (for the 'dir' keyword arguments), maybe even glob
and fnmatch (are the patterns paths?)


What did not get done for 3.6 should be proposed for 3.7.


It looks like what made it to PEP519 was mainly this:

"It is expected that most APIs in Python's standard library that
currently accept a file system path will be updated appropriately to
accept path objects (whether that requires code or simply an update to
documentation will vary)."


To me that says that not being updated (yet) is not a bug.


Was shutil updated at all?  Is unpack_archive the only shutil function not
updated?  If so, I could see the omission as a bug.

If the patch for 30218 were applied in 3.6, would the doc
https://docs.python.org/3/library/shutil.html#shutil.unpack_archive
need to be changed, with a note "Added in 3.6.2: filename can be any
pathlike object"?  If so, it is an enhancement.


Regardless of bugfix vs enhancement semantics, that seems like a good
thing to do.


Enhancing public APIs in normal (non-provisional) modules in bugfix 
releases has turned out to be a bad thing to do.  Hence the policy to 
not do that.  The few exceptions have been necessary to fix a bug that 
needed to be fixed, and could not reasonably be fixed otherwise.



[1] e.g. in this thread somewhere:
https://mail.python.org/pipermail/python-ideas/2016-April/039827.html



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Koos Zevenhoven]

On Thu, May 4, 2017 at 1:07 AM, Terry Reedy  wrote:
> On 5/3/2017 2:15 PM, Brett Cannon wrote:
>>
>> My allergies have hit me hard so I'm not thinking at full capacity, but
>> did we ever decide if supporting os.PathLike in the stdlib was viewed as an
>> enhancement or bugfix? Specifically I'm thinking of
>> https://bugs.python.org/issue30218 for adding support to
>> shutil.unpack_archive() and whether it should be backported to 3.6.
>
>
> On the face of it, that particular issue looks like an enhancement that
> should have gone into 3.6

Agreed.

> (if ever), but did not.  I notice that
> https://www.python.org/dev/peps/pep-0519/#implementation
> did not include "Update shutil", so it was not done, at least not
> completely.

Shutil was among the most important to be updated, IMO.

I had made some sort of list of affected modules elsewhere [1]:
ntpath, posixpath, os.scandir, os.[other stuff], DirEntry (tempted to
say os.DirEntry, but that is
not true), shutil.[stuff], (io.)open, fileinput, filecmp, zipfile,
tarfile, tempfile (for the 'dir' keyword arguments), maybe even glob
and fnmatch (are the patterns paths?)

It looks like what made it to PEP519 was mainly this:

"It is expected that most APIs in Python's standard library that
currently accept a file system path will be updated appropriately to
accept path objects (whether that requires code or simply an update to
documentation will vary)."

> Was shutil updated at all?  Is unpack_archive the only shutil function not
> updated?  If so, I could see the omission as a bug.
>
> If the patch for 30218 were applied in 3.6, would the doc
> https://docs.python.org/3/library/shutil.html#shutil.unpack_archive
> need to be changed, with a note "Added in 3.6.2: filename can be any
> pathlike object"?  If so, it is an enhancement.

Regardless of bugfix vs enhancement semantics, that seems like a good
thing to do.

-- Koos

[1] e.g. in this thread somewhere:
https://mail.python.org/pipermail/python-ideas/2016-April/039827.html


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Victor Stinner]

If you start to backport support for the fspath protocol, be prepared
to have to backport it *many* places. I expect that slowly in the near
future, many functions will be patched to support the fspath protocol.

I suggest to only do that in master. It's not that hard to cast
manually to a string: it's just str(path), no?

Victor

2017-05-03 20:15 GMT+02:00 Brett Cannon :
> My allergies have hit me hard so I'm not thinking at full capacity, but did
> we ever decide if supporting os.PathLike in the stdlib was viewed as an
> enhancement or bugfix? Specifically I'm thinking of
> https://bugs.python.org/issue30218 for adding support to
> shutil.unpack_archive() and whether it should be backported to 3.6.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Terry Reedy]

On 5/3/2017 2:15 PM, Brett Cannon wrote:
My allergies have hit me hard so I'm not thinking at full capacity, but 
did we ever decide if supporting os.PathLike in the stdlib was viewed as 
an enhancement or bugfix? Specifically I'm thinking of 
https://bugs.python.org/issue30218 for adding support to 
shutil.unpack_archive() and whether it should be backported to 3.6.


On the face of it, that particular issue looks like an enhancement that 
should have gone into 3.6 (if ever), but did not.  I notice that

https://www.python.org/dev/peps/pep-0519/#implementation
did not include "Update shutil", so it was not done, at least not 
completely.


Was shutil updated at all?  Is unpack_archive the only shutil function 
not updated?  If so, I could see the omission as a bug.


If the patch for 30218 were applied in 3.6, would the doc
https://docs.python.org/3/library/shutil.html#shutil.unpack_archive
need to be changed, with a note "Added in 3.6.2: filename can be any 
pathlike object"?  If so, it is an enhancement.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Koos Zevenhoven]

On Wed, May 3, 2017 at 9:15 PM, Brett Cannon  wrote:
> My allergies have hit me hard so I'm not thinking at full capacity, but did
> we ever decide if supporting os.PathLike in the stdlib was viewed as an
> enhancement or bugfix? Specifically I'm thinking of
> https://bugs.python.org/issue30218 for adding support to
> shutil.unpack_archive() and whether it should be backported to 3.6.
>

Haha, not sure if you have actual allergy, or if you just remembered
the fspath discussions nightmare ;).

You're obviously not asking me, but I'd say anything in the 3.6 stdlib
that takes a path, but not os.PathLike, should count as a bug.
Otherwise there will just be surprises with code that works on 3.7 but
not on the famously fspath-enabled 3.6. I'm not even 100% sure that
backporting to 3.5 would be (or would have been) a bad idea; 3.5 will
stick around for quite some time still.

-- Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Brett Cannon]

My allergies have hit me hard so I'm not thinking at full capacity, but did
we ever decide if supporting os.PathLike in the stdlib was viewed as an
enhancement or bugfix? Specifically I'm thinking of
https://bugs.python.org/issue30218 for adding support to
shutil.unpack_archive() and whether it should be backported to 3.6.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com