Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Nick Coghlan]

On 9 May 2017 at 05:08, Chris Barker  wrote:
> On Fri, May 5, 2017 at 9:30 PM, Nick Coghlan  wrote:
>>
>> By contrast, 3.6 users can just unconditionally call os.fspath(), and
>> know the result will work with all stdlib APIs, without allowing
>> nonsense like attempting to use "str(str)" as a filesystem path.
>>
>> Doing that implicitly in various APIs is certainly a nice UX
>> enhancement, but it's the replacement of str with os.fspath as the
>> recommended coercion function that removes the major barrier to
>> pathlib adoption.
>
>
> honestly, I'm not sure about that -- burying a str() call in a lib is
> clearly risky, but using one in your user code, not so much -- you usually
> know you have a path-like object (if not a Path) itself. the extra need for
> wrapping str() around things was a barrier to pathlib adoption -- and it
> doesn't take much of a barrier to reduce the adoption of something new.

Exactly - the key barrier to adoption was the lack of "os.fspath()",
since it meant that neither the standard library *nor* third party
helper libraries could reliably deal with the conversion task on
behalf of their users. Instead, people had to carefully manage their
own code, ensuring both that they used str() in the appropriate
places, *and* that they didn't accidentally call str() on the wrong
thing.

os.fspath() changes that to be much simpler and friendlier to an
iterative development process:

1. By default, expect things to just work
2. If they blow up with TypeError, wrap the offending argument in a
call to os.fspath()
3. (optional) file an RFE with the maintainers of the affected API to
call os.fspath() implicitly on that argument

The number of APIs that fall into the second category is going to be
non-zero for quite a while (both in the standard library and
elsewhere), but enough folks proceeding to step 3 will bring it down
over time.

Outside customers complaining to their commercial support vendors
though, it's rarely helpful to insist to maintainers (whether of
standard library modules or third party ones) that their failure to
adopt a feature only added in the latest version of Python now counts
as a bug in their software.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

[Nick Coghlan]

On 8 May 2017 at 15:34, Nick Coghlan  wrote:
> On 7 May 2017 at 15:22, INADA Naoki  wrote:
>> ## Background
>>
>> Locale coercion in current PEP 538 has some downsides:
>>
>> * If user set `LANG=C LC_DATE=ja_JP.UTF-8`, locale coercion may
>>   overrides LC_DATE.
>
> The fact it sets "LC_ALL" has previously been raised as a concern with
> PEP 538, so it probably makes sense to drop that aspect and just
> override "LANG". The scenarios where it makes a difference are
> incredibly obscure (involving non-default SSH locale forwarding
> settings for folks using SSH on Mac OS X to connect to remote Linux
> systems), while just setting "LANG" will be sufficient to address the
> "LANG=C" case that is the main driver for the PEP.

It occurs to me we can even still handle the forwarded
"LC_CTYPE=UTF-8" case by changing the locale coercion to set LC_CTYPE
& LANG, rather than just setting LANG as I suggested above.

That way `LANG=C LC_DATE=ja_JP.UTF-8` would still respect the explicit
LC_DATE setting, `LC_CTYPE=C` would be handled the same way as
`LANG=C`, and LC_ALL=C would continue to provide a way to force the C
locale even for LC_CTYPE without needing to be aware of the Python
specific PYTHONCOERCECLOCALE setting.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Chris Barker]

On Fri, May 5, 2017 at 9:30 PM, Nick Coghlan  wrote:

> By contrast, 3.6 users can just unconditionally call os.fspath(), and
> know the result will work with all stdlib APIs, without allowing
> nonsense like attempting to use "str(str)" as a filesystem path.
>


> Doing that implicitly in various APIs is certainly a nice UX
> enhancement, but it's the replacement of str with os.fspath as the
> recommended coercion function that removes the major barrier to
> pathlib adoption.


honestly, I'm not sure about that -- burying a str() call in a lib is
clearly risky, but using one in your user code, not so much -- you usually
know you have a path-like object (if not a Path) itself. the extra need for
wrapping str() around things was a barrier to pathlib adoption -- and it
doesn't take much of a barrier to reduce the adoption of something new.

The goal is clearly that ALL stdlib function take a path_like -- but if
we're not there yet, we're not there yet :-(

-CHB






>
> > Is there any chance of it breaking already working code? That would be
> the
> > one compelling reason to not back-port.
>
> "Don't make the third party compatibility testing matrix impossibly
> complicated" is our other major reason not to backport new features.
>
> We already have platform providers supporting 3.6.0 in the wild
> (Heroku and AWS Lambda), and we know 3.6.1 is going to be in at least
> Fedora 26, and probably Ubuntu 17.10.
>
> We want CI providers like Travis/GitLab/Circle CI/etc to be able to
> provide a *single* 3.6.x release (typically the most recent one), and
> have the guarantee be "code that passes on this maintenance release
> will only fail on earlier maintenance releases if you hit a genuine
> bug in those releases".
>
> Changing a function signature from accepting "Union[str, bytes]" (or
> potentially even just "str") to instead accepting "os.PathLike"
> doesn't count as fixing a genuine bug in the earlier releases - it
> just means we made the API more permissive in a maintenance release.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

[Chris Barker]

On Fri, May 5, 2017 at 2:34 PM, Terry Reedy  wrote:

> So it would be really great if any updates to shutils (and other stdlib
>> packages) to support the new protocol be back-ported.
>>
>
> Not if they change the language.


we're not talking about language changes here -- we are talking about
updates to the stdlib.

Eric wrote:

The problem is that when someone writes code a year from now and tests it
> under Python 3.6.2, then a customer of theirs finds it doesn't work in
> 3.6.1. This will happen if 3.6.2 supports Path parameters to functions that
> 3.6.1 does not.
>

This does, of course apply to the stdlib -- not just the language.

We've burned ourselves before with this, most famously with True and False
> in 2.2.1: http://python-history.blogspot.com/2013/11/the-history-of-
> bool-true-and-false.html


I'd argue that adding three built-ins is a much bigger deal than enhancing
a subset of one standard package -- not that the same issues wouldn't arise.

I think it's still worth, but I can certainly see why it might not be.

Too bad we didn't do a bit more testing for completely when it was beta

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

[INADA Naoki]

>>> On platforms where they would have no effect (e.g. Mac OS X, iOS, Android,
>>> Windows) these preprocessor variables would always be undefined.
>>
>> Why ``--with[out]-c-locale-coercion`` have no effect on macOS, iOS and 
>> Android?
>
> On these three, we know the system encoding is UTF-8, so we never
> interpreted the C locale as meaning "ascii" in the first place.
>
>> On Android, locale coercion fixes readline.  Do you mean locale
>> coercion happen always
>> regardless this configuration option?
>
> Right, the change for Android is that we switch to calling
> 'setlocale(LC_ALL, "C.UTF-8")' during interpreter startup instead of
> 'setlocale(LC_ALL, "")'. That change is guarded by "#ifdef
> __ANDROID__", rather than either of the new conditionals.
>
>> On macOS, ``LC_ALL=C python`` doesn't make Python's stdio to
>> ``ascii:surrogateescape``?
>
> Similar to Android, CPython itself is hardcoded to assume UTF-8 on Mac
> OS X, since that's a platform API guarantee that users can't change.
>
>> Even so, locale coercion may fix libraries like readline, curses.
>> While C locale is less common on macOS, I don't understand any
>> reason to disable it on macOS.
>
> My understanding is that other libraries and applications also
> automatically use UTF-8 for system interfaces on Mac OS X and iOS. It
> could be that that understanding is wrong, and locale coercion would
> provide a benefit there as well.
>
> (Checking the draft implementation, it turns out I haven't actually
> implemented the configure logic to make those config settings platform
> dependent yet - they're currently only undefined on Windows by
> default, since that doesn't use the autotools based build system)
>

I tried Python 3.6 on macOS 10.11 El Capitan.

  $ LANG=C python3 -c 'import locale; print(locale.getpreferredencoding())'
  US-ASCII

And interactive shell (which uses readline by default) doesn't accept non-ASCII
input anymore.
https://www.dropbox.com/s/otshuzhnw7a71n5/macos-c-locale-readline.gif?dl=0

I think many problems with C locale are same on macOS too.
So I don't think no special casing is required on macOS.

Regards,
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com