Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

2017-05-09 Thread Nick Coghlan 
On 9 May 2017 at 21:57, Nick Coghlan  wrote:
> The essential related changes to the reference implementation can be seen 
> here:
>
> * Always set "surrogateescape" for coercion target locales,
> independently of whether or not coercion occurred:
> https://github.com/ncoghlan/cpython/commit/188e7807b6d9e49377aacbb287c074e5cabf70c5
> * Stop setting LC_ALL:
> https://github.com/python/peps/commit/2f530ce0d1fd24835ac0c6f984f40db70482a18f

Sorry, I just noticed the copy & paste error when posting that second
link. The correct link is:
https://github.com/ncoghlan/cpython/commit/476a78133c94d82e19b89f50036cecd9b4214e7a

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 proposal: don't default to Optional if argument default is None

2017-05-09 Thread Guido van Rossum 
On Tue, May 9, 2017 at 7:37 PM, Nick Coghlan  wrote:

> On 10 May 2017 at 08:51, Brett Cannon  wrote:
> > On Tue, 9 May 2017 at 11:11 Carl Meyer  wrote:
> >> It might be nice to have a less verbose syntax for Optional, but that
> >> can be a separate discussion.
> >
> > You should be able to do that today with `from typing import Optional as
> Eh`
> > or whatever your preferred optional/maybe name is. :)
>
> While "from typing import Optional as Opt" can indeed help, perhaps
> PEP 505 should be updated to discuss this point in addition to the
> current proposals for None-aware binary operators?
>
> If it included a ? prefix operator as a shorthand for
> "typing.Optional[]", that would shorten affected declarations
> back to:
>
> def handle_employee(e: ?Employee = None) -> None: ...
>

I really don't want to go there. And this idea should definitely not be a
condition for removing the existing PEP 484 feature. Whatever gets done
syntax-wise won't affect anyone who needs any kind of backward
compatibility anyways, and that's very important for practical adoption of
PEP 484.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 proposal: don't default to Optional if argument default is None

2017-05-09 Thread Nick Coghlan 
On 10 May 2017 at 08:51, Brett Cannon  wrote:
> On Tue, 9 May 2017 at 11:11 Carl Meyer  wrote:
>> It might be nice to have a less verbose syntax for Optional, but that
>> can be a separate discussion.
>
> You should be able to do that today with `from typing import Optional as Eh`
> or whatever your preferred optional/maybe name is. :)

While "from typing import Optional as Opt" can indeed help, perhaps
PEP 505 should be updated to discuss this point in addition to the
current proposals for None-aware binary operators?

If it included a ? prefix operator as a shorthand for
"typing.Optional[]", that would shorten affected declarations
back to:

def handle_employee(e: ?Employee = None) -> None: ...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-05-09 Thread Naomi Seyfer 
Stay tuned for the pep that allows callable to take keyword args. 

> On May 9, 2017, at 3:59 PM, Brett Cannon  wrote:
> 
> The idea seems reasonable to me when viewing type hints as a form of 
> documentation as it helps remind people how they are expected to call the 
> final function.
> 
> One worry I do have, though, is Callable doesn't support keyword-only 
> parameters, so declared_type won't work in all cases without Callable gaining 
> such support (for those that don't know, Callable didn't start with that 
> support as Callable has been meant for callback scenarios up to this point).
> 
>> On Tue, 9 May 2017 at 10:21 Guido van Rossum  wrote:
>> There's a PR to the peps proposal here:
>> https://github.com/python/peps/pull/242
>> 
>> The full text of the current proposal is below. The motivation for this is 
>> that for complex decorators, even if the type checker can figure out what's 
>> going on (by taking the signature of the decorator into account), it's 
>> sometimes helpful to the human reader of the code to be reminded of the type 
>> after applying the decorators (or a stack thereof). Much discussion can be 
>> found in the PR. Note that we ended up having `Callable` in the type because 
>> there's no rule that says a decorator returns a function type (e.g. 
>> `property` doesn't).
>> 
>> This is a small thing but I'd like to run it by a larger audience than the 
>> core mypy devs who have commented so far. There was a brief discussion on 
>> python-ideas (my original, favorable reply by Nick, my response).
>> 
>> Credit for the proposal goes to Naomi Seyfer, with discussion by Ivan 
>> Levkivskyi and Jukka Lehtosalo.
>> 
>> If there's no further debate here I'll merge it into the PEP and an 
>> implementation will hopefully appear in the next version of the typing 
>> module (also hopefully to be included in CPython 3.6.2 and 3.5.4).
>> 
>> Here's the proposed text (wordsmithing suggestions in the PR please):
>> 
>> +Decorators
>> +--
>> +
>> +Decorators can modify the types of the functions or classes they
>> +decorate. Use the ``decorated_type`` decorator to declare the type of
>> +the resulting item after all other decorators have been applied::
>> +
>> + from typing import ContextManager, Iterator, decorated_type
>> + from contextlib import contextmanager
>> +
>> + class DatabaseSession: ...
>> +
>> + @decorated_type(Callable[[str], ContextManager[DatabaseSession]])
>> + @contextmanager
>> + def session(url: str) -> Iterator[DatabaseSession]:
>> + s = DatabaseSession(url)
>> + try:
>> + yield s
>> + finally:
>> + s.close()
>> +
>> +The argument of ``decorated_type`` is a type annotation on the name
>> +being declared (``session``, in the example above). If you have
>> +multiple decorators, ``decorated_type`` must be topmost. The
>> +``decorated_type`` decorator is invalid on a function declaration that
>> +is also decorated with ``overload``, but you can annotate the
>> +implementation of the overload series with ``decorated_type``.
>> +
>> 
>> -- 
>> --Guido van Rossum (python.org/~guido)
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-05-09 Thread Brett Cannon 
The idea seems reasonable to me when viewing type hints as a form of
documentation as it helps remind people how they are expected to call the
final function.

One worry I do have, though, is Callable doesn't support keyword-only
parameters, so declared_type won't work in all cases without Callable
gaining such support (for those that don't know, Callable didn't start with
that support as Callable has been meant for callback scenarios up to this
point).

On Tue, 9 May 2017 at 10:21 Guido van Rossum  wrote:

> There's a PR to the peps proposal here:
> https://github.com/python/peps/pull/242
>
> The full text of the current proposal is below. The motivation for this is
> that for complex decorators, even if the type checker can figure out what's
> going on (by taking the signature of the decorator into account), it's
> sometimes helpful to the human reader of the code to be reminded of the
> type after applying the decorators (or a stack thereof). Much discussion
> can be found in the PR. Note that we ended up having `Callable` in the type
> because there's no rule that says a decorator returns a function type (e.g.
> `property` doesn't).
>
> This is a small thing but I'd like to run it by a larger audience than the
> core mypy devs who have commented so far. There was a brief discussion on
> python-ideas (my original
> ,
> favorable reply
>  by
> Nick, my response
> ).
>
> Credit for the proposal goes to Naomi Seyfer, with discussion by Ivan
> Levkivskyi and Jukka Lehtosalo.
>
> If there's no further debate here I'll merge it into the PEP and an
> implementation will hopefully appear in the next version of the typing
> module (also hopefully to be included in CPython 3.6.2 and 3.5.4).
>
> Here's the proposed text (wordsmithing suggestions in the PR please):
>
> +Decorators
> +--
> +
> +Decorators can modify the types of the functions or classes they
> +decorate. Use the ``decorated_type`` decorator to declare the type of
> +the resulting item after all other decorators have been applied::
> +
> + from typing import ContextManager, Iterator, decorated_type
> + from contextlib import contextmanager
> +
> + class DatabaseSession: ...
> +
> + @decorated_type(Callable[[str], ContextManager[DatabaseSession]])
> + @contextmanager
> + def session(url: str) -> Iterator[DatabaseSession]:
> + s = DatabaseSession(url)
> + try:
> + yield s
> + finally:
> + s.close()
> +
> +The argument of ``decorated_type`` is a type annotation on the name
> +being declared (``session``, in the example above). If you have
> +multiple decorators, ``decorated_type`` must be topmost. The
> +``decorated_type`` decorator is invalid on a function declaration that
> +is also decorated with ``overload``, but you can annotate the
> +implementation of the overload series with ``decorated_type``.
> +
>
> --
> --Guido van Rossum (python.org/~guido )
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 proposal: don't default to Optional if argument default is None

2017-05-09 Thread Brett Cannon 
On Tue, 9 May 2017 at 11:11 Carl Meyer  wrote:

> On 05/09/2017 10:28 AM, Guido van Rossum wrote:
> > There's a proposal to change one detail of PEP 484. It currently says:
> >
> > An optional type is also automatically assumed when the default
> value is
> > |None|, for example::
> >
> > |def handle_employee(e: Employee = None): ... |
> >
> > This is equivalent to::
> >
> > |def handle_employee(e: Optional[Employee] = None) -> None: ... |
> >
> >
> > Now that we've got some experience actually using Optional with mypy
> > (originally mypy ignored Optional), we're beginning to think that this
> > was a bad idea. There's more discussion at
> > https://github.com/python/typing/issues/275 and an implementation of the
> > change (using a command-line flag) in
> > https://github.com/python/mypy/pull/3248.
> >
> > Thoughts? Some function declarations will become a bit more verbose, but
> > we gain clarity (many users of annotations don't seem to be familiar
> > with this feature) and consistency (since this rule doesn't apply to
> > variable declarations and class attribute declarations).
>
> I've been code-reviewing a lot of diffs adding type coverage over the
> last few months, and implicit-Optional has been among the most common
> points of confusion. So I favor this change.
>

I personally like the shorthand, but the type hints I have written are in
my own projects so my experience with others is zilch and shouldn't count
for much.


>
> It might be nice to have a less verbose syntax for Optional, but that
> can be a separate discussion.
>

You should be able to do that today with `from typing import Optional as
Eh` or whatever your preferred optional/maybe name is. :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] mention aenum in the Enum docs?

2017-05-09 Thread Brett Cannon 
On Tue, 9 May 2017 at 10:59 Mark Lawrence via Python-Dev <
python-dev@python.org> wrote:

> On 09/05/2017 18:05, Ethan Furman wrote:
> > A comment on a recent SO answer [1] wondered why my aenum library wasn't
> > mentioned in the docs to help guide people that needed/wanted more
> > advanced Enum options to it.  I responded that Python was not in the
> > habit of mentioning third-party libraries in the docs.
> >
> > However, I thought I would double-check here to see if it might be a
> > good idea.
> >
> > Pros:
> > - drop-in replacement for the stdlib Enum
> > - has many advanced features such as
> >- auto __init__ building
> >- multi-value members
> >- duplicate value but non-aliasing members
> >- etc.
> > - I'm the primary/only maintainer for both
> >
> > Cons:
> > - third-party library
> >
> > Thoughts?
> >
> > --
> > ~Ethan~
> >
> >
> > [1] http://stackoverflow.com/a/43855536/208880
>
> The precedent is all ready set as the third-party regex module gets a
> mention here https://docs.python.org/3/library/re.html and the requests
> package here https://docs.python.org/3/library/urllib.request.html.
>

Mark's right that we already do this for select modules. Since aenum is by
the author of enum to add things that don't fit in the stdlib and has been
out for years (i.e. by you, Ethan :) , I see nothing wrong with linking to
it.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-05-09 Thread Abdur-Rahmaan Janhangeer 
I'm really new to the mailing list. Can someone just summarise the
preceding message in 5 to 10 lines like what it is, what type is it or when
does it happen

Thanks for all,

Abdur-Rahmaan Janhangeer
Vacoas,
Mauritius
https://abdurrahmaanjanhangeer.wordpress.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 proposal: don't default to Optional if argument default is None

2017-05-09 Thread Carl Meyer 
On 05/09/2017 10:28 AM, Guido van Rossum wrote:
> There's a proposal to change one detail of PEP 484. It currently says:
> 
> An optional type is also automatically assumed when the default value is
> |None|, for example::
> 
> |def handle_employee(e: Employee = None): ... |
> 
> This is equivalent to::
> 
> |def handle_employee(e: Optional[Employee] = None) -> None: ... |
> 
> 
> Now that we've got some experience actually using Optional with mypy
> (originally mypy ignored Optional), we're beginning to think that this
> was a bad idea. There's more discussion at
> https://github.com/python/typing/issues/275 and an implementation of the
> change (using a command-line flag) in
> https://github.com/python/mypy/pull/3248.
> 
> Thoughts? Some function declarations will become a bit more verbose, but
> we gain clarity (many users of annotations don't seem to be familiar
> with this feature) and consistency (since this rule doesn't apply to
> variable declarations and class attribute declarations).

I've been code-reviewing a lot of diffs adding type coverage over the
last few months, and implicit-Optional has been among the most common
points of confusion. So I favor this change.

It might be nice to have a less verbose syntax for Optional, but that
can be a separate discussion.

Carl



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] mention aenum in the Enum docs?

2017-05-09 Thread Mark Lawrence via Python-Dev 

On 09/05/2017 18:05, Ethan Furman wrote:
A comment on a recent SO answer [1] wondered why my aenum library wasn't 
mentioned in the docs to help guide people that needed/wanted more 
advanced Enum options to it.  I responded that Python was not in the 
habit of mentioning third-party libraries in the docs.


However, I thought I would double-check here to see if it might be a 
good idea.


Pros:
- drop-in replacement for the stdlib Enum
- has many advanced features such as
   - auto __init__ building
   - multi-value members
   - duplicate value but non-aliasing members
   - etc.
- I'm the primary/only maintainer for both

Cons:
- third-party library

Thoughts?

--
~Ethan~


[1] http://stackoverflow.com/a/43855536/208880


The precedent is all ready set as the third-party regex module gets a 
mention here https://docs.python.org/3/library/re.html and the requests 
package here https://docs.python.org/3/library/urllib.request.html.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 484 proposal: don't default to Optional if argument default is None

2017-05-09 Thread Guido van Rossum 
There's a proposal to change one detail of PEP 484. It currently says:

An optional type is also automatically assumed when the default value is
None, for example::

  def handle_employee(e: Employee = None): ...

This is equivalent to::

  def handle_employee(e: Optional[Employee] = None) -> None: ...


Now that we've got some experience actually using Optional with mypy
(originally mypy ignored Optional), we're beginning to think that this was
a bad idea. There's more discussion at
https://github.com/python/typing/issues/275 and an implementation of the
change (using a command-line flag) in
https://github.com/python/mypy/pull/3248.

Thoughts? Some function declarations will become a bit more verbose, but we
gain clarity (many users of annotations don't seem to be familiar with this
feature) and consistency (since this rule doesn't apply to variable
declarations and class attribute declarations).

-- 
--Guido van Rossum (python.org/~guido )
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-05-09 Thread Guido van Rossum 
There's a PR to the peps proposal here:
https://github.com/python/peps/pull/242

The full text of the current proposal is below. The motivation for this is
that for complex decorators, even if the type checker can figure out what's
going on (by taking the signature of the decorator into account), it's
sometimes helpful to the human reader of the code to be reminded of the
type after applying the decorators (or a stack thereof). Much discussion
can be found in the PR. Note that we ended up having `Callable` in the type
because there's no rule that says a decorator returns a function type (e.g.
`property` doesn't).

This is a small thing but I'd like to run it by a larger audience than the
core mypy devs who have commented so far. There was a brief discussion on
python-ideas (my original
,
favorable reply
 by
Nick, my response
).

Credit for the proposal goes to Naomi Seyfer, with discussion by Ivan
Levkivskyi and Jukka Lehtosalo.

If there's no further debate here I'll merge it into the PEP and an
implementation will hopefully appear in the next version of the typing
module (also hopefully to be included in CPython 3.6.2 and 3.5.4).

Here's the proposed text (wordsmithing suggestions in the PR please):

+Decorators
+--
+
+Decorators can modify the types of the functions or classes they
+decorate. Use the ``decorated_type`` decorator to declare the type of
+the resulting item after all other decorators have been applied::
+
+ from typing import ContextManager, Iterator, decorated_type
+ from contextlib import contextmanager
+
+ class DatabaseSession: ...
+
+ @decorated_type(Callable[[str], ContextManager[DatabaseSession]])
+ @contextmanager
+ def session(url: str) -> Iterator[DatabaseSession]:
+ s = DatabaseSession(url)
+ try:
+ yield s
+ finally:
+ s.close()
+
+The argument of ``decorated_type`` is a type annotation on the name
+being declared (``session``, in the example above). If you have
+multiple decorators, ``decorated_type`` must be topmost. The
+``decorated_type`` decorator is invalid on a function declaration that
+is also decorated with ``overload``, but you can annotate the
+implementation of the overload series with ``decorated_type``.
+

-- 
--Guido van Rossum (python.org/~guido )
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] mention aenum in the Enum docs?

2017-05-09 Thread Ethan Furman 
A comment on a recent SO answer [1] wondered why my aenum library wasn't mentioned in the docs to help guide people that 
needed/wanted more advanced Enum options to it.  I responded that Python was not in the habit of mentioning third-party 
libraries in the docs.


However, I thought I would double-check here to see if it might be a good idea.

Pros:
- drop-in replacement for the stdlib Enum
- has many advanced features such as
  - auto __init__ building
  - multi-value members
  - duplicate value but non-aliasing members
  - etc.
- I'm the primary/only maintainer for both

Cons:
- third-party library

Thoughts?

--
~Ethan~


[1] http://stackoverflow.com/a/43855536/208880
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "make touch" replaced with "make regen-all"

2017-05-09 Thread Jeremy Kloth 
On Tue, May 9, 2017 at 5:20 AM, Victor Stinner  wrote:
> This change broke the Coverage job on Travis CI, because sysconfig.py
> uses get_config_var('AST_H_DIR') to build sysconfig.get_python_inc():
> http://bugs.python.org/issue30273
>
> sysconfig was modified in 2012. "Include" was replaced with
> get_config_var('AST_H_DIR') by:
> https://hg.python.org/cpython/rev/998c8a8f2aea
> => see http://bugs.python.org/issue15366
>
> To fix the Coverage job, I reverted this change:
> https://github.com/python/cpython/commit/b109a1d3360fc4bb87b9887264e3634632d392ca
>
> I'm unable to reproduce http://bugs.python.org/issue15366 bug: "venv
> assumes header files in sys._home + '/Include' ".

To reproduce, Python need to be compiled {builddir} != {srcdir} as
noted in the README.
I've submitted a PR that fixes this (previous?) regression and still
permits the Makefile changes.

> It seems like venv and virtualend have been updated in the meanwhile
> to add ${venv}/include to the include directories when building an
> extension.

Yes, http://bugs.python.org/issue16116

-- 
Jeremy Kloth
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

2017-05-09 Thread Nick Coghlan 
Hi folks,

Enough changes have accumulated in PEP 538 since the start of the
previous thread that it seems sensible to me to start a new thread
specifically covering the current design (which aims to address all
the concerns raised in the previous thread).

I haven't requoted the PEP in full since it's so long, but will
instead refer readers to the web version:
https://www.python.org/dev/peps/pep-0538/

I also generated a diff covered the full changes to the PEP text:

* https://gist.github.com/ncoghlan/1067805fe673b3735ac854e195747493/revisions
(this is the diff covering the last few days of changes

Summarising the key technical changes:

* to make the runtime behaviour independent of whether or not locale
coercion took place, stdin and stderr now always have
"surrogateescape" as their error handler in the potential coercion
target locales. This means Python will behave the same way regardless
of whether the locale gets set externally (e.g. by a parent Python
process or a container image definition) or implicitly during CLI
startup
* for the full locales, the interpreter now sets LC_CTYPE and LANG,
*not* LC_ALL. This means LC_ALL is once again a full locale override,
and also means that CPython won't inadvertently interfere with other
locale categories like LC_MONETARY, LC_NUMERIC, etc
* the reference implementation has been refactored so the bulk of the
new code lives in the shared library and is exposed to the linker via
a couple of underscore prefixed API symbols
(_Py_LegacyLocaleDetected() and _Py_CoerceLegacyLocale()). While the
current PEP still keeps them private, it would be straightforward to
make them public for use in embedding applications if we decided we
wanted to do so.
* locale coercion and warnings are now enabled by default on all
platforms that use the autotools-based build chain - the assumption
that some platforms didn't need them turned out to be incorrect

In addition to being updated to cover the above changes, the Rationale
section of the PEP has also been updated to explain why it doesn't
propose setting PYTHONIOENCODING, and to walk through some examples of
the problems with GNU readlines compatibility when the current locale
isn't set correctly.

The essential related changes to the reference implementation can be seen here:

* Always set "surrogateescape" for coercion target locales,
independently of whether or not coercion occurred:
https://github.com/ncoghlan/cpython/commit/188e7807b6d9e49377aacbb287c074e5cabf70c5
* Stop setting LC_ALL:
https://github.com/python/peps/commit/2f530ce0d1fd24835ac0c6f984f40db70482a18f

(There are also some smaller cleanup commits that can be seen by
browsing that branch on GitHub)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "make touch" replaced with "make regen-all"

2017-05-09 Thread Victor Stinner 
2017-05-04 19:51 GMT+02:00 Raymond Hettinger :
> Yes.  It is perfectly reasonable to backport improvements to the tooling as 
> long as it doesn't break anyone's existing build process.

I pushed my change to 2.7, 3.5, 3.6 and master (3.7) branches: "make"
doesn't try to regenerate generated files based on file modification
time, "make regen-all" is now required instead.

This change broke the Coverage job on Travis CI, because sysconfig.py
uses get_config_var('AST_H_DIR') to build sysconfig.get_python_inc():
http://bugs.python.org/issue30273

sysconfig was modified in 2012. "Include" was replaced with
get_config_var('AST_H_DIR') by:
https://hg.python.org/cpython/rev/998c8a8f2aea
=> see http://bugs.python.org/issue15366

To fix the Coverage job, I reverted this change:
https://github.com/python/cpython/commit/b109a1d3360fc4bb87b9887264e3634632d392ca

I'm unable to reproduce http://bugs.python.org/issue15366 bug: "venv
assumes header files in sys._home + '/Include' ".

It seems like venv and virtualend have been updated in the meanwhile
to add ${venv}/include to the include directories when building an
extension.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-05-09 Thread Nick Coghlan 
On 9 May 2017 at 13:44, Nick Coghlan  wrote:
> On 8 May 2017 at 15:34, Nick Coghlan  wrote:
>> On 7 May 2017 at 15:22, INADA Naoki  wrote:
>>> ## Background
>>>
>>> Locale coercion in current PEP 538 has some downsides:
>>>
>>> * If user set `LANG=C LC_DATE=ja_JP.UTF-8`, locale coercion may
>>>   overrides LC_DATE.
>>
>> The fact it sets "LC_ALL" has previously been raised as a concern with
>> PEP 538, so it probably makes sense to drop that aspect and just
>> override "LANG". The scenarios where it makes a difference are
>> incredibly obscure (involving non-default SSH locale forwarding
>> settings for folks using SSH on Mac OS X to connect to remote Linux
>> systems), while just setting "LANG" will be sufficient to address the
>> "LANG=C" case that is the main driver for the PEP.
>
> It occurs to me we can even still handle the forwarded
> "LC_CTYPE=UTF-8" case by changing the locale coercion to set LC_CTYPE
> & LANG, rather than just setting LANG as I suggested above.
>
> That way `LANG=C LC_DATE=ja_JP.UTF-8` would still respect the explicit
> LC_DATE setting, `LC_CTYPE=C` would be handled the same way as
> `LANG=C`, and LC_ALL=C would continue to provide a way to force the C
> locale even for LC_CTYPE without needing to be aware of the Python
> specific PYTHONCOERCECLOCALE setting.

I've posted an updated reference implementation that works this way,
and it turned out to have some rather nice benefits: not only did it
make the handling of full locales (C.UTF-8, C.utf8) and partial
locales (UTF-8) more consistent (allowing for a net deletion of code),
it also meant I no longer needed a custom test case in _testembed to
check the locale warning. Instead, the affected test cases now just
set "LC_ALL" as a locale override that switches off CPython's locale
coercion without also switching off the locale warning.

Code changes: 
https://github.com/ncoghlan/cpython/commit/476a78133c94d82e19b89f50036cecd9b4214e7a

Rather than posting the PEP updates here though, I'll start a new
thread that explains what has changed since my initial posting to
python-dev back in March.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com