Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-03-14 Thread Nick Coghlan 
On 15 March 2017 at 06:22, Chris Barker  wrote:

> So the question nis -- is anyone counting on errors in this case? i.e., is
> a sysadmin thinking:
>
> "I want an ASCII-only system, so I'll set the locale, and now I can expect
> any program running on this system that is not ascii compatible to fail."
>
> I honestly don't know if this is common -- but I would argue that trying
> to run a unicode-aware program on an ASCII-only system could be considered
> a mis-configuration as well.
>

>From a mainstream Linux point of view, it's not common - on systemd-managed
systems, for example, the only way to get the C locale these days is to
either specify it in /etc/locale.conf, or to set it specifically in the
environment. Upstart was a little less reliable about that, and sysvinit
was less reliable still, but the trend is definitely towards making C.UTF-8
the assumed default, rather than "C". Even glibc itself would quite like to
get to a point where you only get the C locale if you explicitly ask for
it: https://sourceware.org/glibc/wiki/Proposals/C.UTF-8

The main practical objection that comes up in relation to "UTF-8
everywhere" isn't to do with UTF-8 per se, but rather with the size of the
collation tables needed to do "proper" sorting of Unicode code points.
However, there's a neat hack in the design of UTF-8 where sorting the
encoded bytes by byte value is equivalent to sorting the decoded text by
the Unicode code point values, which means that "LC_COLLATE=C" sorting by
byte value, and "LC_COLLATE=C.UTF-8" sorting by "Unicode code point value"
give the same results.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-03-14 Thread Nick Coghlan 
On 15 March 2017 at 00:17, Nick Coghlan  wrote:

> On 13 March 2017 at 23:31, Random832  wrote:
>
>> On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote:
>> > But locale coercing works nice on platforms like android.
>> > So how about simplified version of PEP 538?  Just adding configure
>> > option for locale coercing
>> > which is disabled by default.  No envvar options and no warnings.
>>
>> A configure option just kicks the decision to packagers - either no-one
>> uses it (and thus it solves nothing) or people do use it (and any
>> problems it causes won't be mitigated at all)
>>
>
> Distro packagers have narrower user bases and a better known set of
> compatibility constraints than upstream, so kicking platform integration
> related config decisions downstream to us(/them) is actually a pretty
> reasonable thing for upstream to do :)
>
> For example, while I've been iterating on the reference implementation for
> 3.7, Charalampos Stratakis has been iterating on the backport patch for
> Fedora 26, and he's found that we really need the PEP's "disable the C
> locale warning" config option to turn off the CLI's coercion warning in
> addition to the warning in the shared library, as leaving it visible breaks
> build processes for other packages that check that there aren't any
> messages being emitted to stderr (or otherwise care about the exact output
> from build tools that rely on the system Python 3 runtime).
>

The build processes that broke due to the warning were judged to be a bug
in autoconf rather than a problem with the warning itself:
http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=commit;h=883a2abd5af5c96be894d5ef7ee6e9a2b8e64307

So we're going to leave this as it is in the PEP for now (i.e. the locale
coercion warning always happens unless you preconfigure a locale other than
C), but keep an eye on it to see if it causes any other problems.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-03-14 Thread Chris Barker 
There was a bunch of discussion about all this a while back, in which I
think these points were addressed:

However, in some cases the C locale is a normal environment for system
> services, cron scripts, distro package builds and whatnot.
>

Indeed it is. But:

if you run a Python (or any) program that is expecting an ASCII-only
locale, then it will work jsut fine with any ascii-compatible locale. -- so
no problem there.

On the other hand, if you run a program that is expectign a unicode-aware
locale, then it might barf unexpectently if run on a ASCII-only locale. A
lot of people do in fiact have these issues (which are due to
mis-configuration of the host system, which is indeed not properly Python's
problem).

So if we do all this, then:

A) mis-configured systems will magically work (sometimes)

 This is a Good Thing.

and

B) If someone runs a python program that is expecting Unicode support on an
properly configured ASCII-only system, then it will mostly "just work" --
after all a lot of C APIs are simply char*, who cares what the encoding is?
It would not, however, fail if when a non-ascii value is used somewhere it
shouldn't.

So the question nis -- is anyone counting on errors in this case? i.e., is
a sysadmin thinking:

"I want an ASCII-only system, so I'll set the locale, and now I can expect
any program running on this system that is not ascii compatible to fail."

I honestly don't know if this is common -- but I would argue that trying to
run a unicode-aware program on an ASCII-only system could be considered a
mis-configuration as well.

Also -- many programs will just be writing bytes to the system without
checking encoding anyway. So this would simply let Python3 programs behave
like most others...

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-03-14 Thread Random832 
On Tue, Mar 14, 2017, at 10:17, Nick Coghlan wrote:
> It's not that you *can't* run Python 3 in that kind of environment, and
> it's not that there are never any valid reasons to do so. It's that lots
> of
> things that you'd typically expect to work are going to misbehave (one I
> discovered myself yesterday is that the GNU readline problems reported in
> interactive mode on Android also show up when you do either "LANG=C
> python2" or "LANG=C python3" on traditional Linux and attempt to *edit*
> lines containing multi-byte characters)

It occurs to me that (at least for readline... and maybe also as a
general proxy for whether the rest should be done) detecting the IUTF8
terminal flag (which, properly, controls basic non-readline-based line
editing such as backspace) may be worthwhile.

(And maybe Readline itself should be doing this, more or less
independent of Python. But that's a discussion for elsewhere)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.13

2017-03-14 Thread Raúl Cumplido 
Hi Jorge,

This is the mailing list for the Python language development itself. It's
used for discussion of features and topics on the development of Python.
You can use other mailing lists for questions as:
python-h...@python.org --> ask for help to other people on the python
community
tu...@python.org --> if you are learning 

Kind Regards,
Raul

On Tue, Mar 14, 2017 at 2:32 PM,  wrote:

>
>
> Hi,
>
> I dowloaded the Python 2.7.13 and install it as root . it installed in my
> directory:
>
> /usr/src/Python-2.7.13
>
> Then I typed:
>
> python and I had:
>
> Python 2.7.13 (default, Mar 14 2017, 09:30:46)
> [GCC 4.9.2 20150212 (Red Hat 4.9.2-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
>
> Then I did:
>
> import numpy
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named numpy
>
>
> import scpy
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named scpy
>
>
> import basemap
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named basemap
>
>
> Please, what can I do to solve these.
>
>
> Conrado
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/raulcumpl
> ido%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.13

2017-03-14 Thread Oleg Broytman 
Hello.

   This mailing list is to work on developing Python (adding new
features to Python itself and fixing bugs); if you're having problems
learning, understanding or using Python, please find another forum.
Probably python-list/comp.lang.python mailing list/news group is the
best place; there are Python developers who participate in it; you may
get a faster, and probably more complete, answer there. See
http://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Tue, Mar 14, 2017 at 11:32:36AM -0300, jorge.conr...@cptec.inpe.br wrote:
> I dowloaded the Python 2.7.13 and install it as root . it installed in my
> directory:
> 
> /usr/src/Python-2.7.13
> 
> Then I typed:
> 
> python and I had:
> 
> Python 2.7.13 (default, Mar 14 2017, 09:30:46)
> [GCC 4.9.2 20150212 (Red Hat 4.9.2-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> 
> Then I did:
> 
> import numpy
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named numpy
> 
> import scpy
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named scpy
> 
> import basemap
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named basemap
> 
> Please, what can I do to solve these.

   These modules are not in the standard library, you have to download
and install them separately. I recommend you to learn what is PyPI and
how to use `pip install`.

> Conrado

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 2.7.13

2017-03-14 Thread jorge . conrado 



Hi,

I dowloaded the Python 2.7.13 and install it as root . it installed in 
my directory:


/usr/src/Python-2.7.13

Then I typed:

python and I had:

Python 2.7.13 (default, Mar 14 2017, 09:30:46)
[GCC 4.9.2 20150212 (Red Hat 4.9.2-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.


Then I did:

import numpy
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named numpy


import scpy
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named scpy


import basemap
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named basemap


Please, what can I do to solve these.


Conrado
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Regarding writing tests for module tabnanny

2017-03-14 Thread Jaysinh Shukla 

On Monday 13 March 2017 10:11 PM, Brett Cannon wrote:
These questions are best asked on the core-mentorship mailing list, 
Jaysinh, but to quickly answer your question:


Thanks Brett for replaying. I will take care of this next time.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] A big THANK YOU to the maintainers of What's New

2017-03-14 Thread Chris Angelico 
A bit of a thankless job, updating What's New for a bugfix release,
but can be so important. Today I was trying to figure out why a Python
script behaved differently on my dev system and my server, even when I
used Python 3.4 on both ends - but it was 3.4.4 on one and 3.4.2 on
the other. My first port of call:

https://docs.python.org/3.4/whatsnew/changelog.html#python-3-4-4

Search for 'argparse'. Find this:

Issue #9351: Defaults set with set_defaults on an argparse subparser
are no longer ignored when also set on the parent parser.

Bingo. That's where the difference came from.

So, thank you to the release managers and those who volunteer to trawl
the tracker and put stuff into What's New! It is very much
appreciated.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-03-14 Thread Nick Coghlan 
On 13 March 2017 at 23:31, Random832  wrote:

> On Mon, Mar 13, 2017, at 04:37, INADA Naoki wrote:
> > But locale coercing works nice on platforms like android.
> > So how about simplified version of PEP 538?  Just adding configure
> > option for locale coercing
> > which is disabled by default.  No envvar options and no warnings.
>
> A configure option just kicks the decision to packagers - either no-one
> uses it (and thus it solves nothing) or people do use it (and any
> problems it causes won't be mitigated at all)
>

Distro packagers have narrower user bases and a better known set of
compatibility constraints than upstream, so kicking platform integration
related config decisions downstream to us(/them) is actually a pretty
reasonable thing for upstream to do :)

For example, while I've been iterating on the reference implementation for
3.7, Charalampos Stratakis has been iterating on the backport patch for
Fedora 26, and he's found that we really need the PEP's "disable the C
locale warning" config option to turn off the CLI's coercion warning in
addition to the warning in the shared library, as leaving it visible breaks
build processes for other packages that check that there aren't any
messages being emitted to stderr (or otherwise care about the exact output
from build tools that rely on the system Python 3 runtime).

However, when it comes to choosing the upstream config defaults, it's
important to keep in mind that one of the explicit goals of the PEP is to
modify PEP 11 to *formally drop upstream support* for running Python 3 in
the legacy C locale without using PEP 538, PEP 540 or a combination of the
two to assume UTF-8 instead of ASCII for system interfaces.

It's not that you *can't* run Python 3 in that kind of environment, and
it's not that there are never any valid reasons to do so. It's that lots of
things that you'd typically expect to work are going to misbehave (one I
discovered myself yesterday is that the GNU readline problems reported in
interactive mode on Android also show up when you do either "LANG=C
python2" or "LANG=C python3" on traditional Linux and attempt to *edit*
lines containing multi-byte characters), so you really need to know what
you're doing in order to operate under those constraints.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com