[issue27066] SystemError if custom opener returns -1

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added comments on Rietveld.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Hugh C. Lauer]

Hugh C. Lauer added the comment:

Thanks, I will try it when I get out of airports.

Hugh Lauer

On May 19, 2016 11:56 AM, Zachary Ware  wrote:

>

>
> Zachary Ware added the comment:
>
> Here's some pretty compelling evidence for a bad PYTHONHOME setting:
>
> C:\>set PYTHONHOME=C:\path\not\here
>
> C:\>py -3.5
> Fatal Python error: Py_Initialize: unable to load the file system codec
> ImportError: No module named 'encodings'
>
> Current thread 0x1db4 (most recent call first):
>
> C:\>set PYTHONHOME=C:\Python27\
>
> C:\>py -3.5
> Fatal Python error: Py_Initialize: unable to load the file system codec
>   File "C:\Python27\\lib\encodings\__init__.py", line 123
> raise CodecRegistryError,\
> ^
> SyntaxError: invalid syntax
>
> Current thread 0x0664 (most recent call first):
>
> C:\>set PYTHONHOME=
>
> C:\>py -3.5
> Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit 
> (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>>
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26536] Add the SIO_LOOPBACK_FAST_PATH option to socket.ioctl

[Daniel Stokes]

Daniel Stokes added the comment:

I have uploaded a new patch with the suggested changes, thanks for the review.

--
Added file: http://bugs.python.org/file42908/loopback_fast_path_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27066] SystemError if custom opener returns -1

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

Here's a proposed fix.

--
assignee:  -> barry
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file42907/27066-1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22558] Missing doc links to source code

[Yoni Lavi]

Yoni Lavi added the comment:

Friendly ping

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27066] SystemError if custom opener returns -1

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

On May 20, 2016, at 12:12 AM, ppperry wrote:

>Also, `OSError [Errno 0] Error` isn't the most helpful error message.

No, definitely not. ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27066] SystemError if custom opener returns -1

[ppperry]

ppperry added the comment:

Also, `OSError [Errno 0] Error` isn't the most helpful error message.

--
nosy: +ppperry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27066] SystemError if custom opener returns -1

[Barry A. Warsaw]

New submission from Barry A. Warsaw:

Let's say you use a custom opener, and that opener happens to return exactly 
-1.  You end up with a SystemError because NULL got returned without an 
exception being set:

def negative(fname, flags):
return -1


with open('/tmp/foo.txt', 'w', encoding='utf-8', opener=negative) as fp:
print('oops', file=fp)


% python3 /tmp/foo.py 
Traceback (most recent call last):
  File "/tmp/foo.py", line 5, in 
with open('/tmp/foo.txt', 'w', encoding='utf-8', opener=negative) as fp:
SystemError:  returned NULL without setting an error


Anything else and you get a relatively decent exception.  E.g. return -2 and 
you get an OSError.  Raise an exception and you get that exception.

The problem is pretty clear to see; when an opener is set, after coercing the 
fd to an integer, the check is made for that integer being -1, and then it 
jumps right to the exit.

Let's say you return some non-integer, like 'foo'.  Then the _PyLong_AsInt() 
will fail and a proper exception will be set.  So I think the "if (self->fd == 
-1)" clause just needs to check for an exception set first and set one if there 
isn't one before it does the "goto error".  I guess you'd want to see the same 
exception as if it returned say, -2:

Traceback (most recent call last):
  File "/tmp/foo.py", line 5, in 
with open('/tmp/foo.txt', 'w', encoding='utf-8', opener=negative) as fp:
OSError: [Errno 0] Error: '/tmp/foo.txt'

--
components: IO
messages: 265901
nosy: barry
priority: normal
severity: normal
stage: needs patch
status: open
title: SystemError if custom opener returns -1
type: crash
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27065] robotparser user agent considered hostile by mod_security rules.

[John Nagle]

New submission from John Nagle:

"robotparser" uses the default Python user agent when reading the "robots.txt" 
file, and there's no parameter for changing that.

Unfortunately, the "mod_security" add-on for Apache web server, when used with 
the standard OWASP rule set, blacklists the default Python USER-AGENT in Rule 
990002, User Agent Identification. It doesn't like certain HTTP USER-AGENT 
values. One of them is "python-httplib2". So any program in Python which 
accesses the web site will trigger this rule and be blocked form access.  

For regular HTTP accesses, it's possible to put a user agent string in the 
Request object and work around this. But "robotparser" has no such option. 

Worse, if "robotparser" has its read of "robots.txt" rejected, it interprets 
that as a "deny all" robots.txt file, and returns False for all "can_fetch()" 
requests.

--
components: Library (Lib)
messages: 265900
nosy: nagle
priority: normal
severity: normal
status: open
title: robotparser user agent considered hostile by mod_security rules.
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27064] Make py.exe default to Python 3 when used interactively

[Steve Dower]

Steve Dower added the comment:

LGTM.

The launcher is in a bit of a funny place wrt versioning, as it is now somewhat 
independent from the Python version it is installed with:

* won't be downgraded by 3.5 (3.4 will still stomp over the file itself)
* has a separate entry in Programs & Features
* won't be uninstalled by the regular Python installer
* (theoretically can be installed separately, but since we don't link to the 
MSI... ;) )
* this includes alpha/beta releases of 3.6

So I wouldn't have an issue with this going into 3.5, in the same way that we 
will update the bundled setuptools/pip in minor releases, but since we don't 
have an official OK for that just putting it into 3.6 is fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27064] Make py.exe default to Python 3 when used interactively

[Paul Moore]

New submission from Paul Moore:

By default, the launcher tries to launch (the latest version of) Python 2 on 
the user's machine. This can be altered with the configuration file, and if the 
user doesn't have Python 2 installed Python 3 will be used. Now that we are at 
Python 3.6, it's about time to change that default to try Python 3 first.

This was discussed on python-ideas in the thread starting at 
https://mail.python.org/pipermail/python-ideas/2016-March/038667.html. My 
summary of the consensus was at 
https://mail.python.org/pipermail/python-ideas/2016-March/038759.html

The key points were:

1. For interactive use, make py.exe launch the latest version
installed on the machine, without special-casing a preference for
Python 2.
2. For shebang lines where an unversioned name is used, retain the
current behaviour (for compatibility with Unix).
3. When the user explicitly chooses a version, or has configured the
launcher via the ini file or environment variables, no change to
current behaviour.
4. The change is small enough that it doesn't need a PEP.

The attached patch implements this behaviour.

I assume the patch is to be applied only to Python 3.6, as it is changed 
behaviour, not a bug fix.

--
assignee: paul.moore
components: Windows
files: launcher.diff
keywords: needs review, patch
messages: 265898
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Make py.exe default to Python 3 when used interactively
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file42906/launcher.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27061] 2.7.11 OS X pkg doesn't put proper version in the receipts

[Ned Deily]

Ned Deily added the comment:

This problem was brought up previously in Issue24052, which is still open.  I 
am closing this a duplicate add adding you to the nosy list there.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> OS X installer provides flat sub-packages with no version 
numbers

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24502] OS X installer provides flat sub-packages with no version numbers

[Ned Deily]

Changes by Ned Deily :


--
nosy: +Paul Bloch
priority: normal -> high
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

STINNER Victor added the comment:

> In short, I replayed exaclty the same scenario. And... Only raytrace remains 
> slower, (...)

Oh, it looks like the reference binary calls the garbage collector less 
frequently than the patched python. In the patched Python, collections of the 
generation 2 are needed, whereas no collection of the generation 2 is needed on 
the reference binary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27062] `inspect` doesn't have `__all__`

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I refer again to https://bugs.python.org/issue26632 :)

--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27063] Some unittest loader tests are silently skipped by mistake

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Due to bug in the decorator some tests in Lib/unittest/test/test_loader.py are 
silently skipped. Proposed patch fixes this as well as other bugs in the 
decorator and tests.

--
components: Tests
files: test_loader.patch
keywords: patch
messages: 265894
nosy: ezio.melotti, michael.foord, rbcollins, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Some unittest loader tests are silently skipped by mistake
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file42905/test_loader.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27062] `inspect` doesn't have `__all__`

[Franklin? Lee]

New submission from Franklin? Lee:

`inspect`'s names are pretty close to unique, except for `stack`, `unwrap`, and 
`trace`. It doesn't define `__all__`, though.

--
components: Library (Lib)
messages: 265893
nosy: leewz
priority: normal
severity: normal
status: open
title: `inspect` doesn't have `__all__`
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27060] Documentation of assertItemsEqual in unittest is VERY misleading in 2.7

[Georg Brandl]

Changes by Georg Brandl :


--
nosy: +michael.foord

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22570] Better stdlib support for Path objects

[Guido van Rossum]

Guido van Rossum added the comment:

Done. The revs are 90e58a77d386, 97198545e6c3, ade839421b8f.

--
resolution: out of date -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27061] 2.7.11 OS X pkg doesn't put proper version in the receipts

[Paul Bloch]

New submission from Paul Bloch:

Python 2.7.6 writes the version number (2.7.6) into the installed receipts. 
Python 2.7.11 calls the version "0". That isn't very helpful for managed 
installs.

--
components: Macintosh
files: typescript.txt
messages: 265891
nosy: Paul Bloch, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: 2.7.11 OS X pkg doesn't put proper version in the receipts
versions: Python 2.7
Added file: http://bugs.python.org/file42904/typescript.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17866] TestCase.assertItemsEqual exists in 2.7, not in 3.3

[Vitaly]

Vitaly added the comment:

I opened http://bugs.python.org/issue27060 regarding the erroneous 
documentation of assertItemsEqual in python 2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17866] TestCase.assertItemsEqual exists in 2.7, not in 3.3

[Vitaly]

Vitaly added the comment:

Python 2.7 documentation is VERY misleading about the functionality of 
assertItemsEqual. It actually claims to compare not only the counts, but the 
actual sorted elements themselves. This documentation mislead my group to use 
this method for comparing the elements. See 
https://hg.python.org/cpython/file/d9921cb6e3cd/Doc/library/unittest.rst:

   .. method:: assertItemsEqual(actual, expected, msg=None)

  Test that sequence *expected* contains the same elements as *actual*,
  regardless of their order. When they don't, an error message listing the
  differences between the sequences will be generated.

  Duplicate elements are *not* ignored when comparing *actual* and
  *expected*. It verifies if each element has the same count in both
  sequences. It is the equivalent of ``assertEqual(sorted(expected),
  sorted(actual))`` but it works with sequences of unhashable objects as
  well.

--
nosy: +vitaly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27060] Documentation of assertItemsEqual in unittest is VERY misleading in 2.7

[Vitaly]

Changes by Vitaly :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27060] Documentation of assertItemsEqual in unittest is VERY misleading in 2.7

[Vitaly]

Changes by Vitaly :


--
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27060] Documentation of assertItemsEqual in unittest is VERY misleading in 2.7

[Vitaly]

New submission from Vitaly:

Python 2.7 documentation is VERY misleading about the functionality of 
assertItemsEqual. The implementation only compares item counts, but the 
documentation actually claims to compare not only the counts, but the actual 
sorted elements themselves. This documentation mislead my group to use this 
method for comparing the elements. See 
https://hg.python.org/cpython/file/d9921cb6e3cd/Doc/library/unittest.rst, which 
is what appears on 
https://docs.python.org/2.7/library/unittest.html#unittest.TestCase.assertItemsEqual:

   .. method:: assertItemsEqual(actual, expected, msg=None)

  Test that sequence *expected* contains the same elements as *actual*,
  regardless of their order. When they don't, an error message listing the
  differences between the sequences will be generated.

  Duplicate elements are *not* ignored when comparing *actual* and
  *expected*. It verifies if each element has the same count in both
  sequences. It is the equivalent of ``assertEqual(sorted(expected),
  sorted(actual))`` but it works with sequences of unhashable objects as
  well.

--
messages: 265889
nosy: vitaly
priority: normal
severity: normal
status: open
title: Documentation of assertItemsEqual in unittest is VERY misleading in 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

[Ilya Kulakov]

Ilya Kulakov added the comment:

Yury, as you suggested posted to python-ideas 
(https://groups.google.com/forum/#!topic/python-ideas/ABOe22Mib44)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

STINNER Victor added the comment:

> Result of the benchmark suite:
>
> slower (3):
>
> * raytrace: 1.06x slower
> * etree_parse: 1.03x slower
> * normal_startup: 1.02x slower

Hum, I recompiled the patched Python, again with PGO+LTO, and ran the same 
benchmark with the same command. In short, I replayed exaclty the same 
scenario. And... Only raytrace remains slower, etree_parse and normal_startup 
moved to the "not significant" list.

The difference in the benchmark result doesn't come from the benchmark. For 
example, I ran gain the normal_startup benchmark 3 times: I got the same result 
3 times.

### normal_startup ###
Avg: 0.295168 +/- 0.000991 -> 0.294926 +/- 0.00048: 1.00x faster
Not significant

### normal_startup ###
Avg: 0.294871 +/- 0.000606 -> 0.294883 +/- 0.00072: 1.00x slower
Not significant

### normal_startup ###
Avg: 0.295096 +/- 0.000706 -> 0.294967 +/- 0.00068: 1.00x faster
Not significant

IMHO the difference comes from the data collected by PGO.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27059] find_spec and find_loader ignore package/path argument

[Brett Cannon]

Brett Cannon added the comment:

So there are 2 problems with your example. One, importlib.util.find_spec() 
doesn't take a path argument but a package argument which is a package name and 
not a file system location 
(https://docs.python.org/3/library/importlib.html#importlib.util.find_spec). 
Two, importlib.find_loader() takes a "path" as in "a package's path" which 
means __path__. What this means it is it expected to be an iterable of 
directories and not a string as it's iterated over 
(https://docs.python.org/3/library/importlib.html#importlib.find_loader).

If you think the docs aren't clear enough then please feel free to submit a 
patch to make them a bit clearer.

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25548] Show the address in the repr for class objects

[ppperry]

ppperry added the comment:

I don't quite get why the memory address is helpful, but you could work around 
this using a custom metaclass:

class IdMeta(type):
def __repr__(cls):
return super().__repr__()[:-1] + " at 0x%x>"%id(cls)
class IdInRepr(metaclass=IdMeta):pass

`IdInRepr` and all its subclasses will then have the memory address in their 
representation:

  

--
nosy: +ppperry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27053] Python Windows x86-64 embeddable zip file missing module encodings

[Steve Dower]

Changes by Steve Dower :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27053] Python Windows x86-64 embeddable zip file missing module encodings

[Roundup Robot]

Roundup Robot added the comment:

New changeset 12cb81e90479 by Steve Dower in branch '3.5':
Issue #27053: Updates make_zip.py to correctly generate library ZIP file.
https://hg.python.org/cpython/rev/12cb81e90479

New changeset a6473f7a8c55 by Steve Dower in branch 'default':
Issue #27053: Updates make_zip.py to correctly generate library ZIP file.
https://hg.python.org/cpython/rev/a6473f7a8c55

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[ppperry]

Changes by ppperry :


--
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27059] find_spec and find_loader ignore package/path argument

[Gerrit Ansmann]

New submission from Gerrit Ansmann:

According to my understanding, the `package`/`path` argument of the 
`find_loader`/`find_spec` command should enable it to find modules in other 
directories than the current one. Instead, these arguments seem to be ignored 
and even changing the working directory from within Python (using os.chdir) 
does not yield the expected results with `find_spec`.

I append a script with several test cases that should succeed in my 
understanding, but some of which don’t.

Python version: 3.4.3.
OS: Ubuntu 14.04.4
Pertinent Stack Overflow question: http://stackoverflow.com/a/37260391/2127008

--
components: Library (Lib)
files: test.py
messages: 265882
nosy: Wrzlprmft, brett.cannon
priority: normal
severity: normal
status: open
title: find_spec and find_loader ignore package/path argument
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file42903/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26369] unicode.decode and str.encode are unnecessarily confusing for non-ascii

[Josh Rosenberg]

Josh Rosenberg added the comment:

Agree with Steven; the whole reason Python 3 changed from unicode and str to 
str and bytes was because having Py2 str be text sometimes, and binary data at 
other times is confusing. The existing behavior can't change in Py2 in any 
meaningful way without breaking existing code, introducing special cases for 
text->text encodings (where Python 3 supports them using the codecs module 
only), behaving in non-obvious ways in corner cases, etc. Silently treating 
str.encode("utf-8") to mean "decode as UTF-8 and throw away the result to 
verify that it's already UTF-8 bytes" is not particularly intuitive either.

It does seem like a doc fix would be useful though; right now, we have only 
"String methods" documented, with no distinction between str and unicode. It 
might be helpful to explicitly deprecate str.encode on str objects, and 
unicode.decode, with a note that while it's meaningful to use these methods in 
Python 2 for text<->text encoding/decoding, the methods don't exist at all in 
Python 3.

Otherwise, yes, if you want consistent text/binary types, that's what Python 3 
is for. Python 2 has tons of flaws when it comes to handling unicode (e.g. csv 
module), and fixing any given single problem (creating backward compatibility 
headaches in the process) is not worth the trouble.

If you're concerned about excessive boilerplate, just write a function (or a 
type) that allows you to perform the tests/conversions you care about as a 
single call. For example, the following seems like it achieves your objectives 
(one line usage, handles str by verifying that it's legal in provided encoding 
in strict mode, dropping/replacing characters in ignore/replace mode, etc.):

def basestringencode(s, encoding=sys.getdefaultencoding(), errors="strict"):
if isinstance(s, str):
# Decode with provided rules, so a str with illegal characters
# raises exception, replaces, ignores, etc. per arguments
s = s.decode(encoding, errors)
return s.encode(encoding, errors)

If you don't want to see UnicodeDecodeError, you either pass 'ignore' for 
errors, or wrap the s.decode step in a try/except and raise a different 
exception type.

The biggest change I could see happening code wise would be a textual change to 
the UnicodeDecodeError error str.encode raises, so str.encode specifically 
replaces the default error message (but not type, for back compat reasons) with 
something like "str.encode cannot perform implicit decode with 
sys.getdefaultencoding(); use .encode only with unicode objects"

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27058] Tkinter's canvas' dashed lines have incorrect segment lengths

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

"Not our fault. Try reporting this to the Tcl team" ;-)

In any case thank you for your report Kevin.

--
resolution: not a bug -> third party
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26369] unicode.decode and str.encode are unnecessarily confusing for non-ascii

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> btw If anyone can find the place in the code (sorry I tried and failed!) 
> where str.encode('utf-8', error=X) is resulting in an implicit call to the 
> equivalent of decode(defaultencoding, errors=strict) (as suggested by the 
> exception message) I think it'll be easier to discuss the details of fixing.

There is no single place. Search lines "str = PyUnicode_FromObject(str);" in 
Modules/_codecsmodule.c.

> But that's not what happens - it *silently works* (is a no-op) as long as you 
> happen to be using ASCII characters so this so-called 'programming bug' will 
> go unnoticed by most programmers (and authors of third party library code you 
> might be relying on!)... but the moment a non-ascii character get introduced 
> suddenly you'll get an exception, maybe in some library code you rely on but 
> can't fix.

The problem is that encoding ASCII str to UTF-8 is legal operation in some 
circumstances and is a programming bug in other. There is no way to distinguish 
these two cases automatically.

As non-English speaker I am familiar with the problems you described. This is a 
bug in the design of Python 2, and the only solution is using Python 3.

You can experiment with your idea, but I'm afraid that the patch will be more 
difficult than you expect and break the tests. I want to warn that even if your 
experiment is quite successful, there is not much chance to take it in 2.7. 
This is more like a new feature than a bug fix. Programs that depend on this 
feature will be incompatible with previous bugfix releases. It is unlikely to 
help the migration on Python 3, but rather would encourage writing code that is 
incompatible with Python 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26369] unicode.decode and str.encode are unnecessarily confusing for non-ascii

[Steven D'Aprano]

Steven D'Aprano added the comment:

Ben, I'm sorry to see you have spent such a long time writing up reasons for 
changing this behaviour. I fear this is a total waste of your time, and ours to 
read it. Python 2.7 is under feature freeze, and changing the behaviour of 
str.encode and unicode.decode is a new feature. So it could only happen in 2.8, 
but there will never be a Python 2.8.

If you want more sensible behaviour, then upgrade to Python 3. If you want to 
improve the docs, then suggest some documentation improvements. But arguing for 
a change in behaviour of Python 2.7 str.encode and unicode.decode is, I fear, a 
waste of everyone's time.

If you still wish to champion this change, feel free to raise the issue on the 
Python-Dev mailing list where the senior developers, including Guido, hang out. 
I doubt it will do any good, but there is at least the theoretical possibility 
that if you convince them that this change will encourage people to migrate to 
Python 3 then you might get your wish.

Just don't hold your breath.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27058] Tkinter's canvas' dashed lines have incorrect segment lengths

[Kevin]

Kevin added the comment:

This problem appears to go deeper than Python. On the TK bug tracker, under the 
issue "-dashofset option doesnt work on ms windows build" 
(https://core.tcl.tk/tk/tktview?name=1055974fff), a commenter mentions:

 > On Windows, only certain dash patterns and no dash offsets are supported.

Which implies that you can't create segments and gaps of arbitrary length 
ratios- you can only select from a limited collection of premade patterns. This 
comment from tkWinDraw.c 
(https://github.com/tcltk/tk/blob/master/win/tkWinDraw.c#L1188) supports this:

 > Below is a simple translation of serveral dash patterns to valid  windows 
 > pen types. Far from complete, but I don't know how to do it better.

The code goes on to invoke CreatePen 
(https://msdn.microsoft.com/en-us/library/windows/desktop/dd183509%28v=vs.85%29.aspx),
 which only allows for four different variations of dashed line styles.

Conclusions:
1) It isn't Python's fault.
2) The guy who wrote the Tk code was aware of the limitation at the time, so 
it's arguably not a bug.

So, this issue ought to be closed; not much we can do, except perhaps petition 
the popular Tkinter reference websites to put up a warning label on their 
descriptions of the `dash` keyword argument.

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27058] Tkinter's canvas' dashed lines have incorrect segment lengths

[Kevin]

Kevin added the comment:

Ok, I've tried testing Serhiy's `canvas_dash.tcl` file, by running:

import Tkinter
root = Tkinter.Tk()
root.tk.eval("source canvas_dash.tcl")
root.mainloop()

(I'm not sure if this counts as "pure" Tcl/Tk. I don't have any experience in 
using Tcl/Tk outside of Python. If there's a way to execute `canvas_dash.tcl` 
straight from the command line, I'm willing to try, but after some cursory 
research I couldn't figure out how to do it.)

The result is the same as before: 18 pixel segments, 6 pixel gaps.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27053] Python Windows x86-64 embeddable zip file missing module encodings

[Wolfgang Langner]

Wolfgang Langner added the comment:

Yes this is the cause. If it is renamed to python36.zip everything works fine.
For the test I have only extracted the distributed zip and executed python.exe. 
Had not looked further into the issue. So only the simple report something did 
not work.

Thanks for looking at it and fixing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27053] Python Windows x86-64 embeddable zip file missing module encodings

[Steve Dower]

Steve Dower added the comment:

The "python35.zip" file contains the standard library and it needs to be 
renamed "python36.zip".

Apparently I hardcoded this somewhere instead of inferring it from the current 
version... I'll fix.

--
assignee:  -> steve.dower
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16468] argparse only supports iterable choices

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +berker.peksag
versions: +Python 3.5, Python 3.6 -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17408] second python execution fails when embedding

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26369] unicode.decode and str.encode are unnecessarily confusing for non-ascii

[Ben Spiller]

Ben Spiller added the comment:

btw If anyone can find the place in the code (sorry I tried and failed!) where 
str.encode('utf-8', error=X) is resulting in an implicit call to the equivalent 
of decode(defaultencoding, errors=strict) (as suggested by the exception 
message) I think it'll be easier to discuss the details of fixing.

Thanks for your reply - yes I'm aware that theoretically you _could_ globally 
change python's default encoding from ascii, but the prevailing view I've heard 
from python developers seems to be that changing it is not a good idea and may 
cause lots of library code to break. Also it's probably not a good idea for 
individual libraries or modules to be changing global state that affects the 
entire python invocation, and it would be nice to find a less fragile and more 
out-of-the-box solution to this. You may well be using different encodings (not 
just utf-8) to be used in different parts of your program - so changing the 
globally-defined default encoding doesn't seem right, especially for a method 
like str.encode method that already takes an 'encoding' argument (used 
currently only for the encoding aspect, not the decoding aspect). 

I do think there's a strong case to be made for changing the str.encode (and 
also unicode.decode) behaviour so that str.encode('utf-8') behaves the same 
whether it's given ascii or non-ascii characters, and also similar to 
unicode.encode('utf-8'). Let me try to persuade you... :)

First, to address the point you made:

> If str.encode() raises a decoding exception, this is a programming bug. It 
> would be bad to hide it.

I totally agree with the general principal of not hiding programming bugs. 
However if calling str.encode for codecs like utf8 (let's ignore base64 for 
now, which is a very different beast) was *consistently* treated as a 
'programming bug' by python and always resulted in an exception that would be 
ok (suboptimal usability imho, but still ok), since programmers would quickly 
spot the problem and fix it. But that's not what happens - it *silently works* 
(is a no-op) as long as you happen to be using ASCII characters so this 
so-called 'programming bug' will go unnoticed by most programmers (and authors 
of third party library code you might be relying on!)... but the moment a 
non-ascii character get introduced suddenly you'll get an exception, maybe in 
some library code you rely on but can't fix. For this reason I don't think 
treating this as a programming bug is helping anyone write more robust python 
code - quite the reverse. Plus I think the behaviour of being a no-op is almost 
always
  'what you would have wanted it to do' anyway, whereas the behaviour of 
throwing an exception almost never is. 

I think we'd agree that changing str.encode(utf8) to throw an exception in 
*all* cases wouldn't be a realistic option since it would certainly break 
backwards compatability in painful ways for many existing apps and library 
code. 

So, if we want to make the behaviour of this important built-in type a bit more 
consistent and less error-prone/fragile for this case then I think the only 
option is making str.encode be a no-op for non-ascii characters (at least, 
non-ascii characters that are valid in the specified encoding), just as it is 
for ascii characters. 

Here's why I think ditching the current behaviour would be a good idea:
- calling str.encode() and getting a DecodeError is confusing ("I asked you to 
encode this string, what are you decoding for?")
- calling str.encode('utf-8') and getting an exception about "ascii" is 
confusing as the only encoding I mentioned in the method call was utf-8
- calling encode(..., errors=ignore) and getting an exception is confusing and 
feels like a bug; I've explicitly specified that I do NOT want exceptions from 
calling this method, yet (because neither 'errors' nor 'encoding' argument gets 
passed to the implicit - and undocumented - decode operation), I get unexpected 
behaviour that is far more likely to break my program than a no-op
- the somewhat surprising behaviour we're talking about is not explicitly 
documented anywhere
- having str.encode throw on non-ascii but not ascii makes it very likely that 
code will be written and shipped (including library code you may have no 
control over) that *appears* to work under normal testing but has *hidden* bugs 
that surface only once non-ascii characters are used. 
- in every situation I can think of, having str.encode(encoding, errors=ignore) 
honour the encoding and errors arguments even for the implicit-decode operation 
is more useful than having it ignore those arguments and throw an exception
- a quick google shows lots of people in the Python community (from newbies to 
experts) are seeing this exception and being confused by it, therefore a lot of 
people's lives might be improved if we can somehow make the situation better :)
- even with the best of intentions (and with code written by senior python 

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[STINNER Victor]

STINNER Victor added the comment:

>> You should now sign the PSF Contributor Agreement
>Done.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15913] PyBuffer_SizeFromFormat is missing

[Berker Peksag]

Changes by Berker Peksag :


--
stage:  -> needs patch
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27058] Tkinter's canvas' dashed lines have incorrect segment lengths

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please test pure Tcl/Tk example?

--
nosy: +serhiy.storchaka
Added file: http://bugs.python.org/file42902/canvas_dash.tcl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Zachary Ware]

Zachary Ware added the comment:

Here's some pretty compelling evidence for a bad PYTHONHOME setting:

C:\>set PYTHONHOME=C:\path\not\here

C:\>py -3.5
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'

Current thread 0x1db4 (most recent call first):

C:\>set PYTHONHOME=C:\Python27\

C:\>py -3.5
Fatal Python error: Py_Initialize: unable to load the file system codec
  File "C:\Python27\\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x0664 (most recent call first):

C:\>set PYTHONHOME=

C:\>py -3.5
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[Michał Bednarski]

Michał Bednarski added the comment:

> You should now sign the PSF Contributor Agreement
Done.

> (Well, in fact it would be better to do that *before* merging your change, 
> but well, your change is short enough ;-))
Well on https://docs.python.org/devguide/patch.html was said "For non-trivial 
changes".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27058] Tkinter's canvas' dashed lines have incorrect segment lengths

[Kevin]

New submission from Kevin:

When creating a dashed line using `Canvas.create_line`. Minimal reproducing 
example:

from Tkinter import *
root = Tk()
canvas = Canvas(root, width=100, height=30, bg="black")
canvas.pack()
canvas.create_line((0,10,100,10), dash=(20,), fill="red")
canvas.create_line((0,20,100,20), dash=(20,20), fill="green")
root.mainloop()

Expected result: each line segment should be 20 pixels wide, separated by gaps 
20 pixels wide.

Actual result: each line segment is 18 pixels wide, separated by gaps 6 pixels 
wide. See attached file for screenshot.

Additional information: this problem appears to only occur on Windows. The 
Stack Overflow Python chat room attempted to replicate this issue, starting 
around here: 
http://chat.stackoverflow.com/transcript/message/30645798#30645798. Users of 
Windows 7, 8, and 10 were able to replicate the incorrect segmenting behavior. 
Users of Linux had correct segment lengths. (some Windows & 2.7 users also 
noticed that their green line was 3 pixel segments with 3 pixel gaps, but this 
seems to be an independent bug which was already fixed somewhere between 2.7.2 
and 2.7.10.)

--
components: Tkinter, Windows
files: output.PNG
messages: 265868
nosy: kms70847, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Tkinter's canvas' dashed lines have incorrect segment lengths
versions: Python 2.7, Python 3.5
Added file: http://bugs.python.org/file42901/output.PNG

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Zachary Ware]

Zachary Ware added the comment:

Also, I suspect your pip issue on Windows 10 is due to the fact that you need 
to either specify the --user flag or run pip as administrator, since your user 
account can't write to the global site-packages under Program Files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[STINNER Victor]

STINNER Victor added the comment:

To list environment variable, you can open cmd.exe and type the command "set".

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Zachary Ware]

Zachary Ware added the comment:

Hugh, could you check your environment and see if you have any PYTHON* 
variables set, particularly PYTHONHOME?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[STINNER Victor]

STINNER Victor added the comment:

> The ioctl is blocked for given file type regardless of request argument. As 
> I'm running Python on non-rooted Android updating policy is not really an 
> option.

Oh. Right :-) My intent was to suggest to report the SELinux issue upstream ;-)

> It will always fail for given file type (such as sockets), while it will work 
> for others (such as regular files). I think that remembering that ioctl 
> doesn't work may be okay, though I may be wrong. Either way if we are wrong 
> whenever ioctl works we'll do one extra syscall.

The purpose of using ioctl() is to *reduce* the number of syscalls. If an 
application mostly use sockets, it will do 3 syscalls per socket (ioctl, fcntl 
get, fcntl set) instead of just 2 (fcntl get, fcntl set) :-/ So I like your 
patch ;-)

I applied your fix to Python 3.5 and 3.6. Thanks for your contribution.

You should now sign the PSF Contributor Agreement:
https://www.python.org/psf/contrib/contrib-form/

(Well, in fact it would be better to do that *before* merging your change, but 
well, your change is short enough ;-))

--
resolution:  -> fixed
status: open -> closed
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Stefan Krah]

Stefan Krah added the comment:

I can only comment on the Windows 10 issue: I think I did the install
with "Right-click -> Run as administrator" and then installed for
all users.  Encodings, pip and everything else works here.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[Roundup Robot]

Roundup Robot added the comment:

New changeset 13c5135d8467 by Victor Stinner in branch '3.5':
Fix os.set_inheritable() on Android
https://hg.python.org/cpython/rev/13c5135d8467

New changeset 783c1b8cdddb by Victor Stinner in branch 'default':
Merge 3.5 (issue #27057)
https://hg.python.org/cpython/rev/783c1b8cdddb

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Hugh C. Lauer]

Hugh C. Lauer added the comment:

On a hunch, I uninstalled Python 3.5.1 and reinstalled it for the current user 
only, to see if it would make a difference in the association of the encodings 
in the Registry.

There was no difference. The original error message still appears.

Regards,

Hugh C. Lauer
Worcester Polytechnic Institute

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[Michał Bednarski]

Michał Bednarski added the comment:

> Hum, I'm surprised that SELinux blocks such safe function. Maybe the SELinux 
> policy should be completed to allow it?
The ioctl is blocked for given file type regardless of request argument. As I'm 
running Python on non-rooted Android updating policy is not really an option.

> But do you expect ioctl() to always fail with EACCESS? Or only fail 
> sometimes, or only on some file descriptors? Your patch remembers that 
> ioctl() fails once and never retries.
It will always fail for given file type (such as sockets), while it will work 
for others (such as regular files). I think that remembering that ioctl doesn't 
work may be okay, though I may be wrong. Either way if we are wrong whenever 
ioctl works we'll do one extra syscall.

> About your patch: please add a comment explaining why you fallback with a 
> reference to this issue ("Issue #27057").
Attached

--
Added file: http://bugs.python.org/file42900/set_inheritable-eacces.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27056] pickle: constant propagation in _Unpickler_Read()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

+1. Similar optimization is used in marshal.

Added comments on Rietveld.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

STINNER Victor added the comment:

Status of the my FASTCALL implementation (34456cce64bb.patch):

* Add METH_FASTCALL calling convention to C functions, similar
  to METH_VARARGS|METH_KEYWORDS
* Clinic uses METH_FASTCALL when possible (it may use METH_FASTCALL
  for all cases in the future)
* Add new C functions:

  - _PyObject_FastCall(func, stack, nargs, kwds): root of the FASTCALL branch
  - PyObject_CallNoArg(func)
  - PyObject_CallArg1(func, arg)

* Add new type flags changing the calling conventions of tp_new, tp_init and
  tp_call:

  - Py_TPFLAGS_FASTNEW
  - Py_TPFLAGS_FASTINIT
  - Py_TPFLAGS_FASTCALL

* Backward incompatible change of Py_TPFLAGS_FASTNEW and Py_TPFLAGS_FASTINIT
  flags: calling explicitly type->tp_new() and type->tp_init() is now a bug
  and is likely to crash, since the calling convention can now be FASTCALL.

* New _PyType_CallNew() and _PyType_CallInit() functions to call tp_new and
  tp_init of a type. Functions which called tp_new and tp_init directly were
  patched.

* New helpers function to parse functions functions:

  - PyArg_ParseStack()
  - PyArg_ParseStackAndKeywords()
  - PyArg_UnpackStack()

* New Py_Build functons:

  - Py_BuildStack()
  - Py_VaBuildStack()

* New _PyStack API to handle a stack:

  - _PyStack_Alloc(), _PyStack_Free(), _PyStack_Copy()
  - _PyStack_FromTuple()
  - _PyStack_FromBorrowedTuple()
  - _PyStack_AsTuple(), _PyStack_AsTupleSlice()
  - ...

* Many changes were done in the typeobject.c file to handle FASTCALL, new
  type flags, handle correctly flags when a new type is created, etc.

* ceval.c: add _PyFunction_FastCall() function (somehow, I only exposed
  existing code)

A large part of the patch changes existing code to use the new calling
convention in many functions of many modules. Some changes were generated
by the Argument Clinic. IMHO the best would be to use Argument Clinic in more
places, rather than patching manually the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

STINNER Victor added the comment:

New patch: 34456cce64bb.patch

$ diffstat 34456cce64bb.patch 
 .hgignore |3 
 Makefile.pre.in   |   37 
 b/Doc/includes/shoddy.c   |2 
 b/Include/Python.h|1 
 b/Include/abstract.h  |   17 
 b/Include/descrobject.h   |   14 
 b/Include/funcobject.h|6 
 b/Include/methodobject.h  |6 
 b/Include/modsupport.h|   20 
 b/Include/object.h|   28 
 b/Lib/json/encoder.py |1 
 b/Lib/test/test_extcall.py|   19 
 b/Lib/test/test_sys.py|6 
 b/Modules/_collectionsmodule.c|   14 
 b/Modules/_csv.c  |   15 
 b/Modules/_ctypes/_ctypes.c   |   12 
 b/Modules/_ctypes/stgdict.c   |2 
 b/Modules/_datetimemodule.c   |   47 
 b/Modules/_elementtree.c  |   11 
 b/Modules/_functoolsmodule.c  |  113 +-
 b/Modules/_io/clinic/_iomodule.c.h|8 
 b/Modules/_io/clinic/bufferedio.c.h   |   42 
 b/Modules/_io/clinic/bytesio.c.h  |   42 
 b/Modules/_io/clinic/fileio.c.h   |   26 
 b/Modules/_io/clinic/iobase.c.h   |   26 
 b/Modules/_io/clinic/stringio.c.h |   34 
 b/Modules/_io/clinic/textio.c.h   |   40 
 b/Modules/_io/iobase.c|4 
 b/Modules/_json.c |   24 
 b/Modules/_lsprof.c   |4 
 b/Modules/_operator.c |   11 
 b/Modules/_pickle.c   |  106 -
 b/Modules/_posixsubprocess.c  |   15 
 b/Modules/_sre.c  |   11 
 b/Modules/_ssl.c  |9 
 b/Modules/_testbuffer.c   |4 
 b/Modules/_testcapimodule.c   |4 
 b/Modules/_threadmodule.c |   32 
 b/Modules/_tkinter.c  |   11 
 b/Modules/arraymodule.c   |   29 
 b/Modules/cjkcodecs/clinic/multibytecodec.c.h |   50 
 b/Modules/clinic/_bz2module.c.h   |8 
 b/Modules/clinic/_codecsmodule.c.h|  318 +++--
 b/Modules/clinic/_cryptmodule.c.h |   10 
 b/Modules/clinic/_datetimemodule.c.h  |8 
 b/Modules/clinic/_dbmmodule.c.h   |   26 
 b/Modules/clinic/_elementtree.c.h |   86 -
 b/Modules/clinic/_gdbmmodule.c.h  |   26 
 b/Modules/clinic/_lzmamodule.c.h  |   16 
 b/Modules/clinic/_opcode.c.h  |   10 
 b/Modules/clinic/_pickle.c.h  |   34 
 b/Modules/clinic/_sre.c.h |  124 +-
 b/Modules/clinic/_ssl.c.h |   74 -
 b/Modules/clinic/_tkinter.c.h |   50 
 b/Modules/clinic/_winapi.c.h  |  124 +-
 b/Modules/clinic/arraymodule.c.h  |   34 
 b/Modules/clinic/audioop.c.h  |  210 ++-
 b/Modules/clinic/binascii.c.h |   36 
 b/Modules/clinic/cmathmodule.c.h  |   24 
 b/Modules/clinic/fcntlmodule.c.h  |   34 
 b/Modules/clinic/grpmodule.c.h|   14 
 b/Modules/clinic/md5module.c.h|8 
 b/Modules/clinic/posixmodule.c.h  |  642 ++-
 b/Modules/clinic/pyexpat.c.h  |   32 
 b/Modules/clinic/sha1module.c.h   |8 
 b/Modules/clinic/sha256module.c.h |   14 
 b/Modules/clinic/sha512module.c.h |   14 
 b/Modules/clinic/signalmodule.c.h |   50 
 b/Modules/clinic/unicodedata.c.h  |   42 
 b/Modules/clinic/zlibmodule.c.h   |   68 -
 b/Modules/itertoolsmodule.c   |   20 
 b/Modules/main.c  |2 
 b/Modules/pyexpat.c   |3 
 b/Modules/signalmodule.c  |9 
 b/Modules/xxsubtype.c |4 
 b/Objects/abstract.c  |  403 ---
 b/Objects/bytesobject.c   |2 
 b/Objects/classobject.c   |   36 
 b/Objects/clinic/bytearrayobject.c.h  |   90 -
 b/Objects/clinic/bytesobject.c.h  |   66 -
 b/Objects/clinic/dictobject.c.h   |   10 
 b/Objects/clinic/unicodeobject.c.h|   10 
 b/Objects/descrobject.c   |  162 +-
 b/Objects/dictobject.c|   26 
 b/Objects/enumobject.c|8 
 b/Objects/exceptions.c|   91 +
 b/Objects/fileobject.c|   29 
 b/Objects/floatobject.c   |   25 
 

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file42898/34456cce64bb.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file42898/34456cce64bb.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26814] [WIP] Add a new _PyObject_FastCall() function which avoids the creation of a tuple or dict for arguments

[STINNER Victor]

STINNER Victor added the comment:

Hi,

I made progress on my FASTCALL branch. I removed tp_fastnew, tp_fastinit and
tp_fastnew fields from PyTypeObject to replace them with new type flags (ex:
Py_TPFLAGS_FASTNEW) to avoid code duplication and reduce the memory footprint.
Before, each function was simply duplicated.

This change introduces a backward incompatibility change: it's not more
possible to call directly tp_new, tp_init and tp_call. I don't know yet if such
change would be acceptable in Python 3.6, nor if it is worth it.

I spent a lot of ot time on the CPython benchmark suite to check for
performance regression. In fact, I spent most of my time to try to understand
why most benchmarks looked completly unstable. I now tuned correctly my
system and patched perf.py to get reliable benchmarks.

On the latest run of the benchmark suite, most benchmarks are faster! I have to 
investigate why 3 benchmarks are still slower. In the run, normal_startup was 
not significant, etree_parse was faster (instead of slower), but raytrace was 
already slower (but only 1.13x slower). It may be the "noise" of the PGO 
compilation. I already noticed that once: see the issue #27056 "pickle: 
constant propagation in _Unpickler_Read()".

Result of the benchmark suite:

slower (3):

* raytrace: 1.06x slower
* etree_parse: 1.03x slower
* normal_startup: 1.02x slower

faster (18):

* unpickle_list: 1.11x faster
* chameleon_v2: 1.09x faster
* etree_generate: 1.08x faster
* etree_process: 1.08x faster
* mako_v2: 1.06x faster
* call_method_unknown: 1.06x faster
* django_v3: 1.05x faster
* regex_compile: 1.05x faster
* etree_iterparse: 1.05x faster
* fastunpickle: 1.05x faster
* meteor_contest: 1.05x faster
* pickle_dict: 1.05x faster
* float: 1.04x faster
* pathlib: 1.04x faster
* silent_logging: 1.04x faster
* call_method: 1.03x faster
* json_dump_v2: 1.03x faster
* call_simple: 1.03x faster

not significant (21):

* 2to3
* call_method_slots
* chaos
* fannkuch
* fastpickle
* formatted_logging
* go
* json_load
* nbody
* nqueens
* pickle_list
* pidigits
* regex_effbot
* regex_v8
* richards
* simple_logging
* spectral_norm
* startup_nosite
* telco
* tornado_http
* unpack_sequence

I know that my patch is simply giant and cannot be merged like that.

Since the performance is still promising, I plan to split my giant
patch into smaller patches, easier to review. I will try to check that
individual patches don't make Python slower. This work will take time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27054] Python installation problem: No module named 'encodings'

[Hugh C. Lauer]

Hugh C. Lauer added the comment:

Upon further investigation, I find lots of encodings in the folder C:\Program 
Files\Python35\Lib\encodings\ and lots of compiled encodings in the subfolder 
__pycache__ .

Somehow, these are not being found by Python itself, or there is something 
missing from the installation.

I investigated the registry. A search for "encodings" turns up lots of registry 
entries but none associated with Python.

Regards,

Hugh C. Lauer
Worcester Polytechnic Institute

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[STINNER Victor]

STINNER Victor added the comment:

Hum, I'm surprised that SELinux blocks such safe function. Maybe the SELinux 
policy should be completed to allow it?

Detect when ioctl() fails with EACCESS and fallback to fnctl() sounds like a 
good option. But do you expect ioctl() to always fail with EACCESS? Or only 
fail sometimes, or only on some file descriptors? Your patch remembers that 
ioctl() fails once and never retries.

About your patch: please add a comment explaining why you fallback with a 
reference to this issue ("Issue #27057").

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27057] os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES

[Michał Bednarski]

New submission from Michał Bednarski:

When SELinux forbids ioctl() it fails with EACCES and whole os.set_inheritable 
raises exception.

As in https://bugs.python.org/issue22258 code was added to fall back to fcntl 
when ioctl() fails with ENOTTY I'm adding EACCES value to same condition to 
fall back to fcntl().

--
files: set_inheritable-eacces.diff
keywords: patch
messages: 265853
nosy: Michał Bednarski
priority: normal
severity: normal
status: open
title: os.set_inheritable(): fall back to fcntl() if ioctl() fails with EACCES
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file42897/set_inheritable-eacces.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27056] pickle: constant propagation in _Unpickler_Read()

[STINNER Victor]

New submission from STINNER Victor:

According to Linux perf, the unpickle_list benchmark (of the CPython benchmark 
suite) heavily depends on the performance of load() and _Unpickler_Read() 
functions. While running benchmarks with PGO+LTO compilation, I noticed a 
difference around 5% because one build uses constant propation on 
_Unpickler_Read() (fast), wheras another build doesn't (slow).

Depending on the result of the PGO, depending on the OS and the compiler, you 
may or may not get this nice optimization. I propose to implement it manually 
to not depend on the compiler.

Attached short patch implements manually the optimization. It looks to have a 
big impact on unpickle_list, but no impact (benchmark is not significant) on 
fastunpickle and pickle_list:
---
$ taskset -c 3 python3 perf.py ../ref_default/rel/python ../default/rel/python 
-b unpickle_list,fastunpickle,pickle_list --rigorous -v 
(...)
Report on Linux smithers 4.4.9-300.fc23.x86_64 #1 SMP Wed May 4 23:56:27 UTC 
2016 x86_64 x86_64
Total CPU cores: 4

### fastunpickle ###
Avg: 0.527359 +/- 0.005932 -> 0.518548 +/- 0.00953: 1.02x faster
Not significant


### pickle_list ###
Avg: 0.269307 +/- 0.017465 -> 0.266015 +/- 0.00423: 1.01x faster
Not significant


### unpickle_list ###
Avg: 0.255805 +/- 0.006942 -> 0.231444 +/- 0.00394: 1.11x faster
Significant (t=21.58)
---

It would be interesting to also evaluate the computed goto optimization for the 
load() function. (And also try computed goto for the re/_sre module, but that's 
a different issue.)

I tuned my system and patched perf.py (of the CPython benchmark suite) to get 
stable benchmark results.

--
files: unpickle_read.patch
keywords: patch
messages: 265852
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: pickle: constant propagation in _Unpickler_Read()
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file42896/unpickle_read.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17408] second python execution fails when embedding

[Denny Weinberg]

Denny Weinberg added the comment:

Hi,

I think that the problem exists also in python 3.5.1

After calling Py_Finalize and Py_Initialize I get the message "attribute of 
type 'NoneType' is not callable" on the datetime.strptime method.

Example:
from datetime import datetime
s = '20160505 16'
refdatim = datetime.strptime(s, '%Y%m%d %H%M%S')

The first call works find but it crashes after the re initialization.

Workaround:
from datetime import datetime
s = '20160505 16'
try:
refdatim = datetime.strptime(s, '%Y%m%d %H%M%S')
except TypeError:
import time
refdatim = datetime.fromtimestamp(time.mktime(time.strptime(s, '%Y%m%d 
%H%M%S')))

Can anyone confirm this bug? Can you tell me if this will be fixed for python 
3.5/3.6/...?

All the other modules are working find after the re initialization but 
datetime.strptime.

--
nosy: +Denny Weinberg
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17408] second python execution fails when embedding

[Palm Kevin]

Changes by Palm Kevin :


--
nosy: +palm.kevin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20699] Document that binary IO classes work with bytes-likes objects

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added few minor comments on Rietveld.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27037] Universal newline support for zipFile.ZipExtFile.read() is not working (deprecated?), and the missing functionality not documented

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The support of the "U" mode will be removed in 3.6 (issue27029).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com