date:20160822


Xiang Zhang added the comment:

v3 applies haypo's suggestion.

--
Added file: http://bugs.python.org/file44192/list_resize_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

2016-08-22 Thread David Edelsohn


David Edelsohn added the comment:

> This bug was marked for 2.7 as well. Is there anything that needs to be done 
> for 2.7?

It would be great if both patches were applied to 2.7 also.

> How does patch 2 make the test_distutils situation worse? Is there anything 
> that should be done to improve things first, or should that just be handled 
> independently? I see that the 2.7 and 3.5 buildbots are currently failing 
> with errors like

distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit 
status 1

while 3.6 is failing with

distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' 
failed with exit status 1

Presumably the 3.6 failure may be helped by Patch 2, since it is complaining 
about the plain “config” path.

Patch2 makes test_distutils worse because LDSHARED refers to a file that only 
exists when Python is installed.  If one tests in tree, the files don't exist.  
This may be related to the existence of BLDSHARED, used to build modules in the 
source tree, which can be overridden separately -- otherwise it defaults to 
LDSHARED.  If the rest of the testsuite ever is cleaned up for AIX, one either 
could skip the parts of test_distutils that require the LDSHARED files or add 
special hooks in the test for AIX.  As you point out, the change doesn't make 
the current testsuite results situation any worse.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken


Martin Panter added the comment:

It looks like the change from plain config to config- was made in Issue 
9807 (r86731, 3.2). So the patch seems reasonable to me; it is just catching up 
with that change.

This bug was marked for 2.7 as well. Is there anything that needs to be done 
for 2.7?

How does patch 2 make the test_distutils situation worse? Is there anything 
that should be done to improve things first, or should that just be handled 
independently? I see that the 2.7 and 3.5 buildbots are currently failing with 
errors like

distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit 
status 1

while 3.6 is failing with

distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' 
failed with exit status 1

Presumably the 3.6 failure may be helped by Patch 2, since it is complaining 
about the plain “config” path.

--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27831] Python 3.4.5 leaks memory when attempting to load non-existent file

2016-08-22 Thread geeknik


New submission from geeknik:

While fuzzing Python 3.4.5 with AFL, ASAN and libdislocator, I fat fingered the 
keyboard ASAN pops up a LeakSanitizer error. Apparently trying to open a 
non-existent file causes Python 3.4.5 to leak >880KB of memory.

./python: can't open file 'id:000109': [Errno 2] No such file or directory

=
==16921==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 620904 byte(s) in 469 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12

Direct leak of 41800 byte(s) in 29 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x769b2f in PyEval_EvalCodeEx /root/python-3.4.5/Python/ceval.c:3588:14

Direct leak of 28160 byte(s) in 7 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x60459c in set_add_key /root/python-3.4.5/Objects/setobject.c:373:12

Direct leak of 26095 byte(s) in 33 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4dc54d in PyMem_RawMalloc /root/python-3.4.5/Objects/obmalloc.c:276:12
#2 0x4dc54d in _PyObject_Malloc /root/python-3.4.5/Objects/obmalloc.c:1314

Direct leak of 18176 byte(s) in 31 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4dc54d in PyMem_RawMalloc /root/python-3.4.5/Objects/obmalloc.c:276:12
#2 0x4dc54d in _PyObject_Malloc /root/python-3.4.5/Objects/obmalloc.c:1314
#3 0x8a80e9 in _PyObject_GC_NewVar 
/root/python-3.4.5/Modules/gcmodule.c:1765:26

Direct leak of 3584 byte(s) in 16 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x636d59 in type_call /root/python-3.4.5/Objects/typeobject.c:883:11

Direct leak of 1472 byte(s) in 15 object(s) allocated from:
#0 0x4bc5ce in realloc (/root/python-3.4.5/python+0x4bc5ce)
#1 0x4da38a in PyMem_Realloc /root/python-3.4.5/Objects/obmalloc.c:308:12

Direct leak of 448 byte(s) in 2 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x836b0e in _Py_InitializeEx_Private 
/root/python-3.4.5/Python/pythonrun.c:418:14
#3 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#4 0x4d9c8e in main /root/python-3.4.5/./Modules/python.c:69:11

Direct leak of 264 byte(s) in 2 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#3 0x4d9c8e in main /root/python-3.4.5/./Modules/python.c:69:11

Direct leak of 224 byte(s) in 1 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x546820 in _PyExc_Init /root/python-3.4.5/Objects/exceptions.c:2509:5
#3 0x836b09 in _Py_InitializeEx_Private 
/root/python-3.4.5/Python/pythonrun.c:416:5
#4 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#5 0x4d9c8e in main /root/python-3.4.5/./Modules/python.c:69:11

Direct leak of 224 byte(s) in 1 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x5468ad in _PyExc_Init /root/python-3.4.5/Objects/exceptions.c:2510:5
#3 0x836b09 in _Py_InitializeEx_Private 
/root/python-3.4.5/Python/pythonrun.c:416:5
#4 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#5 0x4d9c8e in main /root/python-3.4.5/./Modules/python.c:69:11

Direct leak of 224 byte(s) in 1 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x5e7712 in PyModule_Create2 
/root/python-3.4.5/Objects/moduleobject.c:105:9
#3 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#4 0x4d9c8e in main /root/python-3.4.5/./Modules/python.c:69:11

Direct leak of 224 byte(s) in 1 object(s) allocated from:
#0 0x4bc2db in malloc (/root/python-3.4.5/python+0x4bc2db)
#1 0x4da307 in PyMem_Malloc /root/python-3.4.5/Objects/obmalloc.c:299:12
#2 0x545fdf in _PyExc_Init /root/python-3.4.5/Objects/exceptions.c:2494:5
#3 0x836b09 in _Py_InitializeEx_Private 
/root/python-3.4.5/Python/pythonrun.c:416:5
#4 0x4e2e82 in Py_Main /root/python-3.4.5/Modules/main.c:654:5
#5 0x4d9c8e in main

[issue27435] ctypes library loading and AIX - also for 2.7.X (and later)


Martin Panter added the comment:

I had understood that changing find_library() would only be useful in 
combination with the automatic RTDL_MEMBER detection. If you want to mention 
lack of support for AIX in the documentation, that is okay by me. If you want 
to propose an alternative find_library() implementation _without_ affecting the 
CDLL behaviour, that may be okay too.

For the libuuid problem, for 2.7 I would suggest adding an explicit call to 
load libc.a(. . .) or whatever is appropriate.

Personally I don’t have a specific interest in AIX, but I am happy to commit 
patches for AIX if I think they are sensible. I think I mentioned before in the 
other bug that your changes seemed okay for the next Python version. The beta 
deadline for 3.6 is in a few weeks (PEP 494), although after that there is 
still 3.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Roundup Robot added the comment:

New changeset 620b2e554be4 by Victor Stinner in branch 'default':
Issue #27809: builtin___build_class__() uses fast call
https://hg.python.org/cpython/rev/620b2e554be4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27830] Add _PyObject_FastCallKeywords(): pass keyword arguments as (key, value) pairs


STINNER Victor added the comment:

Here is a first patch:

* Add _PyObject_FastCallKeywords(): main entry point
* Add _PyCFunction_FastCallKeywords(): implementation for C function, but 
currently it's just an empty skeleton
* Rename fast_function() to PyFunction_FastCallKeywords() and ajust its API

In this first implementation, _PyObject_FastCallKeywords() has a full 
implementation for Python function, otherwise it only uses 
_PyObject_FastCallDict() fast path if the function has no keyword argument. In 
the last resort, it creates a temporary tuple and maybe also a temporary dict, 
to call PyObject_Call().

To test the patch, I modified do_call() to call _PyObject_FastCallKeywords(). 
I'm not sure that it makes sense in term of performance at this stage.

Serhiy: You can use this first implementation to experimental for fast argument 
parser for keyword arguments passed as (key, value) pairs ;-)

--
keywords: +patch
Added file: http://bugs.python.org/file44191/fastcall_keywords.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27643] test_ctypes fails on AIX with xlc


Martin Panter added the comment:

Now I am confused. In  we have

[Me] If your compiler does not support “signed short” bitfields, maybe we just 
have to accept that ctypes supports it even though the compiler doesn’t, and 
skip the test.

[Michael] Looks like this may be the approach: as signed int, signed short give 
the same message - it seems that xlc (and maybe Sun C) does not accept "short" 
as a bitfield type.

Apparently XLC doesn’t accept signed short bitfields for Michael, but does for 
Eric. What’s going on? Maybe different versions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27824] update ConfigParser docs regarding in-line comments

2016-08-22 Thread Joshua Haas


Joshua Haas added the comment:

I'm not sure it's deprecated so much as moved. Python 3.5 includes an 
additional init parameter, so setting inline_comment_prefixes=';' allows inline 
comments even for values containing the ';' character, displaying the "expected 
behavior" described in my original bug report. They are, however, disabled by 
default, unlike in Python 2.7.

I also have a different proposal I think might be more clear:

Configuration files may include comments, prefixed by specific characters. 
Comments may appear on their own in an otherwise empty line using either '#' or 
';'. In addition, inline comments can be added to the end of a line after a ';' 
as long as the ';' is preceded by a whitespace character. Only the first ';' in 
a line is checked, so an inline comment cannot be added to lines that already 
contain any ';'s.

I'm also wondering if an example would be a good idea, as I don't think any of 
the examples on the doc page include comments:

# this is a comment
; this is also a comment
foo = bar ; inline comment
password = jgfw;as1 ; this will be parsed as part of 'password'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27810] Add METH_FASTCALL: new calling convention for C functions


STINNER Victor added the comment:

According to discussions in the issue #27809, require a Python dict leads to 
inefficient code because a temporary dict may be required. The issue #27830 
proposes to pass keyword arguments as (key, value) pairs.

It should be used for the new METH_FASTCALL calling convention.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27830] Add _PyObject_FastCallKeywords(): pass keyword arguments as (key, value) pairs


New submission from STINNER Victor:

In the issue #27809, I added support for keyword arguments to the function 
_PyObject_FastCallDict() (previously called _PyObject_FastCall). Keyword 
arguments are passed as a Python dictionary (dict).

_PyObject_FastCallDict() is efficient when keyword arguments are not used. But 
when keyword arguments are used, the creation of a temporary dictionary to pass 
keyword arguments can defeat the "fastcall" optimization for positional 
arguments.

See Serhiy Storchaka's comment:
http://bugs.python.org/issue27809#msg273367

Extract: "It may be more efficient to push values and names as continuous 
arrays (issue27213). PyArg_ParseTupleAndKeywords() accepts a tuple and a dict, 
but private function _PyArg_ParseTupleAndKeywordsFast() (issue27574) can be 
changed to accept positional and keyword arguments as continuous arrays: (int 
nargs, PyObject **args, int nkwargs, PyObject **kwnames, PyObject **kwargs)."

And Stefan Behnel's comment:
http://bugs.python.org/issue27809#msg273391

Serhiy proposed to pass keyword arguments as (key, value) pairs in C array. We 
can probably use the simple "stack" format used in Python/ceval.c:

   PyObject* _PyObject_FastCallKeywords(PyObject *func, PyObject **stack, int 
nargs, int nkwargs)

where nargs is number of positional arguments at the beginning of stack, and 
nkwargs is the number of (key, value) pairs at the end of stack.

This function would be the fundation for a new C calling convention: 
METH_FASTCALL, see the issue #27810.

--
components: Interpreter Core
messages: 273408
nosy: haypo, scoder, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add _PyObject_FastCallKeywords(): pass keyword arguments as (key, value) 
pairs
type: performance
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Roundup Robot added the comment:

New changeset ef0110a52e24 by Victor Stinner in branch 'default':
PyEval_CallObjectWithKeywords() uses fast call with kwargs
https://hg.python.org/cpython/rev/ef0110a52e24

New changeset 5587d0dfab4c by Victor Stinner in branch 'default':
Issue #27809: Use _PyObject_FastCallDict()
https://hg.python.org/cpython/rev/5587d0dfab4c

New changeset 0c65a2089f00 by Victor Stinner in branch 'default':
Add _PyErr_CreateException()
https://hg.python.org/cpython/rev/0c65a2089f00

New changeset c53c532de995 by Victor Stinner in branch 'default':
Issue #27809: PyErr_SetImportError() uses fast call
https://hg.python.org/cpython/rev/c53c532de995

New changeset 614dd914c21e by Victor Stinner in branch 'default':
Issue #27809: tzinfo_reduce() uses fast call
https://hg.python.org/cpython/rev/614dd914c21e

New changeset f48ae71e8a8f by Victor Stinner in branch 'default':
Issue #27809: _csv: _call_dialect() uses fast call
https://hg.python.org/cpython/rev/f48ae71e8a8f

New changeset 60660890459f by Victor Stinner in branch 'default':
Issue #27809: methodcaller_reduce() uses fast call
https://hg.python.org/cpython/rev/60660890459f

New changeset bf2d88cf039a by Victor Stinner in branch 'default':
PyEval_CallObjectWithKeywords() doesn't inc/decref
https://hg.python.org/cpython/rev/bf2d88cf039a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

Stefan Behnel: "I like the oneArg/noArg etc. macros. We use something similar 
in Cython."

Oh, nice :-) Since they macros are private, I pushed fastcall_macros.patch. We 
can still rework them later if needed.


"You can even use them to inline the METH_O and METH_NOARGS call cases 
(although I use inline functions for that in Cython)."

Hum, I'm not sure that it's worth it.


"I just took a quick look at the fastcall_kwargs-2.patch for now. It looks ok 
in general, but it also adds quite some special code for the dict-to-locals 
mapping. Is the keyword argument calling case really that important? I mean, it 
requires creating a dict and filling it, after all. People would rather avoid 
that in the first place if they care about performance. I wouldn't object, I'm 
merely asking..."

Hey, it's nothing new: Python 3.5 does the same in function_call()!? Oh in 
fact, function_call() code is very short compared to fastcall_kwargs-2.patch 
and it doesn't require to modify the hot code of Python/ceval.c! So I pushed a 
very small change (change 1aefb4c4a7b4) which adds support for keyword with the 
sample function_call() code.

Serhiy already told me that creating a dict is overkill... but please read 
again my message msg273370. We cannot break all APIs, we need a transition from 
the existing C APIs using Python dictionaries for keyword arguments.

I reworked my new fastcall APIs to keep the door open for a faster API to pass 
keyword arguments as pairs of (key, value) in an unique C array.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27820] Possible bug in smtplib when initial_response_ok=False

2016-08-22 Thread Mario Colombo


Mario Colombo added the comment:

Yes, this (or something similar) totally bit me, when for another unrelated 
reason 'AUTH PLAIN' authentication failed:

https://gist.github.com/macolo/bf2811c14d985d013dda0741bfd339e0

Python then tries auth_login, but doesn't send 'AUTH LOGIN' to the mail server. 
The second auth method also fails.

--
nosy: +Mario Colombo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Roundup Robot added the comment:

New changeset 1aefb4c4a7b4 by Victor Stinner in branch 'default':
_PyFunction_FastCallDict() supports keyword args
https://hg.python.org/cpython/rev/1aefb4c4a7b4

New changeset 7924eac912fe by Victor Stinner in branch 'default':
Issue #27809: Cleanup _PyEval_EvalCodeWithName()
https://hg.python.org/cpython/rev/7924eac912fe

New changeset 15eab21bf934 by Victor Stinner in branch 'default':
Issue #27809: Use _PyObject_FastCallDict()
https://hg.python.org/cpython/rev/15eab21bf934

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Roundup Robot added the comment:

New changeset 263334a652ac by Victor Stinner in branch 'default':
Rename _PyObject_FastCall() to _PyObject_FastCallDict()
https://hg.python.org/cpython/rev/263334a652ac

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


STINNER Victor added the comment:

Ok, I see much better with concrete commits. I'm really happy that
Cython also benefits from these enhancements.

Note: handling keywords is likely to change quickly ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27660] Replace size_t with Py_ssize_t as the type of local variable in list_resize


STINNER Victor added the comment:

I reviewed list_resize_v2.patch.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27814] contextlib.suppress: Add whitelist argument

2016-08-22 Thread Michael Büsch


Michael Büsch added the comment:

>an implementation of the vastly *less* common pattern:

Ok, here are some numbers.
My codebase is about 32 kLOC.

$ git grep suppressAllExc |wc -l
20
$ git grep contextlib\\.suppress |wc -l
17

(suppressAllExc being my local version to suppress Exception, but exclude a few)

So _neither_ one is pretty common. And that's not surprising, as exceptions 
ought to be properly handled most of the time.
However the ratio between the two forms is about 1:1. So I don't agree that it 
would be "vastly *less* common".

But I do accept if such a thing is not wanted upsteam, Altough it basically is 
a twoliner. I'll simply have to ship my own (I'll have to do that anyway for 
older Python versions. meh :)

Thanks for the discussion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27826] Null-pointer dereference in tuplehash() function

2016-08-22 Thread Raymond Hettinger


Raymond Hettinger added the comment:

Closing as won't fix.  It is impractical to make marshal resilient against 
bytecode hacks and it is likewise impractical to put a NULL pointer check 
in-front of every dereference in the language.

--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27660] Replace size_t with Py_ssize_t as the type of local variable in list_resize

2016-08-22 Thread Raymond Hettinger


Raymond Hettinger added the comment:

This is low priority.  I will look at it during the sprint.  As far as I know 
there is no bug here or performance problem; instead, the patch alters stable 
code and transfers responsibilities.

--
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27643] test_ctypes fails on AIX with xlc


Changes by Eric N. Vander Weele :


Removed file: http://bugs.python.org/file44189/ctypes_test_sign_bitfields.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27643] test_ctypes fails on AIX with xlc


Eric N. Vander Weele added the comment:

> I believe that there is a specific reason that M, N, O, P, Q, R and S 
are "short". [...]

Oops - this was an oversight when I created the patch.  I just uploaded 
ctypes_test_sign_bitfields_2.diff, which is what I originally intended.

--
Added file: http://bugs.python.org/file44190/ctypes_test_sign_bitfields_2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27794] setattr a read-only property; the AttributeError should show the attribute that failed

2016-08-22 Thread Antti Haapala


Antti Haapala added the comment:

I've got one idea about how to implement this, but it would require adding a 
new flag field to PyExc_AttributeError type.

This flag, if set, would tell that the AttributeError in question was raised in 
C descriptor code or under similar circumstances, and that the attribute name 
was not known, and thus it is OK for setattr/delattr and attribute lookups to 
append ": attributename" to the end of the message, then clear the flag; then  
all those places that raise AttributeError in __get__, __set__, __del__ would 
just need to set this flag.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12713] argparse: allow abbreviation of sub commands by users

2016-08-22 Thread Vinay Sajip


Vinay Sajip added the comment:

Mean culpa - I have  jumped the gun on this. Sorry to all for the 
inconvenience. I have just got back after being away for a few days, and will 
revert the change shortly, if no one beats me to it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12713] argparse: allow abbreviation of sub commands by users

2016-08-22 Thread Vinay Sajip


Vinay Sajip added the comment:

*Mea culpa. Autocorrect :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27643] test_ctypes fails on AIX with xlc

2016-08-22 Thread Michael Felt


Michael Felt added the comment:

On 22-Aug-16 19:43, Eric N. Vander Weele wrote:
> Eric N. Vander Weele added the comment:
>
> I came across this issue while researching where to post my patch (having 
> come across this while building Python 2.7 & 3.x on AIX via xlc).
>
>> In general in C, if a bit-field has type “int” without a signed or unsigned 
>> qualifier, it is up to the implementation which mode is chosen.
> Unfortunately, the ISO C standard leaves it up to the compiler to decide 
> whether to default to 'signed' or 'unsigned' for non-qualified bit-field 
> declarations.  gcc defaults to signed 
> (https://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/Non_002dbugs.html); however, 
> xlc defaults to unsigned 
> (https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.3/com.ibm.xlc1313.aix.doc/compiler_ref/opt_bitfields.html).
>
>> [...] having bitfields working regardless of the compiler should be 
>> preferred.
> However, ctypes_test assumes an unqualified bit-field will be signed.  To 
> achieve bit-fields working regardless of the compiler, declarations must 
> explicitly qualifying the sign for bit-fields.  This makes the intent of what 
> is expected explicit and avoids implementation-defined behavior that will 
> differ from one compiler to the next.
>
> With patch ctypes_test_sign_bitfields.diff provided, I have verified 
> ctypes_test passes on Python 2.7, Python 3.4, and Python 3.5 (on AIX & 
> Solaris).  If need be, I'm happy to provide before-&-after output of 
> ctypes_test with & without the patch applied (or verification from others 
> would be greatly appreciated).
>
> --
> Added file: http://bugs.python.org/file44189/ctypes_test_sign_bitfields.diff
I believe that there is a specific reason that M, N, O, P, Q, R and S 
are "short". For AIX/xlc at least "signed" is equivalent to "signed int" 
and not the same size as "unsigned short".

> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27435] ctypes library loading and AIX - also for 2.7.X (and later)

2016-08-22 Thread Michael Felt

Michael Felt added the comment:

On 22-Aug-16 04:04, Martin Panter wrote:
> Martin Panter added the comment:
>
> The ctypes tests all seem to be protected with code that checks for None, and 
> explicitly skip the test in that case. The skip message should be visible 
> when you run the test in verbose mode.
So at least test/test_something.py run cleanly - but why accept code 
that always returns None - for find_library().

Rather than "hard-code" proper strings in a test - shouldn't the test be 
demonstrating the "failure" of find_library() to return a suitable 
response - when it should not be None.

For example, on AIX the IBM provided /usr/lib/libm.a only has static 
members. So, what has always been wrong (and noone noticed, or is it 
noone cared) the "built-in" test in util.py has, and does, always fail:

root@x064:[/data/prj/aixtools/python/python-2.7.12.0/Lib/ctypes]../../python 
util.py
None
None
None
Traceback (most recent call last):
   File "util.py", line 271, in 
 test()
   File "util.py", line 266, in test
 print cdll.LoadLibrary("libm.so")
   File 
"/data/prj/aixtools/python/python-2.7.12.0/Lib/ctypes/__init__.py", line 
440, in LoadLibrary
 return self._dlltype(name)
   File 
"/data/prj/aixtools/python/python-2.7.12.0/Lib/ctypes/__init__.py", line 
362, in __init__
 self._handle = _dlopen(self._name, mode)
OSError:0509-022 Cannot load module .
 0509-026 System error: A file or directory in the path name 
does not exist.

The documentation says:

The exact functionality is system dependent.

On Linux, |find_library()| tries to run external programs 
(|/sbin/ldconfig|, |gcc|, and |objdump|) to find the library file. It 
returns the filename of the library file.
...
On OS X, |find_library()| tries several predefined naming schemes and 
paths to locate the library, and returns a full pathname if successful:
...
On Windows, |find_library()| searches along the system search path, and 
returns the full pathname, but since there is no predefined naming 
scheme a call like |find_library("c")| will fail and return |None|.

Are you saying the documentation should be updated to read:
On AIX, find_library tries to use the Linux code which generally fails 
and will return None.

And the closing bit:
If wrapping a shared library with |ctypes| 
, it /may/ 
be better to determine the shared library name at development time, and 
hardcode that into the wrapper module instead of using |find_library()| 
to locate the library at runtime (so forget about developing code for 
multiple platforms - find_library() cannot give you that support.
p.s. On AIX, you must use a hard-coded name to ctypes.CDLL() - and also 
add an extra flag - because even though we said "the exact functionality 
is system dependent" we do not permit new systems to add corrections.

I apologize that my frustration is more than peeking through. But I am 
frustrated. find_library(), unless I completely misunderstand the 
documentation, is a helper function to releave the programmer of needing 
to know the specifics of multiple platforms. My frustration is 
heightened by what I read about the issues with "uuid" - which also 
calls find_library() - so again, on AIX it always returns None -- a) 
find_library is broken; b) there is no libuuid.so or libuuid.a on AIX 
(and c) 4 of the 5 programs also called do not exist on AIX, and the one 
that does generates a different string than is compared to).

I understand - AIX has not been a major (user) platform for python. I am 
amazed that noone from IBM (AIX) development has never shown more interest.
Disclaimer: yes, I work for IBM - but am not a developer. Customers do 
ask me about "why is python so old, why does it not work with XXX" - and 
only because I am inclined to "get to the bottom" did I even start on this.

If the root cause is: "IBM showed no interest X years ago and we, 
python, have no interest now in IBM or AIX" then please just say so - 
and I shall do other things with my time.

In closing - I thought python was multi-platform and I had hoped to be 
of some help in improving python integration with the platform AIX. 
Please, one way or the other, help me lessen my frustration.

Thank you for your time and patience with my rant!

> We could avoid skipping these tests in 2.7 by adding a special case for AIX 
> that loads the library via CDLL("libc.a(shr[_64].o)", RTLD_MEMBER). Like an 
> extension of the special cases already at 
> ,
>  but we have to set the RTLD_ flag as well as the library name.
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Stefan Behnel added the comment:

I just took a quick look at the fastcall_kwargs-2.patch for now. It looks ok in 
general, but it also adds quite some special code for the dict-to-locals 
mapping. Is the keyword argument calling case really that important? I mean, it 
requires creating a dict and filling it, after all. People would rather avoid 
that in the first place if they care about performance. I wouldn't object, I'm 
merely asking...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


Stefan Behnel added the comment:

I like the oneArg/noArg etc. macros. We use something similar in Cython. You 
can even use them to inline the METH_O and METH_NOARGS call cases (although I 
use inline functions for that in Cython).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken


Eric N. Vander Weele added the comment:

> Patch1 looks fine to me, though I will have to rely on you people to verify 
> that it does what it’s supposed to. Do you want me to commit it straight 
> away, or wait for your follow-up patch?

Independently, I have created a similar patch as well and can also verify that 
it does what it's suppose to do.

> The second of two patches. This patch changes the definition of LDSHARED for 
> AIX in configure to reference the matching installed location as defined in 
> Makefile.pre.in by Patch1. [...]

I can also confirm that Patch2 is necessary as well and does what it's suppose 
to do, having creating this patch independently as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


Stefan Behnel added the comment:

> What do you mean by "I copied your (no-kwargs) implementation"?

I copied what you committed into CPython for _PyFunction_FastCall():

https://github.com/cython/cython/commit/8f3d3bd199a3d7f2a9fdfec0af57145b3ab363ca

and then enabled its usage in a couple of places:

https://github.com/cython/cython/commit/a3cfec8f7bd6d585831dd6669f6dad5f88303c71

especially for all function/method calls that we generate for user code:

https://github.com/cython/cython/commit/a51df339f395634f57b77e3ec13cecb3a28a5462

Note that PyMethod objects get unpacked into function+self right before the 
PyFunction_Check(), so the tuple avoidance optimisation also applies to Python 
method calls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27643] test_ctypes fails on AIX with xlc

Eric N. Vander Weele added the comment:

I came across this issue while researching where to post my patch (having come
across this while building Python 2.7 & 3.x on AIX via xlc).

> In general in C, if a bit-field has type “int” without a signed or unsigned
> qualifier, it is up to the implementation which mode is chosen.

Unfortunately, the ISO C standard leaves it up to the compiler to decide
whether to default to 'signed' or 'unsigned' for non-qualified bit-field
declarations. gcc defaults to signed
(https://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/Non_002dbugs.html); however, xlc
defaults to unsigned
(https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.3/com.ibm.xlc1313.aix.doc/compiler_ref/opt_bitfields.html).

> [...] having bitfields working regardless of the compiler should be preferred.

However, ctypes_test assumes an unqualified bit-field will be signed. To
achieve bit-fields working regardless of the compiler, declarations must
explicitly qualifying the sign for bit-fields. This makes the intent of what
is expected explicit and avoids implementation-defined behavior that will
differ from one compiler to the next.

With patch ctypes_test_sign_bitfields.diff provided, I have verified
ctypes_test passes on Python 2.7, Python 3.4, and Python 3.5 (on AIX &
Solaris). If need be, I'm happy to provide before-&-after output of
ctypes_test with & without the patch applied (or verification from others would
be greatly appreciated).

--
Added file: http://bugs.python.org/file44189/ctypes_test_sign_bitfields.diff

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27794] setattr a read-only property; the AttributeError should show the attribute that failed


Xiang Zhang added the comment:

One solution I can think of is alter the constructor of property() and add an 
optional name attribute to it. If users really care about the exception 
message, they can set the attribute to the property's name. If they don't care, 
everything remains the same as now. This is just like the doc parameter of 
property() right now and the first parameter of type() when it is used to 
create a class dynamically. This is just a suggestion, I am not sure it's worth 
to do it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27794] setattr a read-only property; the AttributeError should show the attribute that failed

2016-08-22 Thread R. David Murray


R. David Murray added the comment:

Not worth it.  It would feel like boilerplate, and the situation where you want 
the information is almost invariably going to be one you didn't anticipate and 
so didn't "bother" with the name.  Either that or you always have to do it, and 
the elegance of your python code takes a hit not commensurate with the benefit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26331] PEP 515: Tokenizer: allow underscores for grouping in numeric literals


Xiang Zhang added the comment:

Hi Georg, I left several comments on Rietveld. Hope it helps.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27828] Regression in http.cookies parsing with single key without value

2016-08-22 Thread Yuriy Zhuravlev


Yuriy Zhuravlev added the comment:

>Any RFC I'm looking at says that the value is required, so your syntax is 
>simply not valid and is rejected by the parser.

Perhaps but this cookies sets not by me (maybe JS ads widget or etc). And I 
can't fix it. I can't auth users because their cookies wrong. And I can't 
detect this situation because all process is silent without exception.

>I am not the biggest expert on cookie syntaxes, but maybe these reports are 
>related:

I suppose you are right.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27784] Random failure of test_TCPServer() of test.test_socketserver.SocketServerTest and test_handle_accept() of test.test_asyncore.TestAPI_UseIPv6Select on FreeBSD buildbots

2016-08-22 Thread Berker Peksag


Berker Peksag added the comment:

Here is another one on koobs-freebsd10: 
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.x%20Shared%203.x/builds/4809/steps/test/logs/stdio
 test_handle_accept was fine this time.

And this one is from koobs-freebsd9:

==
ERROR: test_handle_accept (test.test_asyncore.TestAPI_UseIPv6Select)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/test/test_asyncore.py", 
line 500, in tearDown
asyncore.close_all()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/asyncore.py", 
line 561, in close_all
x.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/asyncore.py", 
line 397, in close
self.socket.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/socket.py", line 
407, in close
self._real_close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/socket.py", line 
401, in _real_close
_ss.close(self)
ConnectionResetError: [Errno 54] Connection reset by peer

--

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/4705/steps/test/logs/stdio

--
components: +Tests
nosy: +berker.peksag
stage:  -> needs patch
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18139] email module should have a way to prepend and insert headers

2016-08-22 Thread Barry A. Warsaw


Barry A. Warsaw added the comment:

I meant .items()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18139] email module should have a way to prepend and insert headers

2016-08-22 Thread Barry A. Warsaw


Barry A. Warsaw added the comment:

I've updated this to Python 3.6, but I don't know if there's time to design an 
API and implementation in the time left before beta 1.  But a use case has come 
up, so I want to reboot this discussion (yes, it should go to email-sig too).

Apparently ARC headers  requires a header order.  In 
Mailman we have a GSoC student who is working on an implementation, and he's 
stuck because he needs to *prepend* ARC headers on the message, but the API 
gives him no direct way to do this.

We do have all the API problems that RDM points out.  I'm not going to suggest 
to the student that they use Message._headers.  It seems like the only way to 
do what he needs is to .get() all the headers in order, then del them all, 
manipulate the list out-of-band, and then re-add them all in the original order 
with the ARC headers at the front.  Yuck.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18139] email module should have a way to prepend and insert headers

2016-08-22 Thread Barry A. Warsaw


Changes by Barry A. Warsaw :


--
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27587] Issues, reported by PVS-Studio static analyzer

2016-08-22 Thread Berker Peksag


Changes by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27587] Issues, reported by PVS-Studio static analyzer


Roundup Robot added the comment:

New changeset 51627344fc58 by Berker Peksag in branch '3.5':
Issue #27587: Move null pointer check earlier in _PyState_AddModule()
https://hg.python.org/cpython/rev/51627344fc58

New changeset 7d90bf4780ff by Berker Peksag in branch 'default':
Issue #27587: Merge from 3.5
https://hg.python.org/cpython/rev/7d90bf4780ff

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27660] Replace size_t with Py_ssize_t as the type of local variable in list_resize


Xiang Zhang added the comment:

Ping.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

fastcall_macros.patch:

* Rename _PyObject_FastCall() to _PyObject_FastCallDict()
* Add _PyObject_FastCall(func, args, nargs) macro
* Add _PyObject_CallArg1(func, arg) macro
* Add _PyObject_CallNoArg(func) macro

The 3 new macros call _PyObject_FastCallDict().

Statistics on calls to these functions and macros:

* _PyObject_CallNoArg: 11
* _PyObject_CallArg1: 8
* _PyObject_FastCall: 5
* _PyObject_FastCallDict: 0   (the function doesn't work yet, it's the purpose 
of this issue, so it's expected that it's unused yet ;-))

_PyObject_CallNoArg() and _PyObject_CallArg1() are much more common and the 
more complex format _PyObject_FastCall() format (more than 1 argument). I like 
having macros with a simple prototype.

--

The C API has a wide choice of functions to call a function, but I still want 
to add  _PyObject_CallNoArg() and _PyObject_CallArg1() to get the best 
performances (avoid layers before reaching the final _PyObject_FastCallDict) 
and have a simple and well defined prototype for the most common function calls.

kwargs:

* PyObject_Call(): root of all call functions in Python 3.5
* PyEval_CallObjectWithKeywords(func, args, kwargs) -> PyObject_Call()

args:

* PyEval_CallObject(func, args)
* PyObject_CallObject(func, args) -> PyEval_CallObjectWithKeywords()

vargs:

* PyObject_CallFunctionObjArgs(func, ...) -> PyObject_Call()

format:

* PyObject_CallFunction(func, format, ...) -> PyObject_Call()
* PyObject_CallMethod(func, method, format, ...) -> PyObject_Call()
* _PyObject_CallMethodId(func, method_id, format, ...) -> PyObject_Call()
* PyEval_CallFunction(func, format, ...) -> PyEval_CallObject()
* PyEval_CallMethod(func, method, format, ...) -> PyEval_CallObject()

Fast call:

* _PyObject_CallNoArg(func) -> _PyObject_FastCallDict()
* _PyObject_CallArg1(func, arg) -> _PyObject_FastCallDict()
* _PyObject_FastCall(func, arg) -> _PyObject_FastCallDict()

--
Added file: http://bugs.python.org/file44188/fastcall_macros.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27824] update ConfigParser docs regarding in-line comments

2016-08-22 Thread R. David Murray


R. David Murray added the comment:

The python3 docs do not mention inline comments.  Perhaps we even want a 
documented deprecation here?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27824] update ConfigParser docs regarding in-line comments

2016-08-22 Thread R. David Murray


R. David Murray added the comment:

Alternate proposal.  Replace that whole paragraph with:

Configuration files may include comments, prefixed by specific characters (# 
and ;). Comments may appear on their own in an otherwise empty line.  If a line 
has a value or section name that does not include a ';', an inline comment can 
be added by entering at least one space followed by a ';' and the comment.  
While supported, inline comments are fragile in the face of values that may 
contain ';'s, and thus are not recommended.

Of course, I'm making a (new) value judgement in that last sentence so lukasz 
may override me :)

--
nosy: +lukasz.langa, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

> _PyObject_FastCallKeywords(PyObject **args, int nargs, PyObject **kwargs, int 
> nkwargs): nkwargs is the number of (key, value) pairs

Hum, I don't know if it's worth to "split" the C array into two parts. Maybe it 
can be just:

_PyObject_FastCallKeywords(PyObject **args, int nargs, int nkwargs);

As current used in Python/ceval.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27818] Speed up number format spec parsing


STINNER Victor added the comment:

-while (pos

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27213] Rework CALL_FUNCTION* opcodes


STINNER Victor added the comment:

See also the issue #27809 "_PyObject_FastCall(): add support for keyword 
arguments" where we discuss if we can completely avoid the creation of a 
temporary dictionary to pass keyword arguments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


STINNER Victor added the comment:

Serhiy: I "moved" your msg273365 to the issue #27809.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

Serhiy Storchaka: "The problem is that passing keyword arguments as a dict is 
not the most efficient way due to an overhead of creating a dict. For now 
keyword arguments are pushed on the stack as interlaced array of keyword names 
and values. It may be more efficient to push values and names as continuous 
arrays (issue27213)."

You describe one specific kind of function call: from Python to Python. Sure, 
in this case, we *can* avoid the creation of a dictionary. Such optimization 
can be implemented in call_function() in Python/ceval.c.

The problem is that in other cases, it's harder to avoid the creation of a 
dictionary:

* C functions defined with METH_KEYWORDS require a dict
* tp_new, tp_init and tp_call slots require a dict

Many C functions "pass through" keyword arguments. Example with the builtin 
sorted() function:

static PyObject *
builtin_sorted(PyObject *self, PyObject *args, PyObject *kwds)
{
...
v = PyObject_Call(callable, newargs, kwds);
...
}

Example of a tp_new slot, the type constructor, type_new():

static PyObject *
type_new(PyTypeObject *metatype, PyObject *args, PyObject *kwds)
{
...
if (winner != metatype) {
if (winner->tp_new != type_new) /* Pass it to the winner */
return winner->tp_new(winner, args, kwds);
metatype = winner;
}
...
if (init_subclass(type, kwds) < 0)
goto error;
...
}

static int
init_subclass(PyTypeObject *type, PyObject *kwds)
{
...
tmp = PyObject_Call(func, tuple, kwds);
...
}

Well, the question can be: where do these keyword argument come from? It's like 
that in 99% of cases, these C functions are called from Python functions which 
use the Python stack for positional and keyword arguments.

The last part of my full fastcall project is to modify tp_new, tp_init and 
tp_call slots to support a fastcall mode which uses the fastcall calling 
convention (add new flags to tp_flags: Py_TPFLAGS_FASTNEW, Py_TPFLAGS_FASTINIT, 
Py_TPFLAGS_FASTCALL).

I'm not sure that this part of my plan is feasible since it is a major backward 
incompatible change: it breaks all C code calling directly tp_new, tp_init and 
tp_call, and such code is common (even inside the stdlib). I plan to open a 
discussion to see if it's worth it. It would require to modify all code calling 
directly tp_new, tp_init and tp_call to use a new wrapper function, function 
which would only be available in the C API of Python 3.6...

--

I understand your idea of avoid dictionaries for keyword parameters, but maybe 
we need an intermediate step and maybe even two functions:

* one function taking a dict: _PyObject_FastCallDict()
* one function taking an array of (key, value) pairs: _PyObject_FastCall()

_PyObject_FastCallDict() would pass directly the Python dict to C functions 
defined with METH_KEYWORDS, to tp_new, tp_init and tp_call slots.

In a perfect world, _PyObject_FastCallDict() wouldn't be needed. In practice, 
dictionaries are used *everywhere* in the current C code base, so we need a 
function for the transition.

_PyObject_FastCallDict() is private, so we can remove it whenever we want.

--

I'm patching more and more code, and I hate having to add an extra ", NULL" 
when calling the current _PyObject_FastCall().

Maybe we should have a short name, _PyObject_FastCall(), for the common case: 
no keyword parameter, and use longer name for keywords: 
_PyObject_FastCallKeywords() (as PyArg_ParseTuple and PyArg_ParseTupleKeywords).

--

To summarize, I propose 3 functions:

* _PyObject_FastCall(PyObject **args, int nargs)
* _PyObject_FastCallKeywords(PyObject **args, int nargs, PyObject **kwargs, int 
nkwargs): nkwargs is the number of (key, value) pairs
* _PyObject_FastCallDict(PyObject **args, int nargs, PyObject *kwargs): kwargs 
is a Python dictionary

METH_FASTCALL would use _PyObject_FastCallKeywords format: "PyObject **kwargs, 
int nkwargs" for keyword parameters.

What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27828] Regression in http.cookies parsing with single key without value


Martin Panter added the comment:

I am not the biggest expert on cookie syntaxes, but maybe these reports are 
related:

Issue 25228: Skip over invalid cookies by searching for a semicolon; needs 
confirming whether it reopens a security hole

Issue 17340: Problem with a comma in (or preceding) a cookie key

Issue 27674: Handling of cookie value with embedded double quote signs

--
nosy: +martin.panter
title: Regression in cookie parsing with single key without value -> Regression 
in http.cookies parsing with single key without value

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27674] Quote mark breaks http.cookies, Cookie.py processing


Changes by Martin Panter :


--
title: Quote mark breaks cookie processing -> Quote mark breaks http.cookies, 
Cookie.py processing

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23930] http.cookies.SimpleCookie doesn't parse comma-only separated cookies correctly


Changes by Martin Panter :


--
title: SimpleCookie doesn't  parse comma-only separated cookies correctly -> 
http.cookies.SimpleCookie doesn't parse comma-only separated cookies correctly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27818] Speed up number format spec parsing

2016-08-22 Thread Serhiy Storchaka


Serhiy Storchaka added the comment:

LGTM. I expect that with the patch for issue27078 the effect of this 
optimization is even more significant.

Years ago I wrote much larger patch that includes similar but even more 
aggressive microoptimization. But the effect was not very impressive for 
str.format. For f-strings it should be more significant. I'm going to revive my 
old patch and compare it with your short patch.

--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25228] Regression in http.cookies parsing with brackets and quotes


Changes by Martin Panter :


--
title: Regression in cookie parsing with brackets and quotes -> Regression in 
http.cookies parsing with brackets and quotes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

Copy of msg273365 from the issue #27128:

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)Date: 
2016-08-22 12:29

The problem is that passing keyword arguments as a dict is not the most 
efficient way due to an overhead of creating a dict. For now keyword arguments 
are pushed on the stack as interlaced array of keyword names and values. It may 
be more efficient to push values and names as continuous arrays (issue27213). 
PyArg_ParseTupleAndKeywords() accepts a tuple and a dict, but private function 
_PyArg_ParseTupleAndKeywordsFast() (issue27574) can be changed to accept 
positional and keyword arguments as continuous arrays: (int nargs, PyObject 
**args, int nkwargs, PyObject **kwnames, PyObject **kwargs). Therefore we will 
be forced either to change the signature of _PyObject_FastCall() and the 
meaning of METH_FASTCALL, or add new _PyObject_FastCallKw() and METH_FASTCALLKW 
for support fast passing keyword arguments. Or may be add yet 
_PyObject_FastCallNoKw() for faster passing only positional arguments without 
an overhead of _PyObject_FastCall(). And make older _PyObject_FastCall() and 
METH_FASTCAL
 L obsolete.

There is yet one possibility. Argument Clinic can generate a dict that maps 
keyword argument names to indices of arguments and tie it to a function. 
External code should map names to indices using this dictionary and pass 
arguments as just a continuous array to function with METH_FASTCALL (raising an 
error if some argument is passed as positional and keyword, or if keyword-only 
argument is passed as positional, etc). In that case the kwargs parameter of 
_PyObject_FastCall() becomes obsolete too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27829] test.regrtest: changed environment variables are not logged


Roundup Robot added the comment:

New changeset 2e23f7b9515c by Victor Stinner in branch 'default':
Issue #27829: libregrtest.save_env: flush stderr
https://hg.python.org/cpython/rev/2e23f7b9515c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1375011] http.cookies, Cookie.py: Improper handling of duplicate cookies


Changes by Martin Panter :


--
title: Improper handling of duplicate cookies -> http.cookies, Cookie.py: 
Improper handling of duplicate cookies

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()

2016-08-22 Thread Serhiy Storchaka


Serhiy Storchaka added the comment:

The problem is that passing keyword arguments as a dict is not the most 
efficient way due to an overhead of creating a dict. For now keyword arguments 
are pushed on the stack as interlaced array of keyword names and values. It may 
be more efficient to push values and names as continuous arrays (issue27213). 
PyArg_ParseTupleAndKeywords() accepts a tuple and a dict, but private function 
_PyArg_ParseTupleAndKeywordsFast() (issue27574) can be changed to accept 
positional and keyword arguments as continuous arrays: (int nargs, PyObject 
**args, int nkwargs, PyObject **kwnames, PyObject **kwargs). Therefore we will 
be forced either to change the signature of _PyObject_FastCall() and the 
meaning of METH_FASTCALL, or add new _PyObject_FastCallKw() and METH_FASTCALLKW 
for support fast passing keyword arguments. Or may be add yet 
_PyObject_FastCallNoKw() for faster passing only positional arguments without 
an overhead of _PyObject_FastCall(). And make older _PyObject_FastCall() and 
METH_FASTCAL
 L obsolete.

There is yet one possibility. Argument Clinic can generate a dict that maps 
keyword argument names to indices of arguments and tie it to a function. 
External code should map names to indices using this dictionary and pass 
arguments as just a continuous array to function with METH_FASTCALL (raising an 
error if some argument is passed as positional and keyword, or if keyword-only 
argument is passed as positional, etc). In that case the kwargs parameter of 
_PyObject_FastCall() becomes obsolete too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27829] test.regrtest: changed environment variables are not logged


New submission from STINNER Victor:

It's common to get a random "env failed" issue when tests are run by buildbots, 
but changed environment variables are not logged, so the "env changed 
"information cannot be used to fix the issue :-/

Example:
--
...
0:07:32 [160/402] test_io failed (env changed) (51 sec)
0:07:33 [161/402] test_threading_local passed
...
0:13:04 [289/402/2] test_ssl failed (env changed) -- running: 
test_multiprocessing_spawn (119 sec)
0:13:05 [290/402/2] test_wsgiref passed -- running: test_multiprocessing_spawn 
(120 sec)
...
2 tests altered the execution environment:
test_io test_ssl
--
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%20CURRENT%20Debug%203.x/builds/932/steps/test/logs/stdio

I expected "Warning -- xxx was modfied by test_xxx" lines, but no, there is no 
such line. I don't know if it's related to stderr buffering or something else?

This buildbot runs tests in subprocesses.

--
components: Tests
messages: 273364
nosy: haypo
priority: normal
severity: normal
status: open
title: test.regrtest: changed environment variables are not logged
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27829] test.regrtest: changed environment variables are not logged


Changes by STINNER Victor :


--
keywords: +buildbot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16113] Add SHA-3 and SHAKE (Keccak) support

2016-08-22 Thread Christian Heimes


Christian Heimes added the comment:

The maintenance burden is minimal. All six algorithms are just variants of the 
same KeccakP-1600 sponge construction with different initialization parameters 
for rate, capacity, delimiter and output size. SHAKEs have no default output 
len and another delimiter as SHA3s. 
https://github.com/gvanas/KeccakCodePackage/blob/master/Modes/KeccakHash.h#L34

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27828] Regression in cookie parsing with single key without value

2016-08-22 Thread SilentGhost


SilentGhost added the comment:

The change was implemented in issue22796 and is listed in the what's new 
document. Any RFC I'm looking at says that the value is required, so your 
syntax is simply not valid and is rejected by the parser.

--
nosy: +SilentGhost, pitrou
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27768] ssl: add public API for IA-32 processor capabilities vector

2016-08-22 Thread Antoine Pitrou


Antoine Pitrou added the comment:

I still think it's a slippery slope to include such specialized APIs that most 
people don't know what to make of.  This reminds me of `RAND_egd()`.

If ChaCha20-Poly1305 is always /at least/ as secure as AES-GCM (and sometimes 
more), then we should simply prioritize it in the cipher list, regardless of a 
potentially poorer performance that probably never has an actual impact on the 
Python-written application (correct me if I'm wrong here).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27768] ssl: add public API for IA-32 processor capabilities vector

2016-08-22 Thread Alex Gaynor


Alex Gaynor added the comment:

In this case, performance is security. Both AES-GCM and ChaCha20-Poly1305 are 
secure. Modulo one thing: GCM in software is hard to implement in 
constant-time, so it's strongly preferable to use it only when there's a 
hardware implementation. It works out nicely that in addition to being 
constant-time, the hardware implementation of GCM is faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27768] ssl: add public API for IA-32 processor capabilities vector

2016-08-22 Thread Antoine Pitrou


Antoine Pitrou added the comment:

Agreed with Gregory.

I'm also surprised you consider selecting a cipher suite based on performance 
considerations rather than security. I'm not sure that's something we want to 
promote.

(btw, AES speed is usually not a critical factor except in very specific 
situations)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Xiang Zhang added the comment:

Thanks for your work too. ;) You does the most important change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Mark Dickinson added the comment:

issue27792_v2.patch LGTM. Thanks for the fix!

--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Roundup Robot added the comment:

New changeset d998d87f0aa0 by Mark Dickinson in branch 'default':
Issue #27792: force int return type for modulo operations involving bools.
https://hg.python.org/cpython/rev/d998d87f0aa0

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27828] Regression in cookie parsing with single key without value

2016-08-22 Thread Yuriy Zhuravlev


New submission from Yuriy Zhuravlev:

After I migrated my site to Python 3.5 from 3.4 many users of my site have not 
been able to login. After small research I found out that users cookies there 
was a key without value and new Python get empty SimpleCookie result. 
Example for Python 3.5.2:

>>> import http.cookies as Cookie
>>> _cookies = Cookie.SimpleCookie()
>>> _cookies.load("rgoods_1=2; __test;")
>>> _cookies


Example for Python 3.4.5:

>>> import http.cookies as Cookie
>>> _cookies = Cookie.SimpleCookie()
>>> _cookies.load("rgoods_1=2; __test;")
>>> _cookies


I don't know many about right cookie format but drop the wrong key for me is 
the best solution. I have seen many discussions about Cookie parsers but I was 
not see words about single keys. If it will be dublicate I am sorry.

--
components: Library (Lib)
messages: 273355
nosy: Yuriy Zhuravlev
priority: normal
severity: normal
status: open
title: Regression in cookie parsing with single key without value
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

@Stefan: Would you mind to review fastcall_kwargs-2.patch please?

--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27809] _PyObject_FastCall(): add support for keyword arguments


STINNER Victor added the comment:

Patch version 2:

* add a fast-path to fastlocals_dict() when total_args==0
* Extract co_varnames assignement out of the loop (it's a constant)
* Cleanup also the code to make it more readable

--
Added file: http://bugs.python.org/file44187/fastcall_kwargs-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Xiang Zhang added the comment:

Oh, sorry. I forgot about long_long. Actually I did think for a while to search 
for a good function here. :( v2 uses long_long now.

> I agree that the fast paths need looking at; it seems there's a fair bit of 
> redundancy there. But not in this issue, please!

Don't worry. It's only a passing mention. Not in this thread. :)

--
Added file: http://bugs.python.org/file44186/issue27792_v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


Roundup Robot added the comment:

New changeset 7dd85b19c873 by Victor Stinner in branch 'default':
Optimize call to Python function without argument
https://hg.python.org/cpython/rev/7dd85b19c873

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


STINNER Victor added the comment:

> FYI: I copied your (no-kwargs) implementation over into Cython and I get 
> around 17% faster calls to Python functions with 2 positional arguments.

Hey, cool! It's always cool to get performance enhancement without having to 
break the C API nor having to modify source code :-)

What do you mean by "I copied your (no-kwargs) implementation"? The whole giant 
patch? Or just a few changes? Which changes?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()


Stefan Behnel added the comment:

FYI: I copied your (no-kwargs) implementation over into Cython and I get around 
17% faster calls to Python functions with 2 positional arguments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Mark Dickinson added the comment:

> BTW, Mark, do you think the fast path in long_mod is really needed?

I agree that the fast paths need looking at; it seems there's a fair bit of 
redundancy there. But not in this issue, please!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Mark Dickinson added the comment:

Thanks for the patch. I think what we really want here is a call to `long_long` 
rather than `PyNumber_Long`; `PyNumber_Long` includes all the conversions using 
`__trunc__`, etc., which we don't need here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27539] negative Fraction ** negative int not normalized


Mark Dickinson added the comment:

Thanks, all. Closing as fixed.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27539] negative Fraction ** negative int not normalized


Roundup Robot added the comment:

New changeset 7eea5b87f5fa by Mark Dickinson in branch '3.5':
Issue #27539: Fix unnormalised Fraction.__pow__ result for negative exponent 
and base. Thanks Vedran Čačić.
https://hg.python.org/cpython/rev/7eea5b87f5fa

New changeset 902e82f71880 by Mark Dickinson in branch 'default':
Issue #27539: Merge from 3.5.
https://hg.python.org/cpython/rev/902e82f71880

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27827] pathlib is_reserved fails for some reserved paths on Windows

2016-08-22 Thread Eryk Sun


New submission from Eryk Sun:

pathlib._WindowsFlavour.is_reserved assumes Windows uses an exact match up to 
the file extension for reserved DOS device names. However, this misses cases 
involving trailing spaces and colons, such as the following examples:

Trailing colon:

>>> pathlib.Path('C:/foo/NUL:').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL:'))
\\.\NUL

Trailing spaces:

>>> pathlib.Path('C:/foo/NUL  ').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL  '))
\\.\NUL

Trailing spaces followed by a file extension:

>>> pathlib.Path('C:/foo/NUL  .txt').is_reserved()
False
>>> print(os.path._getfullpathname('C:/foo/NUL  .txt'))
\\.\NUL

Windows calls RtlIsDosDeviceName_Ustr to check whether a path represents a DOS 
device name. Here's a link to the reverse-engineered implementation of this 
function in ReactOS 4.1:

http://code.reactos.org/browse/reactos/branches/ros-branch-0_4_1/reactos/sdk/lib/rtl/path.c?r=71210#to85

The ReactOS implementation performs the following steps:

* Return false for a UNC or unknown path type or an empty path.
* Strip a final ":" if present. Return false if it was the
  only character.
* Strip trailing dot and space characters.
* Iterate over the path in reverse. If the current character is
  a "\\" or "/" or a ":" drive letter separator (at index 1),
  then if the next character matches the first letter of a DOS
  device name, splice out the base name as a potential match.
  Else return false.
* Return false if the first character at this point does not 
  match the first letter of a DOS device name.
* Remove the file extension, starting at the first dot or colon.
* Remove trailing spaces.
* Return the name offset and length if it equals
* "COM" or "LPT" plus a digit
* "PRN", "AUX", "NUL, or "CON"
* Else return false.

It seems that ":" and "." are effectively equivalent for the purposes of 
is_reserved. Given this is the case, it could return whether 
parts[-1].partition('.')[0].partition(':')[0].rstrip(' ').upper() is in 
self.reserved_names. Or maybe use a regex for the entire check.

If a script is running on Windows, I think the best approach is to call 
os.path.abspath, which calls _getfullpathname. This lets Windows itself 
determine if the path maps to the \\.\ device namespace. However, I realize 
that is_reserved is intended to be cross-platform.

By the way, the comment for this method says that r"foo\NUL" isn't reserved, 
but it is. Maybe the author checked by trying to open NUL in a non-existing foo 
directory. DOS device names are only reserved in practice when opening and 
creating files in existing directories (as opposed to reserved in principle 
with GetFullPathName, which doesn't check for a valid path). NT can thus return 
an error that's consistent with how DOS behaved in the 1980s -- because that's 
really important, you know.

--
components: Library (Lib), Windows
messages: 273344
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: pathlib is_reserved fails for some reserved paths on Windows
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27821] IDLE custom keymaps don't work anymore (Windows, Python 3.6.0a3)

2016-08-22 Thread Vedran Čačić


Vedran Čačić added the comment:

P.S. I tested and no, IDLE Classic Windows (nor anything else except Veky2, I 
deleted Veky) is not added to config-keys.cfg. My config-keys.cfg is


[Veky2]
toggle-auto-coloring = 
tabify-region =  
copy =  
del-word-right = 
save-window-as-file =  
center-insert =  
dedent-region = 
indent-region = 
untabify-region =  
select-all =  
end-of-file =  
open-module =
plain-newline-and-indent =  
close-all-windows =  
paste =  
open-window-from-file =  
python-docs = 
do-nothing = 
cut =  
toggle-tabs =
uncomment-region =  
open-new-window =  
del-word-left = 
find-selection = 
save-window =  
history-previous = 
remove-selection = 
find =  
open-class-browser =
close-window =  
smart-backspace = 
python-context-help = 
save-copy-of-window-as-file =  
redo =  
restart-shell = 
smart-indent = 
goto-line =
comment-region =  
beginning-of-line = 
interrupt-execution =  
find-in-files =  
undo =  
view-restart = 
replace =  
print-window =  
history-next = 
change-indentwidth =
find-again =   
newline-and-indent =  


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27821] IDLE custom keymaps don't work anymore (Windows, Python 3.6.0a3)

2016-08-22 Thread Vedran Čačić


Vedran Čačić added the comment:

Neither. I see _two_ rows, lower one is as you quoted. So:

--Key Set
( ) Use a Built-in Key Set [IDLE Classic Windows --]
(*) Use a  Custom  Key Set [IDLE Classic Windows --]
-

The first button is grayed out. I can select the first radio button and it 
works (I can switch to a different builtin set), but as soon as I select the 
second one, the old behavior returns (select Veky2, OK, doesn't work).

The shortcut I miss the most is Ctrl+Up/Down instead of Alt+p/n for history 
back/forward. But I guess it doesn't really matter much, unless you're willing 
to add it to the builtin set. :-D

I also have customized sys.ps1, sys.ps2 and sys.__interactivehook__, but I 
really think this shouldn't matter for this bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27810] Add METH_FASTCALL: new calling convention for C functions


STINNER Victor added the comment:

> Extensive callback interfaces like map() come to mind, where a large number 
> of calls becomes excessively time critical and might thus have made people 
> implement their own special purpose calling code.

I didn't patch map_next() of Python/bltinmodule.c nor list.sort() for use fast 
call yet. With my full patch, map() and list.sort() are *faster*, but even more 
changes are needed ;-)

https://bugs.python.org/issue26814#msg263999

- map(lambda x: x, list(range(1000))): 18% faster
- sorted(list, key=lambda x: x): 33% faster

I'm now aware of map() or list.sort()/sorted() function rewritten from scratch 
for better performance, but I can imagine that someone did that. But do these 
specialized implementation use PyObject_Call() or "inline" PyCFunction_Call()?

If someone "inlines" PyCFunction_Call(), be prepared to "backward incompatible 
changes", since it's common that CPython internals change in a subtle way.


> However, I don't know any such code (outside of Cython)

I used GitHub to search for such code using "case METH_VARARGS". I found:

* (many copies of the CPython source code)
* 
https://github.com/jhgameboy/ironclad/blob/5892c43b540b216d638e0fed2e6cf3fd8289fdfc/src/CallableBuilder.cs
 : ironclad is a module to call C extensions from IronPython, the last commit 
of the project was in 2011...
* 
https://github.com/pepsipepsi/nodebox_opengl_python3/blob/cfb2633df1055a028672b11311603cc2241a1378/nodebox/ext/psyco/src/c/Objects/pmethodobject.c
 : copy of the psyco module. The last commit in pysco was also in 2011, 
https://bitbucket.org/arigo/psyco/commits/all

In short, I found nothing. But I didn't search far, maybe there are a few other 
code bases using "case METH_VARARGS:"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27810] Add METH_FASTCALL: new calling convention for C functions


Stefan Behnel added the comment:

Extensive callback interfaces like map() come to mind, where a large number of 
calls becomes excessively time critical and might thus have made people 
implement their own special purpose calling code.

However, I don't know any such code (outside of Cython) and I agree that this 
is such a special optimisation that there can't be many cases. If there are 
others, they'll likely be using the obvious fallback already, or otherwise just 
adapt when Py3.6 comes out and move on. The fix isn't complex at all.

I do not consider this a road block.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27810] Add METH_FASTCALL: new calling convention for C functions


STINNER Victor added the comment:

> There is a tiny bit of a backwards compatibility concern as the new function 
> signature would be incompatible with anything we had before,

Right. If you call directly PyCFunction functions, you will likely get quickly 
a crash. But... who call directly PyCFunction functions? Why not using the 30+ 
functions to call functions?

Hopefully, it's easy to support METH_FASTCALL in an existing function getting a 
tuple:

   int nargs = (int)PyTuple_GET_SIZE(args);
   PyObject **stack = _GETITEM(args, 0);
   result = func(self, stack, nargs, kwargs);

I guess that Cython calls directly PyCFunction. cpyext module of PyPy probably 
too. Ok, except of them, are you aware of other projects doing that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27792] bool % int has inconsistent return type.


Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27826] Null-pointer dereference in tuplehash() function


STINNER Victor added the comment:

> And it is hard to protect from such situation in marshal.c.

Python doesn't validate marshal nor bytecode. It's a deliberate choice to get 
best performances.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27506] make bytes/bytearray translate's delete a keyword argument