[issue29274] Change “tests cases” → “test cases”

[Martin Panter]

New submission from Martin Panter:

The “unittest” documentation has “tests cases” written a few times. This 
doesn’t seem right to me, but I thought I should get a second opinion in case I 
missed something.

--
assignee: docs@python
components: Documentation
files: tests-cases.patch
keywords: patch
messages: 285468
nosy: docs@python, martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: Change “tests cases” → “test cases”
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46288/tests-cases.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1621] Do not assume signed integer overflow behavior

[Roundup Robot]

Roundup Robot added the comment:

New changeset dd2c7d497878 by Martin Panter in branch '3.5':
Issues #1621, #29145: Test for str.join() overflow
https://hg.python.org/cpython/rev/dd2c7d497878

New changeset eb6eafafdb44 by Martin Panter in branch 'default':
Issue #1621: Overflow should not be possible in listextend()
https://hg.python.org/cpython/rev/eb6eafafdb44

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29145] failing overflow checks in replace_*

[Roundup Robot]

Roundup Robot added the comment:

New changeset dd2c7d497878 by Martin Panter in branch '3.5':
Issues #1621, #29145: Test for str.join() overflow
https://hg.python.org/cpython/rev/dd2c7d497878

New changeset 0c6ea411af17 by Martin Panter in branch 'default':
Issue #29145: Merge test from 3.6
https://hg.python.org/cpython/rev/0c6ea411af17

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22980] C extension naming doesn't take bitness into account

[Roundup Robot]

Roundup Robot added the comment:

New changeset 80fc40a9ae47 by Martin Panter in branch '3.5':
Issue #22980: Skip a sysconfig test if _ctypes is not available.
https://hg.python.org/cpython/rev/80fc40a9ae47

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29273] test___all__ alters utf8 locale setting

[Martin Panter]

New submission from Martin Panter:

On a Linux computer I have LANG=en_US.utf8 set. Python 3 initializes the locale 
with this setting (see revision 43e32b2b4004):

>>> locale.setlocale(locale.LC_CTYPE)  # Get setting initialized by Python
'en_US.utf8'

In Lib/test/test___all__.py, there is a special case to restore the locale 
after loading the “readline” module (last changed in revision 37ed6eed0595). It 
does this by calling getlocale() and later setlocale(). However the getlocale() 
call translates the setting to UTF-8 in uppercase with a dash, and setlocale() 
keeps it that way:

>>> locale.getlocale(locale.LC_CTYPE)
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_CTYPE, _)
'en_US.UTF-8'

This means that the test framework may complain:

Warning -- locale was modified by test___all__

One subtlety is that I only see the warning when the “tkinter” module is 
unavailable. Otherwise, the test imports Lib/idlelib/iomenu.py, which calls 
setlocale(locale.LC_CTYPE, "") and restores the original “utf8” spelling.

The easiest fix would be to use setlocale(locale.LC_CTYPE) instead of 
getlocale() in test___all__. However I think it may be safe to remove this code 
entirely. The original version of the it was added by Guido in revision 
ba522c74a931 (year 2002). Three hours later is revision c7d5801bbae8, where 
Guido changed the “readline” module to restore the locale after Readline 
library initialization. Since the module is fixed, I don’t think we need the 
test workaround, and I propose to remove it in Python 3, where it causes the 
above conflict.

--
components: Tests
files: readline-getlocale.patch
keywords: patch
messages: 285467
nosy: martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: test___all__ alters utf8 locale setting
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46287/readline-getlocale.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[INADA Naoki]

INADA Naoki added the comment:

1. _PyCFunction_FastCallKeywords() calls _PyStack_AsDict(), then calls 
_PyCFunction_FastCallDict().
2. _PyCFunction_FastCallDict() calls _Py_FastCall_FromArgs() when METH_FASTCALL.
3. _Py_FastCall_FromArgs() calls _PyStack_UnpackDict(), then calls fastcall().

Can you remove this _PyStack_AsDict() & _PyStack_UnpackDict() by creating 
shortcut in _PyCFunction_FastCallKeywords()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix

[Larry Hastings]

Larry Hastings added the comment:

Could one of you recent tagees (Terry, Zach) review the patch?  Hoping to tag 
3.5.3 final in less than 48 hours, and I want to cherry-pick the fix for 
this...!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29006] 2.7.13 _sqlite more prone to "database table is locked"

[Larry Hastings]

Larry Hastings added the comment:

Hoping to tag in less than 48 hours...!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29062] hashlib documentation link error

[Martin Panter]

Martin Panter added the comment:

Looks like Doc/tools/susp-ignored.csv needs updating:

$ make -C Doc/ suspicious
[. . .]
writing output... [ 49%] library/hashlib
WARNING: [library/hashlib:502] ":vatrogasac" found in ">>> cookie = 
b'user:vatrogasac'"
WARNING: [library/hashlib:502] ":vatrogasac" found in 
"user:vatrogasac,349cf904533767ed2d755279a8df84d0"
WARNING: [library/hashlib:502] ":policajac" found in ">>> 
compare_digest(b'user:policajac', sig)"
WARNING: [library/hashlib:646] ":LEAF" found in "... h00 = 
blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH,"
[. . .]
WARNING: Found 4/327 unused rules:
library/hashlib-blake2,,:vatrogasac,>>> cookie = b'user:vatrogasac'
library/hashlib-blake2,,:vatrogasac,user:vatrogasac,349cf904533767ed2d755279a8df84d0
library/hashlib-blake2,,:policajac,>>> compare_digest(b'user:policajac', sig)
library/hashlib-blake2,,:LEAF,h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, 
depth=DEPTH,
build finished with problems, 7 warnings.
make[1]: *** [build] Error 1
make[1]: Leaving directory `/media/disk/home/proj/python/cpython/Doc'
Suspicious check complete; look for any errors in the above output or in 
build/suspicious/suspicious.csv.  If all issues are false positives, append 
that file to tools/susp-ignored.csv.
make: *** [suspicious] Error 1

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions

[INADA Naoki]

INADA Naoki added the comment:

Maybe, we should do

* Make clinic use more METH_FASTCALL
* Use clinic more in builtin methods

before trying this optimization.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23407] os.walk always follows Windows junctions

[Craig Holmquist]

Craig Holmquist added the comment:

FWIW, the only name-surrogate tags in the user-mode SDK headers (specifically 
winnt.h) are IO_REPARSE_TAG_MOUNT_POINT and IO_REPARSE_TAG_SYMLINK, as of at 
least the Windows 8.1 SDK.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions

[INADA Naoki]

INADA Naoki added the comment:

This is proof of concept patch to support LOAD_METHOD & CALL_METHOD for 
METH_FASTCALL methods.
(This patch should be applied after fastcall.patch)

$ ./python -m perf timeit --compare-to `pwd`/python-fastcall -s "d = b''" -- 
"d.decode()"
python-fastcall: . 91.0 ns +- 1.0 ns
python: . 80.3 ns +- 0.3 ns

Median +- std dev: [python-fastcall] 91.0 ns +- 1.0 ns -> [python] 80.3 ns +- 
0.3 ns: 1.13x faster

$ ./python -m perf timeit --compare-to `pwd`/python-fastcall -s "d = b''" -- 
"d.decode('ascii')"
python-fastcall: . 116 ns +- 1 ns
python: . 106 ns +- 1 ns

Median +- std dev: [python-fastcall] 116 ns +- 1 ns -> [python] 106 ns +- 1 ns: 
1.10x faster


Since PyCFunction is lighter than PyMethodObject, performance benefit seems 
smaller than
Python method (up to 20%).

Sadly speaking, there are only few METH_FASTCALL in builtin type.

--
keywords: +patch
Added file: http://bugs.python.org/file46286/loadmethod-fastcall.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[INADA Naoki]

INADA Naoki added the comment:

LGTM

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26296] colorsys rgb_to_hls algorithm error

[Bryan B]

Bryan B added the comment:

Well, the other issue was resolved by updating Python on my computer to 3.6 ;)

Setting up the entire Python build and test environment for an issue this small 
seems a little excessive, especially for a module that seems seldomly used. I'm 
gonna have to be that guy and let this one sit as well.

Sorry :(

--
title: colorys rgb_to_hls algorithm error -> colorsys rgb_to_hls algorithm error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

[Eryk Sun]

Changes by Eryk Sun :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

[Eryk Sun]

Eryk Sun added the comment:

Here's a patch that deletes __classcell__ from the dict before calling type_new.

--
keywords: +patch
Added file: http://bugs.python.org/file46285/issue_29270_01.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26296] colorys rgb_to_hls algorithm error

[Bryan B]

Bryan B added the comment:

Adding myself to this since I'm going to fix another hiccup in this file and I 
might as well clean this up too.

--
nosy: +aarqon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29272] test_logging hangs if /etc/hosts only aliases "localhost" to ::1

[Brian Vandenberg]

New submission from Brian Vandenberg:

On some of the linux boxes on our (air-gapped, if that matters) network it 
looks like some of them were mis-configured and their /etc/hosts file looks 
something like this:

$ cat /etc/hosts
127.0.0.1 snoopy.the.internal.domain snoopy localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

... whereas for other linux machines not exhibiting this problem:

$ cat /etc/hosts
127.0.0.1 woodstock.the.internal.domain woodstock localhost 
localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

It appears this is causing test_logging to hang while running SocketHandlerTest.

Unfortunately I don't have administrative privileges so I'm not able to prove 
I'm right.  I only have circumstantial evidence, but no machine I tested (about 
20 machines) contradicts what I stated above.

--
components: Tests
messages: 285452
nosy: phantal
priority: normal
severity: normal
status: open
title: test_logging hangs if /etc/hosts only aliases "localhost" to ::1
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

> Slower (63):
> - unpickle_pure_python: 671 us +- 16 us -> 952 us +- 9 us: 1.42x slower (+42%)
> ...

This benchmark was run with LTO+PGO. It seems like PGO produced less efficient 
machine code with the patch. Maybe a missed optimization.

Oh, I just realized that I announced that objects with a __call__() are faster, 
but it's not the case yet! Such object still gets slot_tp_call() as tp_call 
which isn't a fastcall!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

[Eryk Sun]

Eryk Sun added the comment:

In 3.6, type_new in Objects/typeobject.c sets the __classcell__ in the dict if 
it's a cell object. It happens that CreateSwappedType in 
Modules/_ctypes/_ctypes.c re-uses the dict to create the swapped type (e.g. big 
endian), which in turn updates the __classcell__. Thus in 
builtin___build_class__ in Python/bltinmodule.c, the check `cell_cls != cls` 
ends up being true, which leads to the observed TypeError. CreateSwappedType 
should be able to avoid this by either deleting "__classcell__" from the dict 
or creating a copy without it before passing it to type_new.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Michael Felt]

Michael Felt added the comment:

I did not go through a whole build process.

Changing:


if _PYTHON_BUILD:
vars['BLDSHARED'] = vars['LDSHARED']
to

if _PYTHON_BUILD:
vars['LDSHARED'] = vars['BLDSHARED']

is not going to help if both variables are wrong in _sysconfigdata*.py

Also,

The patch is not applied in any version - yet the values in _sysconfigdata*.py 
are different, i.e., version dependent (without the patch).

If I errored in my assumption that the file being patched is reading 
_sysconfigdata*.py then I will need to patch and build from scratch to see the 
effect on _sysconfigdata.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Lele Gaifax]

Lele Gaifax added the comment:

Yay, Yury is too fast!! :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Lele Gaifax]

Lele Gaifax added the comment:

FWIW, the problem afflicts other functions, for example the following script 
exhibits a similar difference:

import asyncio

async def coro():
print(asyncio.Task.all_tasks())
print(asyncio.Task.all_tasks(None))

loop = asyncio.get_event_loop()
loop.run_until_complete(coro())

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +ncoghlan, serhiy.storchaka
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] os.cpu_count is problematic on sparc/solaris

[Brian Vandenberg]

Brian Vandenberg added the comment:

> It doesn't cause any real problem with the tests, though.  I routinely run 
> with -j40 on my 2 cpu test box because the test run completes faster that way 
> due to the way many tests spend time waiting for various things.

In my case it did because it caused enough file descriptors to be allocated 
that it hit the cap for max open file handles.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Lele Gaifax]

Changes by Lele Gaifax :


--
nosy: +lelit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Alberto Berti]

Changes by Alberto Berti :


--
nosy: +azazel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Yury Selivanov]

Changes by Yury Selivanov :


--
keywords: +patch
nosy: +inada.naoki
Added file: http://bugs.python.org/file46284/task.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29271] Task.current_task(None) returns unexpected result

[Yury Selivanov]

New submission from Yury Selivanov:

Quoting Alberto Berdi from python-tulip mailing list:

working with the Python 3.6 asyncio implemented in C, i've got what is
for me a strange behavior of the asyncio.Task.current task
function. Executing the following test:

 import asyncio

 async def coro():
 print(asyncio.Task.current_task())
 print(asyncio.Task.current_task(None))
 print(asyncio.Task.current_task(loop=asyncio.get_event_loop()))

 loop = asyncio.get_event_loop()
 loop.run_until_complete(coro())

in Py3.5, I obtain the result:

 
cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>
 
cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>
 
cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:176]>

In Py3.6, i get:

 
cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]>
None
 
cb=[_run_until_complete_cb() at /usr/lib/python3.6/asyncio/base_events.py:176]>

What's going on here? It's still right to expect the same result from the three 
calls or something has changed?
The documentation for the Task.current_task() function seems unchanged between 
the two version.

My python3.6 has version:

$ python3.6 -VV
Python 3.6.0 (default, Dec 29 2016, 04:29:02) 
[GCC 6.2.1 20161215]

--
assignee: yselivanov
components: asyncio
messages: 285445
nosy: gvanrossum, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Task.current_task(None) returns unexpected result
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

[Dave Jones]

New submission from Dave Jones:

While investigating a bug report in one of my libraries 
(https://github.com/waveform80/picamera/issues/355) I've come across a 
behaviour that appears in Python 3.6 but not prior versions. Specifically, 
calling super() in a sub-class of a ctypes scalar type appears to fail at the 
class definition stage. A minimal test case is as follows:

import ctypes as ct

class SuperTest(ct.c_uint32):
def __repr__(self):
return super().__repr__()

This works happily under python 3.2, 3.4, and 3.5 (that I've tested), and also 
under 2.7 (with the appropriate modification to super's arguments). However, 
under 3.6 it elicits the following exception:

Traceback (most recent call last):
  File "py36_ctypes.py", line 3, in 
class SuperTest(ct.c_uint32):
TypeError: __class__ set to  defining 
'SuperTest' as 

Reading through the "What's New" list in 3.6, I thought this might be something 
to do with the PEP-487 implementation (given it modified class construction), 
but having read through the PEP and associated patches I'm not so sure as I 
can't see anything that affects the setting of the "__class__" attribute (but 
don't rule it out on that basis; I'm no expert!).

I'll admit that sub-classing one of ctypes' scalar types is a little odd, but 
given this works in prior versions and there doesn't appear to be anything in 
the documentation banning the practice (that I've found?) this might constitute 
a bug?

--
components: ctypes
messages: 285444
nosy: Dave Jones
priority: normal
severity: normal
status: open
title: super call in ctypes sub-class fails in 3.6
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16189] config/ld_so_aix not found: old dir name

[Martin Panter]

Changes by Martin Panter :


--
title: ld_so_aix not found -> config/ld_so_aix not found: old dir name

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18235] _sysconfigdata.py wrong on AIX installations

[Martin Panter]

Martin Panter added the comment:

So are you saying you tried patching Python 2 and/or 3, but did not see any 
relevant change at all?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken: wrong dir names

[Martin Panter]

Changes by Martin Panter :


--
title: AIX shared library extension modules installation broken -> AIX shared 
library extension modules installation broken: wrong dir names

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29266] test_create_connection_service_name fails if "http" isn't listed in /etc/services

[R. David Murray]

Changes by R. David Murray :


--
components: +asyncio
nosy: +gvanrossum, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29266] test_create_connection_service_name fails if "http" isn't listed in /etc/services

[R. David Murray]

R. David Murray added the comment:

It looks like that test already does so (it expects an OSError when passed the 
service name 'nonsense').

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] os.cpu_count is problematic on sparc/solaris

[R. David Murray]

R. David Murray added the comment:

You don't know it, but you are actually reporting a possible sparc/Solaris 
specific (I think) bug against os.cpu_count.  There was some discussion about 
how cpu_count might be problematic in this regard.

It doesn't cause any real problem with the tests, though.  I routinely run with 
-j40 on my 2 cpu test box because the test run completes faster that way due to 
the way many tests spend time waiting for various things.

--
components: +Library (Lib) -Tests
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29269] test_socket failing in solaris

[Brian Vandenberg]

New submission from Brian Vandenberg:

I started looking into this failure to see if I could figure out why but it 
looks like I'd have to spend more time than I have available to figure out the 
cause.

Environment/setup:
* air-gapped network (no internet access)
* sparc / Solaris 10
* Built with gcc 6.3.0
* Altered configure script to change -std=c99 to -std=gnu99 (see issue 29264)
* The only configure flags used were --prefix and --with-universal-archs=all


When I run test_socket I see the following 4 failures; please note, I'm hand 
typing the results so I may typo something:


ERROR: testCount (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5248, in testCount
  File "(...)/test_socket.py", line 5151, in recv_data
MemoryError

Error: testCount (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 277, in _tearDown
  File "(...)/test_socket.py", line 289, in clientRun
  File "(...)/test_socket.py", line 5241, in _testCount
  File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile
socket.timeout: timed out

Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5318, in testWithTimeout
data = self.recv_data(conn)
  File "(...)/test_socket.py", line 5151, in recv_data
chunk = conn.recv(self.BUFSIZE)
MemoryError

Error: testWithTimeout (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 277, in _tearDown
raise exc
  File "(...)/test_socket.py", line 289, in clientRun
test_func()
  File "(...)/test_socket.py", line 5313, in _testWithTimeout
sent = meth(file)
  File "(...)/Lib/socket.py", line 296, in _sendfile_use_sendfile
socket.timeout: timed out

Error: testCountWithOffset (test.test_socket.SendfileUsingSendfileTest)
Traceback:
  File "(...)/test_socket.py", line 5287, in testCountWithOffset
self.assertEqual(len(data), count)
AssertionError: 4376231 != 17

Ran 539 tests in 69.166s

FAILED (failures=1, errors=4, skipped=324)
test test_socket failed

--
components: Tests
messages: 285440
nosy: phantal
priority: normal
severity: normal
status: open
title: test_socket failing in solaris
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29268] test_spwd fails on solaris using NIS users

[Brian Vandenberg]

New submission from Brian Vandenberg:

On all our solaris 10 machines when I run a simple test program it never 
reports a failure when calling getspnam:

   #include 
   #include 
   int main( int, char** ) {
 spwd *asdf = getspnam( "some_user" );
 if( NULL == sdf ) {
   perror( "getspnam" );
 }
 return 0;
   }

If I run the above program on our linux boxes it fails as expected, but on our 
solaris machines it produces the same information you'd see running "ypcat 
passwd | grep some_user".

I suspect either there's a bug in the solaris implementation of getpwnam() or 
perhaps there's a configuration issue on our solaris machines, though it's also 
possible this is just how it behaves in Solaris (at least with NIS).

As to whether anything should change for test_spwd -- I suspect this will get 
closed as "won't fix" but at least this report may help anyone else running 
into this failure.

--
components: Tests
messages: 285439
nosy: phantal
priority: normal
severity: normal
status: open
title: test_spwd fails on solaris using NIS users
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24875] pyvenv doesn´t install PIP inside a new venv with --system-site-package

[Vinay Sajip]

Vinay Sajip added the comment:

Patch uploaded herewith.

--
assignee:  -> vinay.sajip
stage:  -> patch review
Added file: http://bugs.python.org/file46283/issue-24875-01.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29261] Missing venv/scripts/common after "make install"

[Roundup Robot]

Roundup Robot added the comment:

New changeset 077681e35577 by Vinay Sajip in branch '3.5':
Fixes #29261: added venv/scripts/common to LIBSUBDIRS.
https://hg.python.org/cpython/rev/077681e35577

New changeset f20b2073dd4a by Vinay Sajip in branch '3.6':
Fixes #29261: merged fix from 3.5.
https://hg.python.org/cpython/rev/f20b2073dd4a

New changeset b76c42656639 by Vinay Sajip in branch 'default':
Closed #29261: merged fix from 3.6.
https://hg.python.org/cpython/rev/b76c42656639

--
nosy: +python-dev
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29197] Remove os.path.splitunc()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29197] Remove os.path.splitunc()

[Roundup Robot]

Roundup Robot added the comment:

New changeset 2be38927ea61 by Serhiy Storchaka in branch 'default':
Issue #29197: Removed deprecated function ntpath.splitunc().
https://hg.python.org/cpython/rev/2be38927ea61

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29267] Cannot override some flags in CFLAGS from the command-line

[Brian Vandenberg]

New submission from Brian Vandenberg:

Due to issue 29264 I was attempting to override the build default "-std=c99" 
with:

/path/to/configure (...) CFLAGS=-std=gnu99

... however, the configure script is written like this:

CFLAGS_NODIST="$CFLAGS_NODIST -std=c99"

This causes it to always override my attempt to override the build default.

--
components: Build
messages: 285435
nosy: phantal
priority: normal
severity: normal
status: open
title: Cannot override some flags in CFLAGS from the command-line
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29197] Remove os.path.splitunc()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29266] test_create_connection_service_name fails if "http" isn't listed in /etc/services

[Brian Vandenberg]

New submission from Brian Vandenberg:

One of our solaris machines doesn't have an entry in /etc/services for "http".  
This is causing test_create_connection_service_name to fail.

In my case I can just ignore that particular failure, but as a fix you might 
consider overtly getting valid service names from the host to test.  
Additionally, if you don't already have it you might consider adding a test 
that validates behavior when using a non-existent service.

--
components: Tests
messages: 285434
nosy: phantal
priority: normal
severity: normal
status: open
title: test_create_connection_service_name fails if "http" isn't listed in 
/etc/services
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28911] Clarify the behaviour of assert_called_once_with

[Lisa Roach]

Lisa Roach added the comment:

It took me a little while to wrap my brain around this, but you are definitely 
right that the documentation is not sufficient, your changes are an 
improvement. 

My wonder is, should we change the documentation or be looking at the code 
itself? I have always interpreted the method as asserting that only one call 
was made with the matching signature. If I want to test for call count I can 
just assert mock.call_count == 1.

The code could instead look like this: 

def assert_called_once_with(_mock_self, *args, **kwargs):
self = _mock_self
if not self.mock_calls.count(call(*args, **kwargs)) == 1:
msg = ("Expected '%s' to be called once with %r %r. Called %s times."   
   % (self._mock_name or 'mock', args, kwargs, 
self.mock_calls.count(call(*args,  **kwargs
raise AssertionError(msg)


Then again, if users have been using this to assert that the call_count is one 
(which is likely since it is in the examples documentation), this change would 
break backwards functionality.

Thoughts?

--
nosy: +lisroach

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] test suite is attempting to spawn 258 child processes to run tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] test suite is attempting to spawn 258 child processes to run tests

[Brian Vandenberg]

Brian Vandenberg added the comment:

This is odd.  I just went back and re-ran 3.5.1 to see how many cores and it's 
having the same problem now.  So, scratch that last coment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] test suite is attempting to spawn 258 child processes to run tests

[Brian Vandenberg]

Brian Vandenberg added the comment:

I forgot to mention, this wasn't an issue in 3.5.1 though I never did check how 
many jobs it was using.

I ran into other issues building that version and moved to a newer version 
because at least one of them (logging test race condition) was fixed after 
3.5.1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29265] test suite is attempting to spawn 258 child processes to run tests

[Brian Vandenberg]

New submission from Brian Vandenberg:

I'm attempting to build python 3.6.0 on sparc/solaris 10.  After the initial 
configure/compile complete I ran "make test" and I see:

$ make test
running build
running build_ext
(...)
running build_scripts
copying and adjusting (...)
changing mode of (...)
renaming (...)
(...)
Run tests in parallel using 258 child processes


I'm fairly sure the issue stems from the fact that each core on the machine has 
8 "threads" and there's 32 cores (for a total of 256 virtual cores).

Each core can execute 8 parallel tasks only in very specific circumstances.  
It's intended for use by things like lapack/atlas where you might be doing many 
computations on the same set of data.

Outside of these more restricted circumstances each core can only handle 2 
parallel tasks (or so I gathered from the documentation), so at best this 
machine could handle 64 backgrounded jobs though I normally restrict my builds 
to the actual core count or less.

The most common way to get a "realistic" core count on these machines from 
shell scripts is:

$ core_count=`kstat -m cpu_info | grep core_id | sort -u | wc -l`

... though I'm not sure how the test suite is determining the core count.  I 
didn't see any mention of "kstat" anywhere.

--
components: Tests
messages: 285430
nosy: phantal
priority: normal
severity: normal
status: open
title: test suite is attempting to spawn 258 child processes to run tests
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[JGoutin]

JGoutin added the comment:

Yes, I reported this encoding issue to some of them.

But, there is still some problems : 
- Some libraries are not updated frequently (Or not still maintained), and 
still use fsencode.
- Tests and CI don't see this problem if they don't have a test case for 
filename with accents or other uncommon characters in english.

This problem will not be easy to eliminate totally...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29264] sparc/ffi.c:440 error: 'asm' undeclared

[Brian Vandenberg]

New submission from Brian Vandenberg:

When building pythong 3.6.0 on solaris 10/sparc I'm seeing the following error:

Modules/_ctypes/libffi/src/sparc/ffi.c:440:8: error: 'asm' undeclared (first 
use in this function)
(...)

If I force it to use -std=gnu99 then it seems to compile fine.

--
components: ctypes
messages: 285428
nosy: phantal
priority: normal
severity: normal
status: open
title: sparc/ffi.c:440 error: 'asm' undeclared
type: compile error
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista

[Segev Finer]

Segev Finer added the comment:

I have read some of https://github.com/rprichard/win32-console-docs and it 
documents quite a bunch of nastiness with PROC_THREAD_ATTRIBUTE_HANDLE_LIST in 
Windows Vista/7. Windows is so much fun sometimes :P

Essentially console handles in Windows before Windows 8 are user mode handles 
and not real kernel handles. Those user mode handles are inherited by a 
different mechanism than kernel handles and regardless of 
PROC_THREAD_ATTRIBUTE_HANDLE_LIST, and if passed in 
PROC_THREAD_ATTRIBUTE_HANDLE_LIST will cause it to fail in weird ways. Those 
user mode console handles have the lower two bits set. The lower two bits in 
Windows are reserved for tagging such special handles.

Also in all versions you can't pass in an empty handle list, but a list with 
just a NULL handle works fine.

See: 
https://github.com/rprichard/win32-console-docs/blob/master/src/HandleTests/CreateProcess_InheritList.cc

I attached a version of the patch with a hack around those issues based on what 
I read, but I can't test that it actually fixes the issues since I don't have a 
Windows Vista or 7 system around.

--
Added file: 
http://bugs.python.org/file46282/windows-subprocess-close-fds-v3-vista7-hack.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

> Strange, I'm unable to reproduce the result on a different computer:
>
> haypo@smithers$ ./python -m perf compare_to ref.json tp_fastcall.json  -v
> Median +- std dev: [ref] 20.0 ms +- 0.5 ms -> [tp_fastcall] 20.0 ms +-
> 0.6 ms: 1.00x slower (+0%)
> Not significant!

Oops, I forgot to mention that I tried the telco benchmark.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Strange, I'm unable to reproduce the result on a different computer:

haypo@smithers$ ./python -m perf compare_to ref.json tp_fastcall.json  -v
Median +- std dev: [ref] 20.0 ms +- 0.5 ms -> [tp_fastcall] 20.0 ms +-
0.6 ms: 1.00x slower (+0%)
Not significant!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[STINNER Victor]

STINNER Victor added the comment:

Hum, it was long time ago since I worked on Windows. Well, Python has a "mbcs" 
codec which uses the ANSI code page which exists like "forever". These 
libraries should be patched to use "mbcs" instead of 
sys.getfilesystemencoding().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[STINNER Victor]

STINNER Victor added the comment:

> Temporary fixing issues with some third party libraries which use C code for 
> files I/O (With filename as "mbcs" encoded bytes internally).
>
> Theses libraries generally call 
> "filename.encode(sys.getfilesystemencoding())" or "os.fsencode(filename)" 
> before sending filenames from Python to C code.

Hum, Python lacks a function to encode to/decode from the ANSI code page, 
something like codecs.code_page_encode() / code_page_decode() with CP_ACP. It 
would allow to get the same encoding in UTF-8 and legacy modes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29261] Missing venv/scripts/common after "make install"

[Zachary Ware]

Zachary Ware added the comment:

LGTM.

--
assignee:  -> vinay.sajip
nosy: +vinay.sajip, zach.ware
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[JGoutin]

JGoutin added the comment:

Personally, I call "sys._enablelegacywindowsfsencoding()" for only one reason :
Temporary fixing issues with some third party libraries which use C code for 
files I/O (With filename as "mbcs" encoded bytes internally).

Theses libraries generally call "filename.encode(sys.getfilesystemencoding())" 
or "os.fsencode(filename)" before sending filenames from Python to C code.

Actually, I didn't see any side effect for using this function. Maybe because I 
call it at start before anything else.

Using the environment variable is not easy in my case.


I can look if encoding caching in fsencode is efficient (On Windows). And 
eventually propose a code change for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

I ran benchmarks on tp_fastcall.patch. Hum, it seems like there is a 
performance issue somewhere :-( Almost all benchmarks are much slower with the 
patch.

haypo@speed-python$ python3 -m perf compare_to 
2017-01-11_00-07-default-b9404639a18c.json tp_fastcall_ref_b9404639a18c.json -G
Slower (63):
- unpickle_pure_python: 671 us +- 16 us -> 952 us +- 9 us: 1.42x slower (+42%)
- pickle_pure_python: 1.02 ms +- 0.02 ms -> 1.40 ms +- 0.02 ms: 1.37x slower 
(+37%)
- telco: 16.6 ms +- 0.6 ms -> 22.7 ms +- 0.3 ms: 1.37x slower (+37%)
- xml_etree_generate: 215 ms +- 3 ms -> 290 ms +- 7 ms: 1.35x slower (+35%)
- hexiom: 17.7 ms +- 0.2 ms -> 23.6 ms +- 0.1 ms: 1.33x slower (+33%)
- xml_etree_process: 182 ms +- 3 ms -> 240 ms +- 4 ms: 1.32x slower (+32%)
- genshi_text: 69.0 ms +- 0.8 ms -> 90.4 ms +- 0.8 ms: 1.31x slower (+31%)
- logging_simple: 27.5 us +- 0.4 us -> 35.8 us +- 0.4 us: 1.30x slower (+30%)
(...)

Faster (1):
- spectral_norm: 271 ms +- 2 ms -> 257 ms +- 2 ms: 1.06x faster (-5%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Ok, I pushed a new commit "Add Py_TPFLAGS_HAVE_FASTCALL flag" in my Git branch.

(By the way, I'm using the Git branch as a working copy, it can change anytime 
to get a nice list of commits. If you want reliable changes, use patches 
attached to this issue.)


Serhiy Storchaka!:
> For compatibility with extensions built with older Pythons you should define 
> new type flag and read tp_fastcall only if the flag is set. See for example 
> Py_TPFLAGS_HAVE_FINALIZE.

Wait, there is maybe an issue with the stable API in my design :-/

Let's say that a builtin type NEW defines tp_fastcall and uses 
fastcall_wrapper() for tp_call. A type OLD inherits from NEW and don't override 
tp_call (not tp_fastcall). Type OLD comes from an extension compiled with 
Python 3.6 and use the stable ABI. Does the type OLD contain the field 
tp_fastcall in memory? Is it possible to copy the tp_fastcall filed from type 
NEW into the type OLD?

If not, the fastcall_wrapper() should be smarter to not get tp_fastcall from 
the type OLD but follow the MRO to find the first type with the 
Py_TPFLAGS_HAVE_FASTCALL flag. Is it possible that this short loop to find has 
a cost on performances?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24875] pyvenv doesn´t install PIP inside a new venv with --system-site-package

[Thomas Nyberg]

Thomas Nyberg added the comment:

Hi Vinay,

You should probably upload a patch with the changes you made (however trivial) 
if you want that version considered. Makes it easier to comment and can then 
point to it for others to consider.

Cheers,
Thomas

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There is nothing about this in PEP 7, but I think it would be better to use 
common subset of C and C++. This would allow to migrate to the subset of C++ in 
future. It is sad that PEP 7 allows C99 features not compatible with C++.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

PyArg_UnpackStack() is declared as in the limited API since 3.3. If you want to 
add PyArg_UnpackStack() to the limited API, define it as added in 3.7.

For compatibility with extensions built with older Pythons you should define 
new type flag and read tp_fastcall only if the flag is set. See for example 
Py_TPFLAGS_HAVE_FINALIZE.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions

[STINNER Victor]

Changes by STINNER Victor :


--
dependencies: +Add tp_fastcall to PyTypeObject: support FASTCALL calling 
convention for all callable objects

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26110] Speedup method calls 1.2x

[STINNER Victor]

STINNER Victor added the comment:

I just created the issue #29263: "Implement LOAD_METHOD/CALL_METHOD for C 
functions".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29263] Implement LOAD_METHOD/CALL_METHOD for C functions

[STINNER Victor]

New submission from STINNER Victor:

The issue #29259 implements tp_fastcall on method_descriptor 
(PyMethodDescr_Type). According to http://bugs.python.org/issue26110#msg283093 
it would allow to implement LOAD_METHOD and CALL_METHOD for C functions.

--
messages: 285414
nosy: haypo, inada.naoki, yselivanov
priority: normal
severity: normal
status: open
title: Implement LOAD_METHOD/CALL_METHOD for C functions
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[STINNER Victor]

STINNER Victor added the comment:

> Designated initializers already are used in _asynciomodule.c.

Oh, I didn't notice. Since I don't recall any bug report on compilation failing 
on this module, I guess that it's fine to start to use them in more code.

> But this is one of C99 features that are not compatible with C++. Using them 
> makes harder possible using C++ compiler in future.

Hum, what kind of issue? With which compiler? What is it in the PEP 7 in this 
case?

--
nosy: +benjamin.peterson, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Designated initializers already are used in _asynciomodule.c. But this is one 
of C99 features that are not compatible with C++. Using them makes harder 
possible using C++ compiler in future.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29262] Provide a way to check for *real* typing.Union instances

[flying sheep]

New submission from flying sheep:

typing.Union prevents the use of `isinstance` and `issubclass` via a hook.

This is presumably to prevent errors that would arise if someone tried to do 
issubclass(A, Union[A, B]) or isinstance(A(), Union[A, B]), because Union isn’t 
specified in the PEP to allow a check like this, and doesn’t implement it. 
(Instead it throws said error)

However, as far as I can see there is no blessed way to check if an object was 
returned by Union.__getitem__(). A simple way that works is:

sig = inspect.signature(f)
ann = sig.parameters['arg1'].annotation
is_an_union = isinstance(ann, typing._Union)

but _Union is a private class, and an implementation detail.

is there a blessed way to do this? If not, one should be added.

--
components: Library (Lib)
messages: 285410
nosy: flying sheep
priority: normal
severity: normal
status: open
title: Provide a way to check for *real* typing.Union instances
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

tp_fastcall.patch modifies property_descr_get() to use tp_fastcall if 
available, or use the cached tuple. Once performances are validated (no 
regression), the cached tuple as to pass position arguments, should go away.

By the way, msg285390 uses the following micro-benchmark:
---
import perf
bench = perf.Runner()
bench.timeit("namedtuple.attr",
 "a.a",
 "from collections import namedtuple as n; a = n('n', 'a b c')(1, 
2, 3)",
 duplicate=20)
---

It tests property_descr_get() with operator.itemgetter(). It would be nice to 
benchmark property_descr_get() with other functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[STINNER Victor]

STINNER Victor added the comment:

> The widespread use of PyType_FromSpec() will supersede this issue.

Oh, I didn't know PyType_FromSpec(). It is only used for a very few types in 
the Python stdlib.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[INADA Naoki]

INADA Naoki added the comment:

https://www.python.org/dev/peps/pep-0007/#c-dialect says
"designated initializers (especially nice for type declarations)"

So I think it's allowed explicitly.
To minimize diff, I think we can start using it when adding
slot at bottom of type declaration, or creating completely
new type.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29240] Implementation of the PEP 540: Add a new UTF-8 mode

[STINNER Victor]

STINNER Victor added the comment:

Oh, I just noticed that os.environ uses the hardcoded error handler 
"surrogateescape": it should be replaced with sys.getfilesystemencodeerrors() 
to support UTF-8 Strict mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26858] android: setting SO_REUSEPORT fails

[STINNER Victor]

STINNER Victor added the comment:

> The reason why the changes made by issue 28174 fix only partly the problem 
> should be understood.

The change c1c247cf3488 catchs OSError on sock.setsockopt() in 
asyncio/base_events.py, whereas the test fails while calling sock.getsockopt() 
in test_asyncio/tets_events.py. Extract of the test:

def test_create_server_reuse_port(self):
proto = MyProto(self.loop)
f = self.loop.create_server(
lambda: proto, '0.0.0.0', 0)
server = self.loop.run_until_complete(f)
self.assertEqual(len(server.sockets), 1)
sock = server.sockets[0]
self.assertFalse(
sock.getsockopt(
socket.SOL_SOCKET, socket.SO_REUSEPORT))
server.close()

The change c1c247cf3488 doesn't touch the code of this unit test.

test_base_events.py also calls directly sock.getsockopt(socket.SOL_SOCKET, 
socket.SO_REUSEPORT).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Ok, here is a first patch: tp_fastcall.patch

* Add tp_fastcall to PyTypeObject
* Add fastcall_wrapper() and use it for tp_call in PyType_Ready()
* Modify _PyObject_FastCallDict() and _PyObject_FastCallKeywords() to use 
tp_fastcall if defined
* Modify a lot of types to define tp_fastcall instead of tp_call (to use 
FASTCALL calling convention)
* Add a few new helper functions:

  - _PyObject_FastCall_Prepend() -- similar to _PyObject_Call_Prepend()
  - _Py_FastCall_FromArgs() -- ugly name, sorry about that, it should be 
renamed :-)
  - PyArg_UnpackStack() -- similar to PyArg_UnpackTuple()
  - _PyArg_NoStackKeywords() -- similar to _PyArg_NoKeywords

Hum, I should make sure that these new functions are not public. By the way, I 
should probably rename PyArg_UnpackStack() to _PyArg_UnpackStack().

--
keywords: +patch
Added file: http://bugs.python.org/file46281/tp_fastcall.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29261] Missing venv/scripts/common after "make install"

[Dave Jones]

New submission from Dave Jones:

While attempting to diagnose something (unrelated to this issue) under python 
3.6, I used the following steps to clone and build a non-root python 
installation:

$ mkdir py36
$ hg clone https://hg.python.org/cpython
$ cd cpython
$ hg update 3.6
$ ./configure --prefix=/home/pi/py36
$ make
$ make install

Afterwards I attempted to set up a venv for further testing:

$ /home/pi/py36/bin/python3 -m venv py36env

This succeeded but, unfortunately, activating the venv proved impossible as the 
"activate" script was missing from its "bin" sub-dir. 

After a bit of investigation it appears that "make install" had copied 
"Lib/venv/scripts/posix" but not "Lib/venv/scripts/common" into the 
installation, hence the templates for "activate.csh" and "activate.fish" are 
present, but not the "activate" template.

I had a search around for related issues but only found #22343 which explains 
why "activate" is no longer present (because it's moved from scripts/posix/ to 
scripts/common/) but I don't think is responsible for common not getting copied 
by "make install".

It appears adding "venv/scripts/common" to LIBSUBDIRS in Makefile.pre.in fixes 
the issue (as in the attached patch) but I've no idea if this covers all 
necessary platforms (I'd guess Windows doesn't use the Makefile if #22343 fixed 
things for it?).

--
components: Installation
files: missing_venv_common.diff
keywords: patch
messages: 285403
nosy: Dave Jones
priority: normal
severity: normal
status: open
title: Missing venv/scripts/common after "make install"
type: behavior
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46280/missing_venv_common.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

> * If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses 
> the wrapper)
> 
> Is tp_call set to the wrapper rather then inheriting?

It is set to the wrapper. Defining tp_fastcall should work as defining tp_call: 
it should override the tp_call and tp_fastcall slots.


>  What if tp_call is defined in a superclass?

What is a superclass? Do you have an example?



> * If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call 
> whereas _PyObject_FastCallDict() uses tp_fastcall.
>
> I would consider this as a bug. It would be weird if different ways of 
> calling cause executing different code.

It's a micro-optimization to avoid arguments unpacking/packing, conversions 
needed when you switch from the regular calling convention to the fast call 
convention, or the opposite.

I don't think that types defining tp_call and tp_fastcall will be common.


> What about dynamically changed Python types?

What do you mean? Do you have an example?


> What if you set or delete the __call__ attribute of Python class?

I don't know how these things work :-) Let me try on a patched Python (using 
tp_fastcall):

$ ./python
>>> class A:
...  def __call__(self): print("A")
... 
>>> a=A()
>>> a()
A

>>> A.__call__=lambda self: print("B!")
>>> a()
B!

>>> del A.__call__
>>> a()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'A' object is not callable

It seems like "it just works", but I don't know how it works internally :-)

a() uses a dynamic lookup of a.__class__.__call__, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26858] android: setting SO_REUSEPORT fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Skipping those tests on Android has already been proposed with the existing 
patches in this issue.
The reason why the changes made by issue 28174 fix only partly the problem 
should be understood.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The widespread use of PyType_FromSpec() will supersede this issue. But there is 
a caveat (issue26979).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

* If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses 
the wrapper)

Is tp_call set to the wrapper rather then inheriting? What if tp_call is 
defined in a superclass?

> * If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call 
> whereas _PyObject_FastCallDict() uses tp_fastcall.

I would consider this as a bug. It would be weird if different ways of calling 
cause executing different code.

What about dynamically changed Python types? What if you set or delete the 
__call__ attribute of Python class?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29260] Use designated initializers to define PyTypeObject types

[STINNER Victor]

New submission from STINNER Victor:

Currently, PyTypeObject fields are set in order in the C code. Typical example:

PyTypeObject PyMethod_Type = {
PyVarObject_HEAD_INIT(_Type, 0)
"method",
sizeof(PyMethodObject),
0,
(destructor)method_dealloc, /* tp_dealloc */
0,  /* tp_print */
0,  /* tp_getattr */
0,  /* tp_setattr */
0,  /* tp_reserved */
(reprfunc)method_repr,  /* tp_repr */
0,  /* tp_as_number */
0,  /* tp_as_sequence */
0,  /* tp_as_mapping */
(hashfunc)method_hash,  /* tp_hash */
method_call,/* tp_call */
0,  /* tp_str */
method_getattro,/* tp_getattro */
PyObject_GenericSetAttr,/* tp_setattro */
0,  /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */
method_doc, /* tp_doc */
(traverseproc)method_traverse,  /* tp_traverse */
0,  /* tp_clear */
method_richcompare, /* tp_richcompare */
offsetof(PyMethodObject, im_weakreflist), /* tp_weaklistoffset */
0,  /* tp_iter */
0,  /* tp_iternext */
method_methods, /* tp_methods */
method_memberlist,  /* tp_members */
method_getset,  /* tp_getset */
0,  /* tp_base */
0,  /* tp_dict */
method_descr_get,   /* tp_descr_get */
0,  /* tp_descr_set */
0,  /* tp_dictoffset */
0,  /* tp_init */
0,  /* tp_alloc */
method_new, /* tp_new */
};

The aligned comment is an old practice required to be able to see to which 
field correspond a line.

This syntax usually produces a lot of zeros.

The C standard (C99?) allows to use a more readable syntax:

PyTypeObject PyMethod_Type = {
PyVarObject_HEAD_INIT(_Type, 0)
.tp_name = "method",
.tp_basicsize = sizeof(PyMethodObject),
.tp_dealloc = (destructor)method_dealloc,
.tp_repr = (reprfunc)method_repr,
.tp_hash = (hashfunc)method_hash,
.tp_call = method_call,
.tp_getattro = method_getattro,
...
};

It seems like it's possible to start with positional fields and then switch to 
named fields, so PyVarObject_HEAD_INIT() still works. Maybe maybe 
PyVarObject_HEAD_INIT() should also evolve? Or maybe we need a new macro using 
the ".tp_xxx=...," syntax?

INADA Naoki proposed to use this syntax in my pull request which adds a new 
tp_fastcall field which requires to add many zeros:
https://github.com/python/cpython/pull/65#pullrequestreview-16566423

Example of change:

@@ -370,6 +370,17 @@ PyTypeObject PyMethod_Type = {
 0,  /* tp_init */
 0,  /* tp_alloc */
 method_new, /* tp_new */
+0,  /* tp_free */
+0,  /* tp_is_gc */
+0,  /* tp_bases */
+0,  /* tp_mro */
+0,  /* tp_cache */
+0,  /* tp_subclasses */
+0,  /* tp_weaklist */
+0,  /* tp_del */
+0,  /* tp_version_tag */
+0,  /* tp_finalize */
+(fastternaryfunc)method_fastcall,   /* tp_fastcall */
 };

--
messages: 285399
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: Use designated initializers to define PyTypeObject types
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[Steve Dower]

Steve Dower added the comment:

Windows doesn't use the fs encoding at all until Python code requests/provides 
something in bytes. Except for the caching in fsencode/fsdecode, there's no 
problem setting it once at the start of your program (and it can only be set 
once - there's no parameter and it cannot be undone).

What I'm most interested in is whether caching the encoding in 
fsencode/fsdecode is actually an optimization - if not, remove it, and if so 
make a way to reset it. I'll get around to this sooner or later but I don't 
want to stop someone else from doing it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24875] pyvenv doesn´t install PIP inside a new venv with --system-site-package

[Vinay Sajip]

Vinay Sajip added the comment:

I see why I couldn't reproduce the problem - there's no pip installed in the 
site-packages of the Pythons I was testing with, so the problem didn't show up.

Mark, can you apply the version of EnvBuilder.create that I posted in a message 
above and see if that resolves the issue for you on both OS X and Ubuntu? 
Thomas has confirmed that it works for him.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26439] ctypes.util.find_library fails when ldconfig/glibc not available (e.g., AIX)

[Michael Felt]

Michael Felt added the comment:

OOPS: I have a ... above, meant to be a link to a message.

I also needed to 'patch' util.py - with 32-bit build with something like this:
--- src/python-2.7.13/Lib/ctypes/util.py2016-12-17 20:05:05 +
+++ python-2.7.13.0/Lib/ctypes/util.py  2017-01-13 13:29:12 +
@@ -299,6 +299,10 @@
 print cdll.LoadLibrary("libcrypto.dylib")
 print cdll.LoadLibrary("libSystem.dylib")
 print cdll.LoadLibrary("System.framework/System")
+elif sys.platform[:3] == "aix":
+from ctypes import CDLL
+RTLD_MEMBER =  0x0004
+print CDLL("libc.a(shr.o)", RTLD_MEMBER)
 else:
 print cdll.LoadLibrary("libm.so")
 print cdll.LoadLibrary("libcrypt.so")


And 64-bit with:

1: elif sys.platform[:3] == "aix":
1: from ctypes import CDLL
1: RTLD_MEMBER =  0x0004
1: print CDLL("libc_64.a(shr.o)", RTLD_MEMBER)

The respective output is:

root@x064:[/data/prj/python/python-2.7.13.0]./python Lib/ctypes/util.py
None
None
None


root@x064:[/data/prj/python/python-2.7.12.0]./python Lib/ctypes/util.py
None
None
None


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26439] ctypes.util.find_library fails when ldconfig/glibc not available (e.g., AIX)

[Michael Felt]

Michael Felt added the comment:

I would like to say:
a) I am happy this is being considered for Python3-3.7

b) I still believe it is a "bug" plain and simple - it (find_library()) cannot 
work in "normal" situations. Why "IBM" or others never addressed this is beyond 
me - but - imho - that is a poor argument for not fixing it.

FYI: As I have documented in ... this does not mean that ctypes.CDLL is 
incapable of loading a library. If you know the 'magic', aka what is not being 
provided by this "implementation dependent" module - it is possible to load 
either a .so file (actually a file by any name as long as it is a shared 
archive) - or an AIX-style .a archive with shared libraries stored internally 
as "archive members" - again, any member name is acceptable.

The bug: The "default code" behind an "else:" block depends on the presence of 
the program "/sbin/ldconfig" to create a list of shared libraries. Normally, 
this program is not installed on an AIX system (certainly not production - 
maybe a system with gcc environment and using the gcc ld rather than the AIX ld 
(the latter being normal).

Using python -m trace -C /tmp --count Lib/src/util.py The following (key) 
counts are reported.

This is all that is being called - so when /sbin/ldonfig is not installed 
(which is usual) - so this code can only return 'None'.

Environment: AIX 5.3, AIX 6.1, AIX 7.1 - all identical

root@x064:[/]ls -l /sbin/ld*
ls: 0653-341 The file /sbin/ld* does not exist.

michael@x071:[/home/michael]ls /sbin/ld*
ls: cannot access '/sbin/ld*': A file or directory in the path name does not 
exist.
(GNU coreutils ls here)

root@x062:[/]ls /sbin/ld*
ls: 0653-341 The file /sbin/ld* does not exist.

So, obviously the sys.popen() call below will always 'fail'.


The basic trace from util.py:
1: if os.name == "posix":
   # find and load_version
1: print find_library("m")
1: print find_library("c")
1: print find_library("bz2")


And how this is called:

   else:

1: def _findSoname_ldconfig(name):
3: import struct
3: if struct.calcsize('l') == 4:
   machine = os.uname()[4] + '-32'
   else:
3: machine = os.uname()[4] + '-64'
3: mach_map = {
3: 'x86_64-64': 'libc6,x86-64',
3: 'ppc64-64': 'libc6,64bit',
3: 'sparc64-64': 'libc6,64bit',
3: 's390x-64': 'libc6,64bit',
3: 'ia64-64': 'libc6,IA-64',
   }
3: abi_type = mach_map.get(machine, 'libc6')

   # XXX assuming GLIBC's ldconfig (with option -p)
3: expr = r'\s+(lib%s\.[^\s]+)\s+\(%s' % (re.escape(name), 
abi_type)
3: f = os.popen('LC_ALL=C LANG=C /sbin/ldconfig -p 2>/dev/null')
3: try:
3: data = f.read()
   finally:
3: f.close()
3: res = re.search(expr, data)
3: if not res:
3: return None
   return res.group(1)

1: def find_library(name):
3: return _findSoname_ldconfig(name) or 
_get_soname(_findLib_gcc(name))


And with python2-2.7.13 - the counts start the same - but the result is the 
same - by definition 'None' because subprocess.Popen() also
does not have any output to search...

1: if os.name == "posix":
   # find and load_version
1: print find_library("m")
1: print find_library("c")
1: print find_library("bz2")

   else:

1: def _findSoname_ldconfig(name):
3: import struct
3: if struct.calcsize('l') == 4:
3: machine = os.uname()[4] + '-32'
   else:
   machine = os.uname()[4] + '-64'
3: mach_map = {
3: 'x86_64-64': 'libc6,x86-64',
3: 'ppc64-64': 'libc6,64bit',
3: 'sparc64-64': 'libc6,64bit',
3: 's390x-64': 'libc6,64bit',
3: 'ia64-64': 'libc6,IA-64',
   }
3: abi_type = mach_map.get(machine, 'libc6')

   # XXX assuming GLIBC's ldconfig (with option -p)
3: expr = r'\s+(lib%s\.[^\s]+)\s+\(%s' % (re.escape(name), 
abi_type)

3: env = dict(os.environ)
3: env['LC_ALL'] = 'C'
3: env['LANG'] = 'C'
3: null = open(os.devnull, 'wb')
3: try:
3: with null:
3: p = subprocess.Popen(['/sbin/ldconfig', '-p'],
3:   stderr=null,
3:   stdout=subprocess.PIPE,
3:   env=env)
3:

[issue26858] android: setting SO_REUSEPORT fails

[STINNER Victor]

STINNER Victor added the comment:

IMHO it's ok to skip a few asyncio tests on Android. It's safe if it doesn't 
touch the code of asyncio itself.

It don't really care if the Android version is not checked yet. Moreover, if we 
really want to check the version, we should check the *runtime* verison, not 
the *build* version of Android. We need the discussed platform.android_ver() 
function which doesn't exist yet. So it's fine to postpone the version check in 
asyncio tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29241] sys._enablelegacywindowsfsencoding() don't apply to os.fsencode and os.fsdecode

[STINNER Victor]

STINNER Victor added the comment:

My experience with changing the Python "filesystem encoding" 
(sys.getfilesystemencoding()) at runtime: it doesn't work.

The filesystem encoding must be set as soon as possible and must never change 
later. As soon as possible: before the first call to os.fsdecode(), which is 
implemented in C as Py_DecodeLocale(). For example, the encoding must be set 
before Python imports the first module.

The filesystem encoding must be set before Python decodes *any* operating 
system data: command line arguments, any filename or path, environment 
variables, etc.

Hopefully, Windows provides most operating system data as Unicode directly: 
command line arguments and environment variables are exposed as Unicode for 
example.

os.fsdecode() and os.fsencode() have an important property:
assert os.fsencode(os.fsdecode(data)) == data

On Windows, the other property is maybe more imporant:
assert os.fsdecode(os.fsencode(data)) == data

If the property becomes false, for example if the filesystem encoding is 
changed at runtime, you get mojibake. Example:

* os.fsdecode() decodes the filename b'h\xc3\xa9llo' from UTF-8 => 'h\xe9llo'
* sys._enablelegacywindowsfsencoding()
* os.fsencode() encodes the filename to cp1252 => you get 'h\xc3\xa9llo'
 instead of 'h\xe9llo', say hello to mojibake

--

Sorry, I didn't play with sys._enablelegacywindowsfsencoding() on Windows. I 
don't know if it would "work" if sys._enablelegacywindowsfsencoding() is the 
first instruction of an application. I expect that Python almost decodes 
nothing at startup on Windows, so it may work.

sys._enablelegacywindowsfsencoding() is a private method, so it shouldn't be 
used.

Maybe we could add a "fuse" (flag only sets to 1, cannot be reset to 0) to 
raise an exception if sys._enablelegacywindowsfsencoding() is called "too 
late", after the first call to os.fsdecode() / Py_DecodeLocale()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26858] android: setting SO_REUSEPORT fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

The changes made in issue #28174 fix the problem for the Android x86 platform 
and for the armv7 platform at Android API level 24, but the problem is still 
there on the armv7 platform at Android API level 21.

==
ERROR: test_create_datagram_endpoint_sockopts 
(test.test_asyncio.test_base_events.BaseEventLoopWithS
electorTests)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_base_events.py",
 line 1593,
 in test_create_datagram_endpoint_sockopts
socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

==
ERROR: test_create_server_reuse_port 
(test.test_asyncio.test_events.EPollEventLoopTests)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", 
line 913, in te
st_create_server_reuse_port
socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

==
ERROR: test_create_server_reuse_port 
(test.test_asyncio.test_events.PollEventLoopTests)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", 
line 913, in te
st_create_server_reuse_port
socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

==
ERROR: test_create_server_reuse_port 
(test.test_asyncio.test_events.SelectEventLoopTests)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_asyncio/test_events.py", 
line 913, in te
st_create_server_reuse_port
socket.SOL_SOCKET, socket.SO_REUSEPORT))
OSError: [Errno 92] Protocol not available

--
Ran 1476 tests in 382.973s

FAILED (errors=4, skipped=2)
test test_asyncio failed
test_asyncio failed in 7 min 25 sec

1 test failed:
test_asyncio

Total duration: 7 min 27 sec
Tests result: FAILURE

--
components: +asyncio -Cross-Build
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

I started to work on FASTCALL, because I dislike the "cached tuple" hack used 
in some performance critical code, and the hack causes various kinds of tricky 
but severe issues (can lead to segfault).

Thanks to tp_fastcall, it becomes possible to drop the "cached tuple" hack from 
property_descr_get() *and* keep good performances.

First, a benchmark to show the performance gain of using "cached tuple". I 
modified property_descr_get() to use Python 3.4 code which doesn't have the 
optimization:

$ ./python -m perf compare_to py34.json ref.json 
Median +- std dev: [py34] 75.0 ns +- 1.7 ns -> [ref] 50.0 ns +- 0.9 ns: 1.50x 
faster (-33%)

It's MUCH faster, good job. But it requires complex and fragile code. Ok, let's 
see with operator.itemgetter() supporting tp_fastcall, Python modified to use 
tp_fastcall and without the "cached arg" hack:

$ ./python -m perf compare_to ref.json fastcall_wrapper.json 
Median +- std dev: [ref] 50.0 ns +- 0.9 ns -> [fastcall_wrapper] 48.2 ns +- 1.5 
ns: 1.04x faster (-4%)

It's a little bit faster, but that's not the point. The point is that it isn't 
slower and it doesn't require to modify C code to benefit of the optimization! 
Just to be clear, another benchmark result on property_descr_get() without 
"cache args", without fastcall (py34) and with fastcall ("fastcall_wrapper"):

$ ./python -m perf compare_to py34.json fastcall_wrapper.json 
Median +- std dev: [py34] 75.0 ns +- 1.7 ns -> [fastcall_wrapper] 48.2 ns +- 
1.5 ns: 1.56x faster (-36%)

Summary:

* tp_fastcall avoids to remove the "cached args" hack which will fix severe 
issue in corner cases
* tp_fastcall makes existing code faster for free. I mean, builtin types should 
be modified to support tp_fastcall, most all code *calling* these types don't 
need any change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Naoki wants to use FASTCALL for CALL_METHOD:
http://bugs.python.org/issue26110#msg283093

tp_fastcall should allow to easily implement it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

New submission from STINNER Victor:

A new FASTCALL calling convention was added to Python 3.6. It allows to avoid 
the creation of a temporary tuple to pass positional arguments and a temporary 
dictionary to pass keyword arguments. A new METH_FASTCALL calling convention 
was added for C functions. Most functions now support fastcall, except objects 
with a __call__() method which have to go through slot_tp_call() which still 
requires a tuple and dictionary.

I tried multiple implementations to support fast calls to call the __call__() 
method, but I had practical and technical issues.

First, I tried to reuse the tp_call field to PyTypeObject: it can be a regular 
call (tuple/dict for arguments) or a fast call. I added a flag to the tp_flags 
field. It was tricky to support class inheritance, decide to set or clear the 
flag. But the real blocker issue is fAthat it is obviously breaks the backward 
compatibility: existing code calling directly tp_call with the regular calling 
convention will crash immediatly, and the error is not catched during 
compilation, even if the code is recompiled.

I propose a different design: add a new tp_fastcall field to PyTypeObject and 
use a wrapper for tp_call when tp_fastcall is defined. If a type defines 
tp_fastcall but not, the tp_call wrapper "simply" calls tp_fastcall. Advantages:

* The wrapper is trivial
* Minor changes to PyType_Ready() to support inheritance (simple logic)
* Fully backward compatible
* If tp_call is called directly without keyword arguments, there is no overhead 
but a speedup!

Inheritance:

* If a type only defines tp_call, tp_fastcall is not inherited from the parent: 
tp_fastcall is set to NULL.
* If a type only defines tp_fastcall: tp_fastcall is always use (tp_call uses 
the wrapper)
* If a type defines tp_call and tp_fastcall, PyObject_Call() uses tp_call 
whereas _PyObject_FastCallDict() uses tp_fastcall.

Functions of the C API will be modified to use tp_fastcall if available.

The plan is then to patch most Python types to replace their tp_call with 
tp_fastcall. First, most important (common) types like Python and C functions, 
descriptors, and the various kinds of wrappers should be patched. Later, we 
should maybe discuss on a case by case basis to decide if it's worth it.

I will try to run benchmark before any kind.

--
messages: 285388
nosy: haypo, inada.naoki, serhiy.storchaka
priority: normal
pull_requests: 17
severity: normal
status: open
title: Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for 
all callable objects
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24875] pyvenv doesn´t install PIP inside a new venv with --system-site-package

[Mark Haase]

Mark Haase added the comment:

Hey Thomas and Vinay, thanks for looking into this. Although I discovered this 
problem on my MacOS laptop 6 months ago, today I also reproduced this issue on 
my Ubuntu desktop.

```
/home/mhaase/Downloads $ mkdir temp
/home/mhaase/Downloads $ cd temp
/home/mhaase/Downloads/temp $ pyvenv --system-site-packages pyvenv
/home/mhaase/Downloads/temp $ source pyvenv/bin/activate
(pyvenv) /home/mhaase/Downloads/temp $ pip --version
pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)
(pyvenv) /home/mhaase/Downloads/temp $ python --version
Python 3.5.2
(pyvenv) /home/mhaase/Downloads/temp $ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
(pyvenv) /home/mhaase/Downloads/temp $ uname -av
Linux prodigy 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 
x86_64 x86_64 x86_64 GNU/Linux
```

This bug is really annoying if you work with scientific Python (numpy, scipy, 
sklearn), as those are big, stable packages that I would prefer to install once 
on my system and then access from multiple venvs. The standardization of linux 
wheels in 2016 made this slightly less problematic (don't need to compile all 
of numpy just to create a venv for an ipython notebook), but this is still an 
annoying bug. Unfortunately, I think it lacks the upvotes to get attention from 
the PyPA developers.

Maybe it would help to post this on the 
https://groups.google.com/forum/#!forum/pypa-dev mailing list?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29210] Remove the support of the exclude argument in tarfile.TarFile.add()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29210] Remove the support of the exclude argument in tarfile.TarFile.add()

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4e23948f2df2 by Serhiy Storchaka in branch 'default':
Issue #29210:  Removed support of deprecated argument "exclude" in
https://hg.python.org/cpython/rev/4e23948f2df2

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29099] sqlite3 timestamp converter cannot handle timezone

[Xiang Zhang]

Xiang Zhang added the comment:

timestamptz-3.patch uses ValueError instead of ProgrammingError and treat 
suffix Z as UTC timezone.

--
Added file: http://bugs.python.org/file46279/timestamptz-3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29210] Remove the support of the exclude argument in tarfile.TarFile.add()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29006] 2.7.13 _sqlite more prone to "database table is locked"

[Armin Rigo]

Armin Rigo added the comment:

Managed to write a patch in PyPy that seems to pass all tests including the new 
one, including on Windows.  I know think that dd13098a5dc2 should be backed out 
(i.e. 030e100f048a should be kept).

Reference to the PyPy changes: 
https://bitbucket.org/pypy/pypy/commits/235e8a3889790042b3f148bcf04891b27f97a1fc

Maybe something similar should be added to CPython, to avoid the unexpected 
"database table is locked" case; but such a change should probably be done only 
in trunk, because the <= 2.6 experience seems to suggest it is rare enough in 
practice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29213] python -m venv activate.bat has weird mix of line endings

[Vinay Sajip]

Vinay Sajip added the comment:

I've added a patch, can you confirm if it resolves the issue?

--
assignee:  -> vinay.sajip
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46278/issue-29213-01.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com