[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Manuel Krebber 

Manuel Krebber added the comment:

Alright, I added some tests and tried it again with the patch.

--
Added file: http://bugs.python.org/file46427/slot-wrapper-types.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29379] Custom handlers not used when passing "context" argument to urllib2.urlopen()

2017-01-26 Thread Martin Panter 

Martin Panter added the comment:

I presume this is the same as in Issue 18543 (and a few other duplicates). Let 
me know if I got it wrong.

IMO there is no easy fix. The best solution may be to just document the 
behaviour as a limitation of the API, and design a new/improved API for the 
future (Py 3.7+) that handles these cases.

--
nosy: +martin.panter
resolution:  -> duplicate
status: open -> closed
superseder:  -> urllib.parse.urlopen shouldn't ignore installed opener when 
called with any SSL argument

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

2017-01-26 Thread Martin Panter 

Martin Panter added the comment:

Nothing has been fixed; I don’t see any evidence that this is “out of date”. 
Here is a more complete test:

import urllib.request
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
urllib.request.install_opener(opener)
request = 'https://httpbin.org/cookies/set?name=value'
# Download certifiate chain for https://httpbin.org/ into cacert.pem (I used 
Firefox)
sock = urllib.request.urlopen(request, cafile = 'cacert.pem')
sock.read()  # b'{\n  "cookies": {}\n}\n' (No cookies!)
sock.close()
sock = urllib.request.urlopen(request)  # Default SSL settings
sock.read()  # b'{\n  "cookies": {\n"name": "value"\n  }\n}\n'

I’m not comfortable with applying a patch that “doesn’t mangle [the global 
state] too much”. Anyway, this is the same as Issue 18543, which has slightly 
more recent discussion.

--
nosy: +martin.panter
resolution: out of date -> duplicate
status: pending -> closed
superseder:  -> urllib.parse.urlopen shouldn't ignore installed opener when 
called with any SSL argument

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26355] Emit major version based canonical URLs for docs

2017-01-26 Thread Matthias Bussonnier 

Matthias Bussonnier added the comment:

Here is a patch that add a canonical link to the current documentation that 
should apply cleanly to 3.4 and above. I can do similar for older versions.

--
keywords: +patch
Added file: http://bugs.python.org/file46426/cannonical-doc-for-3.4plus.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25667] Supply dual-stack (IPv4/IPv6) socket bind routine

2017-01-26 Thread Antoine Pitrou 

Changes by Antoine Pitrou :


--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Ivan Levkivskyi 

Ivan Levkivskyi added the comment:

> but an isinstance test seems pretty redundant.

Tests are never redundant :-) Just add one-two asserts that you think should be 
true about issubclass and isinstance with these types (like 
self.assertIsInstance(''.__add__, types.MethodWrapperType)). String 
representation on the contrary is less important.

For some reason your patch is still not recognized by review tool. But don't 
worry about this, if it will not work, I will try to fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26355] Emit major version based canonical URLs for docs

2017-01-26 Thread Matthias Bussonnier 

Matthias Bussonnier added the comment:

Does this have to be implemented on the doc build of EOL pythons versions (like 
2.6), or can it be a script which is ran once on these old docs ? 


One of the issues I had trying to implement that on other projects was that you 
don't know in advance what the future pages will be or if it will be gone. So 
you can go this route[1], in which case it's a 10 line fix that I'm happy to 
contribute also here. 

Typically `https://docs.python.org/2/c-api/string.html` has no cannonical on 3 
whouch you can't know while building with sphinx. 

Or I though about injecting an html comment with a html comment that you search 
and replace once you publish the "new" version and only if the target page 
exists on the new version. 


So : 

- is that ok to have non existing cannonical ? 
- is that ok if adding s on docs is (or requires) a post-sphinx script ? 

1: https://github.com/xonsh/xonsh/pull/1914/files

Thanks

--
nosy: +mbussonn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Manuel Krebber 

Manuel Krebber added the comment:

I added some docs, but I am not sure what I would want to test here. There are 
no tests for types.BuiltinMethodType either. Maybe the string representation of 
it could be tested, but an isinstance test seems pretty redundant. I hope this 
patch file works better, I created the last one with git diff.

--
Added file: http://bugs.python.org/file46425/slot-wrapper-types.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23448] urllib2 needs to remove scope from IPv6 address when creating Host header

2017-01-26 Thread Jonathan Guthrie 

Jonathan Guthrie added the comment:

Michael Sweet's draft RFC requiring that the scope should be included in the 
Host line expired in May 2014 and I can't find where it ever went anywhere.  
Does anyone have any updated information?

--
nosy: +JonathanGuthrie

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29375] httplib: wrong Host header when connecting to IPv6 link-local address

2017-01-26 Thread Jonathan Guthrie 

Jonathan Guthrie added the comment:

Yes, it is more closely related to Issue 23448.  My search for related issues 
apparently wasn't exhaustive enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13285] signal module ignores external signal changes

2017-01-26 Thread Jeroen Demeyer 

Jeroen Demeyer added the comment:

Here is a proposal for an API:

* getsignal: return the Python-level signal handler (this is an existing 
function)

* setsignal: set the Python-level signal handler (but not the OS-level signal 
handler)

* getossignal: get the OS-level signal handler as opaque object

* setossignal: set the OS-level signal handler with an opaque object

Using these primitives, you could implement anything that you want on a higher 
level.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

dict-refactoring-3.patch:

$ ../python.default -m perf compare_to default.json patched2.json -G 
--min-speed=2
Slower (7):
- scimark_lu: 422 ms +- 35 ms -> 442 ms +- 11 ms: 1.05x slower (+5%)
- logging_silent: 736 ns +- 7 ns -> 761 ns +- 21 ns: 1.03x slower (+3%)
- scimark_sor: 482 ms +- 8 ms -> 494 ms +- 7 ms: 1.03x slower (+3%)
- meteor_contest: 200 ms +- 2 ms -> 205 ms +- 2 ms: 1.02x slower (+2%)
- unpickle: 32.2 us +- 0.4 us -> 32.9 us +- 0.5 us: 1.02x slower (+2%)
- unpickle_pure_python: 829 us +- 13 us -> 848 us +- 14 us: 1.02x slower (+2%)
- scimark_sparse_mat_mult: 8.71 ms +- 0.32 ms -> 8.89 ms +- 0.13 ms: 1.02x 
slower (+2%)

Faster (8):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 2 ns: 1.07x faster (-7%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.4 ms +- 0.3 ms: 1.07x faster (-6%)
- call_method: 15.1 ms +- 0.1 ms -> 14.5 ms +- 0.2 ms: 1.04x faster (-4%)
- mako: 40.7 ms +- 0.5 ms -> 39.6 ms +- 0.5 ms: 1.03x faster (-3%)
- scimark_monte_carlo: 266 ms +- 7 ms -> 258 ms +- 6 ms: 1.03x faster (-3%)
- chameleon: 30.4 ms +- 0.4 ms -> 29.6 ms +- 0.4 ms: 1.03x faster (-3%)
- xml_etree_parse: 319 ms +- 11 ms -> 312 ms +- 15 ms: 1.02x faster (-2%)
- pickle_pure_python: 1.28 ms +- 0.03 ms -> 1.26 ms +- 0.02 ms: 1.02x faster 
(-2%)

# microbench

$ ./python -m perf timeit --compare-to=`pwd`/python.default -s 'r=range(1000)' 
-- '{k:k for k in r}'   

python.default: . 60.0 us +- 0.3 us
python: . 61.7 us +- 0.4 us

Median +- std dev: [python.default] 60.0 us +- 0.3 us -> [python] 61.7 us +- 
0.4 us: 1.03x slower (+3%)

$ ./python -m perf timeit --compare-to=`pwd`/python.default -s 'ks=[str(k) for 
k in range(1000)]; d={k:k for k in ks}' -- 'for k in ks: d[k]'  

python.default: . 37.1 us +- 0.9 us
python: . 37.7 us +- 0.9 us

Median +- std dev: [python.default] 37.1 us +- 0.9 us -> [python] 37.7 us +- 
0.9 us: 1.02x slower (+2%)

Hmm, 3% slower?
I'll rerun benchmarks with PGO+LTO build.

--
Added file: http://bugs.python.org/file46424/dict-refactoring-3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Ivan Levkivskyi 

Ivan Levkivskyi added the comment:

Manuel, thank you for a patch!

Two comments:
1. Please produce your patch using Mercurial ``hg diff`` command, so that it 
could be recognized by review tool and merged easily.
2. Your patch should also include few tests (Lib/test/test_types.py) and 
documentation (Doc/library/types.rst)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29300] Modify the _struct module to use FASTCALL and Argument Clinic

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

It looks to me that the Py_buffer converter can be used for all the buffer 
parameter and special purposed converter can be used for the fmt parameter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

>  I think the patch should not be pushed without such analysis. Perhaps 
> Raymond will found a time to do this.

Ok, I won't push until expert's LGTM.

> One possible optimization is removing freeslot completely. because:
> 
> * freeslot is used only when inserting. finding is more important.
> * insertdict() can find dummy quickly, only looking dk_indices.
>
> But before trying optimization,

I found this is not only optimization, but also refactoring.
There is function for finding insertion point already, and insertdict()
and PyDict_SetDefault() use it in some cases.
So this can be done without adding any code.

+ ../python.default -m perf compare_to default.json patched.json -G 
--min-speed=2
Slower (5):
- logging_silent: 736 ns +- 7 ns -> 758 ns +- 15 ns: 1.03x slower (+3%)
- unpickle: 32.2 us +- 0.4 us -> 33.0 us +- 0.3 us: 1.03x slower (+3%)
- unpickle_pure_python: 829 us +- 13 us -> 849 us +- 13 us: 1.02x slower (+2%)
- meteor_contest: 200 ms +- 2 ms -> 204 ms +- 2 ms: 1.02x slower (+2%)
- scimark_sor: 482 ms +- 8 ms -> 493 ms +- 9 ms: 1.02x slower (+2%)

Faster (6):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 0 ns: 1.07x faster (-7%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.7 ms +- 0.3 ms: 1.05x faster (-5%)
- sqlite_synth: 10.3 us +- 0.3 us -> 9.91 us +- 0.21 us: 1.03x faster (-3%)
- mako: 40.7 ms +- 0.5 ms -> 39.8 ms +- 0.3 ms: 1.02x faster (-2%)
- xml_etree_parse: 319 ms +- 11 ms -> 311 ms +- 14 ms: 1.02x faster (-2%)
- chameleon: 30.4 ms +- 0.4 ms -> 29.8 ms +- 0.3 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (53): ...

--
Added file: http://bugs.python.org/file46423/dict-refactoring-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29300] Modify the _struct module to use FASTCALL and Argument Clinic

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

I reformatted docstrings, and wrote a summary line.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29300] Modify the _struct module to use FASTCALL and Argument Clinic

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

Patch version 2 converts most functions and methods to Argument Clinic, except 
of unpack() and unpack_into() which require "*args" support in Argument Clinic: 
see the issue #20291.

--
Added file: http://bugs.python.org/file46422/struct_fastcall-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29379] Custom handlers not used when passing "context" argument to urllib2.urlopen()

2017-01-26 Thread Piotr Dobrogost 

New submission from Piotr Dobrogost:

When urllib2.urlopen() is passed "context" argument the new opener is being 
built (https://hg.python.org/cpython/file/a06454b1afa1/Lib/urllib2.py#l147) and 
used instead custom opener which might had been already installed with 
urllib2.install_opener(). This might lead to problems with proxies if custom 
ProxyHandler was used in custom opener – see 
http://stackoverflow.com/q/36089694/95735 as an example.

--
components: Library (Lib)
messages: 286325
nosy: piotr.dobrogost
priority: normal
severity: normal
status: open
title: Custom handlers not used when passing "context" argument to 
urllib2.urlopen()
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29378] Invalid example in documentation for PyErr_Fetch

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

The example looks correct to me. Could you please add more information? What 
example looks correct to you?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

I agreed that the duplication was made for reasons, but these reasons can be 
not valid or less weighty now, after a number of changes made last years like 
shared keys or compact dicts. An overhead of additional levels of indirection 
and complex code of dk_get_index() can be much larger than the benefit of code 
duplicating.

I think neither macrobenchmarks nor microbenchmarks would not show the true 
effect of this change. The code should be analyzed with more precise tools, 
similarly to how the set implementation was tuned. I think the patch should not 
be pushed without such analysis. Perhaps Raymond will found a time to do this.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

BTW, perturb shift uses (i << 2) + i, instead of i*5.
I feel it doesn't make sense in 21th century.  I'll change it too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27727] Update Tools/freeze to use .vcxproj files

2017-01-26 Thread Kaeptm Blaubaer 

Changes by Kaeptm Blaubaer :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27727] Update Tools/freeze to use .vcxproj files

2017-01-26 Thread Kaeptm Blaubaer 

Changes by Kaeptm Blaubaer :


--
type: behavior -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29378] Invalid example in documentation for PyErr_Fetch

2017-01-26 Thread Kaeptm Blaubaer 

New submission from Kaeptm Blaubaer:

In the example are the pointers to pointers to PyObject too many,
because then PyErr_Fetch and PyErr_Restore would get too many pointers to 
pointers.

--
assignee: docs@python
components: Documentation
messages: 286321
nosy: Kaeptm Blaubaer, docs@python
priority: normal
severity: normal
status: open
title: Invalid example in documentation for PyErr_Fetch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

Digging history, duplicated code is introduced here. (1997-01-17)

https://github.com/python/cpython/commit/99304174680d4c724476dad300ae7fc638842bf0#diff-2131209d0deb0e50c93a88ec6c7b0d52

/* Optimizations based on observations by Jyrki Alakuijala
   (paraphrased):
   - This routine is very heavily used, so should be AFAP
   (As Fast As Possible).
   - Most of the time, the first try is a hit or a definite
   miss; so postpone the calculation of incr until we know the
   first try was a miss.
   - Write the loop twice, so we can move the test for
   freeslot==NULL out of the loop.
   - Write the loop using pointer increments and comparisons
   rather than using an integer loop index.
   Note that it behooves the compiler to calculate the values
   of incr*sizeof(*ep) outside the loops and use this in the
   increment of ep.  I've reduced the number of register
   variables to the two most obvious candidates.

At the time, there was a single lookdict function, and three tries.

* The comment said the first try was for skipping `inc` calculation.
  While `inc` had been removed already, I think this implies skipping
  freeslot==NULL test.

* After first try, there are two loop for skipping freeslot==NULL test
  until first dummy found.  This optimization looks gone.


As I said above, lookdict_unicode_nodummy and lookdict_split only search
from table without dummies.
And lookdict_unicode() and lookdict() are not so important lookmapping() was in 
1997,
duplicated code only for skipping one freeslot==NULL doesn't make sense.

One possible optimization is removing freeslot completely. because:

* freeslot is used only when inserting. finding is more important.
* insertdict() can find dummy quickly, only looking dk_indices.

But before trying optimization, I suggest to remove duplicated code first.
Macro bench doesn't show significant difference at least.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26355] Emit major version based canonical URLs for docs

2017-01-26 Thread Nick Coghlan 

Nick Coghlan added the comment:

Belatedly following up on this, yeah, the RTFD page indicates that the header 
link should look like:

http://docs.python.org/3/">

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29368] Optimize unpickling list-like objects: use extend() instead of append()

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

Updated patch addresses Antoine's comment. It adds the comment explaining a 
fallback.

--
Added file: http://bugs.python.org/file46421/pickle-appends-extend-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

Serhiy Storchaka:
> As for update_one_slot() see also issue5322 and issue25731.

Oh, thanks for the pointers! Now I understand much better these bugs.

I'm quite sure that they are still flaws in this code when a type is modified 
after PyType_Ready(), like sysmodule.c::

/* prevent user from creating new instances */
FlagsType.tp_init = NULL;
FlagsType.tp_new = NULL;


But each time I had to dig into typeobject.c, my head is going to explode :-D I 
may try to understand one more time, but not today ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

As for update_one_slot() see also issue5322 and issue25731.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29358] Add tp_fastnew and tp_fastinit to PyTypeObject, 15-20% faster object instanciation

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

While they are obvious speedup on microbenchmarks, it doesn't speedup "macro" 
benchmarks from performance as much as I expected.

The changes required in typeobject.c to support "tp_new or tp_fastnew" and 
"tp_init or tp_fastinit" are large and very complex. I'm not sure that my code 
is correct. update_one_slot(), add_operators() and PyType_Ready() functions are 
complex beast, sadly almost not documented, except such comment...

/* The __new__ wrapper is not a wrapper descriptor,
   so must be special-cased differently.
   If we don't do this, creating an instance will
   always use slot_tp_new which will look up
   __new__ in the MRO which will call tp_new_wrapper
   which will look through the base classes looking
   for a static base and call its tp_new (usually
   PyType_GenericNew), after performing various
   sanity checks and constructing a new argument
   list.  Cut all that nonsense short -- this speeds
   up instance creation tremendously. */
specific = (void *)type->tp_new;
/* XXX I'm not 100% sure that there isn't a hole
   in this reasoning that requires additional
   sanity checks.  I'll buy the first person to
   point out a bug in this reasoning a beer. */

How am I supposed to be confident in front of such coment :-D

I expect that calling a functions (tp_call) is a more common operation than 
instanciating a new object (tp_new + tp_init). So I don't think that the 
overall change is worth it.

For all these reaons, I close the issue as REJECTED.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

"While I feel your work is great, performance benefit seems very small,
compared complexity of this patch."

I have to agree. I spent a lot of times on benhchmarking these tp_fast* 
changes. While one or two benchmarks are faster, it's not really the case for 
the others.

I also agree with the complexity. In Python 3.6, most FASTCALL changes were 
internals. For example, using PyObject_CallFunctionObjArgs() now uses FASTCALL 
internally, without having to modify callers of the API. I tried to only use 
_PyObject_FastCallDict/Keywords() in a few places where the speedup was 
significant.

The main visible change of Python 3.6 FASTCALL is the new METH_CALL calling 
convention for C function. Your change modifying print() to use METH_CALL has a 
significant impact on the telco benchmark, without no drawback. I tested 
further changes to use METH_FASTCALL in struct and decimal modules, and they 
optimize telco even more.

To continue the optimization work, I guess that using METH_CALL in more cases, 
using Argument Clinic whenever possible, would have a more concrete and 
measurable impact on performances, than this big tp_fastcall patch.

But I'm not ready to abandon the whole approach yet, so I change the status to 
Pending. I may come back in one or two months, to check if I didn't miss 
anything obvious to unlock even more optimizations ;-)

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

While I feel your work is great, performance benefit seems very small,
compared complexity of this patch.

I think AST level optimize is more important. co_const and co_stacksize
can be more small if constant folding is done in AST.

And I think global cache (#10401 or #28158) is important too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28689] OpenSSL 1.1.0c test failures

2017-01-26 Thread Dima Tisnek 

Dima Tisnek added the comment:

1.1.0d is due today. Who wants to test it? :)

--
nosy: +Dima.Tisnek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Manuel Krebber 

Manuel Krebber added the comment:

I would suggest the names SlotWrapperType and MethodWrapperType because I think 
they match their string representations the closest.

For checking whether something is a method/function one could also use 
inspect.isroutine (or inspect.ismethoddescriptor), but that is inconsistent 
with regards to builtin and python methods:

>>> import inspect
>>> inspect.isroutine(object.__init__)
True
>>> inspect.isroutine(object().__init__)
False
>>> class A:
...   def f(self):
... pass
...
>>> inspect.isroutine(A.f)
True
>>> inspect.isroutine(A().f)
True

Maybe a function to detect the second case is needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Manuel Krebber 

Changes by Manuel Krebber :


--
keywords: +patch
Added file: 
http://bugs.python.org/file46420/0001-Added-SlotWrapperType-and-MethodWrapperType-to-the-t.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29270] super call in ctypes sub-class fails in 3.6

2017-01-26 Thread Nick Coghlan 

Nick Coghlan added the comment:

The scrapy case looks to just be the new metaclass constraint that's already 
covered in the "Porting to Python 3.6" guide: 
https://github.com/scrapy/scrapy/pull/2509/files

The ctypes case is more complicated, as its actually *reusing* the same class 
namespace to define two different classes, which is fundamentally incompatible 
with the way zero-argument super works.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

2017-01-26 Thread Eryk Sun 

Eryk Sun added the comment:

> FindFirstFile naturally fails with ERROR_PATH_NOT_FOUND

Getting this error actually depends on the file system. I don't see it with 
NTFS, which returns STATUS_NOT_A_DIRECTORY, which gets translated to 
ERROR_DIRECTORY. On the other hand, VboxSF (VirtualBox shared folder) returns 
STATUS_OBJECT_PATH_NOT_FOUND, which gets translated to ERROR_PATH_NOT_FOUND.

Also, there are many possible errors when trying to list a path on a device 
that isn't managed by a file system (e.g. \\.\PhysicalDrive0). The device can 
return a status value that FindFirstFile doesn't special case, such as 
STATUS_UNSUCCESSFUL, STATUS_NOT_FOUND, or STATUS_OBJECT_NAME_INVALID. The 
NtOpenFile call may even succeed, but then the NtQueryDirectoryFile will 
probably fail with STATUS_INVALID_DEVICE_REQUEST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28518] execute("begin immediate") throwing OperationalError

2017-01-26 Thread Ma Lin 

Ma Lin added the comment:

You can set isolation_level = None in sqlite3.connect() parameters, then 
sqlite3 module will not begin a transaction implicitly.

Another way is disabling auto-begin-transaction temporarily:
sql.isolation_level = None
sql.execute('VACUUM')
sql.isolation_level = ''  # <- note that this is the default value of 
isolation_level

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

2017-01-26 Thread INADA Naoki 

INADA Naoki added the comment:

Another run at my machine. (tp_fastcall-6.patch)

+ ../python.default -m perf compare_to default.json patched.json -G 
--min-speed=2
Slower (8):
- logging_silent: 736 ns +- 7 ns -> 783 ns +- 12 ns: 1.06x slower (+6%)
- unpickle_pure_python: 829 us +- 13 us -> 875 us +- 12 us: 1.06x slower (+6%)
- scimark_sor: 482 ms +- 8 ms -> 507 ms +- 14 ms: 1.05x slower (+5%)
- xml_etree_iterparse: 223 ms +- 6 ms -> 228 ms +- 7 ms: 1.02x slower (+2%)
- scimark_fft: 728 ms +- 16 ms -> 745 ms +- 19 ms: 1.02x slower (+2%)
- scimark_sparse_mat_mult: 8.71 ms +- 0.32 ms -> 8.90 ms +- 0.28 ms: 1.02x 
slower (+2%)
- json_dumps: 29.3 ms +- 0.3 ms -> 29.9 ms +- 0.3 ms: 1.02x slower (+2%)
- pathlib: 51.2 ms +- 0.6 ms -> 52.3 ms +- 0.8 ms: 1.02x slower (+2%)

Faster (8):
- unpack_sequence: 132 ns +- 2 ns -> 123 ns +- 2 ns: 1.08x faster (-7%)
- xml_etree_parse: 319 ms +- 11 ms -> 303 ms +- 11 ms: 1.05x faster (-5%)
- call_simple: 14.3 ms +- 0.5 ms -> 13.7 ms +- 0.5 ms: 1.04x faster (-4%)
- call_method_unknown: 16.6 ms +- 1.1 ms -> 16.0 ms +- 0.2 ms: 1.04x faster 
(-4%)
- call_method: 15.1 ms +- 0.1 ms -> 14.5 ms +- 0.5 ms: 1.04x faster (-4%)
- xml_etree_process: 241 ms +- 4 ms -> 235 ms +- 4 ms: 1.03x faster (-3%)
- call_method_slots: 14.7 ms +- 0.2 ms -> 14.3 ms +- 0.2 ms: 1.02x faster (-2%)
- genshi_xml: 197 ms +- 3 ms -> 193 ms +- 2 ms: 1.02x faster (-2%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Ivan Levkivskyi 

Ivan Levkivskyi added the comment:

Manuel, it would be helpful if you submit a patch.

Guido, this is necessary for ``get_type_hints`` to work nicer with built-in 
methods, see https://github.com/python/typing/pull/368

--
nosy: +gvanrossum, levkivskyi
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17720] pickle.py's load_appends should call append() on objects other than lists

2017-01-26 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

> From PEP 7:

Sorry, I meant PEP 307 of course.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29377] Add the 'wrapper_descriptor' type to the types module

2017-01-26 Thread Manuel Krebber 

New submission from Manuel Krebber:

There currently is no type in the types module for the slot 
wrappers/wrapper_descriptor types.

I would like to have something like

WrapperDescriptor = type(object.__init__)

added to it (or maybe even add it to MethodType). This would be helpful to 
check more easily if some object is a method.

I can create a pull request for this if desired.

--
components: Library (Lib)
messages: 286304
nosy: Wheerd
priority: normal
severity: normal
status: open
title: Add the 'wrapper_descriptor' type to the types module
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

tp_fastcall-6.patch: "call_method: 11.0 ms +- 0.3 ms -> 11.5 ms +- 0.5 ms: 
1.05x slower (+5%)"

I looked at the C code: the patch doesn't change any C function used by this 
benchmark. IHMO it's only a better of code locality. Maybe the PGO compilation 
produced less efficient code with the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23462] All os.exec*e variants crash on Windows

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

> Just hit that bug. Is this really not yet fixed in any newer version?

The bug was fixed in Python 3.6.0. It was fixed by Steve Dower in its 
implementation of the PEP 529, in the change e20c7d8a8187:

-execv(path_char, argvlist);
+#ifdef HAVE_WEXECV
+_wexecv(path->wide, argvlist);
+#else
+execv(path->narrow, argvlist);
+#endif

I'm not sure that it's easy to backport the fix in Python 3.5.

Python 3.6 no walways use Unicode internally to call the Windows API, whereas 
Python 3.5 used bytes or Unicode depending on the arguments.

So I suggest to upgrade to Python 3.6, or use bytes in the meanwhile.

Would it be ok to close the bug?

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23462] All os.exec*e variants crash on Windows

2017-01-26 Thread Andreas Bergmeier 

Andreas Bergmeier added the comment:

Just hit that bug. Is this really not yet fixed in any newer version?

--
nosy: +Andreas Bergmeier

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28518] execute("begin immediate") throwing OperationalError

2017-01-26 Thread Cédric Bellegarde 

Cédric Bellegarde added the comment:

Adding a commit doesn't fix the issue here:
sql.commit()
sql.execute('VACUUM')

And on run:
cannot VACUUM from within a transaction

--
nosy: +Cédric Bellegarde

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

performance results on tp_fastcall-6.patch (diff >= 2%):
---
haypo@speed-python$ python3 -m perf compare_to 
/home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json 
tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=2
Slower (6):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)
- call_method: 11.0 ms +- 0.3 ms -> 11.5 ms +- 0.5 ms: 1.05x slower (+5%)
- scimark_sparse_mat_mult: 8.57 ms +- 0.14 ms -> 8.92 ms +- 0.15 ms: 1.04x 
slower (+4%)
- fannkuch: 938 ms +- 7 ms -> 974 ms +- 11 ms: 1.04x slower (+4%)
- scimark_fft: 667 ms +- 9 ms -> 690 ms +- 10 ms: 1.03x slower (+3%)

Faster (17):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)
- xml_etree_generate: 208 ms +- 4 ms -> 199 ms +- 3 ms: 1.05x faster (-4%)
- logging_silent: 586 ns +- 8 ns -> 562 ns +- 7 ns: 1.04x faster (-4%)
- richards: 147 ms +- 2 ms -> 141 ms +- 3 ms: 1.04x faster (-4%)
- pickle_pure_python: 1.03 ms +- 0.02 ms -> 994 us +- 15 us: 1.04x faster (-4%)
- regex_compile: 379 ms +- 2 ms -> 367 ms +- 4 ms: 1.03x faster (-3%)
- raytrace: 1.08 sec +- 0.01 sec -> 1.04 sec +- 0.02 sec: 1.03x faster (-3%)
- unpickle: 31.0 us +- 0.4 us -> 30.0 us +- 0.4 us: 1.03x faster (-3%)
- pathlib: 42.3 ms +- 0.5 ms -> 41.0 ms +- 0.5 ms: 1.03x faster (-3%)
- scimark_sor: 391 ms +- 7 ms -> 379 ms +- 12 ms: 1.03x faster (-3%)
- genshi_xml: 167 ms +- 2 ms -> 162 ms +- 2 ms: 1.03x faster (-3%)
- django_template: 356 ms +- 5 ms -> 347 ms +- 4 ms: 1.03x faster (-3%)
- regex_dna: 283 ms +- 1 ms -> 276 ms +- 4 ms: 1.03x faster (-3%)
- hexiom: 18.0 ms +- 0.1 ms -> 17.6 ms +- 0.2 ms: 1.02x faster (-2%)
- call_simple: 10.7 ms +- 0.4 ms -> 10.5 ms +- 0.2 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (41): (...)
---

Or less verbose output, only show difference >= 5%:
---
haypo@speed-python$ python3 -m perf compare_to 
/home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json 
tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=5
Slower (2):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)

Faster (3):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)

Benchmark hidden because not significant (59): (...)
---

To be honest, I'm disappointed, I expected more faster benchmarks. Maybe the 
cost of the temporary tuple/dict to pass arguments to function is not as high 
as I expected.

telco is also my favorite benchmark, I'm sad that it's 1.09x slower. I should 
investigate that.

scimark_monte_carlo and scimark_lu benchmarks are still unstable for an unknown 
reasons, so I don't care too much of these two benchmarks.

* 
https://speed.python.org/timeline/#/?exe=5=scimark_monte_carlo=1=50=off=on=on
* 
https://speed.python.org/timeline/#/?exe=5=scimark_lu=1=50=off=on=on

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29328] struct module should support variable-length strings

2017-01-26 Thread Raymond Hettinger 

Raymond Hettinger added the comment:

s/Where/Whether/

--
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29368] Optimize unpickling list-like objects: use extend() instead of append()

2017-01-26 Thread Raymond Hettinger 

Raymond Hettinger added the comment:

This code looks correct and reasonable.  The tests all pass for me.  I think 
you can go forward and apply the patch.

--
assignee:  -> serhiy.storchaka
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22651] Open file in a+ mode reads from end of file in Python 3.4

2017-01-26 Thread STINNER Victor 

STINNER Victor added the comment:

On Python 2, it's easy to workaround the issue, just always go the end 
explicitly:

   f = open(filename, "a+")
   f.seek(0, os.SEEK_END)

Or use the io module which doesn't use the stdio of the C library and has a 
portable and reliable behaviour:

   f = io.open(filename, "a+")


Georg Brandl: "This is a somewhat unfortunate difference between the major C 
libs and Python's new IO, but probably too late to change."

Since it's easy to workaround the issue and only one user complained lat 2 
years (3 years?), sorry, I close now the issue.

I add this bug to my long list of bugs in the C stdio which won't be fixed in 
Python 2...
http://haypo-notes.readthedocs.io/python.html#bugs-in-the-c-stdio-used-by-the-python-i-o

--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com