[issue28637] Python startup performance regression

[Gregory P. Smith]

Gregory P. Smith added the comment:

re: Ethan's question - I think the enum use should be restored in re.

I realize issue28082 (yay palindrome number) is not an urgent change but we 
created IntEnum for the purpose of more identifiable integer constants.

So a microbenchmark of "import re" slows down.  so what?  I don't find this to 
be a big deal.  Other standard library modules also use enum and I expect more 
to do so in the future.  In realistic size programs other things use enum as 
well so there isn't a hit.

PS thanks for the site.py improvements!

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20629] Python ctypes BigEndianStructure bitfield assignment misbehavior in Linux

[Xiang Zhang]

Xiang Zhang added the comment:

The bug is fixed in #23319. More recent Py2.7 and Py3.4+ should get rid of it.

--
nosy: +xiang.zhang
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28645] Drop __aiter__ compatibility layer from 3.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> I'll inc importlib's magic before committing.

Please left good gap for possible bugfixes between current number and 3.7 
number. Set it to 3390.

I'm not experienced with async protocols, but at first glance the patch looks 
technically correct. Please update the documentation. I have the only question 
about tests. Some tests test the behavior with "async __aiter__". With the 
patch them test the behavior with just "__aiter__". Shouldn't these tests be 
duplicated in 3.5-3.6 for __aiter__ without async?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader losing rows with big files and tab delimiter

[SilentGhost]

Changes by SilentGhost :


--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28646] Using a read-only buffer.

[Michael Rolle]

New submission from Michael Rolle:

Suggest that the _CData.from_buffer (source) method allow a read-only
source as an argument, in which case any assignments to
the object would raise some type of exception.
If the source is shared with some writable object,
then changes to said object would be reflected in the
_CData object.  If this behavior is not what you want,
then you could just make a new method called, say,
from_buffer_shared (source).

--
components: ctypes
messages: 280375
nosy: mrolle
priority: normal
severity: normal
status: open
title: Using a read-only buffer.
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20629] Python ctypes BigEndianStructure bitfield assignment misbehavior in Linux

[Michael Rolle]

Michael Rolle added the comment:

As a separate issue, I'd like to find an appropriate package,
other than ctypes, for interpreting data bytes in a consistently
defined manner, independent of the platform I'm running on.
The struct package is perfect where there are no bitfields
involved, i.e., where each item occupies whole bytes.  But
it doesn't support packing/unpacking bitfields.

Actually, ctypes could fit the bill if you specified that bitfields
be allocated from MSB to LSB for BigEndianStructure, and from LSB
to MSB for LittleEndianStructure.  This way, for instance, it wouldn't matter 
if a sequence of 4-bit fields were based on c_ubyte
or c_ushort, etc.  Each pair of fields would be allocated to the
next consecutive byte.  And if the platform native compiler for some strange 
reason doesn't follow either of these rules, then Structure
would follow the platform compiler.

differs from both Big and Little

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recen changes in typing.py

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Hi Ivan, I noticed you used two space indentation in your docs patch.

Can you use 3 spaces instead?
More guidelines here:
https://docs.python.org/devguide/documenting.html?#use-of-whitespace

Thanks :)

--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Terry J. Reedy]

Terry J. Reedy added the comment:

Thanks Irv: Windows specific it appears to be.

Benjamin, Martin, Steve: the 2.7 Windows .msi installer should be fixed to 
include idlelib/help.html before its next release.  Does it use an explicit 
list of file glob patterns to include?

--
nosy: +loewis
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20629] Python ctypes BigEndianStructure bitfield assignment misbehavior in Linux

[Michael Rolle]

Michael Rolle added the comment:

Similar problem with 2.7.8 with cygwin.

My example is:

Python 2.7.8 (default, Jul 25 2014, 14:04:36)
[GCC 4.8.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> class C (BigEndianStructure): _fields_ = (('rva', c_uint, 31), ('fl', 
>>> c_uint, 1))
...
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x00\x00'
0L
0L
>>> x.rva = 256
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x02\x00'
256L
0L
>>> x.rva = 256
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x02\x00'
256L
0L
>>> x.fl = 1
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x02\x00\x01'
65536L
1L
>>> x.fl = 1
>>> buffer(x)[:]; x.rva; x.fl
'\x01\x00\x02\x01'
8388864L
1L
>>> x.fl = 1
>>> buffer(x)[:]; x.rva; x.fl
'\x01\x02\x00\x01'
8454144L
1L
>>> x.fl = 1
>>> buffer(x)[:]; x.rva; x.fl
'\x01\x00\x02\x01'
8388864L
1L
>>> x.rva = 256
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x02\x01'
256L
1L
>>> x.rva = 256
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x02\x00'
256L
0L
>>> x.rva = 256
>>> buffer(x)[:]; x.rva; x.fl
'\x00\x00\x02\x00'
256L
0L

I'm disappointed that this bug hasn't been fixed after two years!

I understand that ctypes might not be portable across different
platforms.  However, the above behavior is clearly wrong.

BTW, I also have Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 
bit (AMD64)] on win32, and this version works fine.

--
nosy: +mrolle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Guido van Rossum]

Guido van Rossum added the comment:

OK. I believe we also found that the extra slowdown is only in a virtualenv
so that may explain that I saw something different.

--
title: Python startup performance regression due to enum in re -> Python 
startup performance regression

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28645] Drop __aiter__ compatibility layer from 3.7

[Yury Selivanov]

New submission from Yury Selivanov:

As discussed in issue #27243, we want to drop __aiter__ compatibility layer in 
Python 3.7.  Please take a look at the attached patch (I'll inc importlib's 
magic before committing).

--
keywords: +patch
Added file: http://bugs.python.org/file45403/aiter.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28645] Drop __aiter__ compatibility layer from 3.7

[Yury Selivanov]

Changes by Yury Selivanov :


--
assignee: yselivanov
components: Interpreter Core
nosy: haypo, serhiy.storchaka, yselivanov
priority: normal
severity: normal
stage: patch review
status: open
title: Drop __aiter__ compatibility layer from 3.7
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28639] inspect.isawaitable(native_coro) returns int

[Yury Selivanov]

Yury Selivanov added the comment:

Merged! Thank you, Justin.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28639] inspect.isawaitable(native_coro) returns int

[Roundup Robot]

Roundup Robot added the comment:

New changeset bb6ad816a43c by Yury Selivanov in branch '3.5':
Issue #28639: Fix inspect.isawaitable to always return bool
https://hg.python.org/cpython/rev/bb6ad816a43c

New changeset 6540adb8722a by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #28639)
https://hg.python.org/cpython/rev/6540adb8722a

New changeset 8e3d359cc73b by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28639)
https://hg.python.org/cpython/rev/8e3d359cc73b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recen changes in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Sorry, here is the patch with corrected formatting and base classes for 
Coroutine.

--
Added file: http://bugs.python.org/file45402/recent-typing-docs-v2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28003] PEP 525 asynchronous generators implementation

[Roundup Robot]

Roundup Robot added the comment:

New changeset 9f32ef6b210b by Yury Selivanov in branch '3.6':
Issue #28003: Make WrappedVal, ASend and AThrow GC types
https://hg.python.org/cpython/rev/9f32ef6b210b

New changeset 6f51b495656c by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28003)
https://hg.python.org/cpython/rev/6f51b495656c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recen changes in typing.py

[Ivan Levkivskyi]

New submission from Ivan Levkivskyi:

Here is the patch with recent additions/changes in typing.py

Summary: Tuple and Callable are now classes, generic type aliases, added 
Coroutine, extended docs for get_type_hints.

--
assignee: docs@python
components: Documentation
files: recent-typing-docs.diff
keywords: patch
messages: 280364
nosy: docs@python, gvanrossum, levkivskyi
priority: normal
severity: normal
status: open
title: Document recen changes in typing.py
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45401/recent-typing-docs.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[Yury Selivanov]

Changes by Yury Selivanov :


--
priority: release blocker -> normal
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[Roundup Robot]

Roundup Robot added the comment:

New changeset 345904bd0456 by Yury Selivanov in branch '3.6':
Issue #26081: Fix refleak in _asyncio.Future.__iter__().throw.
https://hg.python.org/cpython/rev/345904bd0456

New changeset b95aa07d by Yury Selivanov in branch 'default':
Merge 3.6 (issue #26081)
https://hg.python.org/cpython/rev/b95aa07d

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression due to enum in re

[Ethan Furman]

Ethan Furman added the comment:

Does this mean we can put enum back in re?

--
title: Python startup performance regression -> Python startup performance 
regression due to enum in re

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28555] provid also sha-1 and sha-256 also on download links

[Benjamin Peterson]

Benjamin Peterson added the comment:

If python.org can be MITMed, it doesn't matter how secure the hash is.

On Tue, Nov 8, 2016, at 11:17, Big Stone wrote:
>
> Big Stone added the comment:
>
> I fear GPG is not easy stuff for Windows users.
>
> I fear a bunch of people on this network can circomvent DNS and make
> python.org points to the wrong place.
>
> sha-1 instead of md5 would have been an improvement.
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28607] C implementation of parts of copy.deepcopy

[Rasmus Villemoes]

Rasmus Villemoes added the comment:

New version, addressing (hopefully) all review comments.

--
Added file: http://bugs.python.org/file45400/deepcopy.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[Yury Selivanov]

Yury Selivanov added the comment:

This patch introduced multiple refleaks in test_asyncgen.

--
priority: normal -> release blocker
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26801] Fix shutil.get_terminal_size() to catch AttributeError

[Yury Selivanov]

Yury Selivanov added the comment:

Ah, never mind, the commit message has a wrong issue number:

Issue #26801: Added C implementation of asyncio.Future.

Closing this one, will re-open #26081.

--
priority: release blocker -> normal
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26801] Fix shutil.get_terminal_size() to catch AttributeError

[Yury Selivanov]

Changes by Yury Selivanov :


--
nosy: +larry, ned.deily
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26801] Fix shutil.get_terminal_size() to catch AttributeError

[Yury Selivanov]

Yury Selivanov added the comment:

This patch introduced multiple refleaks in test_asyncgen.

--
nosy: +yselivanov
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[Eric V. Smith]

Eric V. Smith added the comment:

This file is derived from my namedlist project on PyPI.

I've stripped out the namedlist stuff, and just left namedtuple. I've also 
removed the Python 2 support, and I've removed support for default parameters. 
After that surgery, I have not tested it very well.

Those are my excuses for why the code is more complex that it would be if I 
were writing it from scratch.

Anyway, instead of using eval() of a string to create the new() function, I use 
ast manipulations to generate a function that does all of the correct type 
checking. It calls eval() too, of course, but with a code object.

I originally wrote this as an exercise in learning how to generate AST's. I 
can't say it's the best way to solve this problem, and I haven't benchmarked it 
ever. So just consider it as a proof of concept, or ignore it if you're not 
interested.

--
nosy: +eric.smith
Added file: http://bugs.python.org/file45399/namedtuple1.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26182] Deprecation warnings for the future async and await keywords in Python 3.6

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7a996e826f83 by Yury Selivanov in branch '3.6':
Issue #26182: Fix ia refleak in code that raises DeprecationWarning.
https://hg.python.org/cpython/rev/7a996e826f83

New changeset 7b0e79e7f567 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #26182)
https://hg.python.org/cpython/rev/7b0e79e7f567

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28643] Broken makefile depends for profile-opt target

[Neil Schemenauer]

Neil Schemenauer added the comment:

Okay, my initial idea was wrong (I blame years of not having to look at 
Makefiles).  I think the attached patch works.  It uses a "stamp" file to 
record the fact that the profiled build is complete.

The fix is sub-optimal because changing some source code and re-running "make" 
will not rebuild as needed.  That would require proper dependencies in the 
Makefile, rather than treating "make" as an imperative scripting language (e.g. 
recursively calling make to sequence things).  Given that the profile optimised 
build is unlikely to be used during development, I guess that's not so bad.

--
Added file: http://bugs.python.org/file45398/profile-opt-stamp.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

Guido: "So what does that benchmark measure? For me, python 3.6 startup is 44 
ms and python 2.7 startup is 78 ms (real time; user time is proportionally 
less)."

It measures something like "time python -c pass".

The performance module creates a virtual environment to run benchmarks to get 
reproductible timings. If you benchmark the Python startup time using your 
system Python, the timing depends on the number of installed .pth files.

The code of python_startup benchmark:
https://github.com/python/performance/blob/master/performance/benchmarks/bm_python_startup.py

Note: I didn't write the benchmark, it comes from the old benchmark suite.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader losing rows with big files and tab delimiter

[Marc Garcia]

Marc Garcia added the comment:

Sorry, my fault. It looks like having quotes in the file was the problem. As 
mentioned, adding the quoting parameter fixes the problem.

I'd assume that if quotes are not paired, csv should raise an exception. And I 
don't think that all the different chunks of the file I tested, had always an 
even number of quotes.

Also, I don't understand why using the default delimiter worked well, and with 
tab delimiter the problem happened.

I'll take a look in more detail, but I'm closing this issue.

Thank you all a lot for the help!

--
resolution:  -> not a bug
status: open -> closed
title: csv reader loosing rows with big files and tab delimiter -> csv reader 
losing rows with big files and tab delimiter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[SilentGhost]

SilentGhost added the comment:

so using quoting=csv.QUOTE_NONE should solve the immediate problem of "losing" 
lines then, I'm not sure csv module ever supported dealing with corrupted files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file45397/patch.json.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

>> Do you mean comparison between current Python with PGO and patched
>> Python without PGO?
>
> Yes.

Ok, here you have. As expected, PGO compilation is faster than default 
compilation with my patch. PGO implements more optimization than just 
__attribute__((hot)), it also optimizes branches for example.

haypo@smithers$ python3 -m perf compare_to pgo.json.gz patch.json.gz -G 
--min-speed=5
Slower (56):
- regex_effbot: 4.30 ms +- 0.26 ms -> 5.77 ms +- 0.33 ms: 1.34x slower
- telco: 16.0 ms +- 1.1 ms -> 20.6 ms +- 0.4 ms: 1.29x slower
- xml_etree_process: 174 ms +- 15 ms -> 218 ms +- 29 ms: 1.25x slower
- xml_etree_generate: 205 ms +- 16 ms -> 254 ms +- 4 ms: 1.24x slower
- unpickle_list: 6.04 us +- 1.12 us -> 7.47 us +- 0.18 us: 1.24x slower
- call_simple: 10.6 ms +- 1.4 ms -> 13.1 ms +- 0.3 ms: 1.24x slower
- mako: 33.5 ms +- 0.3 ms -> 41.3 ms +- 0.9 ms: 1.23x slower
- pathlib: 37.0 ms +- 2.3 ms -> 44.7 ms +- 2.0 ms: 1.21x slower
- sqlite_synth: 7.56 us +- 0.20 us -> 8.97 us +- 0.18 us: 1.19x slower
- unpickle: 24.2 us +- 3.9 us -> 28.7 us +- 0.3 us: 1.18x slower
- chameleon: 23.4 ms +- 2.6 ms -> 27.4 ms +- 1.5 ms: 1.17x slower
- spectral_norm: 214 ms +- 7 ms -> 249 ms +- 9 ms: 1.17x slower
- nqueens: 210 ms +- 2 ms -> 244 ms +- 36 ms: 1.16x slower
- unpickle_pure_python: 717 us +- 10 us -> 831 us +- 66 us: 1.16x slower
- pickle: 18.7 us +- 4.3 us -> 21.6 us +- 3.3 us: 1.15x slower
- sympy_expand: 829 ms +- 39 ms -> 957 ms +- 28 ms: 1.15x slower
- genshi_text: 73.1 ms +- 3.2 ms -> 84.3 ms +- 1.1 ms: 1.15x slower
- pickle_list: 6.82 us +- 0.20 us -> 7.86 us +- 0.05 us: 1.15x slower
- sympy_str: 372 ms +- 28 ms -> 428 ms +- 3 ms: 1.15x slower
- xml_etree_parse: 231 ms +- 7 ms -> 266 ms +- 9 ms: 1.15x slower
- call_method_slots: 14.0 ms +- 1.3 ms -> 16.1 ms +- 1.2 ms: 1.15x slower
- sympy_sum: 169 ms +- 6 ms -> 194 ms +- 19 ms: 1.15x slower
- logging_format: 29.3 us +- 2.5 us -> 33.7 us +- 1.6 us: 1.15x slower
- logging_simple: 25.7 us +- 2.1 us -> 29.3 us +- 0.4 us: 1.14x slower
- genshi_xml: 159 ms +- 15 ms -> 182 ms +- 1 ms: 1.14x slower
- xml_etree_iterparse: 178 ms +- 3 ms -> 203 ms +- 5 ms: 1.14x slower
- pickle_pure_python: 1.06 ms +- 0.17 ms -> 1.21 ms +- 0.16 ms: 1.14x slower
- logging_silent: 618 ns +- 11 ns -> 705 ns +- 62 ns: 1.14x slower
- hexiom: 19.0 ms +- 0.2 ms -> 21.7 ms +- 0.2 ms: 1.14x slower
- html5lib: 184 ms +- 11 ms -> 209 ms +- 31 ms: 1.14x slower
- call_method: 14.3 ms +- 0.7 ms -> 16.3 ms +- 0.1 ms: 1.14x slower
- django_template: 324 ms +- 18 ms -> 368 ms +- 3 ms: 1.14x slower
- sympy_integrate: 37.9 ms +- 0.3 ms -> 43.0 ms +- 2.7 ms: 1.13x slower
- deltablue: 15.0 ms +- 2.0 ms -> 16.9 ms +- 1.0 ms: 1.12x slower
- call_method_unknown: 16.0 ms +- 0.4 ms -> 17.9 ms +- 0.2 ms: 1.12x slower
- 2to3: 611 ms +- 12 ms -> 677 ms +- 57 ms: 1.11x slower
- regex_compile: 300 ms +- 3 ms -> 332 ms +- 21 ms: 1.11x slower
- json_loads: 50.5 us +- 2.5 us -> 55.8 us +- 1.2 us: 1.10x slower
- unpack_sequence: 111 ns +- 5 ns -> 122 ns +- 1 ns: 1.10x slower
- pickle_dict: 53.2 us +- 3.7 us -> 58.1 us +- 3.7 us: 1.09x slower
- scimark_sor: 420 ms +- 60 ms -> 458 ms +- 12 ms: 1.09x slower
- scimark_lu: 398 ms +- 74 ms -> 434 ms +- 18 ms: 1.09x slower
- regex_dna: 227 ms +- 1 ms -> 247 ms +- 9 ms: 1.09x slower
- pidigits: 266 ms +- 33 ms -> 290 ms +- 10 ms: 1.09x slower
- chaos: 243 ms +- 2 ms -> 265 ms +- 3 ms: 1.09x slower
- crypto_pyaes: 197 ms +- 16 ms -> 215 ms +- 28 ms: 1.09x slower
- dulwich_log: 129 ms +- 15 ms -> 140 ms +- 8 ms: 1.08x slower
- sqlalchemy_imperative: 50.8 ms +- 0.9 ms -> 55.0 ms +- 1.8 ms: 1.08x slower
- meteor_contest: 173 ms +- 22 ms -> 187 ms +- 5 ms: 1.08x slower
- sqlalchemy_declarative: 268 ms +- 11 ms -> 290 ms +- 3 ms: 1.08x slower
- tornado_http: 335 ms +- 4 ms -> 361 ms +- 3 ms: 1.08x slower
- python_startup: 20.6 ms +- 0.6 ms -> 22.1 ms +- 0.9 ms: 1.08x slower
- python_startup_no_site: 8.37 ms +- 0.08 ms -> 9.00 ms +- 0.07 ms: 1.08x slower
- go: 518 ms +- 36 ms -> 557 ms +- 39 ms: 1.07x slower
- raytrace: 1.14 sec +- 0.08 sec -> 1.22 sec +- 0.02 sec: 1.07x slower
- scimark_fft: 594 ms +- 29 ms -> 627 ms +- 13 ms: 1.06x slower

Benchmark hidden because not significant (8): fannkuch, float, json_dumps, 
nbody, regex_v8, richards, scimark_monte_carlo, scimark_sparse_mat_mult

--
Added file: http://bugs.python.org/file45396/pgo.json.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[Matthew Barnett]

Matthew Barnett added the comment:

I split the file into sections, each containing no more 1000 lines, and tried 
reading each section. Attached is a zip file of those that didn't return the 
expected number of rows.

The problem appears to be due to unclosed quotes, which cause following lines 
to be consumed as part of the field.

It looks a little strange, so I wonder if the file has got corrupted somewhere.

--
nosy: +mrabarnett
Added file: http://bugs.python.org/file45395/allCountries_sections.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18233] SSLSocket.getpeercertchain()

[Mariusz Masztalerczuk]

Mariusz Masztalerczuk added the comment:

ping! :) 
Could someone look at my changes? :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26934] android: test_faulthandler fails

[STINNER Victor]

STINNER Victor added the comment:

buggy_raise_3.patch LGTM but I don't think that it's worth it to add 
@requires_raise to support directly. I suggest to only add it to 
test_faulthandler.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

Cool, thanks Serhiy for the site enhancement :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[STINNER Victor]

STINNER Victor added the comment:

Elvis Pranskevichus added the comment:
> I'm actually working on a tool that parses Misc/NEWS, cpython and peps repos, 
> and roundup to gather and organize the changes to make the news editor's job 
> easier.

I don't understand why developers don't do it during the cycle :-/ I'm
trying to keep What's Up In Python X.Y? up to date.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28610] Provide PDB hook to customize how to find source files

[Xavier de Gaye]

Xavier de Gaye added the comment:

This is a simple code object compiled from a source (the string), a module is 
quite different and more complex. The patch uses a fake module Loader to use 
linecache, there is no gain in going any further and pulling from the importlib 
machinery, I think.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27243] __aiter__ should return async iterator instead of awaitable

[Roundup Robot]

Roundup Robot added the comment:

New changeset a0d50aad7b02 by Yury Selivanov in branch '3.6':
Issue #27243: Change PendingDeprecationWarning -> DeprecationWarning.
https://hg.python.org/cpython/rev/a0d50aad7b02

New changeset 9550f0d22d27 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #27243)
https://hg.python.org/cpython/rev/9550f0d22d27

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28643] Broken makefile depends for profile-opt target

[Neil Schemenauer]

New submission from Neil Schemenauer:

I notice that after running "make" then running "make install", the build will 
go through the whole compile/profile/compile process again.  This is really 
infuriating behaviour, given the extremely long make time for the profiled 
optimized build.

The problem appears to me to be that the "profile-opt" target does not have 
proper dependencies.  I think changing it to:

profile-opt: $(BUILDPYTHON)

should fix it.  If my "make install" ever finishes, maybe I will test it. ;-)

--
components: Build
messages: 280342
nosy: nascheme
priority: normal
severity: normal
stage: needs patch
status: open
title: Broken makefile depends for profile-opt target
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28494] is_zipfile false positives

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The problem is that the zipfile module supports even not well-formed archives, 
with a data appended past a comment, and with truncated comment. There are 
special tests for this, and the proposed patch breaks these tests: 
test_comments, test_ignores_newline_at_end, 
test_ignores_stuff_appended_past_comments. See issue10694 and issue1622.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28538] _socket module cross-compilation error on android-24

[Xavier de Gaye]

Xavier de Gaye added the comment:

Patch attached.

--
components: +Cross-Build
keywords: +patch
nosy: +Alex.Willmer, doko
stage: needs patch -> patch review
Added file: http://bugs.python.org/file45394/if_nameindex.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think that's all with this issue.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18317] gettext: DoS via crafted Plural-Forms

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The DoS as well as other flaws is fixed in issue28563 by implementing a 
complete parser for GNU gettext plural form expressions.

--
nosy: +serhiy.storchaka
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed
superseder:  -> Arbitrary code execution in gettext.c2py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Roundup Robot]

Roundup Robot added the comment:

New changeset e0cc3fadd7b3 by Serhiy Storchaka in branch '2.7':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/e0cc3fadd7b3

New changeset 7e66c5dc4218 by Serhiy Storchaka in branch '3.3':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/7e66c5dc4218

New changeset 730cf8cdf02c by Serhiy Storchaka in branch '3.4':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/730cf8cdf02c

New changeset e0dd4cc9661a by Serhiy Storchaka in branch '3.5':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/e0dd4cc9661a

New changeset 7ea64ab9a5c8 by Serhiy Storchaka in branch '3.6':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/7ea64ab9a5c8

New changeset 4ca9a4f96edc by Serhiy Storchaka in branch 'default':
Issue #28563: Fixed possible DoS and arbitrary code execution when handle
https://hg.python.org/cpython/rev/4ca9a4f96edc

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28555] provid also sha-1 and sha-256 also on download links

[Big Stone]

Big Stone added the comment:

I fear GPG is not easy stuff for Windows users.

I fear a bunch of people on this network can circomvent DNS and make python.org 
points to the wrong place.

sha-1 instead of md5 would have been an improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28610] Provide PDB hook to customize how to find source files

[Pinku Surana]

Pinku Surana added the comment:

Thanks. This is clever. I've tried it out and it works. Would it be more 
appropriate to use "importlib" and "importlib.abc" to implement a custom loader 
for a string script? It looks like importlib.abc.InspectLoader does the right 
thing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28636] strange issue with Pandas-0.19.1 on Python-3.6.0b3

[Big Stone]

Big Stone added the comment:

Thank you, Victor. So I guess I should close the issue.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28621] Refactor duplicate code calculating digit's bit length

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you Adrian for your contribution.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28585] Restore docstring of os._isdir

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thanks Zachary.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28621] Refactor duplicate code calculating digit's bit length

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1940b72b0a02 by Serhiy Storchaka in branch 'default':
Issue #28621: Sped up converting int to float by reusing faster bits counting
https://hg.python.org/cpython/rev/1940b72b0a02

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28585] Restore docstring of os._isdir

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5b4fa92dac43 by Serhiy Storchaka in branch '3.5':
Issue #28585: Restored docstring of os._isdir().
https://hg.python.org/cpython/rev/5b4fa92dac43

New changeset 3da89b1678da by Serhiy Storchaka in branch '3.6':
Issue #28585: Restored docstring of os._isdir().
https://hg.python.org/cpython/rev/3da89b1678da

New changeset d77fe603cded by Serhiy Storchaka in branch 'default':
Issue #28585: Restored docstring of os._isdir().
https://hg.python.org/cpython/rev/d77fe603cded

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19569] Use __attribute__((deprecated)) to warn usage of deprecated functions and macros

[Arfrever Frehtes Taifersar Arahesis]

Arfrever Frehtes Taifersar Arahesis added the comment:

About MSVC compiler:

https://msdn.microsoft.com/en-us/library/044swk7y.aspx
https://msdn.microsoft.com/en-us/library/2c8f766e.aspx
https://msdn.microsoft.com/en-us/library/d9x1s805.aspx

So both:

#pragma warning(push)
#pragma warning(disable: 4996)
/* Some code */
#pragma warning(pop)

And:

__pragma(warning(push))
__pragma(warning(disable: 4996))
/* Some code */
__pragma(warning(pop))

Would generally work, but only the second form is suitable for usage inside 
macros.


Updated proposition of Py_COMPILER_DIAGNOSTIC_* macros:

#if defined(__GNUC__) && ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ 
>= 6))
#define Py_COMPILER_DIAGNOSTIC_PUSH _Pragma("GCC diagnostic push")
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS _Pragma("GCC 
diagnostic ignored \"-Wdeprecated-declarations\"")
#define Py_COMPILER_DIAGNOSTIC_POP _Pragma("GCC diagnostic pop")
#elif defined(_MSC_VER) && _MSC_VER >= 1300
#define Py_COMPILER_DIAGNOSTIC_PUSH __pragma(warning(push))
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS 
__pragma(warning(disable: 4996))
#define Py_COMPILER_DIAGNOSTIC_POP __pragma(warning(pop))
#else
#define Py_COMPILER_DIAGNOSTIC_PUSH
#define Py_COMPILER_DIAGNOSTIC_IGNORE_DEPRECATED_DECLARATIONS
#define Py_COMPILER_DIAGNOSTIC_POP
#endif

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Roundup Robot]

Roundup Robot added the comment:

New changeset a822818ec74e by Serhiy Storchaka in branch '3.6':
Issue #28637: No longer use re in site.py.
https://hg.python.org/cpython/rev/a822818ec74e

New changeset a4ea837a7f84 by Serhiy Storchaka in branch 'default':
Issue #28637: No longer use re in site.py.
https://hg.python.org/cpython/rev/a4ea837a7f84

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Irv Kalb]

Irv Kalb added the comment:

On my Mac (with version 2.7.12), selecting Help -> IDLE Help brings up a window 
titled "IDLE Help", it has a pop down with "TOC" selected and many pages of 
text documentation.

On my Windows system (also with version 2.7.12), selecting help -> IDLE help 
does nothing.  I tried it from both the Shell window and from an editor window, 
and no documentation window appeared.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What is the average number of columns on the file?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> If I create a new file with all the skipped files, and I read it again in the 
> same way, around 30% of the rows are skipped.

Could you please provide this smaller file? Or better make yet few iterations 
of keeping only skipped lines until the file will decrease to just few lines.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[SilentGhost]

SilentGhost added the comment:

Could you perhaps make the smaller file make available somewhere?

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader loosing rows with big files and tab delimiter

[Marc Garcia]

New submission from Marc Garcia:

I'm using the csv module from Python standard library, to read a 1.4Gb file 
with 11,157,064 of rows. The file is the Geonames dataset for all countries, 
which can be freely downloaded [1].

I'm using this code to read it:

import csv

with open('allCountries.txt', 'r') as fd:
reader = csv.reader(fd, delimiter='\t')
for i, row in enumerate(reader):
pass

print(i + 1)  # prints 10381963
print(reader.line_num)  # prints 11157064

For some reason, there are around 7% of the rows in the files, that are 
skipped. The rows doesn't have anything special (most of them are all ascii 
characters, even if the file is in utf-8).

If I create a new file with all the skipped files, and I read it again in the 
same way, around 30% of the rows are skipped. So many of them weren't returned 
by the iterator when being a part of a bigger file, but now they are.

Note that the attribute line_num has the right number. Also note that if I 
remove the delimiter parameter (tab) from the reader, and it uses the default 
comma, the iteration on the reader doesn't skip any row.

I checked what I think it's the relevant part of the code [2], but I couldn't 
see anything that could cause this bug.


1. http://download.geonames.org/export/dump/allCountries.zip
2. https://hg.python.org/cpython/file/tip/Modules/_csv.c#l787

--
components: Library (Lib)
messages: 280323
nosy: datapythonista
priority: normal
severity: normal
status: open
title: csv reader loosing rows with big files and tab delimiter
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28640] UnicodeDecodeError: 'cp949'

[Steve Dower]

Changes by Steve Dower :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28640] UnicodeDecodeError: 'cp949'

[Steve Dower]

Steve Dower added the comment:

Firstly, welcome to the tracker, and thanks for helping test the early releases 
of Python 3.6.

For future reference, it's much preferred if you can copy-paste error messages 
into the text of the message. Attaching large log files is okay, but 
screenshots should only really be used for things that are not text.

In this case, it would seem to be an issue with the pefile package. It is 
reading a file with a default encoding as part of its setup.py, but since the 
file appears to be part of its own package it really should specify the 
encoding of the file. It's likely that the file "works" under US English 
encoding, but obviously not for other encodings such as CP949. If the authors 
of pefile specify the encoding when they open the file, then this problem will 
go away.

(As it happens, I investigated the possibility of changing Python to handle 
this situation better, and unfortunately it is not feasible. We've provided the 
tools for authors to avoid these issues on their own, but we can't reasonably 
change how they work without breaking a lot of existing code. So it really 
needs to be fixed by the pefile authors here.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28641] Describe PEP 495 features in "What's New in Python 3.6" document

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Paul (p-ganssle), based on your recent experience implementing PEP 495, what 
are the main points that we should cover in What's New?  Also, any 
comments/criticism on the recent changes to the datetime module documentation 
are welcome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28641] Describe PEP 495 features in "What's New in Python 3.6" document

[Alexander Belopolsky]

New submission from Alexander Belopolsky:

See also #27595.

--
assignee: belopolsky
components: Documentation
messages: 280320
nosy: belopolsky, p-ganssle, tim.peters
priority: normal
severity: normal
stage: needs patch
status: open
title: Describe PEP 495 features in "What's New in Python 3.6" document
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Michael Foord]

Michael Foord added the comment:

Sure, go ahead Syed. Feel free to ask any questions you may have.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

@Łukasz

Dug a bit more to it.

Yes it is RPM specific for that case, in the sense that we create different 
subfolders for the debug and the normal(optimized) builds under the build/ dir, 
where the Include directory does not exist. the dtrace wrapper assumes that the 
directory already exists. The issue of course can be reproduced with any 
packaging system (or ways to compile python) that follow this approach, 
although I am not aware currently if this can happen or might be happening 
anywhere else.

By creating the directory (if it doesn't exist) at the Makefile rule for 
generating pydtrace_probes.h, this can be circumvented.

Attaching a patch that fixes this issue for consideration.

--
Added file: 
http://bugs.python.org/file45393/create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28640] UnicodeDecodeError: 'cp949'

[Hojung An]

New submission from Hojung An:

I have Python3.6 and when I try to install pyinstaller using (pip install 
pyinstaller) I receive error messages for:
1) UnicodeDecodeError: 'cp949'
2) Command 'python setup.py egg_info' failed with error code 1

When I inquired about this to webmas...@python.org they asked me to post it 
here as it seemed like a bug.

--
components: Windows
files: unicode_error.jpg
messages: 280317
nosy: Hojung An, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: UnicodeDecodeError: 'cp949'
versions: Python 3.6
Added file: http://bugs.python.org/file45392/unicode_error.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27589] asyncio doc: issue in as_completed() doc

[Guido van Rossum]

Guido van Rossum added the comment:

Well, in that case the idiom will be even simpler: wrap each one in a
coroutine that awaits it and prints the result and then gather() all the
coroutine wrappers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27589] asyncio doc: issue in as_completed() doc

[Hynek Schlawack]

Hynek Schlawack added the comment:

Such an idiom is IMHO not the main usefulness of this function tho.

As an (untested) example, something like

async def f(n):
   await asyncio.sleep(n)
   return n

for f in asyncio.as_completed([f(3), f(2), f(1)]):
print(await f)


will print:

1
2
3

That’s *super* useful if you’re coordinating multiple independent external 
systems and need to process their results as soon as they arrive (and not once 
they’re *all* done).

Maybe it always worked by accident for me but it’s my understanding, that that 
is what this function is for (and I haven’t found another way to achieve it).

That’s why it would be nice if there’d be authoritative docs on what it’s 
supposed to do. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27589] asyncio doc: issue in as_completed() doc

[Guido van Rossum]

Guido van Rossum added the comment:

However, in general you should use a different pattern. Wrap each future in
a coroutine that does whatever you want to do to the first one ready and
maybe cancel the others. Then gather() all coroutines with the flag that
keeps errors.

--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Guido van Rossum]

Guido van Rossum added the comment:

Please make sure this lands in beta 4!

--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Terry J. Reedy]

Terry J. Reedy added the comment:

The effect is limited to venv only in a narrow sense.  The startup time for 
running programs from an IDLE editor is also affected. Many other programs also 
import re either directly or indirectly on startup.  For instance, importing 
argparse imports re. Running Python from Command Prompt:

Python 3.6.0b3 (default, Nov  1 2016, 03:21:01) [MSC v.1900 64 bit (AMD64)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; 're' in sys.modules
False
>>> import argparse
>>> import sys; 're' in sys.modules
True

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[Elvis Pranskevichus]

Elvis Pranskevichus added the comment:

I'm actually working on a tool that parses Misc/NEWS, cpython and peps repos, 
and roundup to gather and organize the changes to make the news editor's job 
easier.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[Mark Dickinson]

Mark Dickinson added the comment:

What no mention of math.tau? It's a PEP-level change!

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[STINNER Victor]

STINNER Victor added the comment:

Holy cow, I missed the PEP 628.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[Elvis Pranskevichus]

Elvis Pranskevichus added the comment:

The page is a work-in-progress.  We'll surely capture all notable changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Wolfgang Maier]

Changes by Wolfgang Maier :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

> What a comment you need Xiang? Isn't existing comment enough?

Serhiy, I mean the case a number starting with 0, e.g. 0123. The plural form is 
a C expression and in C 0123 is an octal number. c2py now interprets it as a 
decimal number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

It seems to me that current regular expression is not correct. It can keep 
trailing whitespaces it a value.

I'm not experienced with venv and don't know what tests are needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file45391/site-not-use-re.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Wolfgang Maier]

Wolfgang Maier added the comment:

@serhiy.storchaka you've beaten me by a few minutes (still waiting for the test 
suite to finish). Your patch is "contaminated" by an additional change in 
collections/__init__.py though so I'm still uploading my version (which also 
tries to stick as close as possible to the existing behaviour though that 
probably won't matter much)

--
Added file: http://bugs.python.org/file45390/no_use_re_in_site_py.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, please ignore it. This is a sample code from other issue (issue28638).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

One of problems with this patch is that it make instantiating a namedtuple much 
slower (due to parsing arguments by Python code). This can be solved by using 
eval() for creating only the __new__ method (see commented out line 
"result.__new__ = eval(...)"). This increases the time of creating named tuple 
class, but it still is faster than with current code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

site-not-use-re.patch: The change on site.py LGTM. I don't see why re was used 
in the first place for such simple text format ;-) I suggest to push the fix 
into Python 3.6 and 3.7.

But what is the change in Lib/collections/__init__.py?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch removes the use of re from site.py.

--
keywords: +patch
nosy: +serhiy.storchaka
Added file: http://bugs.python.org/file45389/site-not-use-re.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[INADA Naoki]

INADA Naoki added the comment:

(tip)
$ ~/local/py37/bin/python3 -m perf timeit -s 'import importlib, functools' -- 
'importlib.reload(functools)'
.
Median +- std dev: 1.21 ms +- 0.01 ms

(namedtuple-no-compile.patch)
$ ~/local/py37/bin/python3 -m perf timeit -s 'import importlib, functools' -- 
'importlib.reload(functools)'
.
Median +- std dev: 677 us +- 8 us

Nice!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a sample patch that make namedtuple() not using dynamic compilation. It 
has rough the same performance effect as inlining the named tuple source, but 
affects all named tuples.

--
Added file: http://bugs.python.org/file45388/namedtuple-no-compile.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Carl Ekerot]

Carl Ekerot added the comment:

Looks good to me. It behaves as intended on every input I can think of.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[INADA Naoki]

INADA Naoki added the comment:

> What is the main culprit, importing the collections module or compiling a 
> named tuple?

In this time, later.
But collections module takes 1+ ms to import too.
I'll try to optimize it.

> Using namedtuple is not new in 3.6, thus this is not a regression that can be 
> fixed at beta stage.

Make sense.

> More general solution would be to make namedtuple() using cached precompiled 
> class and update the cache if it doesn't match namedtuple arguments.

What "precompiled class" means? pyc file? or source string to be
executed?

> Yet one solution is to make namedtuple() not using compiling, but return 
> patched local class. But Raymond is against this.

I'll search the discussion.
I think anther solution is reimplement namedtuple by C.
As far as I remember, attrs [1] does it.

[1] https://pypi.python.org/pypi/attrs

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Wolfgang Maier]

Wolfgang Maier added the comment:

STINNER Victor added the comment:
>BUT when Python is started from a virtual environment (created by the
>"venv" module), the re module is important by default.
>
>haypo@speed-python$ venv/bin/python3 -c 'import sys; print("re" in 
>sys.modules)'
>True

Exciting, I just verified that this is true and running python3 from a venv 
really seems to be the only situation, in which the re module gets imported 
during startup (at least it's only this one branch in site.py that uses it).

If adding a single enum import to re causes such a big startup time difference 
I wonder how much more could be gained for the venv case by not importing re at 
all!

Turns out that the complete code block in site.py that is used by venvs and 
that was partially shown by @haypo is:

CONFIG_LINE = r'^(?P(\w|[-_])+)\s*=\s*(?P.*)\s*$'

def venv(known_paths):
global PREFIXES, ENABLE_USER_SITE

env = os.environ
if sys.platform == 'darwin' and '__PYVENV_LAUNCHER__' in env:
executable = os.environ['__PYVENV_LAUNCHER__']
else:
executable = sys.executable
exe_dir, _ = os.path.split(os.path.abspath(executable))
site_prefix = os.path.dirname(exe_dir)
sys._home = None
conf_basename = 'pyvenv.cfg'
candidate_confs = [
conffile for conffile in (
os.path.join(exe_dir, conf_basename),
os.path.join(site_prefix, conf_basename)
)
if os.path.isfile(conffile)
]

if candidate_confs:
import re
config_line = re.compile(CONFIG_LINE)
virtual_conf = candidate_confs[0]
system_site = "true"
# Issue 25185: Use UTF-8, as that's what the venv module uses when
# writing the file.
with open(virtual_conf, encoding='utf-8') as f:
for line in f:
line = line.strip()
m = config_line.match(line)
if m:
d = m.groupdict()
key, value = d['key'].lower(), d['value']
if key == 'include-system-site-packages':
system_site = value.lower()
elif key == 'home':
sys._home = value

sys.prefix = sys.exec_prefix = site_prefix

# Doing this here ensures venv takes precedence over user-site
addsitepackages(known_paths, [sys.prefix])

# addsitepackages will process site_prefix again if its in PREFIXES,
# but that's ok; known_paths will prevent anything being added twice
if system_site == "true":
PREFIXES.insert(0, sys.prefix)
else:
PREFIXES = [sys.prefix]
ENABLE_USER_SITE = False

return known_paths

So all the re module is good for here is to parse simple config file records 
with key/value pairs separated by '='. ´Shouldn't it be straightforward to 
implement that logic right inside that block directly without requiring a giant 
import?

This should easily be doable for 3.6 still, seems as if it would solve the 
whole issue and probably speed up the performance tests much more than any 
reverted changesets could.

What do you think?

--
nosy: +wolma

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28000] Build fails on AIX with _LINUX_SOURCE_COMPAT flag

[Matthieu S]

Matthieu S added the comment:

Sorry for the very late reply...

Using _LINUX_SOURCE_COMPAT is sometimes preferable, as it greatly improves 
compatibility with code intended for Linux and improves XPG specs compliance.

But I am not sure it should always be enabled.

In our case, we use it as there where issues with AIX malloc and hotshot when 
building without it, but Python seems to globally work fine without, and we did 
not compare performances.

Like you, I don't know why this was overridden, so it might be safer to keep 
the override and just add the #ifdef _LINUX_SOURCE_COMPAT.

This also applies to Python 2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Martin Wagner]

Martin Wagner added the comment:

i have a use-case that requires a behavior that is referenced above as 
--canonicalize-missing. essentially i need to get rid of relative parts in a 
path representation regardless the actual filesystem.

my conclusion was that PurePath could provide that with a 'normpath' method, 
reflecting os.path.normpath's functionality.

(that's from a user perspective, i haven't looked at any implementation details 
of the pathlib module.)

--
nosy: +mwagner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What a comment you need Xiang? Isn't existing comment enough?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28632] configparser does not close files in read

[Petr]

Petr added the comment:

I am sorry, I can only reproduce it in the production environment so far, it 
does only occur on Ubuntu Linux (Python 3.5.1) and I am developing on Windows. 
So right now I cannot narrow it down (it does not occur with simple code, 
unfortunately). This is what I get after some experimentation:

Exception ignored in: <_io.FileIO name='config.ini' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='config.ini' mode='r' 
encoding='UTF-8'>

The only place when I work with config.ini is the read method of ConfigParser, 
so it definitely should be an issue in ConfigParser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Using namedtuple is not new in 3.6, thus this is not a regression that can be 
fixed at beta stage.

Inlining the source of a named tuple class looks ugly solution. It would be 
better to write the source in separate file and import it. Makefile can have a 
rule for recreating this source file if collections.py is changed.

More general solution would be to make namedtuple() using cached precompiled 
class and update the cache if it doesn't match namedtuple arguments.

Yet one solution is to make namedtuple() not using compiling, but return 
patched local class. But Raymond is against this.

--
assignee:  -> rhettinger
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27589] asyncio doc: issue in as_completed() doc

[Armin Ronacher]

Armin Ronacher added the comment:

I am not even sure what the function is supposed to tell me.  The documentation 
is very unclear and the example code does not help.  What is "fs" for instance? 
 And why would it return things that are not from fs?

--
nosy: +aronacher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28563] Arbitrary code execution in gettext.c2py

[Xiang Zhang]

Xiang Zhang added the comment:

LGTM. And I expect there could be a comment about the special decimal number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28638] Creating namedtuple is too slow to be used in common stdlib (e.g. functools)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What is the main culprit, importing the collections module or compiling a named 
tuple?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com