date:20171018


Change by Łukasz Langa :


--
pull_requests: +4015

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible

2017-10-18 Thread Nick Coghlan


Nick Coghlan  added the comment:

To put this another way: I see an uninterruptible infinite loop as a data loss 
bug on par with a segfault, since there's no graceful way to terminate the 
process and allow cleanup code to run.

For segfaults, we're willing to tolerate them, but we expect the reproducers to 
involve arcane coding contortions, not simple expressions like 
"sum(itertools.count())".

Now, the producer side check that Serhiy posted here only addresses part of the 
problem - there's also the question of making the consumption loops more robust 
by having them check for signals, and adding a ThreadExit equivalent to allow 
the interpreter to request shutdown of non-daemon threads other than the main 
thread.

But as long as we think it's a-OK for us to hang a user's session, causing them 
to lose all their unsaved/uncached data, then we're going to resist the extra 
code complexity required to account for these usability concerns. (And I 
realise they're not new concerns - they're just longstanding problems that 
folks have gotten used to tolerating and excusing)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible

2017-10-18 Thread Nick Coghlan


Nick Coghlan  added the comment:

Defensive coding and the complications it brings is a fact of life when 
providing a widely used platform.

Sure, we're free to say "We don't care about minor user experience irritations 
like Ctrl-C not always being reliable, users should just suck it up and cope".

I think "It's your own fault for typing that, just restart your session from 
scratch" is setting the bar too low for ourselves.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested


Łukasz Langa  added the comment:


New changeset 537ed75291ed8a640887f199f98e2e5076ef87b6 by Łukasz Langa (Miss 
Islington (bot)) in branch '3.6':
[3.6] bpo-31457: Make the `LoggerAdapter.manager` property settable (GH-4042) 
(#4043)
https://github.com/python/cpython/commit/537ed75291ed8a640887f199f98e2e5076ef87b6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested


Change by Roundup Robot :


--
pull_requests: +4014

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested


Łukasz Langa  added the comment:


New changeset 0b6a118a45ac2eded1348fea6ed300d5651f7471 by Łukasz Langa in 
branch 'master':
bpo-31457: Make the `LoggerAdapter.manager` property settable (#4042)
https://github.com/python/cpython/commit/0b6a118a45ac2eded1348fea6ed300d5651f7471


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested


Change by Łukasz Langa :


--
pull_requests: +4013

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30140] Binary arithmetic does not always call subclasses first

2017-10-18 Thread Stephan Hoyer


Stephan Hoyer  added the comment:

Ping -- it would be great if someone could take a look at my PR. (I suspect it 
needs more documentation, tips on where to put that would be appreciated.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31454] Include "import as" in tutorial

2017-10-18 Thread Mario Corchero


Change by Mario Corchero :


--
keywords: +patch
pull_requests: +4012
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31454] Include "import as" in tutorial

2017-10-18 Thread Mario Corchero


Change by Mario Corchero :


--
nosy: +mariocj89

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()

2017-10-18 Thread Mario Corchero


Mario Corchero  added the comment:

Wrote https://bugs.python.org/issue31800 without realising this issue was open 
(Thanks for bringing it up Martin Panter).

issue31800 basically just adds the ability to parse NN:NN to the already 
existing python isoformat function when %z is specified.
Rather than adding "%:z".

As available as well in linux strptime: 
http://man7.org/linux/man-pages/man3/strptime.3.html

I'd really like to see a way to parse isoformatted dates and this is the only 
thing in the middle. Happy to continue with the other issue/PR or help out here 
if needed.

Thanks! ^^

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible

2017-10-18 Thread Koos Zevenhoven


Koos Zevenhoven  added the comment:

To repeat one of my points in the linked threads, I'm not convinced that 
infinite iterators are the most common case for the problem of long 
uninterruptible loops. A general mechanism that can be easily used in many 
places with minimal maintenance burden would be nice. It could be used even in 
third-party extension modules.

--
nosy: +koos.zevenhoven

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31742] Default to emitting FutureWarning for provisional APIs

2017-10-18 Thread Ethan Furman


Change by Ethan Furman :


--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9253] argparse: optional subparsers

2017-10-18 Thread paul j3


paul j3  added the comment:

In a recent stackoverflow question a user wanted this optional-subparsers 
ability in Python 2.7.

https://stackoverflow.com/questions/46667843/how-to-set-a-default-subparser-using-argparse-module-with-python-2-7

Short of modifying the _parse_known_args method, the best I could suggest was a 
two stage parsing.  That is, one parser without the subparsers.  This uses 
parse_known_args, and if a 'cmd' is provided passes the 'extras' to one that 
handles subparsers.

---

Another issue which I don't think has been addressed is the 'usage' when 
subparsers are optional.  At least with 3.5, subparsers are displayed with the 
choices: {'cmd1', 'cmd2', ...}, but no indication of being optional.  An 
optional positional (with ? nargs) would normally be displayed as

 prog [-h] [{'one', 'two'}] ...

My guess is that 'usage' adds the [] when positionals nargs='?', without regard 
to the 'required' attribute (I should verify this from code).

I'm undecided as to whether we want the brackets or not.  It's more accurate, 
but makes the usage messier.  And the 'help' grouping for 
'optional-positionals' is the subject of other bug/issue(s).

I haven't checked it the patch has changed this behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible


Serhiy Storchaka  added the comment:

I concur with Raymond. I cited the same arguments in the discussion on 
Python-ideas. But the other solution that was suggested in this discussion will 
add more complexity and can't solve all cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible


Serhiy Storchaka  added the comment:

With optimized repeat():

$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import repeat' 'list(repeat(None, 100))'
/home/serhiy/py/cpython-release/python: . 3.77 ms +- 0.06 ms
/home/serhiy/py/cpython-iter/python: . 3.77 ms +- 0.05 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 3.77 ms +- 0.06 ms -> 
[/home/serhiy/py/cpython-iter/python] 3.77 ms +- 0.05 ms: 1.00x faster (-0%)
Not significant!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible


Serhiy Storchaka  added the comment:

Microbenchmark results:

$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import repeat' 'list(repeat(None, 100))'
/home/serhiy/py/cpython-release/python: . 3.79 ms +- 0.09 ms
/home/serhiy/py/cpython-iter/python: . 4.14 ms +- 0.07 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 3.79 ms +- 0.09 ms -> 
[/home/serhiy/py/cpython-iter/python] 4.14 ms +- 0.07 ms: 1.09x slower (+9%)


$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import cycle, islice' 'list(islice(cycle(range(1000)), 100))'
/home/serhiy/py/cpython-release/python: . 6.88 ms +- 0.30 ms
/home/serhiy/py/cpython-iter/python: . 6.87 ms +- 0.26 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 6.88 ms +- 0.30 ms -> 
[/home/serhiy/py/cpython-iter/python] 6.87 ms +- 0.26 ms: 1.00x faster (-0%)
Not significant!


$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import count, islice' 'list(islice(count(), 100))'
/home/serhiy/py/cpython-release/python: . 26.1 ms +- 0.6 ms
/home/serhiy/py/cpython-iter/python: . 26.3 ms +- 0.6 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 26.1 ms +- 0.6 ms -> 
[/home/serhiy/py/cpython-iter/python] 26.3 ms +- 0.6 ms: 1.01x slower (+1%)
Not significant!


$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import product' 'list(product(range(100), repeat=3))'
/home/serhiy/py/cpython-release/python: . 80.2 ms +- 3.2 ms
/home/serhiy/py/cpython-iter/python: . 80.2 ms +- 1.7 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 80.2 ms +- 3.2 ms -> 
[/home/serhiy/py/cpython-iter/python] 80.2 ms +- 1.7 ms: 1.00x faster (-0%)
Not significant!


$ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from 
itertools import combinations' 'list(combinations(range(23), 10))'
/home/serhiy/py/cpython-release/python: . 177 ms +- 14 ms
/home/serhiy/py/cpython-iter/python: . 169 ms +- 4 ms

Mean +- std dev: [/home/serhiy/py/cpython-release/python] 177 ms +- 14 ms -> 
[/home/serhiy/py/cpython-iter/python] 169 ms +- 4 ms: 1.05x faster (-4%)


The only significant slowdown is for repeat(). But there is possibility to 
optimize this one by reusing an existing counter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible

2017-10-18 Thread Raymond Hettinger


Raymond Hettinger  added the comment:

When I have time, I would like to re-launch a python-dev discussion on this.  
It is my feeling that this solves an invented problem.  In my experience, it 
only ever happens to people who have intentionally trying to create this effect.

Adding this kind of "junk" through-out the code base adds complexity and more 
internal operations, but won't help *any* existing, deployed code.  We're 
making everyone pay for a problem that almost no one has.

Also, if we do care about interruptability, it is unclear whether the 
responsiblity should like with the consumer or the producer of the iterator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible

2017-10-18 Thread Raymond Hettinger


Change by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31778] ast.literal_eval supports non-literals in Python 3

2017-10-18 Thread Yury Selivanov


Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28339] "TypeError: Parameterized generics cannot be used with class or instance checks" in test_functools after importing typing module

2017-10-18 Thread Wilfred Hughes


Wilfred Hughes  added the comment:

Note that this also affects the singledispatch library that backports 
singledispatch to Python 2: https://github.com/python/typing/issues/484

--
nosy: +Wilfred.Hughes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interruptible


Change by Serhiy Storchaka :


--
title: Make itertools iterators interrable -> Make itertools iterators 
interruptible

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interrable


Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4011

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31815] Make itertools iterators interrable


New submission from Serhiy Storchaka :

Proposed PR makes tight C loops with itertools iterators interruptible with 
Ctrl-C. It adds checks for keyboard interrupt in iterators that can produce 
long sequences without advancing other iterators. For performance checks are 
performed only for every 0x1000-th item. If for generating new value other 
iterator should be advanced, the responsibility for checking for keyboard 
interrupt is attributed to that iterator.

This would solve the problem discussed on Python-ideas:

https://mail.python.org/pipermail/python-ideas/2017-October/047412.html
http://permalink.gmane.org/gmane.comp.python.ideas/47429

Example:

>>> import itertools
>>> it = itertools.count()
>>> it in it
^CTraceback (most recent call last):
  File "", line 1, in 
KeyboardInterrupt
>>>

--
components: Extension Modules
messages: 304588
nosy: ncoghlan, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Make itertools iterators interrable
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31814] subprocess_fork_exec more stable with vfork

2017-10-18 Thread Albert Zeyer


New submission from Albert Zeyer :

subprocess_fork_exec currently calls fork().

I propose to use vfork() or posix_spawn() or syscall(SYS_clone, SIGCHLD, 0) 
instead if possible and if there is no preexec_fn. The difference would be that 
fork() will call any atfork handlers (registered via pthread_atfork()), while 
the suggested calls would not.

There are cases where atfork handlers are registered which are not save to be 
called e.g. in multi-threaded environments. In the case of subprocess_fork_exec 
without preexec_fn, there is no need to call those atfork handlers, so avoiding 
this could avoid potential problems. It's maybe acceptable if a pure fork() 
without exec() doesn't work in this case anymore, but there is no reason that a 
fork()+exec() should not work in any such cases. This is fixed by my proposed 
solution.

An example case is OpenBlas and OpenMP, which registers an atfork handler which 
is safe to be called if there are other threads running.
See here:
https://github.com/tensorflow/tensorflow/issues/13802
https://github.com/xianyi/OpenBLAS/issues/240
https://trac.sagemath.org/ticket/22021

About fork+exec without the atfork handlers, see here for alternatives (like 
vfork):
https://stackoverflow.com/questions/46810597/forkexec-without-atfork-handlers/

--
components: Interpreter Core
messages: 304587
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: subprocess_fork_exec more stable with vfork
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

2017-10-18 Thread Christian Heimes


Christian Heimes  added the comment:

- BoringSSL is not a supported libssl/libcrypto library for Python. We only 
support 100% OpenSSL-compatible libraries.
- OpenSSL 1.0.1 is no longer supported by upstream. Python's semi-official 
support policy for 1.0.1 and 0.9.8 is "use at your own risk". You should 
upgrade.
- Python < 2.7.13 does not support OpenSSL 1.1
- Python >= 2.7.13 has #if !defined(OPENSSL_VERSION_1_1) before 
SSL_CTX_set_ecdh_auto()

I added a new test that verifies connections with various curve settings. I 
cannot reproduce your problem with the test case.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

2017-10-18 Thread Christian Heimes


Change by Christian Heimes :


--
keywords: +patch
pull_requests: +4010
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31813] python -m enshure pip stucks

2017-10-18 Thread Serhy Pyton


New submission from Serhy Pyton :

pip stucks, during installing python 3.6.3 in termux and not complete it. 
Posibly it's could be a problem only on meizu m2. Also it produse load in such 
state.

--
components: Installation
files: S71018-182104.jpg
messages: 304585
nosy: Serhy Pyton
priority: normal
severity: normal
status: open
title: python -m enshure pip stucks
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47226/S71018-182104.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31812] Document PEP 545 (documentation translation) in What's New in Python 3.7


New submission from STINNER Victor :

The PEP 545 should be documented in What's New in Python 3.7: mention that the 
Python documentation is now *officially* translated to french and japanese.

--
assignee: docs@python
components: Documentation
messages: 304584
nosy: docs@python, haypo, inada.naoki, mdk
priority: normal
severity: normal
status: open
title: Document PEP 545 (documentation translation) in What's New in Python 3.7
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31811] async and await missing from keyword list in lexical analysis doc

2017-10-18 Thread Colin Dunklau


New submission from Colin Dunklau :

I see that code making async/await real keywords has been merged, but it looks 
like Doc/reference/lexical_analysis.rst doesn't have those added
https://github.com/python/cpython/blob/4a2d00c/Doc/reference/lexical_analysis.rst#keywords


Is that list autogenerated somehow or was it just overlooked?

--
assignee: docs@python
components: Documentation
messages: 304583
nosy: Colin Dunklau, docs@python
priority: normal
severity: normal
status: open
title: async and await missing from keyword list in lexical analysis doc
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31810] Travis CI, buildbots: run "make smelly" to check if CPython leaks symbols


New submission from STINNER Victor :

Recently, a new "cell_set_contents()" public symbol was added by mistake: see 
bpo-30486. It was quickly noticed by doko, and fixed by me (commit 
0ad05c32cc41d4c21bfd78b9ffead519ead475a2).

The problem is that we already have a "make smelly" command to check if Python 
"leaks" symbols, but this check is not run on our CIs (Travis CI or buildbots).

It would be nice to run automatically this test.

--
components: Tests
keywords: buildbot
messages: 304582
nosy: haypo, zach.ware
priority: normal
severity: normal
status: open
title: Travis CI, buildbots: run "make smelly" to check if CPython leaks symbols
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

2017-10-18 Thread Andy


Andy  added the comment:

While debugging I reproduced this on
- 'OpenSSL 1.1.0f  25 May 2017'
- 'OpenSSL 1.0.1f 6 Jan 2014'
- and 'BoringSSL', latest.

using Python 2.7.12, 2.7.13, 2.7.6 and 3.5.3. This was all on Debian.


Note that since I used Python <2.7.14 (or equivalent for 3.x) for all tests, 
the check "... && !defined(OPENSSL_VERSION_1_1)" is missing and therefore the 
bug *always* triggers regardless of OpenSSL version.



I'm not sure I agree that this one curve is a good default. Note that openssl 
has 81 curves currently (openssl ecparam -list_curves) and probably can use 
most of them to connect to a server - accommodating a variety of server setups. 
Restricting this list to one single curve seems suboptimal.

Think about it like this, if tomorrow we find an issue with that particular 
curve, all servers can just migrate to a different one and all clients will be 
able to connect just fine - except those that use Python, they will not be able 
to talk to those servers ever again until they are upgraded. I mean in the end 
it's your call but having a *client* just accepting one single security 
parameter and nothing else doesn't seem right.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31778] ast.literal_eval supports non-literals in Python 3


Serhiy Storchaka  added the comment:

PR 4035 makes ast.literal_eval() more strict.

--
versions:  -Python 3.6, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31778] ast.literal_eval supports non-literals in Python 3


Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4009
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

2017-10-18 Thread Christian Heimes


Christian Heimes  added the comment:

Which version of OpenSSL are you using? Please note that macOS' system python 
uses either an ancient version of OpenSSL 0.9.8 or an ancient version of 
LibreSSL (IIRC 2.3.x).

The code in question is:

if !defined(OPENSSL_NO_ECDH) && !defined(OPENSSL_VERSION_1_1)
/* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use
   prime256v1 by default.  This is Apache mod_ssl's initialization
   policy, so we should be safe. OpenSSL 1.1 has it enabled by default.
 */
#if defined(SSL_CTX_set_ecdh_auto)
SSL_CTX_set_ecdh_auto(self->ctx, 1);
#else
{
EC_KEY *key = EC_KEY_new_by_curve_name(NID_X9_62_prime256v1);
SSL_CTX_set_tmp_ecdh(self->ctx, key);
EC_KEY_free(key);
}
#endif
#endif

The block is executed for all SSLContexts (server and client) because . The 
behavior depends on the version of OpenSSL:

OpenSSL >= 1.1: not executed
OpenSSL >= 1.0.2, < 1.1: SL_CTX_set_ecdh_auto(ctx, 1)
LibreSSL: SSL_CTX_set_ecdh_auto(ctx, 1)
OpenSSL < 1.0.2: hard-code prime256v1

Since we have no mean to distinguish between a server context and a client 
context at the moment, we unconditionally call SSL_CTX_set_ecdh_auto(). It may 
not be perfect under some condition. But it gives most users a sane and secure 
default to start with.

https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_set_ecdh_auto.html

--
nosy: +alex, dstufft, janssen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29854] Segfault when readline history is more then 2 * history size


STINNER Victor  added the comment:

FYI I added the test.pythoninfo utility as a follow-up of this issue to log 
many informations to debug Python, not only the readline version: see bpo-30871.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31809] ssl module unnecessarily pins the client curve when using ECDH

2017-10-18 Thread A

New submission from A :

When using elliptic curves in combination with the ssl module to wrap a socket,
the only curve the client accepts is prime256v1. Expected behavior is to accept
all curves that are on the default list and supported by the server.

We noticed the issue when connecting to a server that is configured with a non
standard curve, we get

[SSL: WRONG_CURVE] wrong curve (_ssl.c:661)

unless we explicitly specify to use the correct curve for the context using

context.set_ecdh_curve("secp521r1")

The bug happens both when using a context or ssl.wrap_socket directly.

The bug is that cpython calls OpenSSL (or whatever lib is linked) server
methods even when the context is purely for client communications. The flow is:

- A ssl.SSLContext gets instantiated that will later be used in
SSLContext.wrap_socket with server_side either True or False (bug happens when
this is client side, so server_side=False).

- In the context's constructor, where it's not yet clear if the context is for
a client or a server socket, there is this code:
https://github.com/python/cpython/blob/b9a860f3bf80b0d4a6c25d0f2f6ef849d9bf3594/Modules/_ssl.c#L2180

That code calls (in many cases, depending on the Python and OpenSSL versions
involved) server side only methods, SSL_CTX_set_ecdh_auto or
SSL_CTX_set_tmp_ecdh.

Those methods should in theory not influence the client side of the context,
there is even a comment in the docs for SSLContext.set_ecdh_curve which does
similar things that says "This setting doesn’t apply to client sockets".
However, this being OpenSSL, this is not true. The methods actually set the
list of acceptable curves that is used for both, server and client side
connections, to exactly a single curve, prime256v1. This happens for both,
SSL_CTX_set_tmp_ecdh and SSL_CTX_set_ecdh_auto.

Versions affected:

OpenSSL has changed the API twice here. Before 1.0.2, SSL_CTX_set_tmp_ecdh was
mandatory, 1.0.2 - <1.1.0 used SSL_CTX_set_ecdh_auto and 1.1.0+ doesn't need
this setting at all.

Python 2.7.14+ added a check for 1.1.0+ in
https://github.com/python/cpython/commit/f1a696efd6ca674579e25de29ec4053ff5a5ade1
so starting from that version the issue only happens when using OpenSSL
<1.1.0. I suspect this is the case for many machines still, including all Macs
(haven't confirmed the actual bug there). For LibreSSL the server side methods
will be called too, I can't confirm if that combination is vulnerable too.

Python 3.5.3 gives me the same error, no reason to believe that higher versions
are not affected.

--
assignee: christian.heimes
components: SSL
messages: 304577
nosy: christian.heimes, gr
priority: normal
severity: normal
status: open
title: ssl module unnecessarily pins the client curve when using ECDH
type: behavior
versions: Python 2.7, Python 3.5

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14465] xml.etree.ElementTree: add feature to prettify XML output


Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
components: +XML
nosy: +serhiy.storchaka
versions: +Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c


Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts


Serhiy Storchaka  added the comment:


New changeset 27b951c63353345cdf7a9a8c4c8133a5dafd6a80 by Serhiy Storchaka 
(Riccardo Coccioli) in branch '2.7':
[2.7] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (#4034)
https://github.com/python/cpython/commit/27b951c63353345cdf7a9a8c4c8133a5dafd6a80


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts


Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts


Serhiy Storchaka  added the comment:


New changeset 97abcabc195b87d6a5562dbb867a469fac27d3f6 by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31334: Fix timeout in select.poll.poll() (GH-3277) (#4033)
https://github.com/python/cpython/commit/97abcabc195b87d6a5562dbb867a469fac27d3f6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31778] ast.literal_eval supports non-literals in Python 3


Serhiy Storchaka  added the comment:

"""
The string or node provided may only consist of the following Python literal 
structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and 
None.
"""

1+1 is not a literal number.

"""
It is not capable of evaluating arbitrarily complex expressions, for example 
involving operators or indexing.
"""

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31778] ast.literal_eval supports non-literals in Python 3

2017-10-18 Thread R. David Murray


R. David Murray  added the comment:

"Safely evaluate an expression node or a string containing a Python expression."

The behavior you are citing matches that documentation, as far as I can see.  
1+1 is an expression involving supported literals.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts

2017-10-18 Thread Riccardo Coccioli


Change by Riccardo Coccioli :


--
pull_requests: +4008

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning

2017-10-18 Thread Marc-Andre Lemburg

Marc-Andre Lemburg  added the comment:

On 18.10.2017 11:45, STINNER Victor wrote:
> Marc-Andre, Ethan: What do you think of removing the deprecation warning from 
> the C (my last commit), leave the deprecation warning in the documentation, 
> and modify time.clock() to become an alias to time.perf_counter()?
> 
> By alias, I really mean time.clock = time.perf_counter, so 
> time.clock.__name__ would say "perf_counter".

That's what I think would be a better solution, since the
absolute value of time.clock() is never used, only the difference.

If you then get better accuracy in that difference, things
can only get better, so this is not really backwards compatibility
issue (nothing gets worse).

Not sure whether the function name would cause an incompatibility
issue. I doubt it, but if it does we could have time.clock()
as function which then simply calls time.perf_counter().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29696] Use namedtuple in string.Formatter.parse iterator response

2017-10-18 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

With the exception of string.FormatterItem, which is the change at hand.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29696] Use namedtuple in string.Formatter.parse iterator response

2017-10-18 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

IMHO this change makes things a bit more consistent. In lots of places when a 
tuple is returned, a `structseq` is used to improve readability on the returned 
result. Some examples of this are:

* grp.struct_group
* os.terminal_size
* pwd.struct_passwd
* resource.struct_rusage
* signal.struct_siginfo
* time.struct_time
* spwd.struct_spwd
* sys.float_info
* sys.int_info
* string.FormatterItem
* sys.hash_info
* sys.getwindowsversion
* sys.flags
* sys.version_info
* sys.thread_info

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning


STINNER Victor  added the comment:

"Initially the time module was a thin wrapper around C and OS time-related 
functions. It may be confusing that the behavior of time.clock() differs from 
the behavior of C's clock()."

Well, there is the theory, and there is the practice. Someone decided to 
implement time.clock() with QueryPerformanceCounter() on Windows, and so 
time.clock() is no more a thin wrapper to the C clock() function since at least 
Python 2.7 (I didn't check earlier versions).

Portability on clocks is even more than complex than in the os module, 
especially when you want the same behaviour on Windows and Unix. The PEP 418 
added new clocks with a better defined behaviour to "fix" this issue.


"The documentation for clock() on MSDN suggests to use GetProcessTimes() for 
the behavior conforming the C standard."

time.process_time() is implemented with GetProcessTimes(). That's why 
time.clock() suggests to either replace it with time.perf_counter() or 
time.process_time().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c


Serhiy Storchaka  added the comment:


New changeset be4e9cc769aac3cb46670c049b9f21e412be53d1 by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31806: Use _PyTime_ROUND_TIMEOUT for the timeout argument parsing in 
more functions (GH-4026) (#4032)
https://github.com/python/cpython/commit/be4e9cc769aac3cb46670c049b9f21e412be53d1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning


Serhiy Storchaka  added the comment:

Initially the time module was a thin wrapper around C and OS time-related 
functions. It may be confusing that the behavior of time.clock() differs from 
the behavior of C's clock().

The documentation for clock() on MSDN suggests to use GetProcessTimes() for the 
behavior conforming the C standard.

https://msdn.microsoft.com/en-us/library/4e2ess30.aspx

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit


Serhiy Storchaka  added the comment:

Maybe the difference is processor depending. I'm going to repeat benchmarking 
on 32-bit processors. Or maybe fast call was optimized in master. Or passing 
via an arguments tuple was pessimized.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning


STINNER Victor  added the comment:

(I reopen the issue since the discussion is not over.)

Marc-Andre Lemburg: "time.cock() is used in a lot of code."

I ran a quick search on GitHub. I found different use cases of time.clock():

1) Measure performance. On Windows, time.clock() has a very good precision, 
*much* better than any other clock. For example, time.process_time() has a 
resolution of 15.6 ms whereas time.clock() has a resolution of 100 ns (or 
0.0001 ms):
https://www.python.org/dev/peps/pep-0564/#windows


2) An explicit check that time.clock() doesn't include sleep. I guess that 
people are using such snippet to explain this behaviour?

https://github.com/pbarton666/PES_Python_examples_and_solutions/blob/master/py_time.py

---

#py_time.py

import time
print(time.clock(), time.time())
time.sleep(1)  #seconds
print(time.clock(), time.time())
---


Ethan: "I agree with MAL; removing functions just makes multi-version code more 
painful."

We have two choices:

* Deprecate and then remove time.clock(): break the backward compatibility -- 
currently chosen solution
* Modify time.clock() to get a portable behaviour: break the backward 
compatibility

Depending which clock we choose for time.clock(), we may break more and less 
code, I would vote for using the time.perf_counter() clock in time.clock(). It 
means no change on Windows, but modify the behaviour on Unix if the code sleeps 
(time.sleep or I/O).

It seems like time.clock() is mostly used for benchmarking, and 
time.perf_counter() is documented as the best clock for such use case. In the 
benchmarks I saw on GitHub, the benchmarked code didn't sleep, it was more 
likely pure CPU-bound.


Marc-Andre, Ethan: What do you think of removing the deprecation warning from 
the C (my last commit), leave the deprecation warning in the documentation, and 
modify time.clock() to become an alias to time.perf_counter()?

By alias, I really mean time.clock = time.perf_counter, so time.clock.__name__ 
would say "perf_counter".

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31786] In select.poll.poll() ms can be 0 if timeout < 0


Serhiy Storchaka  added the comment:

Thank you for your contribution Pablo.

The issue is fixed in 3.6 and 3.7. It is hard to fix it in 2.7.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31808] tarfile.extractall fails to overwrite symlinks

2017-10-18 Thread Frederic Beister


New submission from Frederic Beister :

The fix for https://bugs.python.org/issue10761 somehow didn't make it into 3.5 
and later - I have the same behavior as described there in 3.5.1 and 3.6.1 
(tested on Ubuntu 16.04)

--
components: Library (Lib)
messages: 304563
nosy: Frederic Beister
priority: normal
severity: normal
status: open
title: tarfile.extractall fails to overwrite symlinks
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31786] In select.poll.poll() ms can be 0 if timeout < 0


Serhiy Storchaka  added the comment:


New changeset ed267e3305a54eddae8106bdaae2c62d4c3b7db6 by Serhiy Storchaka in 
branch '2.7':
[2.7] bpo-31786: Make functions in the select module blocking when timeout is a 
small negative value. (GH-4003). (#4031)
https://github.com/python/cpython/commit/ed267e3305a54eddae8106bdaae2c62d4c3b7db6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31803] time.clock() should emit a DeprecationWarning


STINNER Victor  added the comment:

Marc Andre Lemburg: "time.cock() is used in a lot of code. Why can't we simply 
replace the functionality with one of the other functions ? The documentation 
certainly allows for such a change, since it pretty much just says that only 
the delta between two values has a meaning."

Currently time.clock() is the same clock than time.perf_counter() on Windows, 
but it's closer to CPU time as time.process_time() on Unix.

time.perf_counter() and time.process_time() have a different name because the 
they are different clocks and don't measure the same thing. The main obvious 
difference is that time.process_time() doesn't include time elapsed during 
sleep.

>>> import time
>>> c1=time.perf_counter(); p1=time.process_time(); time.sleep(1); 
>>> c2=time.perf_counter(); p2=time.process_time()
>>> c2-c1, p2-p1
(1.0010841980038094, 7.3085998e-05)

I don't see how to modify clock() to make it behave the same on Windows and 
Unix without breaking any application which relies on the current clock() 
behaviour.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c


Change by Roundup Robot :


--
pull_requests: +4006

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31786] In select.poll.poll() ms can be 0 if timeout < 0


Serhiy Storchaka  added the comment:


New changeset 95602b368b87da3702a0f340ded2a23e823bb104 by Serhiy Storchaka 
(Pablo Galindo) in branch '3.6':
[3.6] bpo-31786: Make functions in the select module blocking when timeout is a 
small negative value. (GH-4003). (#4022)
https://github.com/python/cpython/commit/95602b368b87da3702a0f340ded2a23e823bb104


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31676] test.test_imp.ImportTests.test_load_source has side effects


STINNER Victor  added the comment:

I tested "./python -m test -uall -M4G test_imp test_cgitb" on 3.6 and the 
master branch. Both tests now pass correctly. I close the issue.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts


Change by Roundup Robot :


--
keywords: +patch
pull_requests: +4007

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31786] In select.poll.poll() ms can be 0 if timeout < 0


Change by Serhiy Storchaka :


--
pull_requests: +4005

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31780] Using format spec ',x' displays incorrect error message

2017-10-18 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Correct.  Fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c


Change by Roundup Robot :


--
pull_requests: +4004

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31806] Use _PyTime_ROUND_TIMEOUT in _threadmodule.c, timemodule.c and socketmodule.c