[issue29885] Allow GMT timezones to be used in datetime.

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Use pytz (https://pypi.python.org/pypi/pytz).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

The patch contains an example of using _Py_STATICVAR(). But it doesn't use 
_PY_STATICVAR_INIT(). _PY_STATICVAR_INIT() can be used if extract the code of 
getting _array_reconstructor into separate function.

I like the idea in general, but I want to see more examples. Ideally -- the 
patch should use a new API for *all* static PyObject* variables. This will help 
to design good API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29885] Allow GMT timezones to be used in datetime.

2017-03-22 Thread Decorater

New submission from Decorater:

I noticed that there is no ways to convert local times to GMT if I realize that 
some other object (sometimes from a server) is using GMT and they happen to be 
ahead of my current time. As such it would be great if one can convert their 
current time that can be in an datetime object to an GMT time to see how much 
time has passed since something happened on their zone if so desired (otherwise 
can cause undesired or undefined consequences).

--
components: Library (Lib)
messages: 290027
nosy: Decorater
priority: normal
severity: normal
status: open
title: Allow GMT timezones to be used in datetime.
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29884] faulthandler does not properly restore sigaltstack during teardown

2017-03-22 Thread Christophe Zeitouny

Changes by Christophe Zeitouny :


--
pull_requests: +682

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29884] faulthandler does not properly restore sigaltstack during teardown

2017-03-22 Thread Christophe Zeitouny

Changes by Christophe Zeitouny :


Added file: http://bugs.python.org/file46755/python_reproducer.cpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29884] faulthandler does not properly restore sigaltstack during teardown

2017-03-22 Thread Christophe Zeitouny

New submission from Christophe Zeitouny:

Looks like faulthandler is not properly tearing down its sigaltstack, causing 
potential double-free issues in any application that embeds the Python 
interpreter.
I stumbled upon this when I enabled AddressSanitizer on my application, which 
sets up and tears down a Python interpreter instance at runtime. 
AddressSanitizer complained about a double-free of the stack_t::ss_sp memory 
region.
After close inspection, here's what's happening:

1. When a new thread is created, AddressSanitizer sets up its own alternative 
stack by calling sigaltstack
2. Py_Initialize() is called, which initializes faulthandler, which sets up its 
own alternative stack, therefore overriding the one installed by 
AddressSanitizer
3. Py_Finalize() is called, which deinitializes faulthandler, which merely 
deletes the allocated stack region, but leaves the alternative stack installed. 
Any signal that occurs after this point will be using a memory region it 
doesn't own as stack. dangerous stuff.
4. The thread exits, at which point AddressSanitizer queries sigaltstack for 
the current alternative stack, blindly assumes that it's the same one that it 
installed, and attempts to free the allocated stack region. Therefore causing a 
double free issue

Regardless of the fact that AddressSanitizer should probably not blindly trust 
that the currently installed sigaltstack is the same one it installed earlier, 
the current code in faulthandler leaves the sigaltstack in a very bad state 
after finalizing.
This means that the application that embeds the Python interpreter better hope 
that no signals are raised after it calls Py_Finalize().

I have a patch that fixes this issue. faulthandler will save the previously 
installed alternative stack at initialization time. During deinitialization, it 
will query sigaltstack for the current stack. If it's the same as the one it 
installed,
it will restore the saved previous stack.

'sigaltstack' just sounds like a badly designed API. There is essentially no 
way to use it 'correctly'.
Here's how AddressSanitizer uses it (line 164): 
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/sanitizer_posix_libcdep.cc?view=markup
and here's how the Chrome browser uses it: 
https://chromium.googlesource.com/breakpad/breakpad/+/chrome_43/src/client/linux/handler/exception_handler.cc#149

Notice that my approach is closer to what Chrome does, but in the case where 
the installed stack is no longer ours, I don't disable whatever stack is 
installed. This is because I don't believe that will make much difference. 
Whoever switched out the stack could have
saved our stack somewhere and planned on blindly restoring it upon exit. In 
which case, whatever we do would be overridden.

Attached are a tiny reproducer for the issue, along with the complete analysis 
of what's reported by AddressSanitizer. I'll follow this up by a pull request 
for my changes.

Thanks!
Chris

--
components: Extension Modules
files: python_failure.txt
messages: 290025
nosy: haypo, tich
priority: normal
severity: normal
status: open
title: faulthandler does not properly restore sigaltstack during teardown
type: enhancement
Added file: http://bugs.python.org/file46754/python_failure.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread Nathaniel Smith

Nathaniel Smith added the comment:

Letting Python-level signal handlers run in arbitrary threads is an interesting 
idea, but it's a non-trivial change to Python semantics that may well break 
some programs (previously it was guaranteed that signal handlers couldn't race 
with main thread code), and it doesn't fully fix the bug here (because it's 
common for Python programs to contain threads which don't run Python code, e.g. 
if they use zmq). I can imagine some potential benefits, but I feel like this 
needs some substantial rationale?

OTOH the pthread_kill-based fix I suggested is like 3 lines and AFAICT fixes 
the problem exactly.

> I dislike the idea of transfering the signal to the main thread from another 
> thread in the C signal handler using pthread_kill(). Most code behave very 
> badly when getting a signal, so getting a signal twice is likely to double 
> the pain.

This doesn't make much sense to me... CPython's pretty good at handling EINTR 
these days, and, well, the only case that's different is when you have a signal 
handler that needs to be run and the main thread is blocked in a syscall, which 
is exactly the case that's currently broken? 

> Moreover, pthread_sigmask() can block signals in the main thread for 
> deliberate reasons.

Sure, and with pthread_kill() this would have the effect that the signal would 
be queued by the kernel and delivered after signal is unmasked. That seems like 
pretty sensible semantics to me; in fact it's exactly what you get in 
single-threaded code.

Compare that to right now where AFAICT pthread_sigmask() is pretty useless in 
the presence of threads, because of the thing where if one thread has a signal 
blocked then the kernel will pick another thread to deliver it to.

> From the point of view of the Python signal handler, the current "if 
> (PyThread_get_thread_ident() != main_thread) return 0;" code in the C signal 
> handler is somehow an implicit pthread_sigmask(signal.SIG_BLOCK, range(1, 
> signal.NSIG)) on all threads except of the main thread, whereas Unix gives a 
> fine control on these masks with the pthread_sigmask() function.

That code isn't in the C signal handler, it's in the code that the interpreter 
runs occasionally to check if the C signal handler has been called, right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14965] super() and property inheritance behavior

2017-03-22 Thread Mariatta Wijaya

Changes by Mariatta Wijaya :


--
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Nevada Sanchez

Nevada Sanchez added the comment:

Good point. Updated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24796] Deleting names referencing from enclosed and enclosing scopes

2017-03-22 Thread Mariatta Wijaya

Mariatta Wijaya added the comment:

Thanks for the PR, Ivan. Merged and backported to 3.5 and 3.6.

--
nosy: +Mariatta
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24796] Deleting names referencing from enclosed and enclosing scopes

2017-03-22 Thread Mariatta Wijaya

Changes by Mariatta Wijaya :


--
pull_requests: +681

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24796] Deleting names referencing from enclosed and enclosing scopes

2017-03-22 Thread Mariatta Wijaya

Changes by Mariatta Wijaya :


--
pull_requests: +680

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29728] Expose TCP_NOTSENT_LOWAT

2017-03-22 Thread Mariatta Wijaya

Mariatta Wijaya added the comment:

Thanks, Nathaniel. I merged your patch :)

--
nosy: +Mariatta
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13349] Non-informative error message in index() and remove() functions

2017-03-22 Thread Emmanuel Arias

Emmanuel Arias added the comment:

I agree with Jim Fasarakis-Hilliard this message may be change in new 3.7 
version

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29574] python-3.6.0.tgz permissions borked

2017-03-22 Thread Ned Deily

Ned Deily added the comment:

My apologies for the bad permissions in the tar file.  The newly-released 3.6.1 
tar files should not have this anomaly.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread STINNER Victor

Changes by STINNER Victor :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

The purpose of the issue is to fix memory leaks like issue #21387 (this one is 
not the best example since it seems like the leak doesn't come from a static 
variable.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Tim Peters

Tim Peters added the comment:

See also:

https://en.wikipedia.org/wiki/Hamming_weight

As that says, there are a number of languages and processors with first class 
support for a popcount function.  I've frequently implemented it in Python when 
using integers as integer bitsets (`i` is in the set if and only if bit `2**i` 
is set in the integer), which often - except for finding the cardinality - runs 
much faster than using general Python sets.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

I think that adding bitarray or bitset (or both) in the stdlib would better 
satisfy the needs. There are open issues for adding ability to read or set 
selected bits or range of bits in int or for bitwise operations on bytes. I 
think that bitarray and bitset would provide better interface for these 
operations.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Mark Dickinson

Mark Dickinson added the comment:

Many of those applications are really for bitstrings (chess bitboards, hamming 
distance), which aren't really the same thing as integers.

Nice find for the mathmodule.c case. I'd forgotten about that one (though 
according to git blame, apparently I'm responsible for checking it in). It's a 
fairly obscure corner case, though.

Overall, I'm -1 on adding this: I don't think it meets the bar of being useful 
enough to justify the extra method. I'd suggest that people needing this kind 
of efficient bitstring operation use a 3rd-party bitstring library instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Niklas Fiekas

Niklas Fiekas added the comment:

Searching popcount in Python files on GitHub yields
a considerable number of examples:

https://github.com/search?utf8=%E2%9C%93&q=popcount+extension%3Apy&type=Code

Perhaps intresting:

* In CPython itself: See count_set_bits in
  Modules/mathmodule.c

* Domain specific applications: Bitboards in Chess,
  fairly shuffling cards in Poker, comparing molecules

* Size of bitsets (see bitarray and bitsets I listed above).
  Probably for this reason also as a first class citizen
  in Redis: https://redis.io/commands/bitcount.

Probably most important:

* As the Hamming Distance:
  https://en.wikipedia.org/wiki/Hamming_distance#History_and_applications

---

Btw. not a concrete application. I just stumbled upon this.
popcnt was considered important enough to be included in the
rather limited WebAssembly instruction set:
https://github.com/WebAssembly/spec/raw/master/papers/pldi2017.pdf

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29204] Add code deprecations in ElementTree

2017-03-22 Thread Serhiy Storchaka

Changes by Serhiy Storchaka :


--
pull_requests: +679

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Mark Dickinson

Mark Dickinson added the comment:

Can you give some examples of concrete use-cases? I've spent the last six years 
or so writing scientific applications and parsing all sorts of odd binary 
formats, and haven't needed or wanted a popcount yet.

> (I am not a fan of the arbitrary return value).

Agreed: if this were implemented, I think raising ValueError would be the most 
appropriate thing to do for negative inputs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29816] Get rid of C limitation for shift count in right shift

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

The special case would be not needed if limit Python ints on 32-bit platforms 
to approximately 2**2**28. int.bit_length() could be simpler too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29816] Get rid of C limitation for shift count in right shift

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Updated the PR to divrem1-based version. The drawback is that divrem1 can fail 
with MemoryError while C long long arithmetic always works for integers of the 
size less than 1 exbibyte.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29883] asyncio: Windows Proactor Event Loop UDP Support

2017-03-22 Thread Adam Meily

New submission from Adam Meily:

I am working on a Python 3.5 project that uses asyncio on a Windows system to 
poll both UDP and TCP connections. Multiple sources online say that the Windows 
Proactor event loop, which uses I/O Completion Ports, is considerably faster 
and more efficient than the default Selector event loop. I'm using both UDP and 
TCP connections so I am stuck with the Selector event loop for the time being. 
I've seen the overhead of 128 open UDP/TCP connections on the Selector event 
loop to be near 85%, which I understand is entirely spent in Windows 
proprietary code and not the Python implementation.

I'd like to take a shot at implementing UDP support in the IOCP event loop. It 
looks like there will be a considerable amount of code shared between TCP and 
UDP IOCP so I plan on implementing UDP support directly in 
_ProactorReadPipeTransport and _ProactorBaseWritePipeTransport. I should be 
able to do this by wrapping any TCP/UDP specific function calls in a check of:


if sock.type == socket.SOCK_DGRAM:
# Do UDP stuff
elif sock.type == socket.SOCK_STREAM:
# Do TCP stuff


My initial implementation plan is to:

 - Call datagram_received() instead of data_received() when UDP data is 
available in _ProactorReadPipeTransport._loop_reading().
 - Implement BaseProactorEventLoop._make_datagram_transport().
 - Implement wrappers for WSAConnect, WSARecvFrom, and WSASendTo in _overlapped.
 - Implement sendto() and recvfrom() in IocpProactor, which will use the new 
functions in _overlapped.
 - Implement handling for UDP "connections" in IocpProactor.connect() to call 
WSAConnect(). WSAConnect() appears to always return immediately so the function 
not supporting IOCP shouldn't be an issue. We can't use ConnectEx() for UDP 
because ConnectEx() is for connection-oriented sockets only.

My project is unfortunately tied to Python 3.5. So, if possible, I'd like to 
have UDP support merged into a v3.5 release. I can fork off of master instead 
of v3.5.3 if Python 3.5 support isn't an option.

--
components: asyncio
messages: 290010
nosy: Adam Meily, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: Windows Proactor Event Loop UDP Support
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29757] The loop in utility `socket.create_connection()` swallows previous errors

2017-03-22 Thread Kostis Anagnostopoulos

Kostis Anagnostopoulos added the comment:

> This is a new feature, so we can only push it to 3.7.

As it stands now(b39d4b1c6) it hardly contains any change - just in the case of 
multiple intermediate errors AND final failure, the exception raised is a bit 
different.

AFAICS it would be a "change" for 3.7 if my one of the 3 options of my last 
comment gets implemented.

Any ideas which one to implement?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

I suggest to use my regular expression for haxadedecimals. Your regular 
expression starves from catastrophic backtracking. Compare two examples:

re.match(r'0[xX]_?[\da-fA-F]+(?:_[\da-fA-F]+)*[lL]?'+r'\b', '0x'+'0'*100+'z')
re.match(r'0[xX](?:_?[\da-fA-F]+)+[lL]?'+r'\b', '0x'+'0'*100+'z')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Nevada Sanchez

Nevada Sanchez added the comment:

Thanks, it seems I misspoke. Glad I tested it!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29866] Added datetime_diff to datetime.py.

2017-03-22 Thread Decorater

Decorater added the comment:

an url preview of the bug itself 
https://travis-ci.org/AraHaan/datetime_diff/jobs/213944228

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29878] Add global instances of int 0 and 1

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Yet one idea that can make the code simpler is make PyLong_FromLong(0) and 
PyLong_FromLong(1) never failing. I.e. require NSMALLPOSINTS not less than 2.

> Also consider adding new function PyLong_Increment.  This basic operation is 
> small pain using the current API.  It may also give a small speed benefit.

Smaller pain with using _PyLong_One and Py_SETREF().

Py_SETREF(long_obj, PyNumber_Add(long_obj, _PyLong_One));

Agree that with _PyLong_Increment() it can look better and be faster. But I 
don't know whether incrementing by 1 is enough popular operation. I have 
counted 5 cases in the stdlib (not counting tests): for enumerate, range and 
Counter.

> The problem is to make sure that singletons are created in the right order :-/

Yes, I spent much time for making empty Unicode string singleton always be 
initialized. It can be accessed at very early stage.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29866] Added datetime_diff to datetime.py.

2017-03-22 Thread Decorater

Decorater added the comment:

Yeah, I could. I did just realize that there is a bug in it though that 
sometimes if it is like 58 seconds into a minute and you sleep for 15 that the 
code then thinks an entire minute elapsed when it has not. I need to think of a 
way to bypass that bug first. I found it after running tests many times locally 
(that actually run this time). Before it even gets accepted even with support 
from them I would have to fix that bug somehow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Jim Fasarakis-Hilliard

Changes by Jim Fasarakis-Hilliard :


--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Niklas Fiekas

Changes by Niklas Fiekas :


--
pull_requests: +678

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29882] Add an efficient popcount method for integers

2017-03-22 Thread Niklas Fiekas

New submission from Niklas Fiekas:

An efficient popcount (something equivalent to bin(a).count("1")) would
be useful for numerics, parsing binary formats, scientific applications
and others.

DESIGN DECISIONS

 * Is a popcount method useful enough?
 * How to handle negative values?
 * How should the method be named?

SURVEY

gmpy calls the operation popcount and returns -1/None for negative values:

>>> import gmpy2
>>> gmpy2.popcount(-10)
-1

>>> import gmpy
>>> gmpy.popcount(-10)

>From the documentation [1]:

> If x < 0, the number of bits with value 1 is infinite
> so -1 is returned in that case.

(I am not a fan of the arbitrary return value).

The bitarray module has a count(value=True) method:

>>> from bitarray import bitarray
>>> bitarray(bin(123456789).strip("0b")).count()
16

Bitsets [2] exposes __len__.

There is an SSE4 POPCNT instruction. C compilers call the corresponding
intrinsic popcnt or popcount (with some prefix and suffix) and they
accept unsigned arguments.

Rust calls the operation count_ones [3]. Ones are counted in the binary
representation of the *absolute* value. (I propose to do the same).

Introducing popcount was previously considered here but closed for lack
of a PEP or patch: http://bugs.python.org/issue722647

Sensible names could be bit_count along the lines of the existing
bit_length or popcount for gmpy compability and to distinguish it more.

PERFORMANCE

$ ./python -m timeit "bin(123456789).count('1')"  # equivalent
100 loops, best of 5: 286 nsec per loop
$ ./python -m timeit "(123456789).bit_count()"  # fallback
500 loops, best of 5: 46.3 nsec per loop

[1] https://gmpy2.readthedocs.io/en/latest/mpz.html#mpz-functions
[2] https://pypi.python.org/pypi/bitsets/0.7.9
[3] https://doc.rust-lang.org/std/primitive.i64.html#method.count_ones

--
components: Interpreter Core
messages: 290003
nosy: mark.dickinson, niklasf
priority: normal
severity: normal
status: open
title: Add an efficient popcount method for integers
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

The problem with existing static variables are that they are not properly 
cleared. When the Python interpreter is finalized and reinitialized they can 
contain invalid references. This patch fixes this issue.

> * It requires to write "var.obj" instead of just "var" to access the Python 
> object

You can use a dynamic array of PyObject** instead of a linked list for 
collecting references to "static variables" or use gc_next/gc_refs for managing 
a linked list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Georg Brandl

Georg Brandl added the comment:

> In particular, we must have at least one digit following `0[xXbBoO]` and must 
> be before any underscores.

This is not true (but your test file does the right thing).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29878] Add global instances of int 0 and 1

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

Other common values used in C functions: empty byte string, empty Unicode 
string, empty tuple. The problem is to make sure that singletons are created in 
the right order :-/

This issue reminded me an old idea of writing a generalization of the 
_Py_IDENTIFIER() API. I created an issue #29881: Add a new private API for 
"static C variables" (_PyStaticVar) to clear them at exit. To be clear: it's 
related but different to this issue, the two issues are exclusive.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread STINNER Victor

Changes by STINNER Victor :


--
pull_requests: +677

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread STINNER Victor

Changes by STINNER Victor :


--
nosy: +eric.snow, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

2017-03-22 Thread STINNER Victor

New submission from STINNER Victor:

When I read Serhiy Storshaka's idea in issue #29878, it recalled me an old idea 
of writing a generalization of the _Py_IDENTIFIER() API.

_Py_IDENTIFIER is an API to initialize a static string variable once. The 
_Py_Identifier structure has a "next" field to create a single-linked chained 
list. It allows to clear all variables at exit in 
_PyUnicode_ClearStaticStrings().

I propose a similar API but for any PyObject* object, to be able to clear all 
static variables at exit. It should help to release all memory in Py_Finalize() 
and have a safer Python finalization.

See attached pull request for the API itself.

"Static variables" in C are variables with a limited scope: a single C file or 
a single function.

It seems like the API can remove some lines of code. Example of patch:

@@ -1452,14 +1450,14 @@ compiler_mod(struct compiler *c, mod_ty mod)
 {
 PyCodeObject *co;
 int addNone = 1;
-static PyObject *module;
-if (!module) {
-module = PyUnicode_InternFromString("");
-if (!module)
-return NULL;
+_Py_STATICVAR(module);
+
+if (_PY_STATICVAR_INIT(&module, PyUnicode_InternFromString(""))) {
+return 0;
 }
+
 /* Use 0 for firstlineno initially, will fixup in assemble(). */
-if (!compiler_enter_scope(c, module, COMPILER_SCOPE_MODULE, mod, 0))
+if (!compiler_enter_scope(c, module.obj, COMPILER_SCOPE_MODULE, mod, 0))
 return NULL;
 switch (mod->kind) {
 case Module_kind:

--

Drawbacks of the API:

* It adds one pointer per static variables, so increase the memory footprint of 
8 bytes per variable
* It requires to write "var.obj" instead of just "var" to access the Python 
object


The API doesn't try to remove duplicated objects. I consider that it's not an 
issue since functions like PyLong_FromLong() and 
PyUnicode_InternFromString("c string") already do it for us. Some functions 
create mutable variables like PyImport_Import() which creates an empty list.

--

Note: Eric Snow proposed a solution "solving multi-core Python":

* https://mail.python.org/pipermail/python-ideas/2015-June/034177.html
* http://ericsnowcurrently.blogspot.fr/2016/09/solving-mutli-core-python.html

I'm not sure if this API would help or not to implement such idea, but Eric's 
project is experimental and wasn't taken in account when designing the API.

--
components: Interpreter Core
messages: 28
nosy: haypo
priority: normal
severity: normal
status: open
title: Add a new private API for "static C variables" (_PyStaticVar) to clear 
them at exit
type: resource usage
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29847] Path takes and ignores **kwargs

2017-03-22 Thread Brett Cannon

Brett Cannon added the comment:

Then I vote for Serhiy's idea of simply raising an exception in the concrete 
subclasses when a keyword argument is given.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Jim Fasarakis-Hilliard

Jim Fasarakis-Hilliard added the comment:

That's great, make sure you also take a look at the quick start section of the 
devguide [1] if you need help in any steps :-)

[1]: https://docs.python.org/devguide/#quick-start

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29866] Added datetime_diff to datetime.py.

2017-03-22 Thread Jim Fasarakis-Hilliard

Jim Fasarakis-Hilliard added the comment:

Without stating an opinion on the change, I'd suggest first posting to 
python-ideas (unless you already did so and I missed it :-) to get an initial 
reaction from folks on your idea before coming to b.p.o.

If you have the backing of python-ideas you might have a good chance of having 
your change accepted.

--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Charles Bouchard-Légaré

Charles Bouchard-Légaré added the comment:

Thank you for confirming this.  I'll look into it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Jim Fasarakis-Hilliard

Jim Fasarakis-Hilliard added the comment:

Ah I see now, yes, neither are Reversible, Type, NewType, TYPE_CHECKING and 
DefaultDict. 

Why don't you go ahead an submit a PR for this? :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29640] _PyThreadState_Init and fork race leads to inconsistent key list

2017-03-22 Thread Charalampos Stratakis

Charalampos Stratakis added the comment:

Patch for protecting the key list while forking.

--
Added file: 
http://bugs.python.org/file46753/0001-Protect-key-list-during-fork.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29880] python3.6 install readline ,and then cpython exit

2017-03-22 Thread Zachary Ware

Zachary Ware added the comment:

This is more likely to be a bug in the 'readline' package from PyPI rather than 
in Python itself.  That package has not been updated since 2012 when it was 
renamed to 'gnureadline', and 'gnureadline' has not been updated since 2014.  
If you can still reproduce this issue with the latest 'gnureadline', please 
open an issue with that project.

--
nosy: +zach.ware
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29880] python3.6 install readline ,and then cpython exit

2017-03-22 Thread pz

New submission from pz:

python3.6 -m pip istall readline
# python3.6
Python 3.6.0 (default, Mar 21 2017, 23:23:51)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
*** glibc detected *** python3.6: free(): invalid pointer: 0x7f63bbb5b570 
***
=== Backtrace: =
/lib64/libc.so.6(+0x75f3e)[0x7f63babdaf3e]
/lib64/libc.so.6(+0x78d8d)[0x7f63babddd8d]
python3.6(PyOS_Readline+0x134)[0x44a433]




and after I unistall readline,then enter cpython,the error not occur

# python3.6
Python 3.6.0 (default, Mar 21 2017, 23:23:51)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>>

--
components: 2to3 (2.x to 3.x conversion tool)
messages: 289991
nosy: pz
priority: normal
severity: normal
status: open
title: python3.6 install readline ,and then cpython exit
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29878] Add global instances of int 0 and 1

2017-03-22 Thread Raymond Hettinger

Raymond Hettinger added the comment:

+1 for this idea.  

Also consider adding new function PyLong_Increment.  This basic operation is 
small pain using the current API.  It may also give a small speed benefit.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Nevada Sanchez

Nevada Sanchez added the comment:

The existing regular expressions weren't actually strict enough as is, so I 
made them even more correct. In particular, we must have at least one digit 
following `0[xXbBoO]` and must be before any underscores.

I have a small set of test cases to examine correctness of these regular 
expressions: https://gist.github.com/nevsan/7fc78dc61d309842406d67d6839b9861

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29878] Add global instances of int 0 and 1

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

+1. Please create a PR for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Charles Bouchard-Légaré

Charles Bouchard-Légaré added the comment:

I mean, what is not documented is 'New in 3.5.2'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Jim Fasarakis-Hilliard

Jim Fasarakis-Hilliard added the comment:

I'm guessing you might of missed it, `Text` is documented in the docs for 
Python 3.5 https://docs.python.org/3.5/library/typing.html#typing.Text :-)

--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29879] typing.Text not available in python 3.5.1

2017-03-22 Thread Charles Bouchard-Légaré

New submission from Charles Bouchard-Légaré:

The real issue here is that this is not documented in Doc/library/typing.rst.

--
assignee: docs@python
components: Documentation
messages: 289985
nosy: Charles Bouchard-Légaré, docs@python
priority: normal
severity: normal
status: open
title: typing.Text not available in python 3.5.1
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29816] Get rid of C limitation for shift count in right shift

2017-03-22 Thread Mark Dickinson

Mark Dickinson added the comment:

I much prefer the `divrem1`-based version: it makes fewer assumptions about 
relative sizes of long / long long / size_t and about the number of bits per 
digit. I'd rather not have another place that would have to be carefully 
examined in the future if the number of bits per digit changed again. Overall, 
Objects/longobject.c is highly portable, and I'd like to keep it that way as 
much as possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29878] Add global instances of int 0 and 1

2017-03-22 Thread Mark Dickinson

Mark Dickinson added the comment:

I like it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29877] compileall hangs when accessing urandom even if number of workers is 1

2017-03-22 Thread Dustin Spicuzza

Changes by Dustin Spicuzza :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28331] "CPython implementation detail:" removed when content translated

2017-03-22 Thread INADA Naoki

Changes by INADA Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28331] "CPython implementation detail:" removed when content translated

2017-03-22 Thread INADA Naoki

Changes by INADA Naoki :


--
pull_requests: +676

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29869] Underscores in numeric literals not supported in lib2to3.

2017-03-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Python uses more strong rules for underscores in numerical literals. 
Underscores are acceptable between digits and between the base prefix and 
digit. For example the regular expression for hexadecimals is 
r'0[xX]_?[\da-fA-F]+(?:_[\da-fA-F]+)*[lL]?'.

Underscores also are acceptable in the exponent of float literals:

Exponent = r'[eE][-+]?\d+(?:_\d+)*'

--
nosy: +georg.brandl, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29788] tarfile: Add absolute_path option to tarfile, disabled by default

2017-03-22 Thread Berker Peksag

Changes by Berker Peksag :


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

I created a WIP patch https://github.com/python/cpython/pull/768 to experiment 
allowing signal handlers from any threads. I expect many race conditions since 
I only removed sanity checks without adding new code to handle parallelism.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

Changes by STINNER Victor :


--
pull_requests: +675

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

thread_signal.py: Oops, I forgot to remove the "raise Exception" line from 
sighandler() (I used it for a quick test.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29871] Enable optimized locks on Windows

2017-03-22 Thread Kristján Valur Jónsson

Kristján Valur Jónsson added the comment:

Hi there.
Looking at the API docs today
(https://msdn.microsoft.com/en-us/library/windows/desktop/ms686304(v=vs.85).aspx)
it appears that the timeout case is documented.  I'm fairly sure that it wasn't 
when I implemented it.  There was a good reason for the "2" return code.  The 
idea was:  Either there was an error (-1) or we woke up.  Since there are 
spurious wakeups and stolen wakeups, the predicate must be tested again anyway. 
 The '2' return code would mean that the timeout condition should be tested by 
looking at some external clock.

Now, the api documentation is bad.  Return value is BOOL.
Nonzero means "success" (whatever that means)
Failure means ´zero´
Timeout means FALSE and GetLastError() == ERROR_TIMEOUT

If memory serves, FALSE == 0 on windows

Anyway, I've been out of this part of the code for sufficient time for details 
to be blurry.
My advise:  
1) Check that the API of the function is indeed correct.
2) If there is no bulletproof way of distinguishing timeout from normal return, 
just consider all returns normal (remember, non-error return means that we woke 
up, not that _we_ were signaled)
2) Verify that the code that is failing can indeed support spurious 
wakeups/stolen wakeups.  It used to be that the python condition variables 
didn't have this property, because of the way they were implemented.  This may 
have made people lazy.

K

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

Hum, maybe I found the root issue: the C signal handler calls 
Py_AddPendingCall() which uses a lock to protect a global static pendingcalls 
array (of 32 items). The function tries 100 times in a row to acquire to lock, 
or does nothing (returns -1) if it fails to acquire the lock.

If we start to allow signals from any thread, this shared pendingcalls array 
can quickly become a source of race conditions like deadlocks or ignored 
callbacks.

To avoid deadlocks, IMHO the best is to have a per-thread array which consumes 
512 bytes (on 64-bit, 32 items made of 2 pointers).

--

The _thread module has a strange _thread.interrupt_main() function.

--

>From the point of view of the Python signal handler, the current "if 
>(PyThread_get_thread_ident() != main_thread) return 0;" code in the C signal 
>handler is somehow an implicit pthread_sigmask(signal.SIG_BLOCK, range(1, 
>signal.NSIG)) on all threads except of the main thread, whereas Unix gives a 
>fine control on these masks with the pthread_sigmask() function.

--

The Windows part is more tricky. A Windows Event object (created by 
CreateEvent() and retrieved by _PyOS_SigintEvent()) is used to interrupt a few 
blocking functions:

* my_fgets() used by PyOS_StdioReadline() to implemented "readline" (especially 
for the REPL)
* _PyOS_WindowsConsoleReadline()
* read_console_w() of Modules/_io/winconsole.c
* time.sleep() -- only if it's the main thread and delay != 0 seconds
* _multiprocessing.SemLock.acquire() -- only if called from the main thread
* _winapi.WaitForMultipleObjects()

The event is set by the SIGINT signal handler set by Python.

Extract of pysleep() comment:

/* Allow sleep(0) to maintain win32 semantics, and as decreed
 * by Guido, only the main thread can be interrupted.
 */

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29779] New environment variable PYTHONHISTORY

2017-03-22 Thread Berker Peksag

Changes by Berker Peksag :


--
components: +Library (Lib)
nosy: +berker.peksag
stage:  -> patch review
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

Attached signal_pause_doesnt_wake_up.py is a little bit complex and not sure 
that it's reliable.

I wrote thread_signal.py which should have a more deterministic behaviour. 
Output on Linux:
---
main: main thread 140392674538560
thread: wait
main: spawn thread 140392542746368
main: sleep
main: send signal to thread
main: sleep again
Python signal handler, thread 140392674538560
main: wait thread
thread: done
main: done
---

As expected, the Python signal handler is called in the main thread.

time.sleep() in the thread *is* interrupted by SIGUSR1 (select() fails with 
EINTR), but PyErr_CheckSignals() does nothing since it's not the main thread. 
Then the sleep is automatically restarted (PEP 475).

The code works as expected, but I understand that the behaviour is surprising.

To be honest, I never understood the rationale behind "only execute signal 
handlers in the main thread". I was also surprised to no see a pthread_kill() 
in the C signal handler.

I dislike the idea of transfering the signal to the main thread from another 
thread in the C signal handler using pthread_kill(). Most code behave very 
badly when getting a signal, so getting a signal twice is likely to double the 
pain.

Moreover, pthread_sigmask() can block signals in the main thread for deliberate 
reasons.

If we should change something, I simply suggest to remove the arbitrary 
limitation from the C signal handler. I don't know the consequences yet.

--
Added file: http://bugs.python.org/file46752/thread_signal.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29870] ssl socket leak

2017-03-22 Thread Cory Benfield

Changes by Cory Benfield :


--
nosy: +Lukasa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

2017-03-22 Thread STINNER Victor

STINNER Victor added the comment:

Nathaniel Smith added the comment:
> In any case, the issue here remains that one shouldn't have to use 
> set_wakeup_fd for a signal to interrupt time.sleep etc.

CPython implements a C signal handler which sets a flag and then exits
immediately. The thread getting the signal will probably interrupts
the current blocking syscall with EINTR. CPython detects that a signal
flag was set, calls the *Python* signal handler. If the Python signal
handler raises an exception, the syscall is abandonned. Otherwise, the
syscall is restarted.

So "interrupting sleep" is only reliable if:

* there is only one thread (the "main" thread)
* the syscall is interrupted with EINTR
* the signal handler raises an exception

When you implement an event loop, raising an exception may be the best
design. In asyncio, the Python signal handler calls loop.call_soon()
to execute the final callback (3rd signal handler of the same
signal!). The rule in asynico is not call blocking syscalls, or at
least ensure that they don't block too long (ex: use aiofiles to
access the filesystem).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27863] multiple issues in _elementtree module

2017-03-22 Thread Serhiy Storchaka

Changes by Serhiy Storchaka :


--
pull_requests: +674

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29857] Provide `sys.executable_argv` for host application's command line arguments

2017-03-22 Thread Nick Coghlan

Nick Coghlan added the comment:

OK, I've changed the proposed attribute name to be `sys.executable_argv`, with 
the rationale being that it's "argv as originally seen by sys.executable, 
rather than by Python's __main__ module"

As part of documenting this, both it and the `argv` documentation can make it 
clear that they may be entirely absent if the host application doesn't set them.

--
title: Provide `sys._raw_argv` for host application's command line arguments -> 
Provide `sys.executable_argv` for host application's command line arguments

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com