[issue29730] unoptimal calls to PyNumber_Check

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think it would be better to replace PyNumber_Check() with PyIndex_Check() and 
change error messages to more specific. In IO related functions the accepted 
values are integers and None. In bytes/bytearray methods -- integers and 
bytes-like objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29746] Update marshal docs to Python 3

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +519

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29746] Update marshal docs to Python 3

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +518

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29798] Handle "git worktree" in "make patchcheck"

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +517

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29656] Change "make patchcheck" to be branch aware

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +516

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29656] Change "make patchcheck" to be branch aware

[Nick Coghlan]

Nick Coghlan added the comment:

http://bugs.python.org/issue29798 is a follow-up issue to make the patchcheck 
script compatible with `git worktree`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29656] Change "make patchcheck" to be branch aware

[Nick Coghlan]

Changes by Nick Coghlan :


--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29798] Handle "git worktree" in "make patchcheck"

[Nick Coghlan]

New submission from Nick Coghlan:

While backporting issue 29656 to get "make patchcheck" to play nice with git PR 
branches, I discovered an incompatibility between the way "git worktree" works 
and the assumptions in "patchcheck.py".

Specifically, in a worktree, ".git" is a file, rather than a directory:

$ cat .git
gitdir: /home/ncoghlan/devel/cpython/.git/worktrees/py27

So the current isdir() check should be relaxed to just an exists() check.

--
assignee: ncoghlan
components: Demos and Tools
messages: 289481
nosy: ncoghlan
priority: normal
severity: normal
status: open
title: Handle "git worktree" in "make patchcheck"
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29797] Deadlock with multiprocessing.Queue()

[Tim Peters]

Tim Peters added the comment:

I think this is expected.  Add this as the first line of `simulate()` and the 
problem should go away:

q.cancel_join_thread()

As the docs say, a Queue works with a background thread, which feeds incoming 
data from an internal buffer to a (interprocess) pipe.  By default, a process 
using the Queue attempts to join that thread when the process exits.  But since 
you never take anything off the queue, the thread waits forever, hoping for the 
pipe to drain so it can feed in the rest of its buffer.

But see the docs for why you don't really want to use .cancel_join_thread():  
the process will just exit then, and the data in the internal buffer will most 
likely simply be lost.

A Manager.Queue doesn't have this problem because it runs in its own (Manager) 
process:  q.put() sends the data to that process at once, without buffering 
anything.  So if you have write-only Queues, that's the way to go ;-)

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29797] Deadlock with multiprocessing.Queue()

[Max]

New submission from Max:

Using multiprocessing.Queue() with several processes writing very fast results 
in a deadlock both on Windows and UNIX.

For example, this code:

from multiprocessing import Process, Queue, Manager
import time, sys

def simulate(q, n_results):
for i in range(n_results):
time.sleep(0.01)
q.put(i)

def main():
n_workers = int(sys.argv[1])
n_results = int(sys.argv[2])

q = Queue()
proc_list = [Process(target=simulate, 
args=(q, n_results),
daemon=True) for i in range(n_workers)]

for proc in proc_list:
proc.start()

for i in range(5):
time.sleep(1)
print('current approximate queue size:', q.qsize())
alive = [p.pid for p in proc_list if p.is_alive()]
if alive:
print(len(alive), 'processes alive; among them:', alive[:5])
else:
break

for p in proc_list:
p.join()

print('final appr queue size', q.qsize())


if __name__ == '__main__':
main()


hangs on Windows 10 (python 3.6) with 2 workers and 1000 results each, and on 
Ubuntu 16.04 (python 3.5) with 100 workers and 100 results each. The print out 
shows that the queue has reached the full size, but a bunch of processes are 
still alive. Presumably, they somehow manage to lock themselves out even though 
they don't depend on each other (must be in the implementation of Queue()):

current approximate queue size: 9984
47 processes alive; among them: [2238, 2241, 2242, 2244, 2247]
current approximate queue size: 1
47 processes alive; among them: [2238, 2241, 2242, 2244, 2247]

The deadlock disappears once multiprocessing.Queue() is replaced with 
multiprocessing.Manager().Queue() - or at least I wasn't able to replicate it 
with a reasonable number of processes and results.

--
components: Library (Lib)
messages: 289479
nosy: max
priority: normal
severity: normal
status: open
title: Deadlock with multiprocessing.Queue()
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26121] Use C99 functions in math if available

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Tests are failed only on s390x RHEL 3.x.

http://buildbot.python.org/all/builders/s390x%20RHEL%203.x/builds/446/steps/test/logs/stdio
==
FAIL: test_mtestfile (test.test_math.MathTests)
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_math.py", 
line 1287, in test_mtestfile
'\n  '.join(failures))
AssertionError: Failures in test_mtestfile:
  gam0003: gamma(-inf): expected 'ValueError', got nan (not equal)
  gam0045: gamma(1e-160): expected 1e+160, got 1.0063e+160 (error = 
6.24e+145 (40 ulps); permitted error = 0 or 20 ulps)
  gam0046: gamma(1e-308): expected 1e+308, got 1.0136e+308 (error = 
1.36e+294 (68 ulps); permitted error = 0 or 20 ulps)
  gam0047: gamma(5.6e-309): expected 1.7857142857142848e+308, got 
1.785714285714199e+308 (error = 8.58e+294 (430 ulps); permitted error = 0 or 20 
ulps)
  gam0065: gamma(-1e-160): expected -1e+160, got -1.0063e+160 
(error = 6.24e+145 (40 ulps); permitted error = 0 or 20 ulps)
  gam0066: gamma(-1e-308): expected -1e+308, got -1.0136e+308 
(error = 1.36e+294 (68 ulps); permitted error = 0 or 20 ulps)
  gam0067: gamma(-5.6e-309): expected -1.7857142857142848e+308, got 
-1.785714285714199e+308 (error = 8.58e+294 (430 ulps); permitted error = 0 or 
20 ulps)
  gam0081: gamma(-1.0002): expected 4503599627370495.5, got 
4503599627370484.5 (error = 11 (22 ulps); permitted error = 0 or 20 ulps)
  gam0084: gamma(-100.01): expected -7.540083334883109e-145, got 
-7.540083334882688e-145 (error = 4.21e-158 (296 ulps); permitted error = 0 or 
20 ulps)
  gam0085: gamma(-99.99): expected 7.540083334884096e-145, got 
7.540083334884402e-145 (error = 3.06e-158 (215 ulps); permitted error = 0 or 20 
ulps)
  gam0100: gamma(170.0): expected 4.269068009004705e+304, got 
4.269068009005011e+304 (error = 3.06e+291 (628 ulps); permitted error = 0 or 20 
ulps)
  gam0101: gamma(171.0): expected 7.257415615307999e+306, got 
7.257415615308882e+306 (error = 8.83e+293 (708 ulps); permitted error = 0 or 20 
ulps)
  gam0102: gamma(171.624): expected 1.7942117599248104e+308, got 
1.794211759924979e+308 (error = 1.69e+295 (845 ulps); permitted error = 0 or 20 
ulps)
  gam0120: gamma(-100.5): expected -3.3536908198076787e-159, got 
-3.353690819807666e-159 (error = 1.26e-173 (25 ulps); permitted error = 0 or 20 
ulps)
  gam0121: gamma(-160.5): expected -5.255546447007829e-286, got 
-5.255546447008121e-286 (error = 2.92e-299 (313 ulps); permitted error = 0 or 
20 ulps)
  gam0122: gamma(-170.5): expected -3.3127395215386074e-308, got 
-3.312739521538679e-308 (error = 7.16e-322 (145 ulps); permitted error = 0 or 
20 ulps)
  gam0140: gamma(-63.349078729022985): expected 4.177797167776188e-88, got 
4.177797167776136e-88 (error = 5.19e-102 (93 ulps); permitted error = 0 or 20 
ulps)
  gam0141: gamma(-127.45117632943295): expected 1.183111089623681e-214, got 
1.183111089623727e-214 (error = 4.6e-228 (223 ulps); permitted error = 0 or 20 
ulps)

--

--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

I am embarrassed! That's why I said IIRC... I remembered that either
RichCompare calls RichCompareBool, or the other way around, and I was too
lazy to check :) I did remember about do_richcompare, though!

On Sat, Mar 11, 2017 at 10:07 PM Tim Peters  wrote:

>
> Tim Peters added the comment:
>
> Elliot, PyObject_RichCompareBool calls PyObject_RichCompare.  That in turn
> does some checks, hides a small mountain of tests in the expansions of the
> recursion-checking macros, and calls do_richcompare.  That in turn does
> some useless (in the cases you're aiming at) tests and finally gets around
> to invoking tp_richcompare.  Your patch gets to that final step at once.
>
> I'm surprised you didn't know that ;-)
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

Elliot, PyObject_RichCompareBool calls PyObject_RichCompare.  That in turn does 
some checks, hides a small mountain of tests in the expansions of the 
recursion-checking macros, and calls do_richcompare.  That in turn does some 
useless (in the cases you're aiming at) tests and finally gets around to 
invoking tp_richcompare.  Your patch gets to that final step at once.

I'm surprised you didn't know that ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

Eryk Sun: Thanks for your detailed response. I'm curious, though: can you
figure out why those other two examples *didn't* fail? It's driving me
crazy! For reference: https://bugs.python.org/msg289435

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29656] Change "make patchcheck" to be branch aware

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +515

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29656] Change "make patchcheck" to be branch aware

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +514

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

Ya, that makes sense... I just don't get why it's faster at all, then!
Because if we add the (v==w) check, and the tp_richcompare check, how is
unsafe_object_compare any different from PyObject_RichComareBool? Is it
that we're saving function calls? (PyObject_RichCompareBool calls
do_richcompare, so it's one extra call IIRC).

On Sat, Mar 11, 2017 at 9:32 PM Tim Peters  wrote:

>
> Tim Peters added the comment:
>
> The impact would be small:  it would add one (or so) pointer-equality
> compare that, in practice, will always say "yup, they're equal".  Dirt
> cheap, and the branch is 100% predictable.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29763] test_site failing on AppVeyor

[Zachary Ware]

Zachary Ware added the comment:

This should be fixed by PR624 and PR625.

--
assignee:  -> zach.ware
components: +Tests
stage:  -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29763] test_site failing on AppVeyor

[Zachary Ware]

Changes by Zachary Ware :


--
resolution:  -> fixed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

The impact would be small:  it would add one (or so) pointer-equality compare 
that, in practice, will always say "yup, they're equal".  Dirt cheap, and the 
branch is 100% predictable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

It was a release build -- it would blow up in a debug build.

Now, regarding the fix you propose: I'll benchmark it tomorrow. If the
impact is small, I agree that it would be an elegant fix. If not, however,
perhaps we just have to get rid of unsafe_object_compare entirely? The
common use cases would still work just fine, and maybe we could add a case
for bytes or something to compensate for the loss.

On Sat, Mar 11, 2017 at 9:12 PM Tim Peters  wrote:

>
> Tim Peters added the comment:
>
> Elliot, did you run the example in a release build or a debug build?  I'm
> wondering why this:
>
>assert(v->ob_type == w->ob_type &&
>   v->ob_type->tp_richcompare != NULL &&
>   v->ob_type->tp_richcompare == compare_funcs.key_richcompare);
>
> didn't blow up (in `unsafe_object_compare`).
>
> If that does blow up in a debug build, it suggests "a fix":
> unconditionally check whether the tp_richcompare slot is the expected
> value.  If not, use `PyObject_RichCompareBool(v, w, Py_LT)` instead.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29763] test_site failing on AppVeyor

[Zachary Ware]

Changes by Zachary Ware :


--
pull_requests: +513

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Eryk Sun]

Eryk Sun added the comment:

> assignment of "__lt__" change the value of the tp_richcompare slot? 

Yes. CPython doesn't implement individual dispatching of the rich-comparison 
functions. There's a single tp_richcompare slot, so overriding one rich 
comparison forces the use of slot_tp_richcompare. For built-in types this 
incurs the performance penalty of using a wrapper_descriptor for the other rich 
comparisons. For example, overriding F.__lt__ forces calling float.__gt__ for 
the greater-than comparison. 

Before:

>>> F() > 1

Breakpoint 0 hit
python37_d!float_richcompare:
`6099d930 4489442418  mov dword ptr [rsp+18h],r8d
  ss:0056`cefef280=a1670058
0:000> kc 3
Call Site
python37_d!float_richcompare
python37_d!do_richcompare
python37_d!PyObject_RichCompare
0:000> g
False

After:

>>> F.__lt__ = lambda a, b: 0
>>> F() > 1

Breakpoint 0 hit
python37_d!float_richcompare:
`6099d930 4489442418  mov dword ptr [rsp+18h],r8d
  ss:0056`cefef0a0=a39d7c70
0:000> kc 9
Call Site
python37_d!float_richcompare
python37_d!wrap_richcmpfunc
python37_d!richcmp_gt
python37_d!wrapper_call
python37_d!_PyObject_FastCallDict
python37_d!call_unbound
python37_d!slot_tp_richcompare
python37_d!do_richcompare
python37_d!PyObject_RichCompare

The __gt__ wrapper_descriptor gets bound as a method-wrapper, and the 
method-wrapper tp_call is wrapper_call, which calls the wrapper function (e.g. 
richcmp_gt) with the wrapped function (e.g. float_richcompare). The object ID 
in CPython is the object address, so we can easily get the address of the 
__gt__ wrapper_descriptor to confirm how these C function pointers are stored 
in it:

>>> id(vars(float)['__gt__'])
2154486684248

0:001> ln @@(((PyWrapperDescrObject *)2154486684248)->d_base->wrapper)
(`60a20580)   python37_d!richcmp_gt   |
(`60a205c0)   python37_d!slot_tp_finalize
Exact matches:
python37_d!richcmp_gt (struct _object *, struct _object *, void *)

0:001> ln @@(((PyWrapperDescrObject *)2154486684248)->d_wrapped)
(`6099d930)   python37_d!float_richcompare   |
(`6099e6f0)   python37_d!float_getzero
Exact matches:
python37_d!float_richcompare (struct _object *, struct _object *, int)

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

Elliot, did you run the example in a release build or a debug build?  I'm 
wondering why this:

   assert(v->ob_type == w->ob_type &&
  v->ob_type->tp_richcompare != NULL &&
  v->ob_type->tp_richcompare == compare_funcs.key_richcompare);

didn't blow up (in `unsafe_object_compare`).

If that does blow up in a debug build, it suggests "a fix":  unconditionally 
check whether the tp_richcompare slot is the expected value.  If not, use 
`PyObject_RichCompareBool(v, w, Py_LT)` instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

On Sat, Mar 11, 2017 at 9:01 PM Tim Peters  wrote:

>
> Elliot, I don't care if the example behaves differently.  Although someone
> else may ;-)
>
> The only things `.sort()` has ever tried to guarantee in the presence of
> mutations (of either the list or the elements) during sorting are that (a)
> the implementation won't segfault; and, (b) the list at the end is _some_
> permutation of the input list (no elements are lost or duplicated).
>
> If crazy mutation examples can provoke a segfault, that's possibly "a
> problem" - but different results really aren't (at least not to me).
>
>
That's great to hear. (Of course, one could always remove
unsafe_object_compare from the patch and keep the rest, but that would be a
real shame). I don't think segfaults are possible if the code is
pure-Python, because all the builtin/stdlib functions type-check anyway, so
you would just get an exception. Right? Of course, using the C API you
could probably provoke segfaults, but there are much easier ways to
segfault using the C API :).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

Elliot, I don't care if the example behaves differently.  Although someone else 
may ;-)

The only things `.sort()` has ever tried to guarantee in the presence of 
mutations (of either the list or the elements) during sorting are that (a) the 
implementation won't segfault; and, (b) the list at the end is _some_ 
permutation of the input list (no elements are lost or duplicated).

If crazy mutation examples can provoke a segfault, that's possibly "a problem" 
- but different results really aren't (at least not to me).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

@ppperry, I have no idea what the bulk of the code in typeobect.c is trying to 
do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

I just ran it. With the patched interpreter, I get no error. With the
unpatched interpreter, I get ValueError. :(.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[ppperry]

ppperry added the comment:

Wouldn't the assignment of "__lt__" change the value of the tp_richcompare 
slot? That seems to be what the code in Objects/typeobject.c is doing with  the 
update_slot method and the related helper functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29763] test_site failing on AppVeyor

[Zachary Ware]

Changes by Zachary Ware :


--
pull_requests: +512

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29794] Incorrect error message on invalid __class__ assignments

[Steven D'Aprano]

Steven D'Aprano added the comment:

Your example works because random is a module:

py> from types import ModuleType
py> import random
py> type(random) is ModuleType
True

Since random is an instance of ModuleType, your class M is a subclass of 
ModuleType, and assigning to random.__class__ is allowed.

I'm closing this issue, but if can demonstrate an actual problem, please feel 
free to re-open it.

--
nosy: +steven.daprano
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Tim Peters]

Tim Peters added the comment:

I haven't tried the example, but at this point I'd be surprised if it failed.  
The caching here isn't at level of `__lt__` but at the higher level of 
(invisible from Python code) a type's tp_richcompare slot.  A heap type - 
regardless of whether it derives from a C-level or Python-level type - has to 
be prepared for methods to pop into existence, change, or vanish at (almost) 
any time.  So its tp_richcompare has to be defensive itself.  For example:

>>> class F(float):
... pass
>>> a = F(2)
>>> b = F(3)
>>> a < b
True

Is F.tp_richcompare the same as float.tp_richcompare?  We can't tell from 
Python code, because tp_richcompare isn't exposed.  But, _whatever_ 
F.tp_richcompare is, it notices when relevant new methods are defined (which 
float.tp_richcompare emphatically does not); for example, continuing the above:

>>> F.__lt__ = lambda a, b: 0
>>> a < b
0
>>> del F.__lt__
>>> a < b
True

That said, I know nothing about how comparison internals changed for Python 3, 
so I may just be hallucinating :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29796] test_weakref hangs on AppVeyor (2.7)

[Zachary Ware]

New submission from Zachary Ware:

See PR493 (https://ci.appveyor.com/project/python/cpython/build/2.7.13+.184) 
for an example.  I'd rather not merge PR493, which adds AppVeyor to 2.7, until 
this is resolved.

--
components: Windows
messages: 289461
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: test needed
status: open
title: test_weakref hangs on AppVeyor (2.7)
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29688] Add support for Path.absolute()

[Eryk Sun]

Eryk Sun added the comment:

What's the rationale for not calling self._flavour.pathmod.abspath() to 
implement absolute()? For example:

>>> p = pathlib.Path('C:/con')
>>> p._flavour.pathmod.abspath(p)
'.\\con'
>>> p._from_parts((p._flavour.pathmod.abspath(p),), init=False)
WindowsPath('//./con/')

That's almost right except for an unrelated problem that pathlib shouldn't 
append a trailing slash for \\.\ local device paths. Doing so creates a 
different path, which may be invalid. \\.\con is a symbolic link to 
\Device\ConDrv\Console, and adding a trailing backslash after the "Console" 
filename is invalid. An example where the resulting path is valid but wrong is 
the volume device \\.\C:, which is a link to something like 
\Device\HarddiskVolume2. Appending a backslash refers to the root directory of 
the file system on the volume.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29795] Clarify how to share multiprocessing primitives

[Max]

Max added the comment:

How about inserting this text somewhere:

Note that sharing and synchronization objects (such as `Queue()`, `Pipe()`, 
`Manager()`, `Lock()`, `Semaphore()`) should be made available to a new process 
by passing them as arguments to the `target` function invoked by the `run()` 
method. Making these objects visible through global variables will only work 
when the process was started using `fork` (and as such sacrifices portability 
for no special benefit).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28624] Make the `cwd` argument to `subprocess.Popen` accept a `PathLike`

[Zachary Ware]

Changes by Zachary Ware :


--
pull_requests: +511

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29730] unoptimal calls to PyNumber_Check

[Oren Milman]

Changes by Oren Milman :


--
pull_requests: +510

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29730] unoptimal calls to PyNumber_Check

[Oren Milman]

Oren Milman added the comment:

after some closer examination, ISTM that in Objects/exceptions.c, we can't
remove PyNumber_Check to optimize or simplify the code, as the argument
'filename' can be either an integer type (only in case the error is a
BlockingIOError), or quite anything else, except for None.

We could replace PyNumber_Check(filename) with PyIndex_Check(filename), but
this would change the behavior of BlockingIOError.
IMHO, this isn't that important, and thus we should leave the code in
Objects/exceptions.c as is.

anyway, I would soon create a pull request to remove all other
aforementioned calls to PyNumber_Check.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26121] Use C99 functions in math if available

[Mark Dickinson]

Mark Dickinson added the comment:

So the GitHub branch fails on my OS X 10.10 machine: there are a number of test 
failures for gamma and lgamma (but none for erf and erfc).

Some of the gamma and lgamma test failures are accuracy issues; some are more 
likely to do with errno handling: OS X probably isn't setting errno appropriate 
on domain or range errors, so any implementation using the libm lgamma/tgamma 
would need to add code for error handling.

Given this, I'd prefer to leave lgamma and gamma as-is, using their current 
implementations, at least on OS X; I don't regard the accuracy loss in using 
the libm functions to be within acceptable limits.

There may still be a case for using the libm versions of erf and erfc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29795] Clarify how to share multiprocessing primitives

[Davin Potts]

Davin Potts added the comment:

On Windows, because that OS does not support fork, multiprocessing uses spawn 
to create new processes by default.  Note that in Python 3, multiprocessing 
provides the user with a choice of how to create new processes (i.e. fork, 
spawn, forkserver).

When fork is used, the 'q = Queue()' in this example would be executed once by 
the parent process before the fork takes place, the resulting child process 
continues execution from the same point as the parent when it triggered the 
fork, and thus both parent and child processes would see the same 
multiprocessing.Queue.  When spawn is used, a new process is spawned and the 
whole of this example script would be executed again from scratch by the child 
process, resulting in the child (spawned) process creating a new Queue object 
of its own with no sense of connection to the parent.


Would you be up for proposing replacement text to improve the documentation?  
Getting the documentation just right so that everyone understands it is worth 
spending time on.

--
nosy: +davin
stage:  -> needs patch
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29514] Add a test case that prevents magic number changes in minor releases

[Eric Appelt]

Eric Appelt added the comment:

Thank you for the review and suggestions Nick, Serhiy, and all. I believe that 
I have implemented the suggestions here and on github into a new commit on my 
pull request which moves the test into an existing module and removes the 
notion of a table in favor of a single expected value.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29795] Clarify how to share multiprocessing primitives

[Max]

New submission from Max:

It seems both me and many other people (judging from SO questions) are confused 
about whether it's ok to write this:

from multiprocessing import Process, Queue
q = Queue()

def f():
q.put([42, None, 'hello'])

def main():
p = Process(target=f)
p.start()
print(q.get())# prints "[42, None, 'hello']"
p.join()

if __name__ == '__main__':
main()

It's not ok (doesn't work on Windows presumably because somehow when it's 
pickled, the connection between global queues in the two processes is lost; 
works on Linux, because I guess fork keeps more information than pickle, so the 
connection is maintained).

I thought it would be good to clarify in the docs that all the Queue() and 
Manager().* and other similar objects should be passed as parameters not just 
defined as globals.

--
assignee: docs@python
components: Documentation
messages: 289454
nosy: docs@python, max
priority: normal
severity: normal
status: open
title: Clarify how to share multiprocessing primitives
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29790] Optional use of /dev/random on linux

[STINNER Victor]

STINNER Victor added the comment:

You can already use os.getrandom() if you want /dev/urandom. And as you
wrote, it is easy to inherit from SystemRandom if you need something else
than bytes.

I am strongly opposed to use /dev/random because most users misunderdood
RNG znd so will use /dev/random for *bad* reason.

There is no need to extend the stdlib. I suggest yo close this issue as
WONTFIX.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29793] Convert some builtin types constructors to Argument Clinic

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

str, bytes and bytearray still are not converted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29688] Add support for Path.absolute()

[Brett Cannon]

Changes by Brett Cannon :


--
title: Add support for Path.absolute -> Add support for Path.absolute()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29688] Document Path.absolute

[Brett Cannon]

Brett Cannon added the comment:

I know it has it's uses (avoiding stat calls is one of them), but it is still 
undocumented, untested, and has two comments in it saying it needs work. 
Because of all that it might as well not exist since it doesn't meet our 
standards of quality.

If someone wants to fix all those issues then we can properly document it as 
supported, but if no one is willing to then I don't think we should leave 
unsupported code lying around that people might discover through dir().

And it doesn't serve a _different_ purpose compared to resolve(), it serves a 
_subset_ of resolve()'s purpose since resolve() calls absolute() 
unconditionally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29688] Add support for Path.absolute

[Brett Cannon]

Changes by Brett Cannon :


--
title: Document Path.absolute -> Add support for Path.absolute

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29793] Convert some builtin types constructors to Argument Clinic

[STINNER Victor]

STINNER Victor added the comment:

Cool. You can close my issue, this one supersed it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29722] heapq.merge docs are misleading with the "reversed" flag

[Raymond Hettinger]

Raymond Hettinger added the comment:

The PR was applied to 3.7.  I don't think it is worth backing.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29794] Incorrect error message on invalid __class__ assignments

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29794] Incorrect error message on invalid __class__ assignments

[ppperry]

New submission from ppperry:

If you try to set the __class__ of a type which doesn't support "__class__" 
assignments, you get the error message:

TypeError: __class__ assignment only supported for heap types or ModuleType 
subclasses

However, the actual restriction doesn't require a subclass of "ModuleType"; the 
below code works:

import random
class M(type(random)):pass
random.__class__ = M

Thus the error message is incorrect.

--
components: Interpreter Core
messages: 289448
nosy: ppperry
priority: normal
severity: normal
status: open
title: Incorrect error message on invalid __class__ assignments
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[ppperry]

ppperry added the comment:

What about if one of the relevant comparison functions is implemented in C?

class WackyComparator(int):
def __lt__(self, other):
elem.__class__ = WackyList2
return int.__lt__(self, other)

class WackyList1(list):pass
class WackyList2(list):
def __lt__(self, other):
raise ValueError
lst =
list(map(WackyList1,[[WackyComparator(3),5],[WackyComparator(4),6],[WackyComparator(7),7]]))
random.shuffle(lst)
elem = lst[-1]
lst.sort()

This code raises ValueError, and caching seems like it would cache the
comparator for WackyList1 objects, which is the same as the comparator for
'list' objects -- and midway through comparison, one of them changes type
to WackyList2, which has its own (broken) comparison function.

Python is very very dynamic ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27200] make doctest in CPython has failures

[Marco Buttu]

Changes by Marco Buttu :


--
pull_requests: +509

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29793] Convert some builtin types constructors to Argument Clinic

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
components: +Argument Clinic
nosy: +larry
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29793] Convert some builtin types constructors to Argument Clinic

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +508

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29793] Convert some builtin types constructors to Argument Clinic

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Following PR converts some builtin types constructors to Argument Clinic:

complex.__new__
float.__new__
function.__new__
int.__new__
mappingproxy.__new__
module.__init__
property.__init__
structseq.__new__

--
components: Interpreter Core
messages: 289446
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Convert some builtin types constructors to Argument Clinic
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29790] Optional use of /dev/random on linux

[Nick Coghlan]

Nick Coghlan added the comment:

To provide some additional context to that answer: the problem isn't with folks 
wanting direct access to their hardware entropy devices as such.

There are plenty of options for that (such as exposing it as a file descriptor 
distinct from both /dev/random and /dev/urandom), but going through /dev/random 
(or calling "os.getrandom(n, os.GRND_RANDOM)") isn't one of them.

Instead, /dev/random consumes the same kernel CSPRNG that /dev/urandom does, it 
just adds an extra check to make it block when the kernel's collected entropy 
estimate happens to be low.

So this isn't something we want to expose or explain to Python users in general 
- instead, it's something that only folks doing highly specialised work 
involving hardware security modules might be interested in, and hence can be 
safely left to third party tools and frameworks, rather than being something 
that would ever be shipped by default as part of the reference interpreter and 
standard library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29790] Optional use of /dev/random on linux

[Nick Coghlan]

Nick Coghlan added the comment:

This RFE is unfortunately based on some incorrect assumptions about the way 
Linux kernels use CSPRNGs and entropy pools. Using /dev/random isn't magically 
more secure than /dev/urandom, it just makes your applications less reliable 
for no good reason.

Folks are free to do that through an extension module if they really wish to do 
so, but it's not an option we're interested in supporting in the standard 
library.

This is a good article providing some additional background on that topic: 
http://www.2uo.de/myths-about-urandom/

There was one genuine problem with /dev/urandom (it could return potentially 
predictable values if the entropy pool hadn't been adequately seeded), but 
Victor addressed that in PEP 524 by switching to the blocking variant of the 
getrandom() syscall (when available) rather than using the file descriptor 
interface.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29456] bug in unicodedata.normalize: u1176, u11a7 and u11c3

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +lemburg, loewis
stage:  -> patch review
type:  -> behavior
versions: +Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29456] bug in unicodedata.normalize: u1176, u11a7 and u11c3

[Wonsup Yoon]

Changes by Wonsup Yoon :


--
title: bug in unicodedata.normalize: u1176 -> bug in unicodedata.normalize: 
u1176, u11a7 and u11c3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29792] "Fatal Python error: Cannot recover from stack overflow." from pure Python code

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> duplicate
superseder:  -> Interpreter aborts when chaining an infinite number of 
exceptions

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29792] "Fatal Python error: Cannot recover from stack overflow." from pure Python code

[David MacIver]

David MacIver added the comment:

So it does. My apologies. I'll close this.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29792] "Fatal Python error: Cannot recover from stack overflow." from pure Python code

[Xiang Zhang]

Xiang Zhang added the comment:

Looks same as #6028.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29792] "Fatal Python error: Cannot recover from stack overflow." from pure Python code

[David MacIver]

New submission from David MacIver:

When run under Python 3.6.0 or 3.5.1 (and presumably other versions of Python 
3) the attached code fails with "Fatal Python error: Cannot recover from stack 
overflow." then aborts with a core dump and an error code indicating it got a 
SIGABRT.

On Python 2.7 it instead hangs indefinitely.

Obviously this code is stupid and shouldn't be expected to do anything very 
reasonable - It's shrunk down from what was probably just a bug on my end in a 
larger example - but it seemed like it might be symptomatic of a more general 
class of problems.

--
files: recursionerror.py
messages: 289441
nosy: David MacIver
priority: normal
severity: normal
status: open
title: "Fatal Python error: Cannot recover from stack overflow." from pure 
Python code
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46720/recursionerror.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

PR 614: Objects/tupleobject.c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +507

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com