[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-07-10 Thread Phil Frost
Phil Frost added the comment: Not yet. I had a hell of a time getting gdb to follow gunicorn forks so it would watch the workers but not also every other thing using popen or os.system(). And the lldb package on Alpine doesn't seem to work. So we're currently in the process of testing

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-07 Thread Phil Frost
Phil Frost added the comment: I'm afraid that won't be of much use since the object is statically allocated. Besides, the trouble isn't finding what created the object, but what mutated an int object which should be immutable. -- ___ Python

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-06 Thread Phil Frost
Phil Frost added the comment: Probably too many C extensions to feasibly audit them all. Also we can't rule out something busted in Alpine. So I'm going to set a watchpoint on small_ints[1]->ob_ival with a little gdb script to dump core right when it changes; hopefully inspect

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-06 Thread Phil Frost
Phil Frost added the comment: So this is uminteresting (gdb) p small_ints[0] $2 = (PyIntObject *) 0x558ce65df528 (gdb) p *small_ints[0] $3 = {ob_refcnt = 18, ob_type = 0x7fe019b694c0 , ob_ival = -5} (gdb) p *small_ints[1] $4 = {ob_refcnt = 65, ob_type = 0x7fe019b694c0 , ob_ival = -3

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-06 Thread Phil Frost
Phil Frost added the comment: > Alpine Linux apparently uses musl. Is that well supported and tested? Heh. Not really. Switching to a Debian container is probably just about as much work as building Python from source. Would that be prefera

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-06 Thread Phil Frost
Phil Frost added the comment: skrah: Yes, that's correct. Since I can only produce this bug in production it will take me some days to build and validate a source build. But absent any better ideas, I will try. tim.peters: I've observed this bug across hundreds of EC2 hosts, in dozens

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-05 Thread Phil Frost
Phil Frost added the comment: I should note that while we do observe this issue with Decimal, my troubleshooting so far indicates the logic in decimal.py is fine. It seems instead the instructions in decimal.py aren't being correctly executed. I'm not sure if the problem is limited to only

[issue37168] Decimal divisions sometimes 10x or 100x too large

2019-06-05 Thread Phil Frost
New submission from Phil Frost : We've observed instances of Decimal divisions being incorrect a few times per day in a web application serving millions of requests daily. I've been unable to reproduce the issue but have been investigating core dumps which suggest either some issue in the C