[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[STINNER Victor]

STINNER Victor  added the comment:

See also https://www.python.org/dev/peps/pep-0651/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren
Do you have any ideas to resume this issue again?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Zackery Spytz]

Zackery Spytz  added the comment:

It seems that this issue is a duplicate of bpo-25518.

--
nosy: +ZackerySpytz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

FYI, I've updated the patch to use rlim_cur() on the main thread when 
pthread_get_stacksize_np() is not accurate. This way works pretty well on my 
local machine.
Enjoy your travel!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Sorry about the slow response.

The patch looks OK, although I'm entirely happy yet about the workaround for 
the stack size on the main thread. Primarily because framework builds use a 
non-standard stack size as the default one is too small for the recursion limit 
(IIRC especially with debug builds). 

BTW. My own use case for this feature is primarily the use of python on threads 
created by the system (such as threads created by Cocoa APIs and libdispatch), 
for threads created in Python we can (and do) create them with a large enough 
stack (although this guard is helpful there too).

My next step will be to test the patch locally. This might take a while because 
I will be traveling the rest of this month. Worst case is that I'll report back 
during EuroPython 2018 (starting at 23th of July)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

benchmark result:

22.63911402298 secs  master (+0%)
22.806662271 secs PR 8046 (-0.74%)

def fib(n):
if n == 0:
return 0
if n == 1:
return 1
return fib(n-2) + fib(n-1)

if __name__ == '__main__':
import timeit
print(timeit.timeit("fib(10)", setup="from __main__ import fib"))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren

I updated the PR

Please take a look.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren

I updated the PR and we need some discussion.
I benchmarked new patch with Fibonacci codes.

baseline(master): 21.330535484 
without call sysctlbyname(PR 8046): 22.857963209 secs
with call sysctlbyname(PR 8046): 37.125129114 secs

So my choice is just believe pthread_get_stacksize_np() like rust do. 
(https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/thread.rs#L246)

code:

def fib(n):
if n == 0:
return 0
if n == 1:
return 1
return fib(n-2) + fib(n-1)

if __name__ == '__main__':
import timeit
print(timeit.timeit("fib(10)", setup="from __main__ import fib"))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Ronald Oussoren]

Ronald Oussoren  added the comment:

W.r.t. benchmarks: The full benchmark suite that's often used to assess the 
performance impact on real world code is: http://pyperformance.readthedocs.io

Running this takes some time, and the interesting bit is comparing the 
performance with and without your patch.

You could also test with a small script that uses the timeit module to run some 
code that performs a lot of function calls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren

1. I will try to run a benchmark is there any good benchmark to try it?
2. As far as I know, OS X 10.9 Mavericks has an issue with 
pthread_get_stacksize_np, I added a new commit on PR with this issue.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Have you looked at benchmark results for this patch?

In particular, PyOS_CheckStack is called quite often and I wonder how those 
calls affect performance. If there is a clear performance impact it would be 
useful to store some information in the python thread state.

In what versions of macOS are there problems with pthread_get_stacksize_np() on 
the main thread?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren

I wrote a first proto type PR and it works well on my MacBook Pro.
As you commented, there is a issue with pthread_get_stacksize_np when 
pthread_main_np() == 1 on some old mac system.

I will take a look more deeply at this issue.
If you have any feedback for my first PR please left it. 

Thanks,
Dong-hee

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Change by Dong-hee Na :


--
keywords: +patch
pull_requests: +7655
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Sure. I'm not working on this at the moment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Dong-hee Na]

Dong-hee Na  added the comment:

@ronaldoussoren

Hi, Ronald

Can I take a look this issue?

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33955] Implement PyOS_CheckStack on macOS using pthread_get_stack*_np

[Ronald Oussoren]

New submission from Ronald Oussoren :

The (non-portable) pthread APIs "pthread_get_stacksize_np()" and 
"pthread_get_stackaddr_np()" can be used to implement PyOS_CheckStack on macOS, 
which would give nicer behavior than crashing when the recursion limit is too 
high for the stack size of the current thread.

Creating a patch should be fairly easy, but does require C knowledge.

--
components: Interpreter Core
messages: 320415
nosy: ronaldoussoren
priority: low
severity: normal
stage: needs patch
status: open
title: Implement PyOS_CheckStack on macOS using pthread_get_stack*_np
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com