[issue16565] Increase Py_AddPendingCall array size

Changes by Gregory P. Smith : -- versions: +Python 3.7 -Python 2.7, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker ___

[issue16565] Increase Py_AddPendingCall array size

Felipe Cruz added the comment: Running test_aio_read [New Thread 0x77ff7700 (LWP 20681)] [New Thread 0x761ff700 (LWP 20682)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x761ff700 (LWP 20682)] sem_post () at

[issue16565] Increase Py_AddPendingCall array size

Antoine Pitrou added the comment: Is it with a debug Python build? `PyThread_release_lock (lock=0x0)` is a bit worrying. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16565 ___

[issue16565] Increase Py_AddPendingCall array size

Felipe Cruz added the comment: Yes! 2.7.3 build with pydebug. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16565 ___ ___ Python-bugs-list

[issue16565] Increase Py_AddPendingCall array size

Felipe Cruz added the comment: Without debug backtrace #0 sem_post () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_post.S:34 #1 0x004d456e in PyGILState_Release () #2 0x759b9cdc in notify_func_wrapper (arg=0x78c0) at ../sysdeps/pthread/aio_notify.c:45 #3

[issue16565] Increase Py_AddPendingCall array size

Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16565 ___ ___ Python-bugs-list mailing list

[issue16565] Increase Py_AddPendingCall array size

Amaury Forgeot d'Arc added the comment: This array would still have a fixed size. Could you arrange your code so that you enqueue pending calls elsewhere, and call Py_AddPendingCall only once until the pending items have been processed? -- nosy: +amaury.forgeotdarc

[issue16565] Increase Py_AddPendingCall array size

Antoine Pitrou added the comment: I just took a look at https://github.com/felipecruz/pyaio/commit/127372ba0a6dbca4045256dcd653789ee35f6a78 and it looks a bit silly to me: if the completion handler is called in a separate thread, then the completion handler can just take the GIL and put the

[issue16565] Increase Py_AddPendingCall array size

Felipe Cruz added the comment: Just confirmed that signals is not a viable option. Is too slow, as Antonie already pointed. It's almost 5 times slower than with SIGEV_THREAD. The problem now is: If I use Py_AddPendingCall, the tests can hang sometimes. I can still follow Amaury suggestion

[issue16565] Increase Py_AddPendingCall array size

Antoine Pitrou added the comment: If I try to acquire the GIL PyGILState_Ensure() call PyObject_CallObject and release the GIL in the aio_completion_handler function a SEGFAULT occurs in 2.7.3 but not in 3.3 and newer.. Well, can you post a traceback? --

[issue16565] Increase Py_AddPendingCall array size

New submission from Felipe Cruz: Current pending calls limit is too small and it can be easily reached in very intensive async file io applications. There is a little hack in pyaio[1] which sleeps if Py_AddPendingCall returns 0 but It's not totally clear to me why the size of pendind calls