On 08/15/2011 11:42 AM, mark florisson wrote:
On 15 August 2011 02:22, Nikolaus Rath<nikol...@rath.org> wrote:
mark florisson<markflorisso...@gmail.com> writes:
On 14 August 2011 17:50, Nikolaus Rath<nikol...@rath.org> wrote:
mark florisson<markflorisso...@gmail.com> writes:
On 12 August 2011 20:23, Nikolaus Rath<nikol...@rath.org> wrote:
Hello,
The following segfault is completely incomprehensible to me:
(gdb) bt full
#0 sem_post () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_post.S:34
No locals.
#1 0x000000000050e3e7 in PyThread_release_lock (lock=0x0) at
../Python/thread_pthread.h:374
thelock = 0x0
status = 0
error = 0
#2 0x00000000004c4f7b in PyEval_ReleaseLock () at ../Python/ceval.c:262
No locals.
#3 0x00000000004fbeef in PyThreadState_DeleteCurrent () at
../Python/pystate.c:319
tstate = 0x925da0
#4 0x00000000004fc6a3 in PyGILState_Release (oldstate=PyGILState_UNLOCKED)
at ../Python/pystate.c:651
tcur = 0x925da0
__PRETTY_FUNCTION__ = "PyGILState_Release"
[...]
Why am I getting an error so deep in the python internals?
The error is deep because Python needs to do certain things. So what
it's doing here is destructing your thread state, which means that was
the last call to PyGILState_Ensure(). This means your function was
called from another thread than a Python thread. It looks to me like
the GIL is not initialized (__pyx_gilstate_save =
PyGILState_UNLOCKED). Are you sure you have called
PyEval_EvalInitThreads()?
No, I have not. But isn't Cython supposed to do this automatically when
entering the function if I define the function as "with gil"?
No, it acquires the GIL, but it does not attempt to initialize it.
Seeing how many people have issues with it, I'm thinking that perhaps
we should if the module contains with gil functions, or with gil
blocks in nogil functions. If you're using cython.parallel and with
gil blocks, the GIL will be initialized at module-load time. Note that
initializing the GIL slows down serially executed Python code, so
that's why Cython doesn't initialize it unconditionally. It will have
to be called at module-load time, as it should be called in the main
python thread (or from any other python thread started by the
thread(ing) module).
Oh, wow. That's a tricky one. And it explains why I have the segfault
only in a reduced testcase, but not in the full blown application (the
application probably creates a Python thread before calling into Cython
code). I banged my head against the wall for quite some time already
because of that.
Thanks a lot! It would certainly have cost me a *lot* of time if'd had
to figure that out myself.
I wouldn't mind if Cython inserted the call automatically, but a big
warning in
http://docs.cython.org/src/userguide/external_C_code.html#acquiring-and-releasing-the-gil
(under the "acquiring the GIL") might already be enough.
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
Definitely, I'll do that one of these days. I do think it's rather
common to have with gil functions called as a callback from another
(non-python) thread. I'm not so sure about with gil blocks in nogil
functions.
@Cython-dev: Do we merely want to update the docs, or do we want to
initialize the GIL for either case, or only for the with gil
functions? I'm not entirely sure what the overhead is for
single-threaded code, but I'd say we need to initialize it for both
cases.
I'm not sure if I like magic related to having used "with gil" on a
function signature -- it's a bit too magic and obscure.
I think we should have another directive for this as well as the #ifdef
macro (that is, the directive controls the default value of the C
macro), and document that a bit better.
As for the default value of that, here's two proposals:
Alt. 1) Always initialize the GIL by default.
Alt. 2) Issue a warning whenever a with-gil function is present unless
the directive is explicitly set (one way or the other). Perhaps use the
same warning if "with nogil" is used as well.
I'm in favour of 1), since it would save a lot of frustration among
users, since alt 2) is a rather minor case only costing a little bit of
performance (?), and since in many of the cases where the GIL should not
be initialized, the multithreading support is somewhat likely to turned
off at the CPython level anyway during compilation (?).
Dag Sverre
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
