Evan Jones wrote:
Due to the issue of thread safety in the Python memory allocator, I have
been wondering about thread safety in the rest of the Python
interpreter. I understand that the interpreter is not thread safe, but
I'm not sure that I have seen a discussion of the all areas where this
is an issue. Here are the areas I know of:
This very much depends on the definition of "thread safe". If this mean
"working correctly in the presence of threads", then your statement
is wrong - the interpreter *is* thread-safe. The global interpreter lock
(GIL) guarantees that only one thread at any time can execute critical
operations. Since all threads acquire the GIL before performing such
operations, the interpreter is thread-safe.
1. The memory allocator.
2. Reference counts.
3. The cyclic garbage collector.
4. Current interpreter state is pointed to by a single shared pointer.
This is all protected by the GIL.
5. Many modules may not be thread safe (?).
Modules often release the GIL through BEGIN_ALLOW_THREADS, if they know
that would be safe if another thread would enter the Python interpreter.
Ignoring the issue of #5 for the moment, are there any other areas where
this is a problem? I'm curious about how much work it would be to allow
concurrent execution of Python code.
Define "concurrent". Webster's offers
1. operating or occurring at the same time
Clearly, on a single-processor system, no two activities can execute
concurrently - the processor can do at most one activity at any point
in time.
Perhaps you are asking whether it would be possible to change the
current coarse-grained lock into a more finer-grained lock (as working
without locks is not implementable). This is also known as "free
threading". There have been attempts to implement free threading, and
they have failed.
Note: One of the reasons I am asking is that my memory allocator patch
is that it changes the current allocator from "sort of" thread safe to
obviously unsafe.
The allocator is thread-safe in the presence of the GIL - you are
supposed to hold the GIL before entering the allocator. Due to some
unfortunate historical reasons, there is code which enters free()
without holding the GIL - and that is what the allocator specifically
deals with. Except for this single case, all callers of the allocator
are required to hold the GIL.
However, if it
was one of the components that permitted the interpreter to go
multi-threaded, then it would be worth it.
Again, the interpreter supports multi-threading today. Removing
the GIL is more difficult, though - nearly any container object
(list, dictionary, etc) would have to change, plus the reference
counting (which would have to grow atomic increment/decrement).
Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
