Hi,
if we look at Jython we already have a GIL less implementation of the
Python programming language. I already had a project a few years ago
that was only possible (to be exact: much simpler and therefore much
cheaper) because I could utilize all 16 cores of a server using multiple
threads.
IMHO the GIL is an implementation detail, that is almost invisible
(except for performance, ctypes etc) for the Python developer. Therefore
it is unrelated to the Stackless Python API. Of course we had to adapt
the C-level implementation of Stackless.
But maybe we could base our tasklets on a more low-level construct, if
we have to rework our implementation anyway. I remember Christians(?)
mail about continuelets a few month ago.
Cheers
Anselm
Am 17.04.2014 16:20, schrieb Kristján Valur Jónsson:
Hi there.
At PyCon I was approached by a core developer of CPython who asked me this:
If we had a hypothetical GIL-less CPython, would stackless (tasklets or
co-routines) still be a useful constructs or would real threads be sufficient?
The question is serious and it is possible that a re-architecture of CPython will take
place. This would involve breaking API compatibility in some ways. The question would
be, if it would make sense for such a hypothetical Python to be fully non-recursive in
its execution form, i.e. "stackless". The way CPython uses the c-stack to
mirror recursion on the Python stack is very convenient for the C API.
So anyway, I couldn't answer that question right away and had to think it over.
Here are some of my thoughts so far:
1) Scalability. The reason that libraries like "gevent" exist, to
provide a thread-like execution environment for concurrency. You don't get added
parallelism with gevent over regular threads but you get a program that scales better
because OS threads are expensive.
2) Picklability: In order to pickle (and restore) a python execution frame
it must not be from a recursive invocation
3) Co-operative scheduling: To be able to run multiple execution contexts
without pre-emptive switching is a huge win. If your problem is such that you are
IO bound anyway and cpu doesn"t matter much then this can be a very attractive
programming model.
Any other thoughts? How would applications of stackless python be different if
there were no GIL, i.e. threads could run in parallel (but tasklets were
co-operatively scheduled in each thread as before)? For one thing, I know that
many synchronization privmitives currently written using channels would need
re-writing. They currently rely on tasklet.atomic for atomicitiy which also
prevents thread switching (by preventing the thread from releasing the GIL).
If there were no GIL, we would need to use additional locking if we wanted our
stackless locking primitives (e.g. stacklesslib.locks) to work between tasklets
of different threads.
Food for thought, anyway.
K
Hi there.
At PyCon I was approached by a core developer of CPython who asked me this:
If we had a hypothetical GIL-less CPython, would stackless (tasklets or
co-routines) still be a useful constructs or would real threads be
sufficient?
The question is serious and it is possible that a re-architecture of
CPython will take place. This would involve breaking API compatibility
in some ways. The question would be, if it would make sense for such a
hypothetical Python to be fully non-recursive in its execution form,
i.e. “stackless”. The way CPython uses the c-stack to mirror recursion
on the Python stack is very convenient for the C API.
So anyway, I couldn’t answer that question right away and had to think
it over. Here are some of my thoughts so far:
1) Scalability. The reason that libraries like “gevent” exist, to
provide a thread-like execution environment for concurrency. You don’t
get added parallelism with gevent over regular threads but you get a
program that scales better because OS threads are expensive.
2)Picklability: In order to pickle (and restore) a python execution
frame it must not be from a recursive invocation
3)Co-operative scheduling: To be able to run multiple execution
contexts without pre-emptive switching is a huge win. If your problem
is such that you are IO bound anyway and cpu doesn”t matter much then
this can be a very attractive programming model.
Any other thoughts? How would applications of stackless python be
different if there were no GIL, i.e. threads could run in parallel (but
tasklets were co-operatively scheduled in each thread as before)? For
one thing, I know that many synchronization privmitives currently
written using channels would need re-writing. They currently rely on
tasklet.atomic for atomicitiy which also prevents thread switching (by
preventing the thread from releasing the GIL). If there were no GIL, we
would need to use additional locking if we wanted our stackless locking
primitives (e.g. stacklesslib.locks) to work between tasklets of
different threads.
Food for thought, anyway.
K
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
--
Dipl. Phys. Anselm Kruis science + computing ag
Senior Solution Architect Ingolstädter Str. 22
email [email protected] 80807 München, Germany
phone +49 89 356386 874 fax 737 www.science-computing.de
--
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
