As I told Andrew just right now,
I am not sure if we should instrument STM ourselves.
The matter is quite involved, and already pretty hard for PyPy.
Stackless on top of CPython is much more limited and should investigate
other ways which are orthogonal to STM and can be used in addition.
I'm thinking of something that is already available more or less, but
was never used:
In my post below, I talked about non-shared.
Tonight I had the even simpler idea of using different memory pools
and interpreter instances, together with disjoint globals, but still in
the same process space.
Threads, as they are today in Python, together with the globals and
how everything can reach everything, are the reason why we even _have_
the GIL problem.
In GIL-less Python we don't have this problem if we avoid normal threads.
There are no super-globals across mtasklets which can be freely accessed,
but everything must go through
some enforced protocol between the isolated threadpools.
So if you just use mtasklets and no normal threads at all, you can run
each of these super-threads in its own core, until you need to communicate
between these for some data exchange.
Still very vague, and still no idea how this works with resources like files
and GUI frameworks. That needs some exploration.
I need to play a bit more with the implications...
comments/ideas welcome! - Chris
On 02.01.14 14:31, Kristján Valur Jónsson wrote:
Sounds exciting.
I wonder how much of the STM work can be salvaged and made to work for us?
Are you thinking about “automatic parallelism” like Armin Rigo has
been thinking of, or just explicit tasklets that can, by nature of
just using non-shared objects, can operate gil-less?
non-shared is useful, but remember how everything in python Is
interconnected. So, we might have to put practical limits on
reachability.
For example, a tasklet’s execution frame and hence its locals can be
reached from anyone that has a reference to it. And I’m sure that you
don’t want to mark locals and function arguments as non-shared.
This incredible generalism of Python is what makes it also so hard to
do any optimizations J. Time to remove some of that, perhaps?
K
*From:*[email protected]
[mailto:[email protected]] *On Behalf Of *Christian Tismer
*Sent:* 31. desember 2013 00:56
*To:* The Stackless Python Mailing List
*Subject:* Re: [Stackless] pi-ton 2.9 / Gillesspie
-----------
I want to go for a GIL-Less Python.
That Python uses thread- (or better tasklet-) local storage instead of
the globals.
To make that possible, a completely new storage class needs to be
installed,
probably with a new keyword. This storage class will be automatically used
by tasklets. They will use copy-on-write semantics to make all
modifications
local.
The namespace is named "non-shared".
The idea is to let things by default happen only locally, in a
sub-interpreter.
Objects that are not in the local namespace are installed copy-on-write,
creating a non-shared object after being touched.
The info about being sharable or non-shared is a new attribute for every
object. The implementation is undecided, but may be implicit by using
the memory pool information.
A tasklet that uses "non-shared" data can run without the GIL, until it
writes other data that it does not own privately.
When a tasklet touches data that it does not own, then the regular
actions dig in - either locking, or actions of an STM implementation,
see the PyPy blogs.
But the idea is to de-couple threads of python execution as much as
possible,
without making Python an intractable beast.
For "normal" applications, Stackless 2.9 should just work as expected.
But whenever the effect of an action should be globally visible, an extra
function call is needed that opens the hidden state to others.
This is the rendevous case that we try to avoid, most of the time.
The needed functionality can be folded into Python at some cost for the
general case. But I think it can become very effective on modern hardware
and current software paradigms. Simultaneous access to certain things
can be avoided in over 95 % of typical observations.
Code Name
----------
As always, I am better at assigning a code name than creating a really
one.
My suggestion is:
- Gillespy
This is not only a contribution to Dizzy Gillespie, but also a
word-playing
game for those who wonder what a GIL-less Py should be.
I want to regard this post as an undirected starter.
Opinions?
--
Christian Tismer :^)<mailto:[email protected]>
<mailto:[email protected]>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship*http://starship.python.net/
14482 Potsdam : PGP key ->http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today?http://www.stackless.com/
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
--
Christian Tismer :^) <mailto:[email protected]>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
