Hi Armin, On Tue, May 01, 2012 at 15:55 +0200, Armin Rigo wrote: > Re-hi, > > Now I did a full circle, and I'm wondering again if we couldn't go > with the single primitive of an "atomic" object. You use it as "with > atomic:", in a way quite traditional for Transactional Memory.
i like this, more minimal, more obvious. > Indeed, it seems after all possible to have the following model: a > pypy-stm interpreter with the unmodified thread and threading modules, > with just the addition of "atomic" (from some module, maybe "atomic", > or "transaction" again, or "transact", or a better name). Maybe "atomic" could become a __pypy__ builtin and there could be a "ame" or so package which atomic-using apps could depend on? In any case, I really like the twist of "To remedy the GIL use AME" :) > You can use > it to ensure that a piece of code is run atomically; this would mean, > in the sense of CPython's GIL, that the GIL is not released for the > complete duration of the "with" statement. (On top of CPython, this > cannot be fully achieved without hacking at the interpreter, but to a > limited extent we can emulate it by saying at least the following: two > threads that both want to run "with atomic" are serialized. Easy to > do with just one lock.) > > On top of that, it is possible to write more nicely usable constructs > in pure Python. For example, to write the existing > transaction.add()/run(), we would write code that starts N threads, > each polling a Queue and running the items in a "with atomic" block. > Or we can write "schedule.Runner()" or similar. > > This approach has several advantages: the fact that the thread pooling > logic is written in pure Python and tweakable; but also the fact that > it is compatible with normal threads. This means all the doubts I had > about blocking C calls are already resolved --- if you need really to > do blocking C calls, do them in a separate thread. For example, in > twisted, the loop calling select() would be written in a thread --- or > likely in the main thread --- while a pool of extra threads runs the > actual logic. (Not a normal thread pool, but one which uses "with > atomic" to run each item.) People are already used to this approach > --- with the exception that this gives the additional *huge* benefit > of illusive serial execution, so locks&friends are useless. > > The new things here, when compared to CPython or PyPy-without-STM, are: > > (1) the ability to use multiple cores, by running transactions that go > from one release-the-GIL to the next one (which is already existing as > work-in-progress on CPython for Hardware TM); > > (2) the "with atomic" construct that I already proposed to CPython > last year (and certainly is not new, but inspired directly from > classical TM). > > Does it make any sense to you? To me so far yes. I am wondering how this all applies to the execnet-execution model, btw. (http://codespeak.net/execnet for those who wonder what i mean) remote_exec()s on the same gateway run currently in different threads and thus only send/receive needs to use "with atomic", right? best, holger _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
