As a followup to my earlier post: "pre-emptive micro-threads utilizing shared memory message passing?"
I am actually finding that the biggest hurdle to accomplishing what I want is the lack of ANY type of shared memory -- even if it is limited. I wonder if I might ask a question: Would the following be a possible way to offer a limited type of shared memory: Summary: create a system very, very similar to POSH, but with differences: In detail, here's what I mean: * unlike POSH, utilize OS threads and shared memory (not processes) * Create a special shared memory location where you can place Python objects * Each Python object you place into this location can only be accessed (modified) by 1 thread. * You must manually assign ownership of an object to a particular thread. * The thread that "owns" the object is the only one that can modify it. * You can transfer ownership to another thread (but, as always only the owner can modify it). * There is no GIL when a thread interacts with these special objects. You can have true thread parallelism if your code uses a lot of these special objects. * The GIL remains in place for all other data access. * If your code has a mixture of access to the special objects and regular data, then once you hit a point where a thread starts to interact with data not in the special storage, then that thread must follow GIL rules. Granted, there might be some difficulty with the GIL part... but I thought I might ask anyways. :) > Date: Wed, 28 Jul 2010 22:54:39 +1000 > Subject: Re: [pypy-dev] pre-emptive micro-threads utilizing shared memory > message passing? > From: [email protected] > To: [email protected] > CC: [email protected] > > On 28 July 2010 04:20, Kevin Ar18 <[email protected]> wrote: > > I am attempting to experiment with FBP - Flow Based Programming > > (http://www.jpaulmorrison.com/fbp/ and book: > > http://www.jpaulmorrison.com/fbp/book.pdf) There is something very similar > > in Python: http://www.kamaelia.org/MiniAxon.html Also, there are some > > similarities to Erlang - the share nothing memory model... and on some very > > broad levels, there are similarities that can be found in functional > > languages. > > Does anyone know if there is a central resource for incompatible > python memory model proposals? I know of Jython, Python-Safethread, > and Mont-E. > > I do like the idea of MiniAxon, but let me mention a topic that has > slowly been bubbling to the front of my mind for the last few months. > > Concurrency in the face of shared mutable state is hard. It makes it > trivial to introduce bugs all over the place. Nondeterminacy related > bugs are far harder to test, diagnose, and fix than anything else that > I would almost mandate static verification (via optional typing, > probably) of task noninterference if I was moving to a concurrent > environment with shared mutable state. There might be a reasonable > middle ground where, if a task attempts to violate the required static > semantics, it fails dynamically. At least then, latent bugs make > plenty of noise. An example for MiniAxon (as I understand it, which is > not very well) would be verification that a "task" (including > functions that the task calls) never closes over and yields the same > mutable objects, and never mutates globally reachable objects. > > I wonder if you could close such tasks off with a clever subclass of > the proxy object space that detects and rejects such memory model > violations? With only semantics that make the program deterministic? > > The moral equivalent would be cooperating processes with a large > global (or not) shared memory store for immutable objects, queues for > communication, and the additional semantic that objects in a queue are > either immutable or the queue holds their only reference. The trouble > is that it is so hard to work out what immutable really means. > Non-optional annotations would be not very pythonian. > > -- > William Leslie
_______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
