On Wed, Jul 28, 2010 at 8:33 PM, Kevin Ar18 <[email protected]> wrote: > As a followup to my earlier post: > "pre-emptive micro-threads utilizing shared memory message passing?" > > I am actually finding that the biggest hurdle to accomplishing what I want > is the lack of ANY type of shared memory -- even if it is limited. I wonder > if I might ask a question: > > Would the following be a possible way to offer a limited type of shared > memory: > > Summary: create a system very, very similar to POSH, but with differences: > > In detail, here's what I mean: > * unlike POSH, utilize OS threads and shared memory (not processes) > * Create a special shared memory location where you can place Python objects > * Each Python object you place into this location can only be accessed > (modified) by 1 thread. > * You must manually assign ownership of an object to a particular thread. > * The thread that "owns" the object is the only one that can modify it. > * You can transfer ownership to another thread (but, as always only the > owner can modify it). > > * There is no GIL when a thread interacts with these special objects. You > can have true thread parallelism if your code uses a lot of these special > objects. > * The GIL remains in place for all other data access. > * If your code has a mixture of access to the special objects and regular > data, then once you hit a point where a thread starts to interact with data > not in the special storage, then that thread must follow GIL rules. > > Granted, there might be some difficulty with the GIL part... but I thought I > might ask anyways. :) > >> Date: Wed, 28 Jul 2010 22:54:39 +1000 >> Subject: Re: [pypy-dev] pre-emptive micro-threads utilizing shared memory >> message passing? >> From: [email protected] >> To: [email protected] >> CC: [email protected] >> >> On 28 July 2010 04:20, Kevin Ar18 <[email protected]> wrote: >> > I am attempting to experiment with FBP - Flow Based Programming >> > (http://www.jpaulmorrison.com/fbp/ and book: >> > http://www.jpaulmorrison.com/fbp/book.pdf) There is something very similar >> > in Python: http://www.kamaelia.org/MiniAxon.html Also, there are some >> > similarities to Erlang - the share nothing memory model... and on some very >> > broad levels, there are similarities that can be found in functional >> > languages. >> >> Does anyone know if there is a central resource for incompatible >> python memory model proposals? I know of Jython, Python-Safethread, >> and Mont-E. >> >> I do like the idea of MiniAxon, but let me mention a topic that has >> slowly been bubbling to the front of my mind for the last few months. >> >> Concurrency in the face of shared mutable state is hard. It makes it >> trivial to introduce bugs all over the place. Nondeterminacy related >> bugs are far harder to test, diagnose, and fix than anything else that >> I would almost mandate static verification (via optional typing, >> probably) of task noninterference if I was moving to a concurrent >> environment with shared mutable state. There might be a reasonable >> middle ground where, if a task attempts to violate the required static >> semantics, it fails dynamically. At least then, latent bugs make >> plenty of noise. An example for MiniAxon (as I understand it, which is >> not very well) would be verification that a "task" (including >> functions that the task calls) never closes over and yields the same >> mutable objects, and never mutates globally reachable objects. >> >> I wonder if you could close such tasks off with a clever subclass of >> the proxy object space that detects and rejects such memory model >> violations? With only semantics that make the program deterministic? >> >> The moral equivalent would be cooperating processes with a large >> global (or not) shared memory store for immutable objects, queues for >> communication, and the additional semantic that objects in a queue are >> either immutable or the queue holds their only reference. The trouble >> is that it is so hard to work out what immutable really means. >> Non-optional annotations would be not very pythonian. >> >> -- >> William Leslie > > _______________________________________________ > [email protected] > http://codespeak.net/mailman/listinfo/pypy-dev >
Honestly, that sounds really difficult, out and out removing the GIL would probably be easier. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
