Actors in the actor model are somewhere between a Julia Task (shared memory, concurrent but not parallel) and a Julia Worker (separate memory, concurrent and parallel). Once Julia Tasks are actually parallel as well as concurrent (i.e. can run in separate threads), then the two will be even more similar. I'm not sure that we'll ever get to a straight up actor model since memory isolation is very expensive unless you have universal immutability (like Erlang). But it seems like a good idea to make the difference between local tasks and remote tasks as transparent as possible so that you can program across them uniformly when it makes sense. It's nice to hear that this model is working well for you. By having workers pull work when they're idle, you're effectively using a greedy form of dynamic work scheduling, which is similar to what pmap does.
On Mon, Feb 1, 2016 at 7:48 PM, Lyndon White <[email protected]> wrote: > Hi, > > So I do a lot of batch processing, for machine learning. > I have a lot of RAM, 45Gb, and 12 cores. > > My normal method for parallel processing is to replicate any common shared > read-only memory across all workers, using @everywhere. > Then process my data with pmap, or a variation of my own pmapreduce. > > On my work yesterday, I couldnot replicate my common shared memory across > all workers, as it was too large (17Gb). > So I left it on processor 1, and told the workers to do a remotecall_fetch > to retrieve it. > This seems to have worked very well, as the workers quickly get out of > sync so beg from processor 1 at different times. > And processor 1 is normally unused til the worker are all done anyway. > > I was thinking about it after I went home, and realised that this was a > very rough approximation of the Actor model > <https://en.wikipedia.org/wiki/Actor_model>(If I am recalling the Actor > model correctly). > > What I am thinking, > is I could have 1 worker per service -- where a service is combination of > data and functions that operate on it. > Which is more than 1 worker per core. > When ever a worker needs that data, it remotecall_fetches the function on > the services worker. > (Potentially it does smarter things than a remotecall_fetch, so it is > unblocking.) > > > Is this sensible? > >
