If this could be compiled to run on a Spark-like engine, it would be interesting, from a quick look this doesn't seem easy because this model has arbitrary graphs of fine-grained tasks (more similar to something like Cilk) rather than parallel operations on collections. I think it would require a different engine with different properties about fault tolerance, locality, etc.
Matei On Oct 23, 2013, at 2:33 PM, Alex Boisvert <alex.boisv...@gmail.com> wrote: > (Resending to @apache list instead of old google-group) > > A bit of a random question but I was wondering if there were efforts > underway to generalize / expand the Spark API towards something that would > be similar to the MBrace [1] model ... there's certainly an overlap between > the features of the systems already ... so I guess I'm thinking about an > API that's less centered around RDDs (as a collection) and more towards > distributed dataflow that would feel more like composing Promises/Futures > ... or even generalizing to support various sorts of container/context > monads. > > [1] "MBrace: Cloud Computing with Monads" > http://plosworkshop.org/2013/preprint/dzik.pdf
