On Sat, 2014-03-08 at 08:53 -0800, Andrei Alexandrescu wrote: > On 3/8/14, 3:22 AM, Russel Winder wrote: > > Dataflow is though where "Big Data" is going. There are commercial > > offerings in the JVM space and they are making huge profits on > > licencing, simply because the frameworks work. > > Do you have a couple of relevant links describing dataflow?
First and foremost we have to distinguish dataflow software architectures from dataflow computers. The latter were an alternate hardware architecture that failed to gain traction, but there is an awful lot of literature out there on it. So just searching the Web is likely to give an lot of that especially in the period 1980 to 1995. The dataflow software architectures are modelled directly on the structural concepts of dataflow hardware and so the terminology is exactly the same. However whereas an operator in hardware mean add, multiply, etc. in a software architecture it just means some sequential computation that requires certain inputs and delivers some outputs. The computation must be a process, so effectively a function with no free variables. The GPars version of this is at: http://gpars.codehaus.org/Dataflow http://gpars.org/guide/guide/dataflow.html GPars needs some more work, but I haven't had chance to focus on it recently. This introduces all the cute jargon: http://www.cs.colostate.edu/cameron/dataflow.html Wikipedia has this page: http://en.wikipedia.org/wiki/Dataflow_programming but it is clearly in need of some sub-editing. Hopefully this does as a start. I can try hunt up some other things if that would help. The commercial offering I know something of is called DataRush, it's a product from a subgroup in the Pervasive group for the JVM (and optionally Hadoop): http://en.wikipedia.org/wiki/DataRush_Technology I played with this in 2008 before it was formally released, and on and off since. GPars dataflow should compete with this but they are a company with resources, and GPars has two fairly non-active (due to work commitments) volunteer developers. We had been hoping the fact that GPars is core Groovy technology required for Grails and allt he other Gr8 technology, that people would step up. However the very concept of a concurrency and parallelism framework seems to frighten off even some of the best programmers. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:[email protected] 41 Buckmaster Road m: +44 7770 465 077 xmpp: [email protected] London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
