Someone (Ze Ni, https://www.sics.se/people/ze-ni) has actually attempted such a comparative study as a Masters thesis:
http://www.diva-portal.org/smash/get/diva2:605106/FULLTEXT01.pdf According to this snapshot (c. 2013), Stratosphere is different from Spark in not having an explicit concept of an in-memory dataset (e.g., RDD). In principle this could be argued to be an implementation detail; the operators and execution plan/data flow are of primary concern in the API, and the data representation/materializations are otherwise unspecified. But in practice, for long-running interactive applications, I consider RDDs to be of fundamental, first-class citizen importance, and the key distinguishing feature of Spark's model vs other "in-memory" approaches that treat memory merely as an implicit cache. -- Christopher T. Nguyen Co-founder & CEO, Adatao <http://adatao.com> linkedin.com/in/ctnguyen On Tue, Nov 26, 2013 at 1:26 PM, Matei Zaharia <matei.zaha...@gmail.com>wrote: > I don’t know a lot about it except from the research side, where the team > has done interesting optimization stuff for these types of applications. In > terms of the engine, one thing I’m not sure of is whether Stratosphere > allows explicit caching of datasets (similar to RDD.cache()) and > interactive queries (similar to spark-shell). But it’s definitely an > interesting project to watch. > > Matei > > On Nov 22, 2013, at 4:17 PM, Ankur Chauhan <achau...@brightcove.com> > wrote: > > > Hi, > > > > That's what I thought but as per the slides on > http://www.stratosphere.eu they seem to "know" about spark and the scala > api does look similar. > > I found the PACT model interesting. Would like to know if matei or other > core comitters have something to weight in on. > > > > -- Ankur > > On 22 Nov 2013, at 16:05, Patrick Wendell <pwend...@gmail.com> wrote: > > > >> I've never seen that project before, would be interesting to get a > >> comparison. Seems to offer a much lower level API. For instance this > >> is a wordcount program: > >> > >> > https://github.com/stratosphere/stratosphere/blob/master/pact/pact-examples/src/main/java/eu/stratosphere/pact/example/wordcount/WordCount.java > >> > >> On Thu, Nov 21, 2013 at 3:15 PM, Ankur Chauhan <achau...@brightcove.com> > wrote: > >>> Hi, > >>> > >>> I was just curious about https://github.com/stratosphere/stratosphere > >>> and how does spark compare to it. Anyone has any experience with it to > make > >>> any comments? > >>> > >>> -- Ankur > > > >
