Nice posts. FYI only, you may be interested in [1], which was published a few days earlier.
Keep doing the good work! Davor [1] https://cloud.google.com/blog/big-data/2018/03/joining-a nd-shuffling-very-large-datasets-using-cloud-dataflow On Fri, Mar 23, 2018 at 12:04 AM, John Yang <[email protected]> wrote: > Hi Gon, > > Good catch! It's Spark DSL for the Spark system, and Beam for Nemo. > Will update the articles to make that clear. > > Cheers, > John > > > On Fri, Mar 23, 2018 at 3:59 PM, Byung-Gon Chun <[email protected]> wrote: > > > John, > > > > From the description, it's not clear the programming layer used by Nemo > for > > the experiments. > > It's Beam, not Spark DSL. Right? > > > > Cheers, > > Gon > > > > On Fri, Mar 23, 2018 at 11:35 AM, John Yang <[email protected]> wrote: > > > > > Thanks Gon! > > > Updated as per your suggestion. :) > > > > > > Cheers, > > > John > > > > > > On Fri, Mar 23, 2018 at 11:18 AM, Byung-Gon Chun <[email protected]> > > wrote: > > > > > > > John, thanks for posting the blogs! > > > > The results look great! > > > > > > > > I have a suggestion. In the second blog, it'd be great if you'd > > highlight > > > > the numbers and talk about a speedup (120/18). > > > > > > > > Cheers, > > > > Gon > > > > > > > > > > > > On Fri, Mar 23, 2018 at 10:59 AM, John Yang <[email protected]> > > wrote: > > > > > > > > > Hi Nemo community, > > > > > > > > > > > > > > > > > > > > 2 new blog posts are up! > > > > > > > > > > > > > > > http://nemo.apache.org/blog/2018/03/23/shuffle-on-nemo/ > > > > > > > > > > > > > > > Come and check out how Nemo outperforms Spark in handling a 2TB > > > MapReduce > > > > > job, and harnessing transient resources in datacenters. > > > > > > > > > > > > > > > Feel free to ask us any questions about the results, and stay tuned > > for > > > > > more exciting updates! :) > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > John > > > > > > > > > > > > > > > > > > > > > -- > > > > Byung-Gon Chun > > > > > > > > > > > > > > > -- > > Byung-Gon Chun > > >
