Quite curious about the benefits of using HDFS as shuffle service, also what's the problem of using current shuffle service?
Thanks Saisai On Wed, Apr 27, 2016 at 4:31 AM, Timothy Chen <[email protected]> wrote: > Are you suggesting to have shuffle service persist and fetch data with > hdfs, or skip shuffle service altogether and just write to hdfs? > > Tim > > > > On Apr 26, 2016, at 11:20 AM, Michael Gummelt <[email protected]> > wrote: > > > > Has there been any thought or work on this (or any other networked file > system)? It would be valuable to support dynamic allocation without > depending on the shuffle service. > > > > -- > > Michael Gummelt > > Software Engineer > > Mesosphere > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
