Quite curious about the benefits of using HDFS as shuffle service, also what's the problem of using current shuffle service?
Thanks Saisai On Wed, Apr 27, 2016 at 4:31 AM, Timothy Chen <tnac...@gmail.com> wrote: > Are you suggesting to have shuffle service persist and fetch data with > hdfs, or skip shuffle service altogether and just write to hdfs? > > Tim > > > > On Apr 26, 2016, at 11:20 AM, Michael Gummelt <mgumm...@mesosphere.io> > wrote: > > > > Has there been any thought or work on this (or any other networked file > system)? It would be valuable to support dynamic allocation without > depending on the shuffle service. > > > > -- > > Michael Gummelt > > Software Engineer > > Mesosphere > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > >
