Hi, all See SPARK-1529 for related discussion.
// maropu On Wed, Apr 27, 2016 at 12:27 PM, Saisai Shao <[email protected]> wrote: > Quite curious about the benefits of using HDFS as shuffle service, also > what's the problem of using current shuffle service? > > > Thanks > Saisai > > On Wed, Apr 27, 2016 at 4:31 AM, Timothy Chen <[email protected]> wrote: > >> Are you suggesting to have shuffle service persist and fetch data with >> hdfs, or skip shuffle service altogether and just write to hdfs? >> >> Tim >> >> >> > On Apr 26, 2016, at 11:20 AM, Michael Gummelt <[email protected]> >> wrote: >> > >> > Has there been any thought or work on this (or any other networked file >> system)? It would be valuable to support dynamic allocation without >> depending on the shuffle service. >> > >> > -- >> > Michael Gummelt >> > Software Engineer >> > Mesosphere >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > -- --- Takeshi Yamamuro
