Spark Push-Based Shuffle causing multiple stage failures

2022-05-23 Thread Han Altae-Tran
Hi, First of all, I am very thankful for all of the amazing work that goes into this project! It has opened up so many doors for me! I am a long time Spark user, and was very excited to start working with the push-based shuffle service for an academic paper we are working on, but I encountered som

Re: Spark Push-Based Shuffle causing multiple stage failures

2022-05-25 Thread Han Altae-Tran
yarn-site.xml for NodeManager configurations: > > spark.shuffle.push.server.mergedShuffleFileManagerImpl > org.apache.spark.network.shuffle.RemoteBlockPushResolver > > > On Tue, May 24, 2022 at 3:30 PM Mridul Muralidharan > wrote: > >> +CC zhouye...@gmail.com