Yes Andrew, I saw this page and got it working. I do have a question though. I am manually adding spark-<version>-yarn-shuffle.jar to one of the existing folders(yarn/lib/) which is already in the classpath on each nodemanager. I am wondering what is the best way to add spark-<version>-yarn-shuffle.jar to the classpath on all the nodemanagers.
Thank you On Wed, Jan 7, 2015 at 2:56 PM, Andrew Or <and...@databricks.com> wrote: > Did you end up getting it working? By the way this might be a nicer view > of the docs: > > https://github.com/apache/spark/blob/60e2d9e2902b132b14191c9791c71e8f0d42ce9d/docs/job-scheduling.md > > We will update the latest Spark docs to include this shortly. > -Andrew > > 2015-01-04 4:44 GMT-08:00 Tsuyoshi Ozawa <oz...@apache.org>: > > Please check the document added by Andrew. I could run tasks with Spark >> 1.2.0. >> >> * >> https://github.com/apache/spark/pull/3731/files#diff-c3cbe4cabe90562520f22d2306aa9116R86 >> * >> https://github.com/apache/spark/pull/3757/files#diff-c3cbe4cabe90562520f22d2306aa9116R101 >> >> Thanks, >> - Tsuyoshi >> >> On Sun, Jan 4, 2015 at 11:54 AM, firemonk9 <dhiraj.peech...@gmail.com> >> wrote: >> > I am running into similar problem. Have you found any resolution to this >> > issue ? >> > >> > >> > >> > -- >> > View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Elastic-allocation-spark-dynamicAllocation-enabled-results-in-task-never-being-executed-tp18969p20957.html >> > Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> > For additional commands, e-mail: user-h...@spark.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
