This archive contains all the jars required by Spark runtime, you could zip
all the jars under /jars and upload this archive to HDFS, then
configure spark.yarn.archive with the path of this archive on HDFS.
On Sun, Aug 28, 2016 at 9:59 PM, Srikanth Sampath wrote:
> Hi,
> With SPARK-11157, the bi
Hi,
With SPARK-11157, the big fat assembly jar build was removed.
Has anyone used spark.yarn.archive - the alternative provided and
successfully deployed Spark on a Yarn cluster. If so, what does the archive
contain. What should be the minimal set. Any suggestion is greatly
appreciated.
Thanks
