Xiangrui Meng created SPARK-1921:
------------------------------------
Summary: Allow duplicate jar files among the app jar and secondary
jars in yarn-cluster mode
Key: SPARK-1921
URL: https://issues.apache.org/jira/browse/SPARK-1921
Project: Spark
Issue Type: Sub-task
Components: Deploy
Affects Versions: 1.0.0
Reporter: Xiangrui Meng
Priority: Minor
In yarn-cluster mode, jars are uploaded to a staging folder on hdfs. If there
are duplicates among the app jar and secondary jars, there will be overwrites
that cause inconsistent timestamps. I saw the following message:
{code}
Application application_1400965808642_0021 failed 2 times due to AM Container
for appattempt_1400965808642_0021_000002 exited with exitCode: -1000 due to:
Resource
hdfs://localhost.localdomain:8020/user/cloudera/.sparkStaging/application_1400965808642_0021/app_2.10-0.1.jar
changed on src filesystem (expected 1400998721965, was 1400998723123
{code}
Tested on a CDH-5 quickstart VM.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
