Hi...
After updating the source JARs of an application that launches a second job
while running a MR job, the following error keeps occurring:
org.apache.hadoop.mapred.InvalidJobConfException: cache file
(mapreduce.job.cache.files) scheme: "hdfs", host: "server", port: 9000,
file:
"/tmp/hadoop-yarn/staging/root/.staging/job_1367474197612_0887/libjars/Some.jar",
conflicts with cache file (mapreduce.job.cache.files)
hdfs://server:9000/tmp/hadoop-yarn/staging/root/.staging/job_1367474197612_0888/libjars/Some.jar
at
org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:338)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:273)
at
org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
............
where job_1367474197612_0887 is the name of the initial job,
job_1367474197612_0888 is the name of the subsequent job, and Some.jar is a
JAR file specific to the application.
Any ideas as to how the above error could be eliminated?
Thanks!
