hi, all
I got this while running pig script:
997: Unable to recreate exception from backend error:
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
hdfs://DC-hadoop01:9000/tmp/pig-temp/temp548500412/tmp-1456742965 already exists
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:207)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:188)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:893)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:856)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:830)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Thread.java:722)
But I checked the script , the directory:
hdfs://DC-hadoop01:9000/tmp/pig-temp/temp548500412/tmp-1456742965 is not used
by the script explicitly, so I think it is used by the pig to store tmp results.
But why it exists? Isn't it unique?
Haitao Yao
[email protected]
weibo: @haitao_yao
Skype: haitao.yao.final
