Denes Bodo created OOZIE-3300:
---------------------------------
Summary: Hadoop 3 Pig (with Tez) action shall not use given
distributed cache
Key: OOZIE-3300
URL: https://issues.apache.org/jira/browse/OOZIE-3300
Project: Oozie
Issue Type: Bug
Components: action
Affects Versions: 4.3.1
Environment: Oozie-4.3.1
Hadoop-3
Reporter: Denes Bodo
Assignee: Denes Bodo
When I run my Pig action and set -x Tez to use Tez as the execution engine, I
got the following error:
{code}
Caused by: java.lang.RuntimeException:
org.apache.hadoop.mapred.InvalidJobConfException: cache file
(mapreduce.job.cache.files) scheme: "hdfs" host: "mycluster" port: -1 file:
"/tmp/temp257132233/hive-exec-3.0.0.3.0.0.0-1469.jar" conflicts with cache file
(mapreduce.job.cache.files)
hdfs://mycluster/user/oozie/share/lib/lib_20180611041054/pig/hive-exec-3.0.0.3.0.0.0-1469.jar
{code}
or
{code}
Caused by: java.lang.RuntimeException:
org.apache.hadoop.mapred.InvalidJobConfException: cache file
(mapreduce.job.cache.files) scheme: "hdfs" host: "ns1" port: -1 file:
"/tmp/temp36150863/joda-time-2.9.6.jar" conflicts with cache file
(mapreduce.job.cache.files)
hdfs://ns1/user/oozie/share/lib/lib_20180612003013/pig/joda-time-2.9.6.jar
{code}
When I exclude the conflicting jar, then Pig cannot start due to class not
found exception.
I think a workaround or solution? could be if we exclude the
*mapreduce.job.cache.files* and *mapreduce.job.cache.archives* from
pig.properties file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
