[
https://issues.apache.org/jira/browse/OOZIE-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449700#comment-16449700
]
Denes Bodo commented on OOZIE-3227:
-----------------------------------
Oozie version is 4.3.1 built and used with hadoop 3. In the specific setup
there is a.jar both in
* share/lib/lib_{id}/oozie/a,jar and
* share/lib/lib_{id}/hive2/a.jar.
When starting java action, and using hive2 sharelib for java action
(oozie.action.sharelib.for.java=hive2) and using system lib
(oozie.use.system.libpath=true), the following exception occurs:
{noformat}
ACTION[0003453-180419084807385-oozie-oozi-W@tpch_query1] Error starting action
[tpch_query1]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: cache
file (mapreduce.job.cache.files) scheme: "hdfs" host: "hostname" port: -1 file:
"/user/oozie/share/lib/lib_20180418091531/hive2/a.jar" conflicts with cache
file (mapreduce.job.cache.files)
hdfs://hostname/user/oozie/share/lib/lib_20180418091531/oozie/a.jar]
org.apache.oozie.action.ActionExecutorException: JA009: cache file
(mapreduce.job.cache.files) scheme: "hdfs" host: "hostname" port: -1 file:
"/user/oozie/share/lib/lib_20180418091531/hive2/a.jar" conflicts with cache
file (mapreduce.job.cache.files)
hdfs://hostname/user/oozie/share/lib/lib_20180418091531/oozie/a.jar
at
org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:463)
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:441)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1210)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1392)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.mapred.InvalidJobConfException: cache file
(mapreduce.job.cache.files) scheme: "hdfs" host: "hostname" port: -1 file:
"/user/oozie/share/lib/lib_20180418091531/hive2/a.jar" conflicts with cache
file (mapreduce.job.cache.files)
hdfs://chelsea/user/oozie/share/lib/lib_20180418091531/oozie/a.jar
at
org.apache.hadoop.mapreduce.v2.util.LocalResourceBuilder.createLocalResources(LocalResourceBuilder.java:150)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:502)
at
org.apache.hadoop.mapred.YARNRunner.setupContainerLaunchContextForAM(YARNRunner.java:544)
at
org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:582)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:325)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:254)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:576)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:571)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:571)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:562)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1195)
... 9 more
{noformat}
> Eliminate duplicated dependencies from distributed cache
> --------------------------------------------------------
>
> Key: OOZIE-3227
> URL: https://issues.apache.org/jira/browse/OOZIE-3227
> Project: Oozie
> Issue Type: Sub-task
> Components: core
> Reporter: Denes Bodo
> Assignee: Denes Bodo
> Priority: Major
>
> Using Hadoop 3 it is not allowed to have multiple dependencies with same file
> names on the list of *mapreduce.job.cache.files*.
> The issue occurs when I have the same file name on multiple sharelib folders
> and/or my application's lib folder. This can be avoided but not easy all the
> time.
> I suggest to remove the duplicates from this list.
> A quick workaround for the source code in JavaActionExecutor is like:
> {code}
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.files");
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.archives");
> ......
> private void removeDuplicatedDependencies(JobConf conf, String key) {
> final Map<String, String> nameToPath = new HashMap<>();
> StringBuilder uniqList = new StringBuilder();
> for(String dependency: conf.get(key).split(",")) {
> final String[] arr = dependency.split("/");
> final String dependencyName = arr[arr.length - 1];
> if(nameToPath.containsKey(dependencyName)) {
> LOG.warn(dependencyName + " [" + dependency + "] is already
> defined in " + key + ". Skipping...");
> } else {
> nameToPath.put(dependencyName, dependency);
> uniqList.append(dependency).append(",");
> }
> }
> uniqList.setLength(uniqList.length() - 1);
> conf.set(key, uniqList.toString());
> }
> {code}
> Other way is to eliminate the deprecated
> *org.apache.hadoop.filecache.DistributedCache*.
> I am going to have a deeper understanding how we should use distributed cache
> and all the comments are welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
