[
https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505885#comment-13505885
]
Ryota Egashira commented on OOZIE-1052:
---------------------------------------
did additional test where I commented out following part in
LauncherMapper.hasIdSwap and ran workflow with map-reduce action, which worked.
====
if (counters != null) {
Counters.Group counterGroup = counters.getGroup(COUNTER_GROUP);
if (counterGroup != null) {
swap = counterGroup.getCounter(COUNTER_DO_ID_SWAP) == 1;
}
=====
> HadoopAccessorService.createFileSystem throws exception in map-reduce action,
> failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Fix For: 3.3.0, trunk
>
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks
> OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as
> argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it
> passes new conf as argument, which doesn't include
> OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned
> above
> -------
> FileSystem fs =
> Services.get().get(HadoopAccessorService.class).createFileSystem(user,
> p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce
> action, oozie fails to get external jobID from counters, then accessing to
> HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is
> introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the
> change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
