Ryota Egashira created OOZIE-2246:
-------------------------------------
Summary: CoordinatorInputCheckCommand does not behave properly
when har file is one of data dependency and doesn't exist
Key: OOZIE-2246
URL: https://issues.apache.org/jira/browse/OOZIE-2246
Project: Oozie
Issue Type: Bug
Reporter: Ryota Egashira
Assignee: Ryota Egashira
when har file doesn't exist, FileSystem.createFileSystem in
HadoopAccessorService throw exception, failing CoordActionInputCheck.
thus, even thought there are other data dependencies which already exist, it is
not reflected on DB.
coordinator job cannot start until the har file becomes available anyway, and
once available, this error doesn't happen, so basic functionality is fine, but
it's misleading.
{code}
2014-03-13 22:00:00,051 WARN CallableQueueService$CallableWrapper:542
[pool-2-thread-288] - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
exception callable [coord_action_input], E1021: Coord Action Input Check Error:
org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured:
[Invalid path for the Har Filesystem. No index file in
har://hdfs-dilithiumred-nn1.red.ygrid.yahoo.com:8020/projects/tpoozie/post_tp_rmx_joined_click_trim_har/hourly/data/2014031322/archive.har]
org.apache.oozie.command.CommandException: E1021: Coord Action Input Check
Error: org.apache.oozie.service.HadoopAccessorException: E0902: Exception
occured: [Invalid path for the Har Filesystem. No index file in
har://hdfs-dilithiumred-nn1.red.ygrid.yahoo.com:8020/projects/tpoozie/post_tp_rmx_joined_click_trim_har/hourly/data/2014031322/archive.har]
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:182)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:67)
at org.apache.oozie.command.XCommand.call(XCommand.java:280)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException:
org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured:
[Invalid path for the Har Filesystem. No index file in
har://hdfs-dilithiumred-nn1.red.ygrid.yahoo.com:8020/projects/tpoozie/post_tp_rmx_joined_click_trim_har/hourly/data/2014031322/archive.har]
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:493)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkListOfPaths(CoordActionInputCheckXCommand.java:459)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkResolvedUris(CoordActionInputCheckXCommand.java:429)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkInput(CoordActionInputCheckXCommand.java:259)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:132)
... 6 more
Caused by: org.apache.oozie.service.HadoopAccessorException: E0902: Exception
occured: [Invalid path for the Har Filesystem. No index file in
har://hdfs-dilithiumred-nn1.red.ygrid.yahoo.com:8020/projects/tpoozie/post_tp_rmx_joined_click_trim_har/hourly/data/2014031322/archive.har]
at
org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:430)
at org.apache.oozie.dependency.FSURIHandler.getFileSystem(FSURIHandler.java:134)
at org.apache.oozie.dependency.FSURIHandler.exists(FSURIHandler.java:99)
at
org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:488)
... 10 more
Caused by: java.io.IOException: Invalid path for the Har Filesystem. No index
file in
har://hdfs-dilithiumred-nn1.red.ygrid.yahoo.com:8020/projects/tpoozie/post_tp_rmx_joined_click_trim_har/hourly/data/2014031322/archive.har
at org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:139)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2160)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:303)
at
org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:422)
at
org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:420)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1284)
at
org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
