[
https://issues.apache.org/jira/browse/KYLIN-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048140#comment-17048140
]
ZhouKang commented on KYLIN-4268:
---------------------------------
I think it's worked using solution 1.
So close this issue.
> build job failed caused by hadoop utils update
> ----------------------------------------------
>
> Key: KYLIN-4268
> URL: https://issues.apache.org/jira/browse/KYLIN-4268
> Project: Kylin
> Issue Type: Bug
> Reporter: ZhouKang
> Priority: Major
>
> When we update hadoop package, such as hbase and hive. The job will fill
> cause by:
>
> {code:java}
> // code placeholder
> Exception in thread "main" java.io.FileNotFoundException: File
> file:/home/work/app/kylin-all-2.5.2-2.5.2-HBase2-SNAPSHOT/hbase-2.2.0-3.1.0/lib/hbase-common-2.2.0-3.1.0.jar
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:416)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
> at
> org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:406)
> at
> org.apache.spark.deploy.yarn.Client.org\$apache\$spark\$deploy\$yarn\$Client\$\$distribute\
> {code}
>
> The root cause is kylin will generate the full command as soon as the job
> summited. And the parameter '--jars' use file's absolute path.
>
> there will be 2 ways to fix this problem.
> # keep the old package in the same path
> # change the generation of '--jars' to the stage of job execution.
>
> which is the better choose?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
