[
https://issues.apache.org/jira/browse/KYLIN-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16895966#comment-16895966
]
WangSheng commented on KYLIN-4060:
----------------------------------
Hi, [~wangrupeng], thanks for your advice, and I've already seen the code you
pasted in kylin-2.6.x, it's clearly that this problem has been solved, so I
will close this jira. Anyway, thanks for your kind.
> "Garbage Collection on HDFS" step failed because of hdfs path not exists
> ------------------------------------------------------------------------
>
> Key: KYLIN-4060
> URL: https://issues.apache.org/jira/browse/KYLIN-4060
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v2.4.1
> Reporter: WangSheng
> Priority: Minor
>
> We found a bug recently when we used streaming cube on last job step "Garbage
> Collection on HDFS", the proplem is as blow:
>
> {code:java}
> Drop HDFS path on FileSystem: "hdfs://kylin-cluster"
> HDFS path
> /user/kylin/kylin_home/kylin_metadata/kylin-03c04b31-5d40-441a-a0df-289f5977b733/cube_test/fact_distinct_columns
> not exists.
> File
> /user/kylin/kylin_home/kylin_metadata/kylin-03c04b31-5d40-441a-a0df-289f5977b733/cube_test
> does not exist.
> {code}
> When I check the code and log, I found that the main reason is:
>
> # A build job first submitted, and on step "Update Cube Info", segment
> became "READY";
> # Then a merge job submitted automatically by kylin, include segment on
> step1. The merge job finished quickly, and deleted input segments hdfs path;
> # After merge job finished, the build job continue build, "Hive Cleanup" and
> "Garbage Collection on HBase", failed at last step because the hdfs path is
> deleted on step2.
> Our version is 2.4.x, I'm not sure this if this bug fixed on latest 2.6.x
> version. If not, please assign this Jira to me, thanks!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
