[
https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374760#comment-15374760
]
Zhong Yanghong commented on KYLIN-1883:
---------------------------------------
The better way should be like this:
1. Firstly create a toDeleteSet;
2. Then create a referenceSet;
3. Remove the referenceSet from the toDeleteSet.
> Consensus Problem when running the tool, MetadataCleanupJob
> -----------------------------------------------------------
>
> Key: KYLIN-1883
> URL: https://issues.apache.org/jira/browse/KYLIN-1883
> Project: Kylin
> Issue Type: Bug
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
>
> When do the cleanup, current strategy, to firstly create an active
> referenceSet and then to create the toDeleteSet by checking whether the whole
> set, like dict, is in this referenceSet or not, is not a good solution due to
> consensus issues. For example, before the first step, SEGMENT_A is deleted
> and leave a DICT_A created at the building step. Then the referenceSet will
> not include DICT_A. Between the gap of two steps, SEGMENT_B is starting to
> build. Since DICT_A still exists, it can still be referenced by SEGMENT_B.
> After SEGMENT_B owns the reference to DICT_A, the step two begins. Then
> DICT_A will still be included in the toDeleteSet and will be deleted later.
> Finally SEGMENT_B only owns a reference with no data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
