[
https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-1883:
----------------------------------
Description: When do the cleanup, current strategy, to firstly create an
active referenceSet and then to create the toDeleteSet by checking whether the
whole set, like dict, is in this referenceSet or not, is not a good solution
due to consensus issues. For example, before the first step, SEGMENT_A is
deleted and leave a DICT_A created at the building step. Then the referenceSet
will not include DICT_A. Between the gap of two steps, SEGMENT_B is starting to
build. Since DICT_A still exists, it can still be referenced by SEGMENT_B.
After SEGMENT_B owns the reference to DICT_A, the step two begins. Then DICT_A
will still be included in the toDeleteSet and will be deleted later. Finally
SEGMENT_B only owns a reference with no data. (was: When do the cleanup,
current strategy is )
> Consensus Problem when running the tool, MetadataCleanupJob
> -----------------------------------------------------------
>
> Key: KYLIN-1883
> URL: https://issues.apache.org/jira/browse/KYLIN-1883
> Project: Kylin
> Issue Type: Bug
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
>
> When do the cleanup, current strategy, to firstly create an active
> referenceSet and then to create the toDeleteSet by checking whether the whole
> set, like dict, is in this referenceSet or not, is not a good solution due to
> consensus issues. For example, before the first step, SEGMENT_A is deleted
> and leave a DICT_A created at the building step. Then the referenceSet will
> not include DICT_A. Between the gap of two steps, SEGMENT_B is starting to
> build. Since DICT_A still exists, it can still be referenced by SEGMENT_B.
> After SEGMENT_B owns the reference to DICT_A, the step two begins. Then
> DICT_A will still be included in the toDeleteSet and will be deleted later.
> Finally SEGMENT_B only owns a reference with no data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
