Xingjun Hao created CARBONDATA-4044:
---------------------------------------
Summary: Fix dirty data in indexfile while IUD with stale data in
segment folder
Key: CARBONDATA-4044
URL: https://issues.apache.org/jira/browse/CARBONDATA-4044
Project: CarbonData
Issue Type: Bug
Reporter: Xingjun Hao
XX.mergecarbonindex and XX..segment records the indexfiles list of a segment.
now, we generate xx.mergeindexfile and xx.segment based on filter out all
indexfiles(including carbonindex and mergecarbonindex), which will leading
dirty data when there is stale data in segment folder.
For example, there are a stale index file in segment_0 folder,
"0_1603763776.carbonindex".
While loading, a new carbonindex "0_16037752342.carbonindex" is wrote, when
merge carbonindex files, we expect to only merge 0_16037752342.carbonindex, But
If we filter out all carbonindex in segment folder, both
"0_1603763776.carbonindex" and 0_16037752342.carbonindex will be merged and
recorded into segment file.
While updating, there has same problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
