[
https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525832#comment-16525832
]
Francis Liu commented on HBASE-20704:
-------------------------------------
OK let me see what I can do without adding to much complexity. For scanners the
current expectation is that storefile readers won't get closed on them while
they are using it. My initial look on this was that it would require checks in
a bunch of places wether the stream has been closed. Just a bit concerned that
would affect performance or make things even more complex.
Tho thinking about this a bit more. I think we could address this and
HBASE-20724 in the same manner. If we add in each created compacted storefile
metadata about which storefiles were it's parents. So on the next region open
we can remove the compacted storefiles as part of the region open operation.
What do you guys think?
> Sometimes some compacted storefiles are not archived on region close
> --------------------------------------------------------------------
>
> Key: HBASE-20704
> URL: https://issues.apache.org/jira/browse/HBASE-20704
> Project: HBase
> Issue Type: Bug
> Components: Compaction
> Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0
> Reporter: Francis Liu
> Assignee: Francis Liu
> Priority: Critical
> Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch
>
>
> During region close compacted files which have not yet been archived by the
> discharger are archived as part of the region closing process. It is
> important that these files are wholly archived to insure data consistency. ie
> a storefile containing delete tombstones can be archived while older
> storefiles containing cells that were supposed to be deleted are left
> unarchived thereby undeleting those cells.
> On region close a compacted storefile is skipped from archiving if it has
> read references (ie open scanners). This behavior is correct for when the
> discharger chore runs but on region close consistency is of course more
> important so we should add a special case to ignore any references on the
> storefile and go ahead and archive it.
> Attached patch contains a unit test that reproduces the problem and the
> proposed fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
