[
https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775774#comment-16775774
]
Guanghao Zhang commented on HBASE-20724:
----------------------------------------
The compaction evenet marker was introudced by HBASE-2231 and try to resolve
two case's problem.
case 1: RS crashed. The compacted file can be removed when replay the
compaction event marker.
case 2: RS GC pause long time and zk expired, but it come back again to finish
the compaction. Write compaction event marker will fail and prevent it to
finish the compaction.
For case 1, it still has problem as the HLog may be archived and cannot be
replayed. So it can be resolved by this issue's solution. For case 2, I thought
the compaction event marker is still useful. But we don't need write compacted
files to compaction event marker after this fix.
> Sometimes some compacted storefiles are still opened after region failover
> --------------------------------------------------------------------------
>
> Key: HBASE-20724
> URL: https://issues.apache.org/jira/browse/HBASE-20724
> Project: HBase
> Issue Type: Bug
> Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0
> Reporter: Francis Liu
> Assignee: Guanghao Zhang
> Priority: Critical
> Attachments: HBASE-20724.master.001.patch,
> HBASE-20724.master.002.patch, HBASE-20724.master.003.patch,
> HBASE-20724.master.004.patch, HBASE-20724.master.005.patch,
> HBASE-20724.master.006.patch, HBASE-20724.master.007.patch
>
>
> It is important that compacted storefiles of a given compaction execution are
> wholly opened or archived to insure data consistency. ie a storefile
> containing delete tombstones can be archived while older storefiles
> containing cells that were supposed to be deleted are left unarchived thereby
> undeleting those cells.
> When a server fails compaction markers (in the wal edit) are used to
> determine which storefiles are compacted and should be excluded during region
> open (during failover). But the WALs containing compaction markers can be
> prematurely archived even though there are still compacted storefiles for
> that particular compaction event that hasn't been archived yet. Thus losing
> compaction information that needs to be replayed in the event of an RS crash.
> This is because hlog archiving logic only keeps track of flushed storefiles
> and not compacted ones.
> https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
