[jira] [Assigned] (HBASE-20724) Sometimes some compacted storefiles are still opened after region failover
[ https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-20724: -- Assignee: Guanghao Zhang (was: Francis Liu) > Sometimes some compacted storefiles are still opened after region failover > -- > > Key: HBASE-20724 > URL: https://issues.apache.org/jira/browse/HBASE-20724 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Guanghao Zhang >Priority: Critical > Attachments: HBASE-20724.master.001.patch > > > It is important that compacted storefiles of a given compaction execution are > wholly opened or archived to insure data consistency. ie a storefile > containing delete tombstones can be archived while older storefiles > containing cells that were supposed to be deleted are left unarchived thereby > undeleting those cells. > When a server fails compaction markers (in the wal edit) are used to > determine which storefiles are compacted and should be excluded during region > open (during failover). But the WALs containing compaction markers can be > prematurely archived even though there are still compacted storefiles for > that particular compaction event that hasn't been archived yet. Thus losing > compaction information that needs to be replayed in the event of an RS crash. > This is because hlog archiving logic only keeps track of flushed storefiles > and not compacted ones. > https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20724) Sometimes some compacted storefiles are still opened after region failover
[ https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu reassigned HBASE-20724: --- Assignee: Francis Liu > Sometimes some compacted storefiles are still opened after region failover > -- > > Key: HBASE-20724 > URL: https://issues.apache.org/jira/browse/HBASE-20724 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > > It is important that compacted storefiles of a given compaction execution are > wholly opened or archived to insure data consistency. ie a storefile > containing delete tombstones can be archived while older storefiles > containing cells that were supposed to be deleted are left unarchived thereby > undeleting those cells. > When a server fails compaction markers (in the wal edit) are used to > determine which storefiles are compacted and should be excluded during region > open (during failover). But the WALs containing compaction markers can be > prematurely archived even though there are still compacted storefiles for > that particular compaction event that hasn't been archived yet. Thus losing > compaction information that needs to be replayed. This is because hlog > archiving logic only keeps track of flushed storefiles and not archived ones. > https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
