Francis Liu commented on HBASE-20704:


Are we ensuring this always after this patch? What if the RS going down in 
between a close so not all the compacted files are archived? This issue is 
there with old impl also right (Archive immediately and not by the Discharger 


In both cases if the RS is not gracefully shutdown then the WAL will get 
replayed and the compaction marker gets replayed thus making sure the compacted 
files are never accessed. Or so I'd like to confidently say. But it seems that 
even that part has a bug wherein the WAL containing the compaction marker thats 
needs to be replayed can get archived as sequence id tracking for WAL is only 
tied to memstore flushed, ignoring wether compaction archival for a given 
compaction even has completed. The same can be said for when edits are replayed 
on region open. 

I can think of a few reasons why this was not observed (or not as much) during 
pre-discharger versions. 1. Since we archive soon after compacting the window 
for exposure is pretty small. 2. At least for the delete case assuming the 
common case that the user does not mess with the timestamps. Since the 
compacted storefiles are sorted by seq id and removed in sequence, the 
storefiles containing rows that were deleted are removed first before the 
storefiles containing the corresponding tombstones for those rows. With the 
discharger we skip storefiles if they still have references.

Should we address this issue here? Or create another jira? If another Jira then 
in this one we can prolly add a partial fix wherein the discharger only removes 
contiguous storefiles?




> Sometimes some compacted storefiles are not archived on region close
> --------------------------------------------------------------------
>                 Key: HBASE-20704
>                 URL: https://issues.apache.org/jira/browse/HBASE-20704
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>    Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>            Priority: Critical
>         Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch
> During region close compacted files which have not yet been archived by the 
> discharger are archived as part of the region closing process. It is 
> important that these files are wholly archived to insure data consistency. ie 
> a storefile containing delete tombstones can be archived while older 
> storefiles containing cells that were supposed to be deleted are left 
> unarchived thereby undeleting those cells. 
> On region close a compacted storefile is skipped from archiving if it has 
> read references (ie open scanners). This behavior is correct for when the 
> discharger chore runs but on region close consistency is of course more 
> important so we should add a special case to ignore any references on the 
> storefile and go ahead and archive it. 
> Attached patch contains a unit test that reproduces the problem and the 
> proposed fix.

This message was sent by Atlassian JIRA

Reply via email to