Jingcheng Du commented on HBASE-16578:

Thanks [~huaxiang].
Shouldn't the bulk loaded ref file have a bigger seqId?
I did a test for this case. The test code is modified based on the attached 
TestMobCompaction, and I remove the coprocessor CompactTwoLatestHfilesCopro.
1. put mob cell r1, flush, it will create ref1 and mobFile1.
2. put mob cell r2, flush, it will create ref2 and mobFile2.
3. put normal cell r3, flush, it will create hfile3.
4. minor or major compact, ref1, ref2, hfile3, it creates hfile4.
5. mob compact, it will create mobfile3 and bulkload ref3.
6. Now we have 2 files in the directory, bulkloaded ref3 and hfile4, and put a 
mob cell r_4, it will create ref4 and mobFile4.
6. compact hfile4, ref3, and ref4, try to get the value of r2, it is there.

Could you please share your case, and let me confirm it again? Thanks.

> Mob data loss after mob compaction and normal compcation
> --------------------------------------------------------
>                 Key: HBASE-16578
>                 URL: https://issues.apache.org/jira/browse/HBASE-16578
>             Project: HBase
>          Issue Type: Bug
>          Components: mob
>    Affects Versions: 2.0.0
>            Reporter: huaxiang sun
>            Assignee: Jingcheng Du
>         Attachments: HBASE-16578-V2.patch, HBASE-16578.patch, 
> TestMobCompaction.java, TestMobCompaction.java
> StoreFileScanners on MOB cells rely on the scannerOrder to find the latest 
> cells after mob compaction. The value of scannerOrder is assigned by the 
> order of maxSeqId of StoreFile, and this maxSeqId is valued only after the 
> reader of the StoreFile is created.
> In {{Compactor.compact}}, the compacted store files are cloned and their 
> readers are not created. And in {{StoreFileScanner.getScannersForStoreFiles}} 
> the StoreFiles are sorted before the readers are created and at that time the 
> maxSeqId for each file is -1 (the default value). This will lead  to a chaos 
> in scanners in the following normal compaction. Some older cells might be 
> chosen during the normal compaction.
> We need to create readers either before the sorting in the method 
> {{StoreFileScanner.getScannersForStoreFiles}}, or create readers just after 
> the store files are cloned in {{Compactor.compact}}.

This message was sent by Atlassian JIRA

Reply via email to