ramkrishna.s.vasudevan commented on HBASE-5782:

The problem here is 
HLogKey logKey = makeKey(encodedRegionName, tableName, seqNum, now, clusterId);
doWrite(info, logKey, edits, htd);
We form a key here which is unique based on the seqNum. Now as we do 
appendNoSync() there is no sync called in append flow.  But there is always the 
LogSyncer thread which periodically syncs.
All the edits are added to 
synchronized void append(Entry e) throws IOException {

Here the pendingWrites are obtained and then it is flushed one by one.
pending = logSyncerThread.getPendingWrites();
        // First flush all the pending writes to HDFS. Then 
        // issue the sync to HDFS. If sync is successful, then update
        // syncedTillHere to indicate that transactions till this
        // number has been successfully synced.
        logSyncerThread.hlogFlush(tempWriter, pending);
Here there is no synchronization and hence the pendingwrites are not in 
sequence.  Because it is not in sequence the recovered.edits file is also 
created with no sequence in some cases.
if (key.getLogSeqNum() <= currentEditSeqId) {
                  LOG.debug("******** key is " + key.getLogSeqNum()
                                                + "current edit is " + 
I just added the above log while replayRecoveredEdits and i can find 
******** key is 175current edit is 176
******** key is 174current edit is 176
******** key is 186current edit is 187
******** key is 188current edit is 190
******* key is 189current edit is 190
******** key is 204current edit is 205
******** key is 206current edit is 207
******** key is 217current edit is 218
******** key is 216current edit is 218
******** key is 255current edit is 256

This leads to dataloss.  Correct me if am wrong.

> Not all the regions are getting assigned after the log splitting.
> -----------------------------------------------------------------
>                 Key: HBASE-5782
>                 URL: https://issues.apache.org/jira/browse/HBASE-5782
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 0.94.0
>            Reporter: Gopinathan A
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Blocker
>             Fix For: 0.94.0
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
For more information on JIRA, see: http://www.atlassian.com/software/jira


Reply via email to