[
https://issues.apache.org/jira/browse/HBASE-14028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618938#comment-14618938
]
stack commented on HBASE-14028:
-------------------------------
Latest:
I see a swath of edits that were posted below not showing up on the far end
though they were apparently successfully digested on the remote end (found by
aligning count of edits and some extra logging of sequenceids added in testbed):
2015-07-07 07:16:35,728 DEBUG [RS_LOG_REPLAY_OPS-c2024:16020-0-Writer-2]
wal.WALEditsReplaySink: Replayed 231 edits in 2458ms into
region=IntegrationTestBigLinkedList,\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xF8,1436277280607.bb166b99140bcd32df68676b4e1b60b2.,
hostname=c2025.halxg.cloudera.com,16020,1436278565173, seqNum=320072187,
lastSequenceId=280072763
At the time, the recovering region is flushing -- a few logs are being replayed
into this recovering region concurrently -- which is what is unusual around
this event.
I don't really seen filtering going on sink-side (except if not the primary
replica).
Adding more logging and retrying.
> DistributedLogReplay drops edits when ITBLL 125M
> ------------------------------------------------
>
> Key: HBASE-14028
> URL: https://issues.apache.org/jira/browse/HBASE-14028
> Project: HBase
> Issue Type: Bug
> Components: Recovery
> Affects Versions: 1.2.0
> Reporter: stack
>
> Testing DLR before 1.2.0RC gets cut, we are dropping edits.
> Issue seems to be around replay into a deployed region that is on a server
> that dies before all edits have finished replaying. Logging is sparse on
> sequenceid accounting so can't tell for sure how it is happening (and if our
> now accounting by Store is messing up DLR). Digging.
> I notice also that DLR does not refresh its cache of region location on error
> -- it just keeps trying till whole WAL fails.... 8 retries...about 30
> seconds. We could do a bit of refactor and have the replay find region in new
> location if moved during DLR replay.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
