[
https://issues.apache.org/jira/browse/HBASE-26874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault resolved HBASE-26874.
---------------------------------------
Fix Version/s: 2.6.0
3.0.0-beta-1
Release Note: VerifyReplication can now do multiple recompares (rather
than 1 previously), with exponential backoff. In order to speed up the job when
there are many discrepancies, the recompares can be run in background threads.
New options added: --recompareTries (default 1 if --recompareSleep is > 0,
otherwise 0), --recompareThreads (default 0), --recompareBackoffExponent
(default 1).
Resolution: Fixed
Thanks [~hgromer] for all the work here!
> VerifyReplication recompare async
> ---------------------------------
>
> Key: HBASE-26874
> URL: https://issues.apache.org/jira/browse/HBASE-26874
> Project: HBase
> Issue Type: Improvement
> Reporter: Bryan Beaudreault
> Assignee: Hernan Gelaf-Romer
> Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> VerifyReplication includes an option "sleepMsBeforeReCompare". This is useful
> for helping work around replication lag. However, adding a sleep in a hadoop
> job can drastically slow that job down if there is anything more than a small
> number of invalid results.
> We can mitigate this by doing the recompare in a separate thread. We can
> limit the thread pool and fallback to doing the recompare in the main thread
> if the thread pool is full. This way we offload some of the slowness but
> still retain the same validation guarantees. A configuration can be added to
> control how many threads per mapper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
