[
https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812477#comment-13812477
]
Himanshu Vashishtha commented on HBASE-8741:
--------------------------------------------
Below are some results I got while testing HLogPE, with threads count varying
from 1-5 on a 5 node (1 NN, 4DN) cluster.
IMO, the result are mixed and there is almost negligible much perf hit.
{code}
for i in `seq 1 5` ; do for j in 1 2 3; do
/home/himanshu/hbase-0.97.0-SNAPSHOT/bin/hbase
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -verify
-threads "${i}" -iterations 1000000 -nocleanup -keySize 50 -valueSize 100;
done; done
{code}
||Threads||w/o patch time||w/o patch ops||w/ patch time||w/ patch ops||
|1|530.334s|1885.604ops/s|519.382s|1925.365ops/s|
|1|531.314s|1882.126ops/s|524.750s|1905.669ops/s|
|1|529.636s|1888.089ops/s|537.218s|1861.442ops/s|
|2|796.771s|2510.132ops/s|786.245s|2543.736ops/s|
|2|811.930s|2463.267ops/s|818.789s|2442.632ops/s|
|2|805.139s|2484.043ops/s|792.434s|2523.869ops/s|
|3|948.641s|3162.419ops/s|938.286s|3197.319ops/s|
|3|968.503s|3097.564ops/s|955.333s|3140.266ops/s|
|3|970.692s|3090.579ops/s|949.411s|3159.854ops/s|
|4|648.943s|6163.870ops/s|646.279s|6189.277ops/s|
|4|658.654s|6072.991ops/s|656.277s|6094.987ops/s|
|4|634.568s|6303.501ops/s|669.986s|5970.274ops/s|
|5|722.867s|6916.902ops/s|730.954s|6840.376ops/s|
|5|731.401s|6836.195ops/s|725.907s|6887.935ops/s|
|5|723.812s|6907.871ops/s|718.261s|6961.258ops/s|
> Scope sequenceid to the region rather than regionserver (WAS: Mutations on
> Regions in recovery mode might have same sequenceIDs)
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-8741
> URL: https://issues.apache.org/jira/browse/HBASE-8741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
> Affects Versions: 0.95.1
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Fix For: 0.98.0
>
> Attachments: HBASE-8741-trunk-v6.1-rebased.patch,
> HBASE-8741-trunk-v6.2.1.patch, HBASE-8741-trunk-v6.2.2.patch,
> HBASE-8741-trunk-v6.2.2.patch, HBASE-8741-trunk-v6.3.patch,
> HBASE-8741-trunk-v6.patch, HBASE-8741-v0.patch, HBASE-8741-v2.patch,
> HBASE-8741-v3.patch, HBASE-8741-v4-again.patch, HBASE-8741-v4-again.patch,
> HBASE-8741-v4.patch, HBASE-8741-v5-again.patch, HBASE-8741-v5.patch
>
>
> Currently, when opening a region, we find the maximum sequence ID from all
> its HFiles and then set the LogSequenceId of the log (in case the later is at
> a small value). This works good in recovered.edits case as we are not writing
> to the region until we have replayed all of its previous edits.
> With distributed log replay, if we want to enable writes while a region is
> under recovery, we need to make sure that the logSequenceId > maximum
> logSequenceId of the old regionserver. Otherwise, we might have a situation
> where new edits have same (or smaller) sequenceIds.
> We can store region level information in the WALTrailer, than this scenario
> could be avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file
> which has a trailer and,
> b) completely reading the last wal file (this file would not have the
> trailer, so it needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all
> completed WAL files, and reading the remaining incomplete files.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
