[
https://issues.apache.org/jira/browse/HBASE-11401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045055#comment-14045055
]
stack commented on HBASE-11401:
-------------------------------
Doing [~jeffreyz] suggestion, there is little loss changing the order:
Without patch:
{code}
threads=3, iterations=1000000, syncInterval=0 took 1276.339s 2350.473ops/s
threads=10, iterations=1000000, syncInterval=0 took 1317.274s 7591.435ops/s
threads=25, iterations=1000000, syncInterval=0 took 1422.012s 17580.725ops/s
threads=100, iterations=1000000, syncInterval=0 took 1525.249s 65563.063ops/s
{code}
With patch:
{code}
threads=3, iterations=1000000, syncInterval=0 took 1231.335s 2436.380ops/s
threads=10, iterations=1000000, syncInterval=0 took 1306.012s 7656.897ops/s
threads=25, iterations=1000000, syncInterval=0 took 1421.756s 17583.889ops/s
threads=100, iterations=1000000, syncInterval=0 took 1577.127s 63406.438ops/s
{code}
As concurrency goes up, the patch slows us slightly.
Unfortunately, because we add the memstore before we sync, we cannot remove the
rollback. Let me get some numbers on sync'ing before adding to memstore.
> Issue with seqNo binding for KV mvcc
> ------------------------------------
>
> Key: HBASE-11401
> URL: https://issues.apache.org/jira/browse/HBASE-11401
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.99.0
> Reporter: Anoop Sam John
> Priority: Critical
> Fix For: 0.99.0
>
> Attachments: memstore.txt
>
>
> After HBASE-8763, we have combined KV mvcc and HLog seqNo. This is
> implemented in a tricky way now.
> In HRegion on write path, we first write to memstore and then write to HLog
> finally sync log. So at the time of write to memstore we dont know the WAL
> seqNo. To overcome this, we hold the ref to the KV objects just added to
> memstore and pass those also to write to wal call. Once the seqNo is
> obtained, we will reset the mvcc is those KVs with this seqNo. (While write
> to memstore we wrote kvs with a very high temp value for mvcc so that
> concurrent readers wont see them)
> This model works well with the DefaultMemstore. During the write there wont
> be any concurrent call to snapshot().
> But now we have memstore as a pluggable interface. The above model of late
> binding assumes that the memstore internal datastructure continue to refer to
> same java objects. This might not be true always. Like in HBASE-10713, in
> btw the kvs can be converted into a CellBlock. If we discontinue to refer to
> same KV java objects, we will fail in getting the seqNo assigned as kv mvcc.
> If we were doing write and sync to wal and then write to memstore, this would
> have get solved. But this model we changed (in 94 I believe) for better perf.
> Under HRegion level lock, we write to memstore and then to wal. Finally out
> of lock we do the the log sync. So we can not change it now
> I tried changing the order of ops within the lock (ie. write to log and then
> to memstore) so that we can get the seqNo when write to memstore. But because
> of the new HLog write model, we are not guarenteed to get the write to done
> immediately.
> One possible way can be add a new API in Log level, to get a next seqNo
> alone. Call this first and then using which write to memstore and then to wal
> (using this seqNo). Just a random thought. Not tried.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
