[
https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14460:
--------------------------
Attachment: hack.uncommitted.patch
hack.flamegraph-16593.svg
Patch for discussion. Gets us back to 0.98 speeds; i.e. about 1/3rd slower than
0.94.
Idea is to unshackle Increments and MVCC other than to keep MVCC abreast of
sequenceId change. I can do this if I reorder Increment so the Increment get
and write (as well as sync) are all under the row lock; this makes it so my
read will get the latest always since no concurrent writer on this row
(because I have undone the mvcc connection, I need to read with isolation
UNCOMMITTED). If I reorder Increment so its read, append, sync, then update
memstore, I can undo the crazy +1B and the need of the post-modification of
Cells in MemStore.
The gambit is a slower Increment because all happens under the row lock
including the sync of the write which used to be done on the outside. This
makes it so we don't need MVCC for correctness and so can by-pass the
MVCC-is-a-region-wide-lock phenomenon.
See attached flamegraph. It looks like 0.98 now.
Some basic tests using the above attached IncrementTest (80 concurrent threads
doing an increment over 50k rows) show us doing:
{code}
75th: 3.92218
95th: 5.648627799999997
98th: 8.072542299999984
99th: 23.11843173
{code}
The same test against 0.98 as quoted above shows:
{code}
75th: 4.400081
95th: 6.0390387
98th: 6.7202052
99th: 7.264320360000001
Time: 191.393
{code}
Posting the patch for discussion.
Need to figure downsides. Will study the patch more. Our Increment in memstore
should work as expected when Scanning since we are using the actual assigned
sequenceid. On crash, edit could be in WAL and client may not know it made it
but this has always been an issue.
> [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed
> Increments, CheckAndPuts, batch operations
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-14460
> URL: https://issues.apache.org/jira/browse/HBASE-14460
> Project: HBase
> Issue Type: Bug
> Components: Performance
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Attachments: 0.94.test.patch, 0.98.test.patch,
> 1.0.80.flamegraph-7932.svg, 14460.txt, 98.80.flamegraph-11428.svg,
> HBASE-14460-discussion.patch, client.test.patch,
> flamegraph-13120.svg.master.singlecell.svg, flamegraph-26636.094.100.svg,
> flamegraph-28066.098.singlecell.svg, flamegraph-28767.098.100.svg,
> flamegraph-31647.master.100.svg, flamegraph-9466.094.singlecell.svg,
> hack.flamegraph-16593.svg, hack.uncommitted.patch, m.test.patch,
> region_lock.png, testincrement.094.patch, testincrement.098.patch,
> testincrement.master.patch
>
>
> As reported by 鈴木俊裕 up on the mailing list -- see "Performance degradation
> between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of
> sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to
> 'catch up' to our current point before we can read the last Increment value
> that we need to update.
> We can say that our Increment is just done wrong, we should just be writing
> Increments and summing on read, but checkAndPut as well as batching
> operations have the same issue. Fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
