[
https://issues.apache.org/jira/browse/HBASE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008872#comment-13008872
]
Benoit Sigoure commented on HBASE-2170:
---------------------------------------
bq. This is an impressive number. Just curious if u were able to run the same
benchmark with WAL turned on, and what numbers you see then..
Curiously enough, I see the same numbers.
This is the 1st import I did Thursday (no WAL)
{code}
$ ./src/tsdb import /tmp/data.gz
[...]
2011-03-17 18:45:51,797 INFO [main] TextImporter: ... 1000000 data points in
6688ms (149521.5 points/s)
2011-03-17 18:45:56,836 INFO [main] TextImporter: ... 2000000 data points in
5044ms (198255.4 points/s)
2011-03-17 18:46:01,823 INFO [main] TextImporter: ... 3000000 data points in
4986ms (200561.6 points/s)
2011-03-17 18:46:06,848 INFO [main] TextImporter: ... 4000000 data points in
5025ms (199005.0 points/s)
2011-03-17 18:46:11,865 INFO [main] TextImporter: ... 5000000 data points in
5016ms (199362.0 points/s)
2011-03-17 18:46:14,315 INFO [main] TextImporter: Processed /tmp/data.gz in
29211 ms, 5487065 data points (187842.4 points/s)
2011-03-17 18:46:14,315 INFO [main] TextImporter: Total: imported 5487065 data
points in 29.212s (187838.4 points/s)
{code}
Note: 1 data point = 1 {{KeyValue}}.
I commented out {{dp.setBatchImport(true);}} in
[TextImporter.getDataPoints|https://github.com/stumbleupon/opentsdb/blob/master/src/tools/TextImporter.java#L225]
and ran the same import again. Note: this isn't exactly an apples-to-apples
comparison because I'm going to overwrite existing {{KeyValue}} instead of
creating new ones. The table has {{VERSIONS=>1}} but I think we disabled major
compactions so we don't delete old data (Stack/JD correct me if I'm mistaken
about our setup).
{code}
$ ./src/tsdb import /tmp/data.gz
[...]
2011-03-19 19:09:36,102 INFO [main] TextImporter: ... 1000000 data points in
6699ms (149276.0 points/s)
2011-03-19 19:09:41,101 INFO [main] TextImporter: ... 2000000 data points in
5004ms (199840.1 points/s)
2011-03-19 19:09:46,051 INFO [main] TextImporter: ... 3000000 data points in
4949ms (202061.0 points/s)
2011-03-19 19:09:51,006 INFO [main] TextImporter: ... 4000000 data points in
4955ms (201816.3 points/s)
2011-03-19 19:09:56,017 INFO [main] TextImporter: ... 5000000 data points in
5010ms (199600.8 points/s)
2011-03-19 19:09:58,422 INFO [main] TextImporter: Processed /tmp/data.gz in
29025 ms, 5487065 data points (189046.2 points/s)
2011-03-19 19:09:58,422 INFO [main] TextImporter: Total: imported 5487065 data
points in 29.026s (189041.3 points/s)
{code}
So... this totally surprises me. I expected to see a big performance drop with
the WAL enabled. I wondered if I didn't properly recompile the code or if
something else was still disabling the WAL, but I verified with {{strace}} that
the WAL was turned on in the RPC that was going out:
{code}
$ strace -f -e trace=write -s 4096 ./src/tsdb import /tmp/data.gz
[...]
[pid 21364] write(32,
"\0\0\312\313\0\0\0\3\0\10multiPut\0\0\0\00199\0\0\0\1Btsdb,\0\3\371L\301[\360\0\0\7\0\2;,1300586854474.a2a283a471dfcf5dcda82d05f2d468ed.\0\0\0:\1\r\0\3\371MZ2\200\0\0\7\0\0\216\177\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\1\0\0\0\1\1t\0\0\0(\0\0\6\340\0\0\0,\0\0\0\34\0\0\0\10\0\r\0\3\371MZ2\200\0\0\7\0\0\216\1t\0{\177\377\377\377\377\377\377\377\4\0\0\0\0C>\0\0\0\0\0,\0\0\0\34\0\0\0\10\0\r\0\3\371MZ2\200\0\0\7\0\0\216\1t\1k\177\377\377\377\377\377\377\377\4\0\0\0\0Cd...
{code}
This shows that the WAL is enabled. Having the source of [asynchbase's
{{MultiPutRequest}}|https://github.com/stumbleupon/asynchbase/blob/master/src/MultiPutRequest.java#L274]
greatly helps make sense of this otherwise impossible to understand blob:
* We can easily see where the region name is, it contains an MD5 sum followed
by a period ({{.}}).
* After the region name, the next 4 bytes are the number of edits for this
region: {{\0\0\0:}} = 58
* Then there's a byte with value 1 with the "versioning" of the {{Put}} object:
{{\1}}
* Then there's a the row key of the row we're writing to:
{{\r\0\3\371MZ2\200\0\0\7\0\0\216}} where:
** {{\r}} is a {{vint}} indicating that the key length is 13 bytes
** The first 3 bytes of the row key in OpenTSDB correspond to the metric ID:
{{\0\3\371}}
** The next 4 bytes in OpenTSDB correspond to a UNIX timestamp: {{MZ2\200}}.
Using Python, it's easy to confirm that:
{code}
>>> import struct
>>> import time
>>> struct.unpack(">I", "MZ2\200")
(1297756800,)
>>> time.ctime(*_)
'Tue Feb 15 00:00:00 2011'
{code}
** The next 6 bytes in OpenTSDB correspond to a tag:
*** 3 bytes for a tag name ID: {{\0\0\7}}
*** 3 bytes for a tag value ID: {{\0\0\216}}
* Then we have the timestamp of the edit, which is unset, so it's
{{Long.MAX_VALUE}} which is {{\177\377\377\377\377\377\377\377}}
* Then we have the {{RowLock}} ID. In this case no row lock is involved, so
the value is {{-1L}}: {{\377\377\377\377\377\377\377\377}}
* Then we have one byte indicating whether or not to use the WAL. In this
case, the byte is {{\1}} so the WAL is enabled. □
After undoing my change to test once more with the WAL, I here's the output of
{{strace}}:
{code}
[pid 21727] write(32,
"\0\0\312\313\0\0\0\3\0\10multiPut\0\0\0\00199\0\0\0\1Btsdb,\0\3\371L\301[\360\0\0\7\0\2;,1300586854474.a2a283a471dfcf5dcda82d05f2d468ed.\0\0\0:\1\r\0\3\371MZ2\200\0\0\7\0\0\216\177\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\0\0\0\0\1\1t\0\0\0(\0\0\6\340\0\0\0,\0\0\0\34\0\0\0\10\0\r\0\3\371MZ2\200\0\0\7\0\0\216\1t\0{\177\377\377\377\377\377\377\377\4\0\0\0\0C>\0\0\0\0\0,\0\0\0\34\0\0\0\10\0\r\0\3\371MZ2\200\0\0\7\0\0\216\1t\1k\177\377\377\377\377\377\377\377\4\0\0\0\0Cd...
{code}
Note that the RPC is exactly the same modulo this byte that indicates whether
the WAL is enabled or not. In the last excerpt the byte is {{\0}} (it's the
first byte after the first long sequence of {{\377}}), which indicates that as
expected the WAL is disabled.
In both cases I can consistently do 200k KeyValue inserts per second.
asynchbase was written specifically for high throughput server applications.
I'm running the tests above on a machine with a single crappy Intel E5405
(2.00GHz, 1 physical CPUs, 4 cores/CPU, 1 hardware thread/core = 4 hw threads
total). The HBase cluster I'm writing to is one of StumbleUpon's clusters.
It's a fairly small cluster, but its size is largely irrelevant because the
keys imported here only live in 3 different regions on 3 different servers.
During the import, the client consumes about 130% CPU (or 1 core and a third,
if you prefer).
PS: Stack, I replied here because even though I hate JIRA, it was easier to
format my message here than on the mailing list.
> hbase lightweight client library as a distribution
> ---------------------------------------------------
>
> Key: HBASE-2170
> URL: https://issues.apache.org/jira/browse/HBASE-2170
> Project: HBase
> Issue Type: Wish
> Reporter: Kay Kay
> Attachments: jdepend-report.html
>
>
> As a wish - it would be nice to have a hbase client library (subset of the
> current hbase distribution) that needs to be present at the hbase client
> level to interact with the master/region servers.
> From an app integration - users of hbase can just link against the client
> library as opposed to getting the entire library to link against.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
