BTW, this is not related, but we should not run with dfs.client.read.shortcircuit.skip.checksum=true
This will cause WAL files read with short circuit to bypass checksum verification. We used to put up a warning in HBASE-8322, somehow it seems that part has disappeared from trunk. Enis On Tue, Mar 25, 2014 at 12:29 AM, Andrew Purtell <[email protected]>wrote: > Thought I'd look at how 0.98.0 and 0.98.1 RC1 compare using YCSB. It could > be .1 is just a hair slower than .0, but the results are similar. For some > workloads either .0 or .1 could be slightly ahead run to run. Results from > the most recent runs are presented below. > > *Hardware and Versions* > > Hadoop 2.2.0 > HBase 0.98.1-hadoop2 RC1 > > 5x EC2 cc2.8xlarge > 32 cores > 60 GB RAM > 4 x 840 GB directly attached spinning media, as DN storage volumes > NameNode: 4 GB heap > DataNode: 1 GB heap > Master: 4 GB heap > RegionServer: 8 GB heap, 24 GB bucket cache offheap engine > > 1x EC2 cc2.8xlarge > YCSB > > *Configuration* > > Hadoop and HBase default site configuration, except: > - dfs.client.read.shortcircuit=true > - dfs.client.read.shortcircuit.skip.checksum=true > - dfs.client.domain.socket.data.traffic=true > - dfs.domain.socket.path=... > - dfs.datanode.max.xcievers = 16384 > - dfs.namenode.handler.count = 100 > - ipc.server.tcpnodelay=true > - hbase.ipc.client.tcpnodelay=true > - hbase.bucketcache.ioengine=offheap > - hbase.bucketcache.size=0.9 > > *Methodology* > > Setup: > > 0. Start cluster > 1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" } > 2. YCSB: Preload 100 million rows into table "seed" > 3. shell: compact "seed" > 4. Wait for compaction to complete > 5. shell: create_snapshot "seed", "seed_snap" > 6. shell: disable "seed" > > For each test: > > 7. shell: clone_snapshot "seed_snap", "test" > 8. YCSB: Run test -p operationcount=10000000 -threads 32 -target 50000 > (clamp at ~10k ops/server/sec) > 9. shell: disable "test" > 10. shell: drop "test" > > *Workload A* > > * 0.98.0* > > [OVERALL], RunTime(ms), 200313.0 > [OVERALL], Throughput(ops/sec), 49921.87226989761 > [READ], Operations, 5002419 > [READ], AverageLatency(us), 1136.5890318264023 > [READ], MinLatency(us), 281 > [READ], MaxLatency(us), 1134224 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 2 > [UPDATE], Operations, 4997613 > [UPDATE], AverageLatency(us), 6.608387844356896 > [UPDATE], MinLatency(us), 0 > [UPDATE], MaxLatency(us), 724894 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > * 0.98.1* > > [OVERALL], RunTime(ms), 200323.0 > [OVERALL], Throughput(ops/sec), 49919.38020097542 > [READ], Operations, 5001366 > [READ], AverageLatency(us), 1157.195526182247 > [READ], MinLatency(us), 311 > [READ], MaxLatency(us), 1116840 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 2 > [UPDATE], Operations, 4998666 > [UPDATE], AverageLatency(us), 6.185563908450774 > [UPDATE], MinLatency(us), 0 > [UPDATE], MaxLatency(us), 725483 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > *Workload B* > > * 0.98.0* > > [OVERALL], RunTime(ms), 264508.0 > [OVERALL], Throughput(ops/sec), 37806.039892933295 > [READ], Operations, 9499452 > [READ], AverageLatency(us), 880.201064545618 > [READ], MinLatency(us), 290 > [READ], MaxLatency(us), 1160947 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 1 > [UPDATE], Operations, 500580 > [UPDATE], AverageLatency(us), 12.54600063925846 > [UPDATE], MinLatency(us), 1 > [UPDATE], MaxLatency(us), 741148 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > * 0.98.1* > > [OVERALL], RunTime(ms), 265789.0 > [OVERALL], Throughput(ops/sec), 37623.8294286069 > [READ], Operations, 9499661 > [READ], AverageLatency(us), 884.9811973290415 > [READ], MinLatency(us), 273 > [READ], MaxLatency(us), 1186243 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 1 > [UPDATE], Operations, 500371 > [UPDATE], AverageLatency(us), 16.71922433554303 > [UPDATE], MinLatency(us), 1 > [UPDATE], MaxLatency(us), 923659 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > *Workload C* > > * 0.98.0* > > [OVERALL], RunTime(ms), 275967.0 > [OVERALL], Throughput(ops/sec), 36236.2166490921 > [READ], Operations, 10000000 > [READ], AverageLatency(us), 875.3169807 > [READ], MinLatency(us), 283 > [READ], MaxLatency(us), 1123204 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 1 > > * 0.98.1* > > [OVERALL], RunTime(ms), 274182.0 > [OVERALL], Throughput(ops/sec), 36472.124355355205 > [READ], Operations, 10000000 > [READ], AverageLatency(us), 868.9167299 > [READ], MinLatency(us), 287 > [READ], MaxLatency(us), 1082448 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 1 > > *Workload D* > > * 0.98.0* > > [OVERALL], RunTime(ms), 430518.0 > [OVERALL], Throughput(ops/sec), 23227.83251803641 > [INSERT], Operations, 501258 > [INSERT], AverageLatency(us), 18.74352529036943 > [INSERT], MinLatency(us), 5 > [INSERT], MaxLatency(us), 729485 > [INSERT], 95thPercentileLatency(ms), 0 > [INSERT], 99thPercentileLatency(ms), 0 > [READ], Operations, 9498742 > [READ], AverageLatency(us), 1435.8229302364462 > [READ], MinLatency(us), 278 > [READ], MaxLatency(us), 1086256 > [READ], 95thPercentileLatency(ms), 3 > [READ], 99thPercentileLatency(ms), 13 > > * 0.98.1* > > [OVERALL], RunTime(ms), 473086.0 > [OVERALL], Throughput(ops/sec), 21137.805811205573 > [INSERT], Operations, 500280 > [INSERT], AverageLatency(us), 18.910943871431996 > [INSERT], MinLatency(us), 5 > [INSERT], MaxLatency(us), 741632 > [INSERT], 95thPercentileLatency(ms), 0 > [INSERT], 99thPercentileLatency(ms), 0 > [READ], Operations, 9499720 > [READ], AverageLatency(us), 1574.2442979372024 > [READ], MinLatency(us), 285 > [READ], MaxLatency(us), 1142568 > [READ], 95thPercentileLatency(ms), 6 > [READ], 99thPercentileLatency(ms), 15 > > *Workload E* > > * 0.98.0* > > [OVERALL], RunTime(ms), 1741385.0 > [OVERALL], Throughput(ops/sec), 5742.555494620661 > [INSERT], Operations, 498580 > [INSERT], AverageLatency(us), 12.785394520438045 > [INSERT], MinLatency(us), 5 > [INSERT], MaxLatency(us), 47371 > [INSERT], 95thPercentileLatency(ms), 0 > [INSERT], 99thPercentileLatency(ms), 0 > [SCAN], Operations, 9501420 > [SCAN], AverageLatency(us), 5832.021051169194 > [SCAN], MinLatency(us), 850 > [SCAN], MaxLatency(us), 18383187 > [SCAN], 95thPercentileLatency(ms), 11 > [SCAN], 99thPercentileLatency(ms), 22 > > * 0.98.1* > > [OVERALL], RunTime(ms), 1626498.0 > [OVERALL], Throughput(ops/sec), 6148.178479161978 > [INSERT], Operations, 498248 > [INSERT], AverageLatency(us), 14.589842809203448 > [INSERT], MinLatency(us), 5 > [INSERT], MaxLatency(us), 726310 > [INSERT], 95thPercentileLatency(ms), 0 > [INSERT], 99thPercentileLatency(ms), 0 > [SCAN], Operations, 9501752 > [SCAN], AverageLatency(us), 5451.874070381968 > [SCAN], MinLatency(us), 847 > [SCAN], MaxLatency(us), 1278273 > [SCAN], 95thPercentileLatency(ms), 12 > [SCAN], 99thPercentileLatency(ms), 26 > > *Workload F* > > * 0.98.0* > > [OVERALL], RunTime(ms), 376552.0 > [OVERALL], Throughput(ops/sec), 26556.75710127685 > [READ], Operations, 10000000 > [READ], AverageLatency(us), 1192.262719 > [READ], MinLatency(us), 293 > [READ], MaxLatency(us), 1108215 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 2 > [READ-MODIFY-WRITE], Operations, 4999036 > [READ-MODIFY-WRITE], AverageLatency(us), 1198.5333812359022 > [READ-MODIFY-WRITE], MinLatency(us), 299 > [READ-MODIFY-WRITE], MaxLatency(us), 1070294 > [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1 > [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 2 > [READ-MODIFY-WRITE], 0, 2018024 > [UPDATE], Operations, 4999068 > [UPDATE], AverageLatency(us), 3.940626532785711 > [UPDATE], MinLatency(us), 1 > [UPDATE], MaxLatency(us), 155147 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > * 0.98.1* > > [OVERALL], RunTime(ms), 385273.0 > [OVERALL], Throughput(ops/sec), 25955.621079078992 > [READ], Operations, 10000000 > [READ], AverageLatency(us), 1218.9608977 > [READ], MinLatency(us), 296 > [READ], MaxLatency(us), 1128785 > [READ], 95thPercentileLatency(ms), 1 > [READ], 99thPercentileLatency(ms), 3 > [READ-MODIFY-WRITE], Operations, 5001281 > [READ-MODIFY-WRITE], AverageLatency(us), 1226.2466698031965 > [READ-MODIFY-WRITE], MinLatency(us), 300 > [READ-MODIFY-WRITE], MaxLatency(us), 1129077 > [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1 > [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 3 > [UPDATE], Operations, 5001313 > [UPDATE], AverageLatency(us), 4.26973976633736 > [UPDATE], MinLatency(us), 1 > [UPDATE], MaxLatency(us), 120701 > [UPDATE], 95thPercentileLatency(ms), 0 > [UPDATE], 99thPercentileLatency(ms), 0 > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
