How many times did you run this test? (test suite)
On Mar 25, 2014, at 7:29 AM, Andrew Purtell <[email protected]> wrote:
> Thought I'd look at how 0.98.0 and 0.98.1 RC1 compare using YCSB. It could
> be .1 is just a hair slower than .0, but the results are similar. For some
> workloads either .0 or .1 could be slightly ahead run to run. Results from
> the most recent runs are presented below.
>
> *Hardware and Versions*
>
> Hadoop 2.2.0
> HBase 0.98.1-hadoop2 RC1
>
> 5x EC2 cc2.8xlarge
> 32 cores
> 60 GB RAM
> 4 x 840 GB directly attached spinning media, as DN storage volumes
> NameNode: 4 GB heap
> DataNode: 1 GB heap
> Master: 4 GB heap
> RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
>
> 1x EC2 cc2.8xlarge
> YCSB
>
> *Configuration*
>
> Hadoop and HBase default site configuration, except:
> - dfs.client.read.shortcircuit=true
> - dfs.client.read.shortcircuit.skip.checksum=true
> - dfs.client.domain.socket.data.traffic=true
> - dfs.domain.socket.path=...
> - dfs.datanode.max.xcievers = 16384
> - dfs.namenode.handler.count = 100
> - ipc.server.tcpnodelay=true
> - hbase.ipc.client.tcpnodelay=true
> - hbase.bucketcache.ioengine=offheap
> - hbase.bucketcache.size=0.9
>
> *Methodology*
>
> Setup:
>
> 0. Start cluster
> 1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
> 2. YCSB: Preload 100 million rows into table "seed"
> 3. shell: compact "seed"
> 4. Wait for compaction to complete
> 5. shell: create_snapshot "seed", "seed_snap"
> 6. shell: disable "seed"
>
> For each test:
>
> 7. shell: clone_snapshot "seed_snap", "test"
> 8. YCSB: Run test -p operationcount=10000000 -threads 32 -target 50000
> (clamp at ~10k ops/server/sec)
> 9. shell: disable "test"
> 10. shell: drop "test"
>
> *Workload A*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 200313.0
> [OVERALL], Throughput(ops/sec), 49921.87226989761
> [READ], Operations, 5002419
> [READ], AverageLatency(us), 1136.5890318264023
> [READ], MinLatency(us), 281
> [READ], MaxLatency(us), 1134224
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 2
> [UPDATE], Operations, 4997613
> [UPDATE], AverageLatency(us), 6.608387844356896
> [UPDATE], MinLatency(us), 0
> [UPDATE], MaxLatency(us), 724894
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 200323.0
> [OVERALL], Throughput(ops/sec), 49919.38020097542
> [READ], Operations, 5001366
> [READ], AverageLatency(us), 1157.195526182247
> [READ], MinLatency(us), 311
> [READ], MaxLatency(us), 1116840
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 2
> [UPDATE], Operations, 4998666
> [UPDATE], AverageLatency(us), 6.185563908450774
> [UPDATE], MinLatency(us), 0
> [UPDATE], MaxLatency(us), 725483
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
> *Workload B*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 264508.0
> [OVERALL], Throughput(ops/sec), 37806.039892933295
> [READ], Operations, 9499452
> [READ], AverageLatency(us), 880.201064545618
> [READ], MinLatency(us), 290
> [READ], MaxLatency(us), 1160947
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 1
> [UPDATE], Operations, 500580
> [UPDATE], AverageLatency(us), 12.54600063925846
> [UPDATE], MinLatency(us), 1
> [UPDATE], MaxLatency(us), 741148
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 265789.0
> [OVERALL], Throughput(ops/sec), 37623.8294286069
> [READ], Operations, 9499661
> [READ], AverageLatency(us), 884.9811973290415
> [READ], MinLatency(us), 273
> [READ], MaxLatency(us), 1186243
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 1
> [UPDATE], Operations, 500371
> [UPDATE], AverageLatency(us), 16.71922433554303
> [UPDATE], MinLatency(us), 1
> [UPDATE], MaxLatency(us), 923659
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
> *Workload C*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 275967.0
> [OVERALL], Throughput(ops/sec), 36236.2166490921
> [READ], Operations, 10000000
> [READ], AverageLatency(us), 875.3169807
> [READ], MinLatency(us), 283
> [READ], MaxLatency(us), 1123204
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 1
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 274182.0
> [OVERALL], Throughput(ops/sec), 36472.124355355205
> [READ], Operations, 10000000
> [READ], AverageLatency(us), 868.9167299
> [READ], MinLatency(us), 287
> [READ], MaxLatency(us), 1082448
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 1
>
> *Workload D*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 430518.0
> [OVERALL], Throughput(ops/sec), 23227.83251803641
> [INSERT], Operations, 501258
> [INSERT], AverageLatency(us), 18.74352529036943
> [INSERT], MinLatency(us), 5
> [INSERT], MaxLatency(us), 729485
> [INSERT], 95thPercentileLatency(ms), 0
> [INSERT], 99thPercentileLatency(ms), 0
> [READ], Operations, 9498742
> [READ], AverageLatency(us), 1435.8229302364462
> [READ], MinLatency(us), 278
> [READ], MaxLatency(us), 1086256
> [READ], 95thPercentileLatency(ms), 3
> [READ], 99thPercentileLatency(ms), 13
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 473086.0
> [OVERALL], Throughput(ops/sec), 21137.805811205573
> [INSERT], Operations, 500280
> [INSERT], AverageLatency(us), 18.910943871431996
> [INSERT], MinLatency(us), 5
> [INSERT], MaxLatency(us), 741632
> [INSERT], 95thPercentileLatency(ms), 0
> [INSERT], 99thPercentileLatency(ms), 0
> [READ], Operations, 9499720
> [READ], AverageLatency(us), 1574.2442979372024
> [READ], MinLatency(us), 285
> [READ], MaxLatency(us), 1142568
> [READ], 95thPercentileLatency(ms), 6
> [READ], 99thPercentileLatency(ms), 15
>
> *Workload E*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 1741385.0
> [OVERALL], Throughput(ops/sec), 5742.555494620661
> [INSERT], Operations, 498580
> [INSERT], AverageLatency(us), 12.785394520438045
> [INSERT], MinLatency(us), 5
> [INSERT], MaxLatency(us), 47371
> [INSERT], 95thPercentileLatency(ms), 0
> [INSERT], 99thPercentileLatency(ms), 0
> [SCAN], Operations, 9501420
> [SCAN], AverageLatency(us), 5832.021051169194
> [SCAN], MinLatency(us), 850
> [SCAN], MaxLatency(us), 18383187
> [SCAN], 95thPercentileLatency(ms), 11
> [SCAN], 99thPercentileLatency(ms), 22
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 1626498.0
> [OVERALL], Throughput(ops/sec), 6148.178479161978
> [INSERT], Operations, 498248
> [INSERT], AverageLatency(us), 14.589842809203448
> [INSERT], MinLatency(us), 5
> [INSERT], MaxLatency(us), 726310
> [INSERT], 95thPercentileLatency(ms), 0
> [INSERT], 99thPercentileLatency(ms), 0
> [SCAN], Operations, 9501752
> [SCAN], AverageLatency(us), 5451.874070381968
> [SCAN], MinLatency(us), 847
> [SCAN], MaxLatency(us), 1278273
> [SCAN], 95thPercentileLatency(ms), 12
> [SCAN], 99thPercentileLatency(ms), 26
>
> *Workload F*
>
> * 0.98.0*
>
> [OVERALL], RunTime(ms), 376552.0
> [OVERALL], Throughput(ops/sec), 26556.75710127685
> [READ], Operations, 10000000
> [READ], AverageLatency(us), 1192.262719
> [READ], MinLatency(us), 293
> [READ], MaxLatency(us), 1108215
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 2
> [READ-MODIFY-WRITE], Operations, 4999036
> [READ-MODIFY-WRITE], AverageLatency(us), 1198.5333812359022
> [READ-MODIFY-WRITE], MinLatency(us), 299
> [READ-MODIFY-WRITE], MaxLatency(us), 1070294
> [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1
> [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 2
> [READ-MODIFY-WRITE], 0, 2018024
> [UPDATE], Operations, 4999068
> [UPDATE], AverageLatency(us), 3.940626532785711
> [UPDATE], MinLatency(us), 1
> [UPDATE], MaxLatency(us), 155147
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
> * 0.98.1*
>
> [OVERALL], RunTime(ms), 385273.0
> [OVERALL], Throughput(ops/sec), 25955.621079078992
> [READ], Operations, 10000000
> [READ], AverageLatency(us), 1218.9608977
> [READ], MinLatency(us), 296
> [READ], MaxLatency(us), 1128785
> [READ], 95thPercentileLatency(ms), 1
> [READ], 99thPercentileLatency(ms), 3
> [READ-MODIFY-WRITE], Operations, 5001281
> [READ-MODIFY-WRITE], AverageLatency(us), 1226.2466698031965
> [READ-MODIFY-WRITE], MinLatency(us), 300
> [READ-MODIFY-WRITE], MaxLatency(us), 1129077
> [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1
> [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 3
> [UPDATE], Operations, 5001313
> [UPDATE], AverageLatency(us), 4.26973976633736
> [UPDATE], MinLatency(us), 1
> [UPDATE], MaxLatency(us), 120701
> [UPDATE], 95thPercentileLatency(ms), 0
> [UPDATE], 99thPercentileLatency(ms), 0
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
