[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061757#comment-14061757
]
graham sanderson edited comment on CASSANDRA-7546 at 7/15/14 6:30 AM:
----------------------------------------------------------------------
I tried on 2.1 with pretty similar results (note I used a fresh HeapPool for
each iteration, since it got stuck with the default Native stuff) - although
the 2.1 BTree is much cheaper than the SnapTreeMap
{code}
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 64
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 562ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 238MB vs 8MB; ratio to raw data size =
29.79389619047619
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 480ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 239MB vs 8MB; ratio to raw data size =
29.95105523809524
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 376ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 219MB vs 8MB; ratio to raw data size =
27.37058095238095
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 349ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 219MB vs 8MB; ratio to raw data size =
27.40619619047619
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 268ms maxConcurrency = 1
[junit] Approx allocation = 198MB vs 8MB; ratio to raw data size =
24.820686666666667
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 277ms maxConcurrency = 1
[junit] Approx allocation = 198MB vs 8MB; ratio to raw data size =
24.82146285714286
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 258ms maxConcurrency = 1
[junit] Approx allocation = 190MB vs 8MB; ratio to raw data size =
23.770030476190477
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 234ms maxConcurrency = 1
[junit] Approx allocation = 190MB vs 8MB; ratio to raw data size =
23.769555238095236
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 64
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 3908ms maxConcurrency = 100
[junit] GC for PS Scavenge: 36 ms for 10 collections
[junit] Approx allocation = 2481MB vs 8MB; ratio to raw data size =
309.730900952381
[junit] loopRatio (closest to 1 best) 16.04825 raw 100000/1604825 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 697ms maxConcurrency = 100
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 241MB vs 8MB; ratio to raw data size =
30.141012380952382
[junit] loopRatio (closest to 1 best) 1.00004 raw 287/289 counted 2/2
sync 99712/99713 up 1 down 1
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 867ms maxConcurrency = 99
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 378MB vs 8MB; ratio to raw data size =
47.22428
[junit] loopRatio (closest to 1 best) 2.03201 raw 100000/203201 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 797ms maxConcurrency = 99
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 252MB vs 8MB; ratio to raw data size =
31.476938095238093
[junit] loopRatio (closest to 1 best) 1.17123 raw 22254/30788 counted
7823/8564 sync 74967/77771 up 5846 down 4781
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 545ms maxConcurrency = 96
[junit] Approx allocation = 218MB vs 8MB; ratio to raw data size =
27.227550476190476
[junit] loopRatio (closest to 1 best) 1.09275 raw 100000/109275 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 668ms maxConcurrency = 98
[junit] Approx allocation = 221MB vs 8MB; ratio to raw data size =
27.620777142857143
[junit] loopRatio (closest to 1 best) 1.10094 raw 90437/98693 counted
7860/8165 sync 2719/3236 up 7569 down 7598
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 524ms maxConcurrency = 96
[junit] Approx allocation = 205MB vs 8MB; ratio to raw data size =
25.604992380952382
[junit] loopRatio (closest to 1 best) 1.02883 raw 100000/102883 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 589ms maxConcurrency = 100
[junit] Approx allocation = 217MB vs 8MB; ratio to raw data size =
27.18481619047619
[junit] loopRatio (closest to 1 best) 1.02795 raw 97415/100035 counted
2544/2581 sync 136/179 up 2530 down 2537
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 256
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 685ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 283MB vs 26MB; ratio to raw data size =
10.771714782608695
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 561ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 294MB vs 26MB; ratio to raw data size =
11.198217391304349
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 495ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 273MB vs 26MB; ratio to raw data size =
10.378718550724638
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 539ms maxConcurrency = 1
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 272MB vs 26MB; ratio to raw data size =
10.367084347826086
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 197ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 249MB vs 26MB; ratio to raw data size =
9.492887826086957
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 216ms maxConcurrency = 1
[junit] GC for PS Scavenge: 9 ms for 1 collections
[junit] Approx allocation = 251MB vs 26MB; ratio to raw data size =
9.570564347826087
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 229ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 243MB vs 26MB; ratio to raw data size =
9.23958231884058
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 517ms maxConcurrency = 1
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 243MB vs 26MB; ratio to raw data size =
9.258937101449275
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 256
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 3728ms maxConcurrency = 100
[junit] GC for PS Scavenge: 44 ms for 12 collections
[junit] Approx allocation = 3031MB vs 26MB; ratio to raw data size =
115.17077130434782
[junit] loopRatio (closest to 1 best) 15.84521 raw 100000/1584521 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 479ms maxConcurrency = 100
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 302MB vs 26MB; ratio to raw data size =
11.507159130434783
[junit] loopRatio (closest to 1 best) 1.00188 raw 252/431 counted 7/8
sync 99746/99749 up 4 down 4
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 967ms maxConcurrency = 100
[junit] GC for PS Scavenge: 18 ms for 2 collections
[junit] Approx allocation = 486MB vs 26MB; ratio to raw data size =
18.50185797101449
[junit] loopRatio (closest to 1 best) 2.2405 raw 100000/224050 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 442ms maxConcurrency = 100
[junit] GC for PS Scavenge: 9 ms for 1 collections
[junit] Approx allocation = 314MB vs 26MB; ratio to raw data size =
11.941331304347827
[junit] loopRatio (closest to 1 best) 1.22081 raw 26220/37315 counted
9907/10839 sync 70268/73927 up 7625 down 5981
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 615ms maxConcurrency = 93
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 269MB vs 26MB; ratio to raw data size =
10.220121739130434
[junit] loopRatio (closest to 1 best) 1.08956 raw 100000/108956 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 588ms maxConcurrency = 100
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 271MB vs 26MB; ratio to raw data size =
10.32604347826087
[junit] loopRatio (closest to 1 best) 1.0914 raw 91675/99366 counted
7106/7372 sync 1984/2402 up 6870 down 6917
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 549ms maxConcurrency = 95
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 254MB vs 26MB; ratio to raw data size =
9.666266956521739
[junit] loopRatio (closest to 1 best) 1.02505 raw 100000/102505 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 417ms maxConcurrency = 96
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 255MB vs 26MB; ratio to raw data size =
9.724647246376811
[junit] loopRatio (closest to 1 best) 1.02704 raw 97472/100005 counted
2453/2491 sync 166/208 up 2437 down 2448
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 1024
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 721ms maxConcurrency = 1
[junit] GC for PS Scavenge: 30 ms for 2 collections
[junit] Approx allocation = 504MB vs 99MB; ratio to raw data size =
5.068735019157088
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 636ms maxConcurrency = 1
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 499MB vs 99MB; ratio to raw data size =
5.012685900383142
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 516ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 498MB vs 99MB; ratio to raw data size =
5.003988429118774
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 469ms maxConcurrency = 1
[junit] GC for PS Scavenge: 24 ms for 2 collections
[junit] Approx allocation = 489MB vs 99MB; ratio to raw data size =
4.91579908045977
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 419ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 476MB vs 99MB; ratio to raw data size =
4.782679157088123
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 433ms maxConcurrency = 1
[junit] GC for PS Scavenge: 26 ms for 2 collections
[junit] Approx allocation = 470MB vs 99MB; ratio to raw data size =
4.723301609195402
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 423ms maxConcurrency = 1
[junit] GC for PS Scavenge: 27 ms for 2 collections
[junit] Approx allocation = 465MB vs 99MB; ratio to raw data size =
4.679395708812261
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 445ms maxConcurrency = 1
[junit] GC for PS Scavenge: 35 ms for 2 collections
[junit] Approx allocation = 463MB vs 99MB; ratio to raw data size =
4.654131800766284
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 1024
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 3641ms maxConcurrency = 100
[junit] GC for PS Scavenge: 79 ms for 23 collections
[junit] Approx allocation = 6272MB vs 99MB; ratio to raw data size =
62.99940222222222
[junit] loopRatio (closest to 1 best) 16.84948 raw 100000/1684948 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 637ms maxConcurrency = 100
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 508MB vs 99MB; ratio to raw data size =
5.109580383141762
[junit] loopRatio (closest to 1 best) 1.0001 raw 805/812 counted 3/3 sync
99194/99195 up 1 down 1
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 1261ms maxConcurrency = 100
[junit] GC for PS Scavenge: 37 ms for 4 collections
[junit] Approx allocation = 963MB vs 99MB; ratio to raw data size =
9.67368030651341
[junit] loopRatio (closest to 1 best) 2.30544 raw 100000/230544 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 333ms maxConcurrency = 100
[junit] GC for PS Scavenge: 19 ms for 2 collections
[junit] Approx allocation = 559MB vs 99MB; ratio to raw data size =
5.62049540229885
[junit] loopRatio (closest to 1 best) 1.20045 raw 23136/32826 counted
9312/10027 sync 73535/77192 up 6865 down 5302
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 383ms maxConcurrency = 92
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 476MB vs 99MB; ratio to raw data size =
4.786960459770115
[junit] loopRatio (closest to 1 best) 1.08511 raw 100000/108511 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 491ms maxConcurrency = 100
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 516MB vs 99MB; ratio to raw data size =
5.188691264367816
[junit] loopRatio (closest to 1 best) 1.09487 raw 90981/98803 counted
7476/7714 sync 2486/2970 up 7229 down 7199
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 490ms maxConcurrency = 100
[junit] GC for PS Scavenge: 21 ms for 2 collections
[junit] Approx allocation = 474MB vs 99MB; ratio to raw data size =
4.76750030651341
[junit] loopRatio (closest to 1 best) 1.03162 raw 100000/103162 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 363ms maxConcurrency = 92
[junit] GC for PS Scavenge: 19 ms for 2 collections
[junit] Approx allocation = 480MB vs 99MB; ratio to raw data size =
4.826345440613027
[junit] loopRatio (closest to 1 best) 1.02209 raw 97920/100011 counted
2041/2057 sync 113/141 up 2032 down 2027
[junit]
[junit]
[junit] ==================================================
{code}
was (Author: graham sanderson):
I tried on 2.1 with pretty similar results (note I used a fresh HeapPool for
each iteration, since it got stuck with the default Native stuff)
{code}
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 64
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 562ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 238MB vs 8MB; ratio to raw data size =
29.79389619047619
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 480ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 239MB vs 8MB; ratio to raw data size =
29.95105523809524
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 376ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 219MB vs 8MB; ratio to raw data size =
27.37058095238095
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 349ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 219MB vs 8MB; ratio to raw data size =
27.40619619047619
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 268ms maxConcurrency = 1
[junit] Approx allocation = 198MB vs 8MB; ratio to raw data size =
24.820686666666667
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 277ms maxConcurrency = 1
[junit] Approx allocation = 198MB vs 8MB; ratio to raw data size =
24.82146285714286
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 258ms maxConcurrency = 1
[junit] Approx allocation = 190MB vs 8MB; ratio to raw data size =
23.770030476190477
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 234ms maxConcurrency = 1
[junit] Approx allocation = 190MB vs 8MB; ratio to raw data size =
23.769555238095236
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 64
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 3908ms maxConcurrency = 100
[junit] GC for PS Scavenge: 36 ms for 10 collections
[junit] Approx allocation = 2481MB vs 8MB; ratio to raw data size =
309.730900952381
[junit] loopRatio (closest to 1 best) 16.04825 raw 100000/1604825 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 697ms maxConcurrency = 100
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 241MB vs 8MB; ratio to raw data size =
30.141012380952382
[junit] loopRatio (closest to 1 best) 1.00004 raw 287/289 counted 2/2
sync 99712/99713 up 1 down 1
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 867ms maxConcurrency = 99
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 378MB vs 8MB; ratio to raw data size =
47.22428
[junit] loopRatio (closest to 1 best) 2.03201 raw 100000/203201 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 797ms maxConcurrency = 99
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 252MB vs 8MB; ratio to raw data size =
31.476938095238093
[junit] loopRatio (closest to 1 best) 1.17123 raw 22254/30788 counted
7823/8564 sync 74967/77771 up 5846 down 4781
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 545ms maxConcurrency = 96
[junit] Approx allocation = 218MB vs 8MB; ratio to raw data size =
27.227550476190476
[junit] loopRatio (closest to 1 best) 1.09275 raw 100000/109275 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 668ms maxConcurrency = 98
[junit] Approx allocation = 221MB vs 8MB; ratio to raw data size =
27.620777142857143
[junit] loopRatio (closest to 1 best) 1.10094 raw 90437/98693 counted
7860/8165 sync 2719/3236 up 7569 down 7598
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 524ms maxConcurrency = 96
[junit] Approx allocation = 205MB vs 8MB; ratio to raw data size =
25.604992380952382
[junit] loopRatio (closest to 1 best) 1.02883 raw 100000/102883 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 589ms maxConcurrency = 100
[junit] Approx allocation = 217MB vs 8MB; ratio to raw data size =
27.18481619047619
[junit] loopRatio (closest to 1 best) 1.02795 raw 97415/100035 counted
2544/2581 sync 136/179 up 2530 down 2537
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 256
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 685ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 283MB vs 26MB; ratio to raw data size =
10.771714782608695
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 561ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 294MB vs 26MB; ratio to raw data size =
11.198217391304349
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 495ms maxConcurrency = 1
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 273MB vs 26MB; ratio to raw data size =
10.378718550724638
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 539ms maxConcurrency = 1
[junit] GC for PS Scavenge: 13 ms for 1 collections
[junit] Approx allocation = 272MB vs 26MB; ratio to raw data size =
10.367084347826086
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 197ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 249MB vs 26MB; ratio to raw data size =
9.492887826086957
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 216ms maxConcurrency = 1
[junit] GC for PS Scavenge: 9 ms for 1 collections
[junit] Approx allocation = 251MB vs 26MB; ratio to raw data size =
9.570564347826087
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 229ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 243MB vs 26MB; ratio to raw data size =
9.23958231884058
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 517ms maxConcurrency = 1
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 243MB vs 26MB; ratio to raw data size =
9.258937101449275
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 256
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1
[junit] original code:
[junit] Duration = 3728ms maxConcurrency = 100
[junit] GC for PS Scavenge: 44 ms for 12 collections
[junit] Approx allocation = 3031MB vs 26MB; ratio to raw data size =
115.17077130434782
[junit] loopRatio (closest to 1 best) 15.84521 raw 100000/1584521 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 479ms maxConcurrency = 100
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 302MB vs 26MB; ratio to raw data size =
11.507159130434783
[junit] loopRatio (closest to 1 best) 1.00188 raw 252/431 counted 7/8
sync 99746/99749 up 4 down 4
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 16
[junit] original code:
[junit] Duration = 967ms maxConcurrency = 100
[junit] GC for PS Scavenge: 18 ms for 2 collections
[junit] Approx allocation = 486MB vs 26MB; ratio to raw data size =
18.50185797101449
[junit] loopRatio (closest to 1 best) 2.2405 raw 100000/224050 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 442ms maxConcurrency = 100
[junit] GC for PS Scavenge: 9 ms for 1 collections
[junit] Approx allocation = 314MB vs 26MB; ratio to raw data size =
11.941331304347827
[junit] loopRatio (closest to 1 best) 1.22081 raw 26220/37315 counted
9907/10839 sync 70268/73927 up 7625 down 5981
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 256
[junit] original code:
[junit] Duration = 615ms maxConcurrency = 93
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 269MB vs 26MB; ratio to raw data size =
10.220121739130434
[junit] loopRatio (closest to 1 best) 1.08956 raw 100000/108956 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 588ms maxConcurrency = 100
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 271MB vs 26MB; ratio to raw data size =
10.32604347826087
[junit] loopRatio (closest to 1 best) 1.0914 raw 91675/99366 counted
7106/7372 sync 1984/2402 up 6870 down 6917
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 256) partitions = 1024
[junit] original code:
[junit] Duration = 549ms maxConcurrency = 95
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 254MB vs 26MB; ratio to raw data size =
9.666266956521739
[junit] loopRatio (closest to 1 best) 1.02505 raw 100000/102505 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 417ms maxConcurrency = 96
[junit] GC for PS Scavenge: 12 ms for 1 collections
[junit] Approx allocation = 255MB vs 26MB; ratio to raw data size =
9.724647246376811
[junit] loopRatio (closest to 1 best) 1.02704 raw 97472/100005 counted
2453/2491 sync 166/208 up 2437 down 2448
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 1 THREAD; ELEMENT SIZE 1024
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 721ms maxConcurrency = 1
[junit] GC for PS Scavenge: 30 ms for 2 collections
[junit] Approx allocation = 504MB vs 99MB; ratio to raw data size =
5.068735019157088
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 636ms maxConcurrency = 1
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 499MB vs 99MB; ratio to raw data size =
5.012685900383142
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 516ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 498MB vs 99MB; ratio to raw data size =
5.003988429118774
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 469ms maxConcurrency = 1
[junit] GC for PS Scavenge: 24 ms for 2 collections
[junit] Approx allocation = 489MB vs 99MB; ratio to raw data size =
4.91579908045977
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 419ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 476MB vs 99MB; ratio to raw data size =
4.782679157088123
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 433ms maxConcurrency = 1
[junit] GC for PS Scavenge: 26 ms for 2 collections
[junit] Approx allocation = 470MB vs 99MB; ratio to raw data size =
4.723301609195402
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 423ms maxConcurrency = 1
[junit] GC for PS Scavenge: 27 ms for 2 collections
[junit] Approx allocation = 465MB vs 99MB; ratio to raw data size =
4.679395708812261
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 445ms maxConcurrency = 1
[junit] GC for PS Scavenge: 35 ms for 2 collections
[junit] Approx allocation = 463MB vs 99MB; ratio to raw data size =
4.654131800766284
[junit] loopRatio (closest to 1 best) 1.0 raw 100000/100000 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --------------------------------------------------
[junit] 100 THREADS; ELEMENT SIZE 1024
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1
[junit] original code:
[junit] Duration = 3641ms maxConcurrency = 100
[junit] GC for PS Scavenge: 79 ms for 23 collections
[junit] Approx allocation = 6272MB vs 99MB; ratio to raw data size =
62.99940222222222
[junit] loopRatio (closest to 1 best) 16.84948 raw 100000/1684948 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 637ms maxConcurrency = 100
[junit] GC for PS Scavenge: 29 ms for 2 collections
[junit] Approx allocation = 508MB vs 99MB; ratio to raw data size =
5.109580383141762
[junit] loopRatio (closest to 1 best) 1.0001 raw 805/812 counted 3/3 sync
99194/99195 up 1 down 1
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 16
[junit] original code:
[junit] Duration = 1261ms maxConcurrency = 100
[junit] GC for PS Scavenge: 37 ms for 4 collections
[junit] Approx allocation = 963MB vs 99MB; ratio to raw data size =
9.67368030651341
[junit] loopRatio (closest to 1 best) 2.30544 raw 100000/230544 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 333ms maxConcurrency = 100
[junit] GC for PS Scavenge: 19 ms for 2 collections
[junit] Approx allocation = 559MB vs 99MB; ratio to raw data size =
5.62049540229885
[junit] loopRatio (closest to 1 best) 1.20045 raw 23136/32826 counted
9312/10027 sync 73535/77192 up 6865 down 5302
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 256
[junit] original code:
[junit] Duration = 383ms maxConcurrency = 92
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 476MB vs 99MB; ratio to raw data size =
4.786960459770115
[junit] loopRatio (closest to 1 best) 1.08511 raw 100000/108511 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 491ms maxConcurrency = 100
[junit] GC for PS Scavenge: 22 ms for 2 collections
[junit] Approx allocation = 516MB vs 99MB; ratio to raw data size =
5.188691264367816
[junit] loopRatio (closest to 1 best) 1.09487 raw 90981/98803 counted
7476/7714 sync 2486/2970 up 7229 down 7199
[junit]
[junit]
[junit] Threads = 100 elements = 100000 (of size 1024) partitions = 1024
[junit] original code:
[junit] Duration = 490ms maxConcurrency = 100
[junit] GC for PS Scavenge: 21 ms for 2 collections
[junit] Approx allocation = 474MB vs 99MB; ratio to raw data size =
4.76750030651341
[junit] loopRatio (closest to 1 best) 1.03162 raw 100000/103162 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 363ms maxConcurrency = 92
[junit] GC for PS Scavenge: 19 ms for 2 collections
[junit] Approx allocation = 480MB vs 99MB; ratio to raw data size =
4.826345440613027
[junit] loopRatio (closest to 1 best) 1.02209 raw 97920/100011 counted
2041/2057 sync 113/141 up 2032 down 2027
[junit]
[junit]
[junit] ==================================================
{code}
> AtomicSortedColumns.addAllWithSizeDelta has a spin lock that allocates memory
> -----------------------------------------------------------------------------
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: graham sanderson
> Assignee: graham sanderson
> Attachments: suggestion1.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
