Was this on a uniform scale? Did you do a test with a non-uniform scale?
...jim
On 3/11/2016 3:10 PM, Laurent Bourgès wrote:
Jim,
Here are MapBench results comparing Marlin 0.7.3.2 (before) vs patched
0.7.3.3 (latest patch) on JDK 1.8_72:
Synthetic results:
Tests 36 12 12 12
Threads 4 1 2 4
Before = Marlin 0.7.3.2 <http://0.7.3.2>:
*Med 168.535 165.403 166.940 173.263 *
*Pct95 170.256 166.011 167.495 177.260
FPS 43.200 43.404 43.203 42.993 *
Patched = Marlin 0.7.3.3 <http://0.7.3.3>:
*Med 169.481 166.344 168.096 174.004 *
*Pct95 170.295 166.800 168.751 175.335
FPS 42.194 42.340 42.211 42.031
*
The difference is very minor ~ 1/2ms in global average (36 or 12 tests)
ie within error margin !
Only FPS (based on median timings) seem indicating that Marlin 0.7.3.2
is 'very slightly' faster.*
*
*
*
Complete results:
Before = Marlin 0.7.3.2 <http://0.7.3.2>:
Test Threads Ops
Med Pct95 Avg StdDev Min Max FPS(med) TotalOps
[ms/op]
CircleTests.ser 1 146 71.663
72.225 71.735 0.321 71.162 73.045 13.954 146
CircleTests.ser 2 292 72.652
73.153 72.683 0.284 72.001 73.818 13.764 292
CircleTests.ser 4 584 73.675
75.495 73.944 1.152 72.320 91.951 13.573 584
EllipseTests-fill-false.ser 1 35 295.363
297.853 295.594 0.764 295.097 298.280 3.386 35
EllipseTests-fill-false.ser 2 70 294.418
295.059 294.345 0.613 293.191 295.685 3.397 70
EllipseTests-fill-false.ser 4 140 295.378
303.661 298.108 14.352 293.428 422.684 3.385 140
EllipseTests-fill-true.ser 1 25 436.748
437.658 436.816 0.445 436.180 438.120 2.290 25
EllipseTests-fill-true.ser 2 50 449.424
451.158 449.191 2.426 446.658 458.887 2.225 50
EllipseTests-fill-true.ser 4 100 513.228
520.717 514.442 3.673 497.246 533.654 1.948 100
dc_boulder_2013-13-30-06-13-17.ser 1 123 82.827
83.351 82.818 0.278 82.156 83.620 12.073 123
dc_boulder_2013-13-30-06-13-17.ser 2 246 83.840
84.142 83.796 0.255 83.175 84.301 11.928 246
dc_boulder_2013-13-30-06-13-17.ser 4 492 84.501
86.983 85.032 1.522 83.268 105.370 11.834 492
dc_boulder_2013-13-30-06-13-20.ser 1 240 42.740
43.061 42.702 0.281 42.126 44.237 23.397 240
dc_boulder_2013-13-30-06-13-20.ser 2 480 43.979
44.183 44.001 0.135 43.253 45.934 22.738 480
dc_boulder_2013-13-30-06-13-20.ser 4 960 44.466
45.612 44.584 0.943 43.499 65.537 22.489 960
dc_shp_alllayers_2013-00-30-07-00-43.ser 1 264 39.132
39.390 39.070 0.258 38.541 39.631 25.554 264
dc_shp_alllayers_2013-00-30-07-00-43.ser 2 528 39.543
40.107 39.599 0.338 38.938 40.982 25.289 528
dc_shp_alllayers_2013-00-30-07-00-43.ser 4 1056 40.429
41.233 40.504 0.769 39.267 62.092 24.735 1056
dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 828.145
829.424 828.236 0.701 826.863 830.275 1.208 25
dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 829.794
831.670 829.920 1.326 827.584 833.811 1.205 50
dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 835.829
859.380 841.386 10.144 831.585 863.487 1.196 100
dc_spearfish_2013-11-30-06-11-15.ser 1 822 12.424
12.518 12.438 0.039 12.396 12.721 80.492 822
dc_spearfish_2013-11-30-06-11-15.ser 2 1644 12.472
12.621 12.499 0.059 12.429 12.860 80.177 1644
dc_spearfish_2013-11-30-06-11-15.ser 4 3288 12.499
12.888 12.602 0.407 12.416 31.385 80.004 3288
dc_spearfish_2013-11-30-06-11-19.ser 1 1647 6.368
6.431 6.374 0.023 6.353 6.548 157.047 1647
dc_spearfish_2013-11-30-06-11-19.ser 2 3294 6.370
6.435 6.375 0.025 6.350 7.025 156.982 3294
dc_spearfish_2013-11-30-06-11-19.ser 4 6588 6.379
6.443 6.392 0.235 6.353 24.259 156.755 6588
dc_topp:states_2013-11-30-06-11-06.ser 1 791 13.258
13.435 13.285 0.080 13.194 13.576 75.424 791
dc_topp:states_2013-11-30-06-11-06.ser 2 1582 13.383
13.541 13.366 0.095 13.207 13.680 74.720 1582
dc_topp:states_2013-11-30-06-11-06.ser 4 3164 13.318
13.529 13.353 0.373 13.205 31.555 75.084 3164
dc_topp:states_2013-11-30-06-11-07.ser 1 1221 8.385
8.555 8.415 0.082 8.329 9.306 119.254 1221
dc_topp:states_2013-11-30-06-11-07.ser 2 2442 8.382
8.563 8.419 0.077 8.336 8.680 119.303 2442
dc_topp:states_2013-11-30-06-11-07.ser 4 4884 8.454
8.666 8.473 0.253 8.332 19.872 118.291 4884
test_z_625k.ser 1 71 147.778
148.236 147.762 0.324 146.781 148.338 6.767 71
test_z_625k.ser 2 142 149.019
149.311 149.002 0.218 148.444 149.558 6.711 142
test_z_625k.ser 4 284 150.994
152.514 151.185 1.326 149.368 171.277 6.623 284
Scores:
Tests 36 12 12 12
Threads 4 1 2 4
*Med 168.535 165.403 166.940 173.263 *
*Pct95 170.256 166.011 167.495 177.260
FPS 43.200 43.404 43.203 42.993 *
Patched = Marlin 0.7.3.3 <http://0.7.3.3>:
Test Threads Ops
Med Pct95 Avg StdDev Min Max FPS(med) TotalOps
[ms/op]
CircleTests.ser 1 141 74.384
74.900 74.404 0.304 73.822 75.428 13.444 141
CircleTests.ser 2 282 75.187
75.542 75.200 0.212 74.667 76.527 13.300 282
CircleTests.ser 4 564 78.456
79.359 78.047 1.544 75.002 101.072 12.746 564
EllipseTests-fill-false.ser 1 35 293.541
294.431 293.563 0.710 292.023 294.603 3.407 35
EllipseTests-fill-false.ser 2 70 294.290
296.877 294.578 1.547 291.809 297.005 3.398 70
EllipseTests-fill-false.ser 4 140 293.914
297.667 294.655 2.639 292.165 319.054 3.402 140
EllipseTests-fill-true.ser 1 25 435.423
436.001 435.500 0.372 435.051 436.990 2.297 25
EllipseTests-fill-true.ser 2 50 446.800
448.429 446.907 1.960 444.852 455.553 2.238 50
EllipseTests-fill-true.ser 4 100 510.117
512.215 510.355 2.718 507.249 533.946 1.960 100
dc_boulder_2013-13-30-06-13-17.ser 1 123 85.126
85.689 85.185 0.279 84.913 87.077 11.747 123
dc_boulder_2013-13-30-06-13-17.ser 2 246 85.914
86.276 85.944 0.150 85.715 86.638 11.640 246
dc_boulder_2013-13-30-06-13-17.ser 4 492 86.984
87.633 87.053 1.207 85.843 109.197 11.496 492
dc_boulder_2013-13-30-06-13-20.ser 1 240 43.508
44.306 43.638 0.348 43.308 46.308 22.984 240
dc_boulder_2013-13-30-06-13-20.ser 2 480 44.452
44.909 44.500 0.206 44.109 45.435 22.496 480
dc_boulder_2013-13-30-06-13-20.ser 4 960 45.897
46.231 45.823 0.849 44.521 66.349 21.788 960
dc_shp_alllayers_2013-00-30-07-00-43.ser 1 265 39.453
40.006 39.472 0.279 39.037 40.216 25.346 265
dc_shp_alllayers_2013-00-30-07-00-43.ser 2 530 40.118
40.689 40.114 0.329 39.268 41.028 24.927 530
dc_shp_alllayers_2013-00-30-07-00-43.ser 4 1060 40.700
41.181 40.735 0.744 40.026 62.110 24.570 1060
dc_shp_alllayers_2013-00-30-07-00-47.ser 1 25 832.892
833.254 832.836 0.321 832.124 833.376 1.201 25
dc_shp_alllayers_2013-00-30-07-00-47.ser 2 50 836.860
838.171 836.882 0.903 834.890 838.289 1.195 50
dc_shp_alllayers_2013-00-30-07-00-47.ser 4 100 837.405
843.556 837.978 4.264 835.075 858.780 1.194 100
dc_spearfish_2013-11-30-06-11-15.ser 1 818 12.826
12.948 12.843 0.057 12.804 13.663 77.969 818
dc_spearfish_2013-11-30-06-11-15.ser 2 1636 12.851
12.941 12.854 0.047 12.801 13.271 77.816 1636
dc_spearfish_2013-11-30-06-11-15.ser 4 3272 12.843
12.989 12.879 0.325 12.813 28.151 77.866 3272
dc_spearfish_2013-11-30-06-11-19.ser 1 1606 6.525
6.589 6.532 0.027 6.507 6.734 153.267 1606
dc_spearfish_2013-11-30-06-11-19.ser 2 3212 6.523
6.573 6.527 0.025 6.502 6.992 153.309 3212
dc_spearfish_2013-11-30-06-11-19.ser 4 6424 6.526
6.578 6.538 0.233 6.503 23.834 153.242 6424
dc_topp:states_2013-11-30-06-11-06.ser 1 781 13.488
13.596 13.469 0.084 13.335 13.678 74.141 781
dc_topp:states_2013-11-30-06-11-06.ser 2 1562 13.502
13.632 13.491 0.089 13.343 13.760 74.060 1562
dc_topp:states_2013-11-30-06-11-06.ser 4 3124 13.499
13.635 13.503 0.295 13.344 26.397 74.081 3124
dc_topp:states_2013-11-30-06-11-07.ser 1 1209 8.648
8.719 8.626 0.074 8.489 8.913 115.627 1209
dc_topp:states_2013-11-30-06-11-07.ser 2 2418 8.652
8.718 8.626 0.072 8.500 8.811 115.578 2418
dc_topp:states_2013-11-30-06-11-07.ser 4 4836 8.659
8.730 8.647 0.269 8.494 23.649 115.487 4836
test_z_625k.ser 1 69 150.311
151.154 150.448 0.408 149.847 151.557 6.653 69
test_z_625k.ser 2 138 152.004
152.252 152.013 0.114 151.744 152.410 6.579 138
test_z_625k.ser 4 276 153.049
154.245 153.266 1.418 152.792 175.424 6.534 276
Scores:
Tests 36 12 12 12
Threads 4 1 2 4
*Med 169.481 166.344 168.096 174.004 *
*Pct95 170.295 166.800 168.751 175.335
FPS 42.194 42.340 42.211 42.031
*
Cheers,
Laurent
