Myasuka commented on pull request #15119:
URL: https://github.com/apache/flink/pull/15119#issuecomment-806641673
> Unfortunately, changing the histogram implementation from
`com.codahale.metrics.Histogram + SlidingTimeWindowReservoir` to
`DescriptiveStatisticsHistogram + CircularDoubleArray` gets a worse performance
regression via flink-benchmarks, still investigating ..
It was a false alarm. The actual reason is the newly introduced tracking
iterator#next latency would have impacts over the performace:
Benchmark | (backendType) | original Score | latency-tracking enabled Score
(Flink internal histogram) | latency-tracking enabled Score
(codahle implemented histogram) | performance change (Flink internal
histogram) | performance change (codahle implemented histogram)
-- | -- | -- | -- | -- | -- | --
ListStateBenchmark.listAdd | HEAP | 9199.445 | 7613.489 | 7411.939 | -17.24%
| -19.43%
ListStateBenchmark.listAddAll | HEAP | 1072.198 | 1088.434 | 1047.93 | 1.51%
| -2.26%
ListStateBenchmark.listAppend | HEAP | 4375.627 | 4136.901 | 4052.241 |
-5.46% | -7.39%
ListStateBenchmark.listGet | HEAP | 4419.291 | 4065.197 | 4003.472 | -8.01%
| -9.41%
ListStateBenchmark.listGetAndIterate | HEAP | 4279.762 | 3978.864 | 3927.868
| -7.03% | -8.22%
ListStateBenchmark.listUpdate | HEAP | 4307.681 | 3597.863 | 3631.94 |
-16.48% | -15.69%
MapStateBenchmark.mapAdd | HEAP | 4428.129 | 4295.446 | 4244.579 | -3.00% |
-4.15%
MapStateBenchmark.mapContains | HEAP | 3731.079 | 3609.574 | 3564.076 |
-3.26% | -4.48%
MapStateBenchmark.mapEntries | HEAP | 22218.71 | 12270.087 | 9920.191 |
-44.78% | -55.35%
MapStateBenchmark.mapGet | HEAP | 3485.223 | 3483.249 | 3402.05 | -0.06% |
-2.39%
MapStateBenchmark.mapIsEmpty | HEAP | 5004.396 | 4887.583 | 4721.653 |
-2.33% | -5.65%
MapStateBenchmark.mapIterator | HEAP | 21284.53 | 12807.682 | 10396.586 |
-39.83% | -51.15%
MapStateBenchmark.mapKeys | HEAP | 24892.573 | 13020.899 | 10985.759 |
-47.69% | -55.87%
MapStateBenchmark.mapPutAll | HEAP | 2429.054 | 2216.094 | 2185.309 | -8.77%
| -10.03%
MapStateBenchmark.mapRemove | HEAP | 4735.704 | 4132.611 | 4238.824 |
-12.74% | -10.49%
MapStateBenchmark.mapUpdate | HEAP | 3553.116 | 3440.258 | 3414.649 | -3.18%
| -3.90%
MapStateBenchmark.mapValues | HEAP | 24845.875 | 12207.236 | 10276.476 |
-50.87% | -58.64%
ValueStateBenchmark.valueAdd | HEAP | 5044.144 | 4807.602 | 4673.127 |
-4.69% | -7.36%
ValueStateBenchmark.valueGet | HEAP | 5030.982 | 4807.588 | 4693.427 |
-4.44% | -6.71%
ValueStateBenchmark.valueUpdate | HEAP | 5020.6 | 4785.519 | 4628.28 |
-4.68% | -7.81%
ListStateBenchmark.listAdd | ROCKSDB | 678.07 | 652.355 | 665.843 | -3.79% |
-1.80%
ListStateBenchmark.listAddAll | ROCKSDB | 383.417 | 351.778 | 376.966 |
-8.25% | -1.68%
ListStateBenchmark.listAppend | ROCKSDB | 656.482 | 640.139 | 648.939 |
-2.49% | -1.15%
ListStateBenchmark.listGet | ROCKSDB | 198.101 | 195.144 | 196.885 | -1.49%
| -0.61%
ListStateBenchmark.listGetAndIterate | ROCKSDB | 196.901 | 193.225 | 195.766
| -1.87% | -0.58%
ListStateBenchmark.listUpdate | ROCKSDB | 676.918 | 660.558 | 659.313 |
-2.42% | -2.60%
MapStateBenchmark.mapAdd | ROCKSDB | 563.874 | 563.195 | 562.425 | -0.12% |
-0.26%
MapStateBenchmark.mapContains | ROCKSDB | 72.388 | 71.969 | 72.988 | -0.58%
| 0.83%
MapStateBenchmark.mapEntries | ROCKSDB | 501.586 | 485.41 | 475.739 | -3.22%
| -5.15%
MapStateBenchmark.mapGet | ROCKSDB | 72.206 | 71.506 | 71.793 | -0.97% |
-0.57%
MapStateBenchmark.mapIsEmpty | ROCKSDB | 65.273 | 64.77 | 64.823 | -0.77% |
-0.69%
MapStateBenchmark.mapIterator | ROCKSDB | 504.318 | 488.65 | 482.553 |
-3.11% | -4.32%
MapStateBenchmark.mapKeys | ROCKSDB | 508.078 | 494.691 | 488.075 | -2.63% |
-3.94%
MapStateBenchmark.mapPutAll | ROCKSDB | 166.555 | 162.329 | 159.749 | -2.54%
| -4.09%
MapStateBenchmark.mapRemove | ROCKSDB | 591.476 | 574.673 | 568.482 | -2.84%
| -3.89%
MapStateBenchmark.mapUpdate | ROCKSDB | 570.014 | 566.411 | 561.183 | -0.63%
| -1.55%
MapStateBenchmark.mapValues | ROCKSDB | 502.832 | 489.179 | 484.986 | -2.72%
| -3.55%
ValueStateBenchmark.valueAdd | ROCKSDB | 577.017 | 550.798 | 555.582 |
-4.54% | -3.71%
ValueStateBenchmark.valueGet | ROCKSDB | 933.708 | 918.24 | 909.075 | -1.66%
| -2.64%
ValueStateBenchmark.valueUpdate | ROCKSDB | 583.396 | 562.835 | 554.475 |
-3.52% | -4.96%
We still can see the Flink internal implementation behaves a bit better than
`com.codahale.metrics.Histogram`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
