Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/1517#issuecomment-186244959 @fhueske: So far, I have only run benchmarks on my work laptop and home laptop. I have used the newly created `ReducePerformance`, with varying the `(number of elements / number of different keys)` ratio, and also the memory usage (between 250 MB - 6 GB). The hash-based strategy was always faster, up to 3 times, and mostly by a factor of 1.5 - 2.5. I have also used `ReduceHashTableTest.testWithIntPair`. To make this a benchmark test, the lines that deal with the `reference` hash table for checking the correct output should be commented out, and the time measurement commented in. (This only tests the hash table, not end-to-end; and it can't be directly compared with the sort based reduce.) I have also used `HashTablePerformanceComparison` to compare with the other hash tables. This has the disadvantage that the workload here is not characteristic of a reduce (eg. no processRecordWithReduce calls, because the other hash tables don't have that). I have also benchmarked using my game theory stuff (which originally prompted me to start working on this), and the hash-based combiner was faster here as well. > Did you check the combine rate (input / output ratio) compared to the sort-based strategy? I didn't measure it directly, but `ReducePerformance` shows the difference nicely when the number of input elements is large enough that the sort-based strategy has to emit often, but the number of different keys is small enough that the hash-based strategy can fit all the keys in memory. In this case, the speedup of the final reduce phase is striking. > Have you checked heap memory consumption / GC activity compared to the sort-based strategy? I haven't actually checked it, but it should be the same or better as with the sort-based strategy: - enabled object reuse: no allocations at all - disabled object reuse: one allocation per input element, and one allocation per reduce step The sort-based strategy might additionally have allocations inside maybe the sorter and/or MergeIterator.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---