sunhaibotb commented on issue #7959: [FLINK-11876] Introduce new TwoInputSelectable, BoundedOneInput and BoundedTwoInput interfaces URL: https://github.com/apache/flink/pull/7959#issuecomment-474219792 See the following links for the code of runtime implemention and benchmark. - [All code](https://github.com/sunhaibotb/flink/commit/949ac8f18445b53ce9ba5621742b2f4b25c97a6d) - [Benchmark executor](https://github.com/sunhaibotb/flink-benchmarks/commit/939966cb316b22cfbfc8e2fb1ed4662eb40aa8a6) - [Quick link to class StreamTwoInputSelectableProcessor](https://github.com/sunhaibotb/flink/commit/949ac8f18445b53ce9ba5621742b2f4b25c97a6d#diff-df3f9646d0e6ff0139bf5f1d905db9f1) - [Quick link to class NetworkInput](https://github.com/sunhaibotb/flink/commit/949ac8f18445b53ce9ba5621742b2f4b25c97a6d#diff-3cdf32a5b4473b83107ffaef64b0e177) Benchmark for `StreamTwoInputSelectableProcessor` and `StreamTwoInputProcessor` - Measurement: throughput - Reading Mode: ANY - Testing Scheme * Mocks `InputChannel` which be constructed using static buffers to avoid the network overhead. * Creates two SingleInputGates for each input so that `StreamTwoInputSelectableProcessor` and `StreamTwoInputProcessor` have the same structure containing`UnionInputGate`. * Creates one `InputChannel` for each `SingleInputGate`, and fills the data buffer of the `InputChannel` using the record containing a long value. * Uses JMH to execute the benchmark: @Fork value: 10 @Fork warmups: 1 Number of warmup iterations: 0 Number of measurement iterations: 1 Records per forked invocation: 500,000,000 - Environment * JDK: OracleJDK 1.8.0_202, OpenJDK 1.8.0_40 * OS: Linux 3.10.0 * CPU: 96 cores Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz - Results * `StreamTwoInputProcessor` | *21338.181 ±(99.9%) 373.656 ops/ms [Average]* | (min, avg, max) = (21038.197, 21338.181, 21975.121), stdev = 247.151 | CI (99.9%): [20964.524, 21711.837] (assumes normal distribution) * `StreamTwoInputSelectableProcessor` | *22000.049 ±(99.9%) 191.925 ops/ms [Average]* | (min, avg, max) = (21806.272, 22000.049, 22181.491), stdev = 126.946 | CI (99.9%): [21808.125, 22191.974] (assumes normal distribution) - Summary * `StreamTwoInputSelectableProcessor` is about 3% faster than `StreamTwoInputProcessor` in throughput. * Repeated tests show that the test results are basically stable. * Using OpenJDK, the results are roughly the same. - Result Analysis * The reason for being a bit faster is that `StreamTwoInputSelectableProcessor` uses hot loop to process `StreamRecord` as much as possible, and `StreamTwoInputProcessor` exit the loop after processing a `StreamRecord`. * If `StreamTwoInputSelectableProcessor` exit the loop after processing a `StreamRecord`, it is about 6% slower than `StreamTwoInputProcessor`. After efforts, attempts have been made to implement the code which may be friendly to JIT, failing to eliminate the performance regression. - Concerns * If the actor model which is introduced in the future needs to exit the loop after processing a `StreamRecord`, there will be regression in performance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
