1996fanrui opened a new pull request, #21697: URL: https://github.com/apache/flink/pull/21697
## What is the purpose of the change Improve Flink performance by reducing the average depth of the call stack needed to produce a record for NetworkInput. ## Brief change log NetworkInput#emitNext() should push records to DataOutput within a while loop. > Note: currently, this PR based on the #21690, and this PR should just include the last commit finally. ## Verifying this change Here are the benchmark results obtained by running the [InputBenchmark#mapSink](https://github.com/apache/flink-benchmarks/blob/0bafe0e85700c889894324aadb70302381f98e03/src/main/java/org/apache/flink/benchmark/InputBenchmark.java#L55) with env.disableOperatorChaining(). And I run it 4 times on My Mac. ``` Before the proposed change, the avg is 12429.0605 ops/ms, here is detailed results: Benchmark (sourceType) Mode Cnt Score Error Units InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 12339.771 ± 414.649 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 12687.872 ± 320.084 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 12256.445 ± 512.219 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 12432.154 ± 405.083 ops/ms After the proposed change, the avg is 13836.845 ops/ms, here is detailed results: Benchmark (sourceType) Mode Cnt Score Error Units InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 13092.451 ± 490.886 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 13881.138 ± 370.249 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 13960.280 ± 389.505 ops/ms InputBenchmark.mapSink F27_UNBOUNDED thrpt 30 14413.511 ± 727.844 ops/ms ``` The proposed change increases throughput by 11.3%. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not documented -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
