wsry commented on issue #11088: [FLINK-16012][runtime] Reduce the default number of buffers per channel from 2 to 1 URL: https://github.com/apache/flink/pull/11088#issuecomment-587227214 Theoretically, reducing the number of buffers may break the data processing pipeline which can influence the performance. For verification, I hava tested the change using the flink micro benchmark and a simple benchmark job. Unfortunately, regressions are seen for both tests. For micro benchmark, the following are some results with regression (Because of the unstable result, I run each test three times.): Using 2 buffer: ``` Benchmark (channelsFlushTimeout) (writers) Mode Cnt Score Error Units networkThroughput 1000,100ms 1 thrpt 30 15972.952 ± 752.985 ops/ms networkThroughput 1000,100ms 4 thrpt 30 27650.498 ± 713.728 ops/ms networkThroughput 1000,100ms 1 thrpt 30 15566.705 ± 2007.335 ops/ms networkThroughput 1000,100ms 4 thrpt 30 27769.195 ± 1632.614 ops/ms networkThroughput 1000,100ms 1 thrpt 30 15598.175 ± 1671.515 ops/ms networkThroughput 1000,100ms 4 thrpt 30 27499.901 ± 1035.415 ops/ms ``` Using 1 buffer： ``` Benchmark (channelsFlushTimeout) (writers) Mode Cnt Score Error Units networkThroughput 1000,100ms 1 thrpt 30 13116.610 ± 325.587 ops/ms networkThroughput 1000,100ms 4 thrpt 30 22837.502 ± 1024.360 ops/ms networkThroughput 1000,100ms 1 thrpt 30 11924.883 ± 1038.508 ops/ms networkThroughput 1000,100ms 4 thrpt 30 22823.586 ± 892.918 ops/ms networkThroughput 1000,100ms 1 thrpt 30 12960.345 ± 1596.465 ops/ms networkThroughput 1000,100ms 4 thrpt 30 23028.803 ± 933.609 ops/ms ``` From the above results, we can see about 20% performance regression. For the benchmark job, there are also regressions (about 10% - 20%) in some cases where input channel numbers are small, for example 2 input channels, which means the number of buffer can be used is limited.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
