zhijiangW commented on issue #6417: [FLINK-9913][runtime] Improve output serialization only once in RecordWriter URL: https://github.com/apache/flink/pull/6417#issuecomment-413468551 @pnowojski , I think I got the reason of regression in some non-broadcast cases. The key point is when to call `RecordSerializer#prune()` which is used to decrease the intermediate serialization buffer. In the past, the prune method was only called after the target copying buffer is full and the record is full. But now after emitting each record we will call to prune method in order to narrow down the intermediate buffer ASAP. So the performance may be regression in sensitive job scenarios. I will modify to keep the previous behavior of calling prune in serializer. Thanks for your benchmark to find this potential issue. I only verified the broadcast scenarios in benchmark before and the obvious advantage may hide the potential regression. :)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
