pnowojski commented on a change in pull request #6417: [FLINK-9913][runtime]
Improve output serialization only once in RecordWriter
URL: https://github.com/apache/flink/pull/6417#discussion_r217654112
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java
##########
@@ -113,11 +113,7 @@ public void broadcastEmit(T record) throws IOException,
InterruptedException {
* This is used to send LatencyMarks to a random target channel.
*/
public void randomEmit(T record) throws IOException,
InterruptedException {
- serializer.serializeRecord(record);
-
- if
(copyFromSerializerToTargetChannel(rng.nextInt(numChannels))) {
- serializer.prune();
- }
+ emit(record, new int[] { rng.nextInt(numChannels) });
Review comment:
Wait a minute here.
```
emit(record, new int[] { rng.nextInt(numChannels) });
```
@NicoK is an actual performance regression. Creating a single element int
array (once per every record!) actually reduces the throughput by about 10%-20%
(tested during one of the hackathons). Please revert to using
`copyFromSerializerToTargetChannel(rng.nextInt(numChannels))`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
