pnowojski commented on a change in pull request #7713: [FLINK-10995][runtime]
Copy intermediate serialization results only once for broadcast mode
URL: https://github.com/apache/flink/pull/7713#discussion_r328127065
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/BroadcastRecordWriter.java
##########
@@ -18,30 +18,163 @@
package org.apache.flink.runtime.io.network.api.writer;
+import org.apache.flink.annotation.VisibleForTesting;
import org.apache.flink.core.io.IOReadableWritable;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferConsumer;
import java.io.IOException;
+import java.util.Optional;
+
+import static org.apache.flink.util.Preconditions.checkState;
/**
* A special record-oriented runtime result writer only for broadcast mode.
*
- * <p>The BroadcastRecordWriter extends the {@link RecordWriter} and handles
{@link #emit(IOReadableWritable)}
- * operation via {@link #broadcastEmit(IOReadableWritable)} directly in a more
efficient way.
+ * <p>The BroadcastRecordWriter extends the {@link RecordWriter} and maintain
a single {@link BufferBuilder}
+ * for all the channels. Then the serialization results need be copied only
once to this buffer which would be
+ * shared for all the channels in a more efficient way.
*
* @param <T> the type of the record that can be emitted with this record
writer
*/
public class BroadcastRecordWriter<T extends IOReadableWritable> extends
RecordWriter<T> {
+ /** The current buffer builder shared for all the channels. */
+ private Optional<BufferBuilder> bufferBuilder = Optional.empty();
Review comment:
I think the decision was to enforce code style changes only for new PRs, not
the pending ones, so I wouldn't enforce the change here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
