NicoK edited a comment on issue #6417: [FLINK-9913][runtime] Improve output serialization only once in RecordWriter URL: https://github.com/apache/flink/pull/6417#issuecomment-416236083 I don't quite get the argument with the `pruneBuffer()` call having this much overhead...at least for small records since its implementation only prunes if the buffer is larger than 5MiB: ``` public void pruneBuffer() { if (this.buffer.length > PRUNE_BUFFER_THRESHOLD) { ... ``` well, actually, this brings it down to the overhead from `org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer#prune`: ``` @Override public void prune() { serializationBuffer.pruneBuffer(); dataBuffer = serializationBuffer.wrapAsByteBuffer(); } ``` but to be honest, `serializationBuffer.wrapAsByteBuffer();` also just sets two position numbers -> if we are that sensitive to small changes, we indeed should think about optimising this one call to `tryFinishCurrentBufferBuilder()` which I mentioned above(?)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
