[
https://issues.apache.org/jira/browse/FLINK-11775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813295#comment-16813295
]
Stefan Richter commented on FLINK-11775:
----------------------------------------
[~lzljs3620320] I think in general the performance optimization makes sense.
There is one question for me that I don't understand from the proposal. Are you
suggesting:
a) {{AbstractPagedOutputView implements DataOutputView, MemorySegmentWritable}}
or
b) {{AbstractPagedOutputView implements DataOutputView}} and {{DataOutputView
extends MemorySegmentWritable}}
I am assuming a) and that sounds resaonable to me, but b) does not sound good.
Can you give an example of where and how you would use this new functionality.
Would you have to cast to {{MemorySegmentWritable}} or are we already typed to
{{AbstractPagedOutputView}}.
In general, please be aware that MemorySegment is tagged as {{@Internal}} and
we should not leak it through a new public interface, so at least I would
suggest to also tag {{MemorySegmentWritable}} as internal.
> Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
> bytes
> -----------------------------------------------------------------------------------
>
> Key: FLINK-11775
> URL: https://issues.apache.org/jira/browse/FLINK-11775
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Operators
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
>
> Blink new binary format is based on MemorySegment.
> Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
> bytes
> {code:java}
> /**
> * Provides the interface for write(Segment).
> */
> public interface MemorySegmentWritable {
> /**
> * Writes {@code len} bytes from memory segment {@code segment} starting at
> offset {@code off}, in order,
> * to the output.
> *
> * @param segment memory segment to copy the bytes from.
> * @param off the start offset in the memory segment.
> * @param len The number of bytes to copy.
> * @throws IOException if an I/O error occurs.
> */
> void write(MemorySegment segment, int off, int len) throws IOException;
> }{code}
>
> If we want to write a Memory Segment to DataOutputView, we need to copy bytes
> to byte[] and then write it in, which is less effective.
> If we let AbstractPagedOutputView have a write(MemorySegment) interface, we
> can copy it directly.
> We need to ensure this in network serialization, batch operator calculation
> serialization, Streaming State serialization to avoid new byte[] and copy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
