[
https://issues.apache.org/jira/browse/FLINK-19023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen resolved FLINK-19023.
----------------------------------
Resolution: Fixed
Fixed for 1.12 via 2c273f86e41866bc737de1686aa0925a4749671b
> Remove pruning of Record Serializer Buffer
> ------------------------------------------
>
> Key: FLINK-19023
> URL: https://issues.apache.org/jira/browse/FLINK-19023
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Major
> Fix For: 1.12.0
>
>
> Currently, the {{SpanningRecordSerializer}} prunes its internal serialization
> buffer under special circumstances:
> - The buffer becomes larger than a certain threshold (5MB)
> - The full record end lines up exactly with a full buffer length (this
> change got introduced at some point, it is not clear what the purpose is)
> This optimization virtually never kicks in (because of the second condition)
> and also seems unnecessary. There is only a single serializer on the sender
> side, so this will not help to reduce the maximum memory footprint needed in
> any way.
> NOTE: A similar optimization on the reader side
> ({{SpillingAdaptiveSpanningRecordDeserializer}}) makes sense, because
> multiple parallel deserializers run in order to piece together the records
> when retrieving buffers from the network in arbitrary order. Truncating
> buffers (or spilling) there helps reduce the maximum required memory
> footprint.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
