[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709603#comment-15709603
]
ASF GitHub Bot commented on FLINK-4705:
---------------------------------------
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/2617
Hi @StephanEwen, should we endeavor to get this into 1.2 or defer to the
next release?
> Instrument FixedLengthRecordSorter
> ----------------------------------
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Sub-task
> Components: Local Runtime
> Affects Versions: 1.2.0
> Reporter: Greg Hogan
> Assignee: Greg Hogan
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
