[jira] [Updated] (FLINK-4705) Instrument FixedLengthRecordSorter
[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4705:
--
Labels: auto-unassigned pull-request-available (was:
pull-request-available stale-assigned)
> Instrument FixedLengthRecordSorter
> --
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Task
>Affects Versions: 1.2.0
>Reporter: Greg Hogan
>Assignee: Greg Hogan
>Priority: Major
> Labels: auto-unassigned, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-4705) Instrument FixedLengthRecordSorter
[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4705:
--
Labels: pull-request-available stale-assigned (was: pull-request-available)
> Instrument FixedLengthRecordSorter
> --
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Task
>Affects Versions: 1.2.0
>Reporter: Greg Hogan
>Assignee: Greg Hogan
>Priority: Major
> Labels: pull-request-available, stale-assigned
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-4705) Instrument FixedLengthRecordSorter
[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-4705:
--
Labels: pull-request-available (was: )
> Instrument FixedLengthRecordSorter
> --
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Task
>Affects Versions: 1.2.0
>Reporter: Greg Hogan
>Assignee: Greg Hogan
>Priority: Major
> Labels: pull-request-available
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-4705) Instrument FixedLengthRecordSorter
[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Hogan updated FLINK-4705:
--
Issue Type: Sub-task (was: Improvement)
Parent: FLINK-4860
> Instrument FixedLengthRecordSorter
> --
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Sub-task
> Components: Local Runtime
>Affects Versions: 1.2.0
>Reporter: Greg Hogan
>Assignee: Greg Hogan
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
