[
https://issues.apache.org/jira/browse/FLINK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee updated FLINK-11857:
---------------------------------
Description:
We need a sorter to take full advantage of the high performance of the Binary
format.
For Sort, Sort Aggregation, SortMergeJoin: we need a BinaryExternalSorter to
sort BinaryRows.
For Hash Aggregation：We store the data in HashMap in KeyValue format. When
memory is not enough, we spill all the data in memory onto disk and degenerate
it into Sort Aggregation. So we need a BufferedKVExternalSorter to write the
data that already in memory to disk, and then carry out Sort Merge.
was:We need a sorter to take full advantage of the high performance of the
Binary format.
> Introduce BinaryExternalSorter to batch table runtime
> -----------------------------------------------------
>
> Key: FLINK-11857
> URL: https://issues.apache.org/jira/browse/FLINK-11857
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Operators
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
>
> We need a sorter to take full advantage of the high performance of the Binary
> format.
> For Sort, Sort Aggregation, SortMergeJoin: we need a BinaryExternalSorter to
> sort BinaryRows.
> For Hash Aggregation：We store the data in HashMap in KeyValue format. When
> memory is not enough, we spill all the data in memory onto disk and
> degenerate it into Sort Aggregation. So we need a BufferedKVExternalSorter to
> write the data that already in memory to disk, and then carry out Sort Merge.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
