[
https://issues.apache.org/jira/browse/CALCITE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184954#comment-17184954
]
Rui Wang edited comment on CALCITE-4193 at 8/26/20, 6:23 AM:
-------------------------------------------------------------
>In particular, we have no mechanism to serialize and deserialize values, and
>no on-disk format, and no scheduler that can handle asynchronous i/o.
Will it work if hide such details under EnumerableExternalSort.current(),
EnumerableExternalSort.next(), etc.? In this case, Calcite does not care about
how serialize and deserialize values, etc. is done.
was (Author: amaliujia):
>In particular, we have no mechanism to serialize and deserialize values, and
>no on-disk format, and no scheduler that can handle asynchronous i/o.
Will it work if hide such details under EnumerableExternalSort.current(),
EnumerableExternalSort.next(), etc.? Calcite does not care about how serialize
and deserialize values, etc. is done?
> Implement new sort operator: EnumerableExternalSort
> ---------------------------------------------------
>
> Key: CALCITE-4193
> URL: https://issues.apache.org/jira/browse/CALCITE-4193
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Ruben Q L
> Priority: Major
>
> Sometimes we need to sort a big volume of data which does not fit into
> memory. In this situation EnumerableSort will cause an OutOfMemoryError.
> The solution for such a scenario will be using a different sorting algorithm:
> [External Sort|https://en.wikipedia.org/wiki/External_sorting].
> The goal of the current ticket is to implement a new operator
> (EnumerableExternalSort) to provide this feature.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
