[
https://issues.apache.org/jira/browse/CALCITE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184952#comment-17184952
]
Julian Hyde commented on CALCITE-4193:
--------------------------------------
The Enumerable convention was designed for in-memory data, on the JVM heap. It
wasn’t intended for external algorithms.
In particular, we have no mechanism to serialize and deserialize values, and no
on-disk format, and no scheduler that can handle asynchronous i/o.
We ought to be putting our effort into building a convention with a
byte-oriented data format.
> Implement new sort operator: EnumerableExternalSort
> ---------------------------------------------------
>
> Key: CALCITE-4193
> URL: https://issues.apache.org/jira/browse/CALCITE-4193
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Ruben Q L
> Priority: Major
>
> Sometimes we need to sort a big volume of data which does not fit into
> memory. In this situation EnumerableSort will cause an OutOfMemoryError.
> The solution for such a scenario will be using a different sorting algorithm:
> [External Sort|https://en.wikipedia.org/wiki/External_sorting].
> The goal of the current ticket is to implement a new operator
> (EnumerableExternalSort) to provide this feature.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
