[
https://issues.apache.org/jira/browse/IMPALA-10025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-10025.
------------------------------------
Resolution: Invalid
I'm going to do this in the first version of the patch
> Avoid rebuilding in-memory heap during output phase of top-n
> ------------------------------------------------------------
>
> Key: IMPALA-10025
> URL: https://issues.apache.org/jira/browse/IMPALA-10025
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> In the patch for IMPALA-9853, we reuse some code in the output phase that
> necessitated building the in-memory heap from the sorter's output. This has
> some inherent overhead that gets worse for larger limits and/or partition
> counts.
> It would be better to have the sorter do a full sort on partition/order by
> columns and then apply the limit while streaming the results back from the
> sorter. In combination with IMPALA-10023 this would let us gracefully degrade
> to doing something closer to a regular sort and probably let us bump
> ANALYTIC_PUSHDOWN_THRESHOLD.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
