[
https://issues.apache.org/jira/browse/IMPALA-5004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661063#comment-16661063
]
Sahil Takiar commented on IMPALA-5004:
--------------------------------------
After discussing this some more. We agreed that the patch itself is still
useful, but that we should bump up the default value to ensure we don't hit any
performance regressions. I've updated the review on Gerrit to have a default of
512MB.
> Switch to sorting node for large TopN queries
> ---------------------------------------------
>
> Key: IMPALA-5004
> URL: https://issues.apache.org/jira/browse/IMPALA-5004
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 2.9.0
> Reporter: Lars Volker
> Assignee: Sahil Takiar
> Priority: Major
>
> As explained by [~tarmstrong] in IMPALA-4995:
> bq. We should also consider switching to the sort operator for large limits.
> This allows it to spill. The memory requirements for TopN also are
> problematic for large limits, since it would allocate large vectors that are
> untracked and also require a large amount of contiguous memory.
> There's already logic to select TopN vs. Sort:
> [planner/SingleNodePlanner.java#L289|https://github.com/apache/incubator-impala/blob/master/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java#L289]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
