[
https://issues.apache.org/jira/browse/IMPALA-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho resolved IMPALA-2885.
--------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.7.0
The other patch I was working on was abandoned. So closing this for now. The
code has changed a lot since then and will reopen Jira if there is any clean up
opportunity in the future.
> Scanners store per-split objects in per-query object pool
> ---------------------------------------------------------
>
> Key: IMPALA-2885
> URL: https://issues.apache.org/jira/browse/IMPALA-2885
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.5.0
> Reporter: Tim Armstrong
> Assignee: Michael Ho
> Priority: Minor
> Labels: resource-management
> Fix For: Impala 2.7.0
>
>
> Various scanners store control structures in RuntimeState::object_pool_ to be
> cleaned up at the end of the query. Since some of these control structures
> are allocated for every input split, a small amount of memory is wasted on
> control structures that are no longer needed. If a large number of scan
> ranges and columns are processed in a query, this can add megabytes or 10s of
> megabytes to the query's memory consumption.
> I added some logging and saw that for a largish scan there was 10000+ objects
> in the object pool.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
