Tim Armstrong created IMPALA-5844:
-------------------------------------
Summary: Fix management of FunctionContext "local" allocations.
Key: IMPALA-5844
URL: https://issues.apache.org/jira/browse/IMPALA-5844
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
FunctionContexts associated with expressions make two kind of allocations with
very different lifecycles. One type of allocation is owned and managed by the
expression, while the other "local" allocation is implicitly transferred to the
Impala daemon after control flow returns from the expression. Both are
currently allocated from the same pool.
RowBatches returned from plan nodes may reference variable-length data stored
in local allocations so this memory should be attached to the RowBatches.
One approach here is:
* Separate local and other allocations to allocate from different MemPools.
* Manage local allocations in bulk by clearing, freeing, or transferring data
from that MemPool, similar to other memory that would be allocated from a
MemPool.
I think there are some potential wrinkles here and work to set up correct
MemPools for all places that use expressions but I think the high-level
approach should work.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
