[
https://issues.apache.org/jira/browse/IMPALA-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-2756:
----------------------------------
Issue Type: Improvement (was: Bug)
> Rethink UDF interfaces
> ----------------------
>
> Key: IMPALA-2756
> URL: https://issues.apache.org/jira/browse/IMPALA-2756
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.3.0
> Reporter: Michael Ho
> Priority: Critical
> Labels: incompatibility, resource-management
>
> In the next update of the UDF interfaces, we want to make sure that Status is
> returned for certain functions such as FunctionContext::Allocate(),
> Reallocate() and may be Free().
> Currently, these functions can return a valid pointer to a buffer even if the
> memory limit has been exceeded. It relays the error status (e.g. memory limit
> exceeded) via the FunctionContext object and relies on various poll points in
> the code to call QueryMaintenance() to catch the problem asynchronously and
> abort the query.
> The long term goal would be to enforce the memory limit synchronously in
> these functions (i.e. essentially removing Allocate and replacing it with
> TryAllocate()) and get rid of all the poll loop of status check in
> QueryMaintenance() if possible. This would also give us a tighter bound on
> the memory consumption, potentially avoiding some corner cases in which
> impalad may overrun the process memory limit.
> Please see IMPALA-2620 and http://gerrit.cloudera.org:8080/#/c/1445 for
> discussions about this problem.
> Also, FunctionContext::Allocate(), Reallocate() should support 64-bit
> allocations so the parameter 'size' has to be 64-bit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
