Alexander Saydakov created IMPALA-10956:
-------------------------------------------

             Summary: datasketches UDFS: memory leak and merge overhead
                 Key: IMPALA-10956
                 URL: https://issues.apache.org/jira/browse/IMPALA-10956
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 3.4.0
            Reporter: Alexander Saydakov


I believe that there are memory leaks in aggregate UDFs that are using Apache 
Datasketches. Sketch or union objects are placed in a buffer to maintain the 
state of the aggregation, and later deallocated without calling destructors for 
those objects.

Also there is unnecessary overhead during the merge phase of some of those 
UDFs, when an instance of a union is created for every merge operation and 
finalized by calling a quite expensive get_result() every time as well. This 
second issue is about performance, not a bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to