leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size URL: https://github.com/apache/incubator-druid/issues/6743#issuecomment-485046057 @gianm This topic has been inactive for a while, but the issue still exists. As we have been integrating our C++ versions of our sketches into PostgreSQL we have learned about how they solve this problem. Custom UDFs perform memory allocations and destruction using the Postgres supplied "palloc" and "pfree" functions. This way, Postgres can track and manage the allocation and free processes in the context of queries or transactions. This way, when the contexts are closed, all the allocated memory can be guaranteed to be freed. Inside a specific aggregation function every allocation for space (for an array for example) is performed via these palloc and pfree functions. The aggregation function does not have to live in a contiguous chunk of memory, which make the design of the aggregation function much easier.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
