leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size URL: https://github.com/apache/incubator-druid/issues/6743#issuecomment-452073707 @gianm > Do you know if there's a way to use this functionality to track per-query memory usage? That would be ideal, since we don't want excessive memory use by one query to affect another unrelated query. The mechanism I was referring to is in [Bits.java](http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/nio/Bits.java#l618) and is for the JVM. It is not thread specific so not directly useful on a query basis. However, Druid could create its own query direct-memory tracking mechanism as an "extension" of the underlying JVM mechanism. The way we leverage the nio/Bits tracking mechanism can be found in [memory/NioBits](https://github.com/DataSketches/memory/blob/master/src/main/java/com/yahoo/memory/NioBits.java). > have you got any more thoughts about how we might allow aggregators to allocate extra memory? No conclusions just yet. I will be studying this this week.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
