leventov commented on issue #8126: [Proposal] BufferAggregator support for growable sketches. URL: https://github.com/apache/incubator-druid/issues/8126#issuecomment-518326114 > Then presenting a solution to this problem using Oak to manage the off-heap index, handling growable sketches using Memory, and showing that it performs as good as or better than the existing implementation would be a win-win-win solution, correct? Yes. I just want to warn you about trying to bring all of that at once - the extent of the change may become unbearable. So even if it incurs more work, it's better to make those changes separately: e. g. first, the transition to `Memory` (in the given subsystem), then - Oak. Doing it in the opposite order may be harder because Oak is already based on `Memory`, despite there is a method [`unsafeByteBufferView`](https://datasketches.github.io/api/memory/snapshot/apidocs/org/apache/datasketches/memory/Memory.html#unsafeByteBufferView-long-int-). > Any documentation on how we should configure the cluster to allow it running in off-heap mode? Not sure what do you mean by that. It's not possible to use the off-heap incremental index for indexing even when all aggregations are constant-sized - it's not implemented.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
