b-slim commented on issue #7676: Add OakIncrementalIndex to Druid URL: https://github.com/apache/incubator-druid/pull/7676#issuecomment-494027530 @sanastas that is simple not true, some of those locks are part of `allocate` [calls](https://github.com/yahoo/Oak/blob/master/core/src/main/java/com/oath/oak/NativeAllocator/OakNativeMemoryAllocator.java#L94-L100 ) and some are part of[ iterator loop](https://github.com/yahoo/Oak/blob/master/core/src/main/java/com/oath/oak/InternalOakMap.java#L272-L278) to do some stuff that i doubt it is static ... Looking at the allocator briefly i see bunch of red flags, like possible internal fragmentations up to factor 2 and not clear way to deal with external fragmentations ... Memory Management is a very hard thing in general and make it lock free is not going to be simple task .... Writing one form scratch is a common source of bugs, In fact most of the big data projects avoid this task and use the Netty allocator. The take our of this, current state is not lock free thus lets avoid confusion, (FYI even on the oak github page i do not see it called lock-free so not sure why you are calling it lock free). But the most important thing to add here is how OKA memory management will fit with how Druid manages Direct memory.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
