sanastas commented on issue #5698: Oak: New Concurrent Key-Value Map URL: https://github.com/apache/incubator-druid/issues/5698#issuecomment-513532033 Hi again, Just to prove what I have said about memory usage in numbers, hereby please find the graph presenting the measurements of memory usage of OakIncrementalIndex vs existing IncrementalIndex. The experiment presents memory usage measured for an invocation of IncrementalIngestionBenchmark. The horizontal axis presents the number of rows used in each experiment. The vertical axis presents GBs. Black dashed line present only data size - just keys and values (IndexRow and Aggregators). Blue line presents the total memory consumption for OakIncrementalIndex (both off-heap and on-heap). Red line presents the total memory consumption for Druid's current IncrementalIndex (just on-heap). One can see that IncrementalIndex metadata takes almost the same size as data itself. In total OakIncrementalIndex use much less memory than IncrementalIndex. [Memory Usage (Metadata vs Data).pdf](https://github.com/apache/incubator-druid/files/3414395/Memory.Usage.Metadata.vs.Data.pdf)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
