richardstartin edited a comment on issue #8822: optimize numeric column null value checking for low filter selectivity (more rows) URL: https://github.com/apache/incubator-druid/pull/8822#issuecomment-549985610 Cool heatmaps! Iteration should definitely perform better than calls to `contains` because it reduces the complexity of each call from logarithmic in the number of non empty 16 bit blocks to constant, obviously with a small set up cost. This is the same principle as #6764 which removes binary searches during bitmap construction. It looks like the operation in question (`VectorSelectorUtils.populateNullVector`) is actually simulating the extraction of a mask from the bitmap at the offset of the current vector. Perhaps, with the ability to skip to the next non empty vector (given a vector width), it would be quite easy to implement this as an iterator which returns masks on each call to `next`. Perhaps you could plug this in to Druid's vectorized query engine. cc @lemire.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
