Jackie-Jiang opened a new pull request #6262: URL: https://github.com/apache/incubator-pinot/pull/6262
## Description Enhance the forward index reader by specializing the case for each numBits (from 1 to 31) compression. The int value reader has 3 methods: - Bulk read 32 values - Read value without buffer boundary check - Read value with buffer boundary check The new forward index reader uses the 3 methods based on the passed in doc ids: - For sequential doc ids, use bulk read - For non-sequential doc ids, use unchecked read if possible, and only use checked read for the last few values We observe up to 5x improvement for the reading, detailed perf number can be found here: https://docs.google.com/spreadsheets/d/1o66G7Jw_7K4swEcp1FYCzgr0MF9JBAAM6KMGOXBSdIA/edit?usp=sharing ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
