parthchandra opened a new pull request, #756: URL: https://github.com/apache/datafusion-comet/pull/756
## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/679 and https://github.com/apache/datafusion-comet/issues/670 ## Rationale for this change Improves performance of reading decimals in Comet vector ## What changes are included in this PR? There are two changes 1. For values with precision less than `Decimal.MAX_LONG_DIGITS` we will read the values into a long rather than into a BigInteger/BigDecimal 2. While reading the decimal buffer from native to jvm, we now read the entire buffer at one time instead of one value at a time. We were incurring a bounds check in JVM code for every read from native that is now reduced from once per value to once per vector. Caveat: the code now uses `num_elements_in_vector * size of element = 4K * 4 = 16` bytes more than previously. ## How are these changes tested? Existing tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
