sandugood commented on PR #4672: URL: https://github.com/apache/datafusion-comet/pull/4672#issuecomment-4750836190
Thank you for the review @andygrove I have changed my approach and bechmarked it. My current approach is about using `arrow` internal types and buffers (essentially `NullBuffer`, from the underlying `BooleanBufer`, which is constructed from the bitmap, although with bits flipped), without materializing and looping through `Vec<>` which was still an improvement, but not a drastic one. Posting the benchmarking results, which I ran using `cargo bench --bench array_element_append`. Posting result from `with_nulls` version of benches (`no_nulls` wasnt affected with this PR) | array_type | main (current) | incoming PR | time_reduce | | ---| --- | --- | --- | | i32 | 29.274μs | 2.4256μs | -91.71% | | i64 | 27.260μs |4.7887μs | -82.43% | | f64 | 29.352μs | 4.6621μs | -84.11% | | date32 | 26.863μs | 2.5345μs | -89.89% | | timestamp | 45.818μs | 4.3659μs | -90.47% | cc @mbutrovich, if you could take a quick look at this, please. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
