Dandandan edited a comment on pull request #8960: URL: https://github.com/apache/arrow/pull/8960#issuecomment-749214350
Did a small test against master on your branch against DataFusion to see the impact (merged master to this branch). From profiling I know query 1 is spending quite some time in filtering. Master ``` Query 1 iteration 0 took 895.2 ms Query 1 iteration 1 took 900.6 ms Query 1 iteration 2 took 877.8 ms Query 1 iteration 3 took 882.3 ms Query 1 iteration 4 took 870.9 ms Query 1 iteration 5 took 876.8 ms Query 1 iteration 6 took 875.2 ms Query 1 iteration 7 took 879.5 ms Query 1 iteration 8 took 864.7 ms Query 1 iteration 9 took 874.1 ms Query 1 avg time: 879.70 ms ``` This PR ``` Query 1 iteration 0 took 688.4 ms Query 1 iteration 1 took 688.1 ms Query 1 iteration 2 took 679.2 ms Query 1 iteration 3 took 688.1 ms Query 1 iteration 4 took 687.9 ms Query 1 iteration 5 took 694.7 ms Query 1 iteration 6 took 701.7 ms Query 1 iteration 7 took 708.3 ms Query 1 iteration 8 took 704.2 ms Query 1 iteration 9 took 702.1 ms Query 1 avg time: 694.28 ms ``` Looks like a pretty decent speedup @jorgecarleitao ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
