fatemehp commented on PR #14509: URL: https://github.com/apache/arrow/pull/14509#issuecomment-1292687122
Here are benchmark results before and after the change proposed in this pull request. Only including the Skip since this change does not affect the read performance. We get up to 15X reduction when the batch size (last parameter) is 1, which means we are repeatedly re-allocating the scratch space. ``` PARAMETERS: 1) definition level 2) repetition level 3) 1 for Skip, 0 for ReadBatch (here we only have Skip) 4) batch size BEFORE ------------------------------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------------------------------- BM_Skip/0/0/1/1/iterations:1000 150319842 ns 149567587 ns 1000 BM_Skip/0/0/1/1000/iterations:1000 244565 ns 244931 ns 1000 BM_Skip/0/0/1/10000/iterations:1000 115395 ns 115924 ns 1000 BM_Skip/0/0/1/100000/iterations:1000 115241 ns 115916 ns 1000 BM_Skip/1/0/1/1/iterations:1000 149224507 ns 148683644 ns 1000 BM_Skip/1/0/1/1000/iterations:1000 805812 ns 805417 ns 1000 BM_Skip/1/0/1/10000/iterations:1000 702999 ns 700108 ns 1000 BM_Skip/1/0/1/100000/iterations:1000 654163 ns 651947 ns 1000 BM_Skip/1/1/1/1/iterations:1000 165600118 ns 164864530 ns 1000 BM_Skip/1/1/1/1000/iterations:1000 1130975 ns 1130252 ns 1000 BM_Skip/1/1/1/10000/iterations:1000 1009628 ns 1009589 ns 1000 BM_Skip/1/1/1/100000/iterations:1000 1029064 ns 1028726 ns 1000 AFTER ------------------------------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------------------------------- BM_Skip/0/0/1/1/iterations:1000 10280337 ns 10234495 ns 1000 BM_Skip/0/0/1/1000/iterations:1000 101228 ns 101436 ns 1000 BM_Skip/0/0/1/10000/iterations:1000 96565 ns 96648 ns 1000 BM_Skip/0/0/1/100000/iterations:1000 96598 ns 96814 ns 1000 BM_Skip/1/0/1/1/iterations:1000 25771605 ns 25718891 ns 1000 BM_Skip/1/0/1/1000/iterations:1000 651609 ns 650940 ns 1000 BM_Skip/1/0/1/10000/iterations:1000 660890 ns 654217 ns 1000 BM_Skip/1/0/1/100000/iterations:1000 640417 ns 636855 ns 1000 BM_Skip/1/1/1/1/iterations:1000 36639124 ns 36433537 ns 1000 BM_Skip/1/1/1/1000/iterations:1000 978058 ns 976403 ns 1000 BM_Skip/1/1/1/10000/iterations:1000 997193 ns 996529 ns 1000 BM_Skip/1/1/1/100000/iterations:1000 999080 ns 993296 ns 1000 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
