austin3dickey opened a new issue, #35917: URL: https://github.com/apache/arrow/issues/35917
### Describe the bug, including details regarding any error messages, version, and platform. For a while[^1] now, the TPCH-02, TPCH-10, and TPCH-13 queries have been failing validation during R benchmarking. Here is [a link](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-arm64-m6g-linux-compute/builds/2797#01888349-b91d-4cca-9757-04766a04f4c6) to a recent build, and here is [the benchmarking code](https://github.com/voltrondata-labs/arrowbench/blob/main/R/bm-tpc-h.R) that is run. Each of these queries fails with the same error for all the cases that we test: `format=native` and `format=parquet`, and also `scale_factor=1` and `scale_factor=10`. TPCH-02: ``` Error in eval(bm$after_each, envir = ctx) : The answer does not match Calls: run_bm -> run_iteration -> eval -> eval In addition: Warning message: In eval(bm$after_each, envir = ctx) : old vs new s_comment - old[1, ] l, ironic instructions cajole + new[1, ] uriously regular requests hag - old[2, ] es. furiously silent deposits among the deposits haggle furiously a + new[2, ] efully express instructions. regular requests against the slyly fin - old[3, ] ar, regular requests nag blithely special accounts. final deposits impress carefully. ironic, + new[3, ] etect about the furiously final accounts. slyly ironic pinto beans sleep inside the furiously - old[4, ] s sleep according to the quick requests. carefully + new[4, ] ackages boost blithely. blithely regular deposits c [... truncated] Execution halted ``` TPCH-10: ``` Error in eval(bm$after_each, envir = ctx) : The answer does not match Calls: run_bm -> run_iteration -> eval -> eval In addition: Warning message: In eval(bm$after_each, envir = ctx) : old vs new c_comment - old[1, ] ep. blithely regular foxes promise slyly furiously ironic depend + new[1, ] sits. slyly regular requests sleep alongside of the regular inst - old[2, ] endencies sleep. slyly express deposits nag carefully around the even tithes. slyly regular + new[2, ] ggle carefully enticing requests. final deposits use bold, bold pinto beans. ironic, idle re - old[3, ] tes. final instructions nag quickly according to + new[3, ] need to boost against the slyly regular account - old[4, ] ost carefully. slyly regular packages cajole about the blithely final ideas. permanently daring depos [... truncated] Execution halted ``` TPCH-13: ``` Error in eval(bm$after_each, envir = ctx) : The answer does not match Calls: run_bm -> run_iteration -> eval -> eval In addition: Warning message: In eval(bm$after_each, envir = ctx) : old vs new c_count old[1, ] 0 - old[2, ] 10 + new[2, ] 9 - old[3, ] 9 + new[3, ] 10 old[4, ] 11 old[5, ] 8 old[6, ] 12 old vs new c_count old[7, ] 13 old[8, ] 19 old[9, ] 7 - old[10, ] 18 + new[10, ] 17 - old[11, ] 20 + new[11, ] 18 - old[12, ] 14 + new[12, ] 20 old[13, ] 15 - old[14, ] 17 + new[14, ] 14 old[15, ] 16 old[16, ] 21 old[17, ] 22 old vs new c_count old[33, ] 33 old[34, ] 34 old[35, ] 35 - old[36, ] 36 + new[36, ] 1 - old[37, ] 1 + new[37, ] 36 - old[38, ] 37 + new[38, ] 38 - old[39, ] 38 + new[39, ] 37 - old[40, ] 41 + new[40, ] 40 - old[41, ] 40 + new[41, ] 41 old[42, ] 39 `old$c_count[1:6]`: 0 10 9 11 8 12 `new$c_count[1:6]`: 0 9 10 11 8 12 `old$c_count[7:17]`: 13 19 7 18 20 14 15 17 16 21 [... truncated] Execution halted ``` [^1]: I will look into the first time this failed, and report back. ### Component(s) Benchmarking, R -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
