andygrove opened a new pull request, #7654: URL: https://github.com/apache/datafusion-comet/pull/7654
## Which issue does this PR close? Backport of #7552 to `branch-1.2`. Closes #7551 on this branch. ## Rationale for this change `arr[i]` fails in Comet when `i` is not an INT, while Spark accepts any integer type there, for example `sort_array(collect_list(x))[floor(count(*) / 2)]`, since `floor` returns a BIGINT (#7551, `priority:high`). `branch-1.2` has the same `CometGetArrayItem` serde and the same `ListExtract` native expression. See #7552 for the full analysis. ## What changes are included in this PR? Cherry-pick of c40c8c2822 (`git cherry-pick -x`) with no conflicts and no adaptations. The diff against the source commit is identical apart from line numbers and blob hashes. `CometGetArrayItem` casts a non-INT index to INT with the legacy cast, which keeps the low 32 bits of a BIGINT as Spark's conversion does, before passing it to `ListExtract`. It comes with the source PR's new cases in the `get_array_item.sql` and `get_array_item_ansi.sql` SQL file tests. Other release branches: `branch-1.2` is the newest release branch, so there is no newer one to port to. Whether `branch-1.1` needs a backport is not part of this PR. ## How are these changes tested? On this branch, with JDK 17 and the default Spark profile: `./mvnw test-compile` succeeds, and the `get_array_item.sql` and `get_array_item_ansi.sql` file tests pass (2 tests, 0 failures). No native source changed. With `spark/src/main` reverted to `branch-1.2`'s version, both file tests fail, so the branch has the bug and the tests catch it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
