manuzhang opened a new pull request, #5000: URL: https://github.com/apache/datafusion-comet/pull/5000
## Which issue does this PR close? Relates to [apache/datafusion-comet#4180](https://github.com/apache/datafusion-comet/issues/4180). ## Rationale for this change `spark.sql.legacy.followThreeValuedLogicInArrayExists` changes the result of `exists` when no predicate result is true and at least one result is null. The existing SQL test did not exercise that divergent behavior under both configuration values, and result comparison alone did not directly assert that the JVM codegen dispatcher ran. ## What changes are included in this PR? - Run `expressions/array/exists.sql` with both values of `spark.sql.legacy.followThreeValuedLogicInArrayExists`. - Add column and literal no-match-plus-null cases, plus an always-null predicate case. - Add a focused Scala test that runs `ArrayExists` under both config values inside `assertCodegenRan` and `checkSparkAnswerAndOperator`, so silent fallback to Spark fails the test. - Document the config's semantics, JVM codegen-dispatch path, disabled-dispatcher fallback behavior, and test coverage in the supported Spark configurations contributor guide. ## How are these changes tested? - `make core` - Focused `CometSqlFileTestSuite exists` matrix on Spark 3.4.3, 3.5.8, 4.0.2, and 4.1.2: both configuration cases passed on every profile. - `CometCodegenHOFSuite` on Spark 3.4.3, 3.5.8, 4.0.2, and 4.1.2: 5 tests passed on every profile. - Documentation generation completed for all four Spark profiles under JDK 17; the final HTML render was not available because `sphinx-build` is not installed locally. - Spotless and `git diff --check` - The pre-commit Rust formatting and Clippy checks passed. The hook's Scala/scalafix step could not resolve the unavailable `org.scalameta:semanticdb-scalac_2.13.17:4.13.6` artifact; the commit was amended with `--no-verify` after the focused cross-version tests above passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
