andygrove commented on code in PR #509:
URL: https://github.com/apache/datafusion-comet/pull/509#discussion_r1629447204
##########
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##########
@@ -712,17 +712,6 @@ class CometSparkSessionExtensions
}
override def apply(plan: SparkPlan): SparkPlan = {
- // DataFusion doesn't have ANSI mode. For now we just disable CometExec
if ANSI mode is
- // enabled.
- if (isANSIEnabled(conf)) {
- if (COMET_ANSI_MODE_ENABLED.get()) {
- logWarning("Using Comet's experimental support for ANSI mode.")
- } else {
- logInfo("Comet extension disabled for ANSI mode")
- return plan
- }
- }
Review Comment:
> If I understand correctly we have not set up CI yet with ANSI enabled.
Well, we have the Spark 4 tests where ANSI is enabled and that is catching
issues for sure.
I noticed that we were explicitly disabled ANSI mode in CometTestBase and I
have removed that so we use the default ANSI mode for whatever Spark version we
are running against. This should mean that all of our unit tests will now run
with ANSI enabled when running against Spark 4+. Let's see how many issues this
finds 🌑
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
