parthchandra commented on issue #1637: URL: https://github.com/apache/datafusion-comet/issues/1637#issuecomment-3876859283
https://github.com/apache/datafusion-comet/pull/3454 enables spark sql tests with ansi mode enabled and enables ignored tests. The following tests fail (analysis by Claude Code) Tests Specific to Spark 4.0.1 (15 Total) CollectLimitExec / Data Ordering (3 tests) File: sql/core/src/test/scala/org/apache/spark/sql/execution/InsertSortForLimitAndOffsetSuite.scala - root LIMIT preserves data ordering with CollectLimitExec - middle LIMIT preserves data ordering with the extra sort - middle OFFSET preserves data ordering with the extra sort Collation Support (2 tests) File: sql/core/src/test/scala/org/apache/spark/sql/collation/CollationSuite.scala - hash join should be used for collated strings if sort merge join is not forced - rewrite with collationkey shouldn't disrupt multiple join conditions Aggregation & Features (4 tests) - SPARK-35332: Make cache plan disable configs configurable - check AQE - File: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala - SPARK-47430 Support GROUP BY MapType - File: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala - SPARK-43402: FileSourceScanExec supports push down data filter with scalar subquery - File: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala - SPARK-48655: order by on partition keys should not introduce additional shuffle - File: sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala --- 2. SubqueryBroadcastExec Support (3 tests) Related Issues https://github.com/apache/datafusion-comet/issues/242 and https://github.com/apache/datafusion-comet/issues/1737 These tests fail because Comet doesn't support SubqueryBroadcastExec: File: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala (2 tests) - partition pruning in broadcast hash joins (#1737) - SPARK-35568: Fix UnsupportedOperationException when enabling both AQE and DPP (#1737) File: sql/core/src/test/scala/org/apache/spark/sql/execution/RemoveRedundantProjectsSuite.scala (1 test) - join with ordering requirement (#242) --- 3. Other New Issues in 4.0.1 (3 tests) Extended Explain Output - Test: SPARK-47289: extended explain info - File: sql/core/src/test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala AQE Join Strategy Changes - Test: v1 write with AQE changing SMJ to BHJ - File: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/V1WriteCommandSuite.scala Hive Non-UTF8 Strings - Test: SPARK-30201 HiveOutputWriter standardOI should use ObjectInspectorCopyOption.DEFAULT - File: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
