comphead commented on issue #2214:
URL:
https://github.com/apache/datafusion-comet/issues/2214#issuecomment-3294213466
```
test("invalid CometHashAggregate") {
sql("CREATE TABLE t(id int, value int) USING PARQUET")
sql("INSERT INTO t VALUES (1, 10)")
sql("INSERT INTO t VALUES (2, 20)")
sql("INSERT INTO t VALUES (3, 10)")
sql("INSERT INTO t VALUES (4, 30)")
sql("INSERT INTO t VALUES (5, 20)")
sql("INSERT INTO t VALUES (6, 10)")
sql("select * from t").write.mode("overwrite").parquet("/tmp/test_11")
withSQLConf(
CometConf.COMET_EXEC_SHUFFLE_ENABLED.key -> "true",
"spark.comet.cast.allowIncompatible" -> "true",
"spark.comet.enabled" -> "true",
"spark.comet.exec.replaceSortMergeJoin" -> "true",
"spark.comet.exec.shuffle.enableFastEncoding" -> "true",
"spark.comet.exec.shuffle.enabled" -> "true",
"spark.comet.exec.shuffle.fallbackToColumnar" -> "true",
"spark.comet.explain.native.enabled" -> "true",
"spark.comet.explainFallback.enabled" -> "true",
"spark.shuffle.manager" ->
"org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager",
"spark.comet.logFallbackReasons.enabled" -> "true") {
spark.read.parquet("/tmp/test_11").createOrReplaceTempView("t2")
spark.sql("select count(id) from t2 group by id").explain("extended")
sql("drop table t")
}
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- !CometHashAggregate [id#43, count#51L], Final, [id#43], [count(id#43)]
+- CometExchange hashpartitioning(id#43, 10), ENSURE_REQUIREMENTS,
CometNativeShuffle, [plan_id=141]
+- !CometHashAggregate [id#43], Partial, [id#43],
[partial_count(id#43)]
+- CometScan [native_iceberg_compat] parquet [id#43] Batched: true,
DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1
paths)[file:/tmp/test_11], PartitionFilters: [], PushedFilters: [], ReadSchema:
struct<id:int>
```
This is interesting as `!` usually says the reference from preceding plan
node is missing, which looks okay here
and it still warns without AQE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
