andygrove commented on issue #44:
URL:
https://github.com/apache/datafusion-comet/issues/44#issuecomment-2361585685
This issues appears to be resolved, so I will close this issue. Thanks @okue
for reporting it.
```
scala> Seq(15901).toDF("dt").write.parquet("/tmp/dt")
24/09/19 10:45:43 INFO core/src/lib.rs: Comet native library version 0.3.0
initialized
24/09/19 10:45:43 WARN CometSparkSessionExtensions$CometExecRule: Comet
cannot execute some parts of this plan natively (set
spark.comet.explainFallback.enabled=false to disable this logging):
Execute InsertIntoHadoopFsRelationCommand [COMET: Execute
InsertIntoHadoopFsRelationCommand is not supported]
+- WriteFiles [COMET: WriteFiles is not supported]
+- LocalTableScan [COMET: LocalTableScan is not supported]
scala> spark.read.schema("dt
date").parquet("/tmp/dt").createOrReplaceTempView("v")
24/09/19 10:45:50 WARN CometSparkSessionExtensions$CometExecRule: Comet
cannot execute some parts of this plan natively (set
spark.comet.explainFallback.enabled=false to disable this logging):
Execute CreateViewCommand [COMET: Execute CreateViewCommand is not
supported]
+- CreateViewCommand
+- LogicalRelation
scala> val df = spark.sql("select min(dt) from v")
df: org.apache.spark.sql.DataFrame = [min(dt): date]
scala> df.show(false)
+----------+
|min(dt) |
+----------+
|2013-07-15|
+----------+
scala> df.explain()
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- !CometHashAggregate [min#26], Final, [min(dt#7)]
+- CometExchange SinglePartition, ENSURE_REQUIREMENTS,
CometNativeShuffle, [plan_id=176]
+- !CometHashAggregate [dt#7], Partial, [partial_min(dt#7)]
+- CometScan parquet [dt#7] Batched: true, DataFilters: [], Format:
CometParquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/dt],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<dt:date>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
