andygrove opened a new issue, #2623:
URL: https://github.com/apache/datafusion-comet/issues/2623
### What is the problem the feature request solves?
## Input Schema
This is just showing fields referenced from the query:
```
Created table test1 with schema:
c3: IntegerType
c6: DoubleType
```
## SQL
```
SELECT c6, c3, c6 <= c3 FROM test1 ORDER BY c6, c3;
```
### Spark Plan
```
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
*(2) Sort [c6#128 ASC NULLS FIRST, c3#125 ASC NULLS FIRST], true, 0
+- AQEShuffleRead coalesced
+- ShuffleQueryStage 0
+- Exchange rangepartitioning(c6#128 ASC NULLS FIRST, c3#125 ASC
NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=834]
+- *(1) Project [c6#128, c3#125, (c6#128 <= cast(c3#125 as
double)) AS (c6 <= c3)#346]
+- *(1) ColumnarToRow
+- FileScan parquet [c3#125,c6#128] Batched: true,
DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1
paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test1.parquet],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c3:int,c6:double>
+- == Initial Plan ==
Sort [c6#128 ASC NULLS FIRST, c3#125 ASC NULLS FIRST], true, 0
+- Exchange rangepartitioning(c6#128 ASC NULLS FIRST, c3#125 ASC NULLS
FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=820]
+- Project [c6#128, c3#125, (c6#128 <= cast(c3#125 as double)) AS (c6
<= c3)#346]
+- FileScan parquet [c3#125,c6#128] Batched: true, DataFilters: [],
Format: Parquet, Location: InMemoryFileIndex(1
paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test1.parquet],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c3:int,c6:double>
```
### Comet Plan
```
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
*(1) CometColumnarToRow
+- CometSort [c6#128, c3#125, (c6 <= c3)#354], [c6#128 ASC NULLS FIRST,
c3#125 ASC NULLS FIRST]
+- AQEShuffleRead coalesced
+- ShuffleQueryStage 0
+- CometExchange rangepartitioning(c6#128 ASC NULLS FIRST,
c3#125 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, CometNativeShuffle,
[plan_id=899]
+- CometProject [c6#128, c3#125, (c6 <= c3)#354], [c6#128,
c3#125, (knownfloatingpointnormalized(normalizenanandzero(c6#128)) <=
knownfloatingpointnormalized(normalizenanandzero(cast(c3#125 as double)))) AS
(c6 <= c3)#354]
+- CometScan [native_iceberg_compat] parquet
[c3#125,c6#128] Batched: true, DataFilters: [], Format: CometParquet, Location:
InMemoryFileIndex(1
paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test1.parquet],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c3:int,c6:double>
+- == Initial Plan ==
CometSort [c6#128, c3#125, (c6 <= c3)#354], [c6#128 ASC NULLS FIRST,
c3#125 ASC NULLS FIRST]
+- CometExchange rangepartitioning(c6#128 ASC NULLS FIRST, c3#125 ASC
NULLS FIRST, 200), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=886]
+- CometProject [c6#128, c3#125, (c6 <= c3)#354], [c6#128, c3#125,
(knownfloatingpointnormalized(normalizenanandzero(c6#128)) <=
knownfloatingpointnormalized(normalizenanandzero(cast(c3#125 as double)))) AS
(c6 <= c3)#354]
+- CometScan [native_iceberg_compat] parquet [c3#125,c6#128]
Batched: true, DataFilters: [], Format: CometParquet, Location:
InMemoryFileIndex(1
paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test1.parquet],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c3:int,c6:double>
```
First difference at row 71:
Spark: `0.0,-2048876980,false`
Comet: `-0.0,-2035576013,false`
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
