andygrove opened a new issue, #1824:
URL: https://github.com/apache/datafusion-comet/issues/1824
### Describe the bug
```
2025-05-30T18:23:30.5178844Z
2025-05-30T18:23:30.6051928Z [0m[[0m[0minfo[0m] [0m[0m[31m-
SPARK-32038: NormalizeFloatingNumbers should work on distinct aggregate ***
FAILED *** (509 milliseconds)[0m[0m
2025-05-30T18:23:30.6060839Z [0m[[0m[0minfo[0m] [0m[0m[31m Results
do not match for query:[0m[0m
2025-05-30T18:23:30.6064796Z [0m[[0m[0minfo[0m] [0m[0m[31m Timezone:
sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]][0m[0m
2025-05-30T18:23:30.6068178Z [0m[[0m[0minfo[0m] [0m[0m[31m Timezone
Env: [0m[0m
2025-05-30T18:23:30.6068782Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6069646Z [0m[[0m[0minfo[0m] [0m[0m[31m == Parsed
Logical Plan ==[0m[0m
2025-05-30T18:23:30.6070401Z [0m[[0m[0minfo[0m] [0m[0m[31m 'Sort [1
ASC NULLS FIRST], true[0m[0m
2025-05-30T18:23:30.6071332Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
'Aggregate [1], ['uid, unresolvedalias('count(distinct 'score))][0m[0m
2025-05-30T18:23:30.6072262Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
'UnresolvedRelation [view], [], false[0m[0m
2025-05-30T18:23:30.6072975Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6073653Z [0m[[0m[0minfo[0m] [0m[0m[31m ==
Analyzed Logical Plan ==[0m[0m
2025-05-30T18:23:30.6074711Z [0m[[0m[0minfo[0m] [0m[0m[31m uid:
string, count(DISTINCT score): bigint[0m[0m
2025-05-30T18:23:30.6075609Z [0m[[0m[0minfo[0m] [0m[0m[31m Sort
[uid#493096 ASC NULLS FIRST], true[0m[0m
2025-05-30T18:23:30.6076871Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
Aggregate [uid#493096], [uid#493096, count(distinct score#493097) AS
count(DISTINCT score)#493101L][0m[0m
2025-05-30T18:23:30.6077970Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
SubqueryAlias view[0m[0m
2025-05-30T18:23:30.6078759Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
View (`view`, [uid#493096, score#493097])[0m[0m
2025-05-30T18:23:30.6079697Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- Project [_1#493091 AS uid#493096, _2#493092 AS score#493097][0m[0m
2025-05-30T18:23:30.6080601Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- LocalRelation [_1#493091, _2#493092][0m[0m
2025-05-30T18:23:30.6081233Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6081843Z [0m[[0m[0minfo[0m] [0m[0m[31m ==
Optimized Logical Plan ==[0m[0m
2025-05-30T18:23:30.6082618Z [0m[[0m[0minfo[0m] [0m[0m[31m Sort
[uid#493096 ASC NULLS FIRST], true[0m[0m
2025-05-30T18:23:30.6083769Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
Aggregate [uid#493096], [uid#493096, count(distinct score#493097) AS
count(DISTINCT score)#493101L][0m[0m
2025-05-30T18:23:30.6085382Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
Project [_1#493091 AS uid#493096, _2#493092 AS score#493097][0m[0m
2025-05-30T18:23:30.6086322Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
LocalRelation [_1#493091, _2#493092][0m[0m
2025-05-30T18:23:30.6086946Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6087532Z [0m[[0m[0minfo[0m] [0m[0m[31m ==
Physical Plan ==[0m[0m
2025-05-30T18:23:30.6088301Z [0m[[0m[0minfo[0m] [0m[0m[31m
AdaptiveSparkPlan isFinalPlan=true[0m[0m
2025-05-30T18:23:30.6089016Z [0m[[0m[0minfo[0m] [0m[0m[31m +- ==
Final Plan ==[0m[0m
2025-05-30T18:23:30.6089647Z [0m[[0m[0minfo[0m] [0m[0m[31m *(4)
CometColumnarToRow[0m[0m
2025-05-30T18:23:30.6090590Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
CometSort [uid#493096, count(DISTINCT score)#493101L], [uid#493096 ASC NULLS
FIRST][0m[0m
2025-05-30T18:23:30.6091770Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
AQEShuffleRead coalesced[0m[0m
2025-05-30T18:23:30.6092510Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- ShuffleQueryStage 2[0m[0m
2025-05-30T18:23:30.6093824Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometColumnarExchange rangepartitioning(uid#493096 ASC NULLS FIRST, 5),
ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=967521][0m[0m
2025-05-30T18:23:30.6096048Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(3) HashAggregate(keys=[uid#493096], functions=[count(distinct
score#493104)], output=[uid#493096, count(DISTINCT score)#493101L])[0m[0m
2025-05-30T18:23:30.6097391Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(3) ColumnarToRow[0m[0m
2025-05-30T18:23:30.6098200Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- AQEShuffleRead coalesced[0m[0m
2025-05-30T18:23:30.6098788Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- ShuffleQueryStage 1[0m[0m
2025-05-30T18:23:30.6100300Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometColumnarExchange hashpartitioning(uid#493096, 5),
ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=967449][0m[0m
2025-05-30T18:23:30.6102391Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(2) HashAggregate(keys=[uid#493096],
functions=[partial_count(distinct score#493104)], output=[uid#493096,
count#493107L])[0m[0m
2025-05-30T18:23:30.6103719Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(2) CometColumnarToRow[0m[0m
2025-05-30T18:23:30.6104995Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometHashAggregate [uid#493096, score#493104],
[uid#493096, score#493104][0m[0m
2025-05-30T18:23:30.6106080Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- AQEShuffleRead coalesced[0m[0m
2025-05-30T18:23:30.6106990Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- ShuffleQueryStage 0[0m[0m
2025-05-30T18:23:30.6108493Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometColumnarExchange
hashpartitioning(uid#493096, score#493104, 5), ENSURE_REQUIREMENTS,
CometColumnarShuffle, [plan_id=967350][0m[0m
2025-05-30T18:23:30.6110297Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(1) HashAggregate(keys=[uid#493096,
knownfloatingpointnormalized(normalizenanandzero(score#493097)) AS
score#493104], functions=[], output=[uid#493096, score#493104])[0m[0m
2025-05-30T18:23:30.6111750Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(1) Project [_1#493091 AS
uid#493096, _2#493092 AS score#493097][0m[0m
2025-05-30T18:23:30.6112619Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- *(1) LocalTableScan [_1#493091,
_2#493092][0m[0m
2025-05-30T18:23:30.6113492Z [0m[[0m[0minfo[0m] [0m[0m[31m +- ==
Initial Plan ==[0m[0m
2025-05-30T18:23:30.6114773Z [0m[[0m[0minfo[0m] [0m[0m[31m
CometSort [uid#493096, count(DISTINCT score)#493101L], [uid#493096 ASC NULLS
FIRST][0m[0m
2025-05-30T18:23:30.6115816Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
CometColumnarExchange rangepartitioning(uid#493096 ASC NULLS FIRST, 5),
ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=966971][0m[0m
2025-05-30T18:23:30.6116869Z [0m[[0m[0minfo[0m] [0m[0m[31m +-
HashAggregate(keys=[uid#493096], functions=[count(distinct score#493104)],
output=[uid#493096, count(DISTINCT score)#493101L])[0m[0m
2025-05-30T18:23:30.6117921Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometColumnarExchange hashpartitioning(uid#493096, 5), ENSURE_REQUIREMENTS,
CometColumnarShuffle, [plan_id=966969][0m[0m
2025-05-30T18:23:30.6119888Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- HashAggregate(keys=[uid#493096], functions=[partial_count(distinct
score#493104)], output=[uid#493096, count#493107L])[0m[0m
2025-05-30T18:23:30.6121130Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometHashAggregate [uid#493096, score#493104], [uid#493096,
score#493104][0m[0m
2025-05-30T18:23:30.6122069Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- CometColumnarExchange hashpartitioning(uid#493096, score#493104, 5),
ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=966946][0m[0m
2025-05-30T18:23:30.6123359Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- HashAggregate(keys=[uid#493096,
knownfloatingpointnormalized(normalizenanandzero(score#493097)) AS
score#493104], functions=[], output=[uid#493096, score#493104])[0m[0m
2025-05-30T18:23:30.6124492Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- Project [_1#493091 AS uid#493096, _2#493092 AS
score#493097][0m[0m
2025-05-30T18:23:30.6125725Z [0m[[0m[0minfo[0m] [0m[0m[31m
+- LocalTableScan [_1#493091, _2#493092][0m[0m
2025-05-30T18:23:30.6126395Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6126737Z [0m[[0m[0minfo[0m] [0m[0m[31m ==
Results ==[0m[0m
2025-05-30T18:23:30.6127057Z [0m[[0m[0minfo[0m] [0m[0m [0m
2025-05-30T18:23:30.6127496Z [0m[[0m[0minfo[0m] [0m[0m[31m ==
Results ==[0m[0m
2025-05-30T18:23:30.6127950Z [0m[[0m[0minfo[0m] [0m[0m[31m !==
Correct Answer - 2 == == Spark Answer - 2 ==[0m[0m
2025-05-30T18:23:30.6128995Z [0m[[0m[0minfo[0m] [0m[0m[31m !struct<>
struct<uid:string,count(DISTINCT score):bigint>[0m[0m
2025-05-30T18:23:30.6129997Z [0m[[0m[0minfo[0m] [0m[0m[31m
[abellina,2] [abellina,2][0m[0m
2025-05-30T18:23:30.6130548Z [0m[[0m[0minfo[0m] [0m[0m[31m
![mithunr,1] [mithunr,2] (QueryTest.scala:267)[0m[0m
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
