[
https://issues.apache.org/jira/browse/SPARK-61527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61527:
-----------------------------------
Labels: pull-request-available sql (was: sql)
> approx_top_k counts positive and negative zero as separate items
> ----------------------------------------------------------------
>
> Key: SPARK-61527
> URL: https://issues.apache.org/jira/browse/SPARK-61527
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Environment: {code}
> Spark: 4.3.0
> Commit: 98bc0212c94705434471627068f41e374ccab143
> Java: OpenJDK 17.0.17
> OS: macOS 15.7.4
> Master: local[2]
> {code}
> Reporter: Yibo Dong
> Priority: Major
> Labels: pull-request-available, sql
>
> {{approx_top_k}} treats positive zero ({{0.0}}) and negative zero ({{-0.0}})
> as separate items, although Spark SQL considers them equal.
> For two input rows containing {{0.0}} and {{-0.0}}, {{approx_top_k(v, 2,
> 10)}} returns two items with count 1 each, while {{GROUP BY}} returns one
> group with count 2.
> h2. Steps to Reproduce
> Save the following SQL as {{tmp.sql}}:
> {code}
> WITH input AS (
> SELECT /*+ REPARTITION(3) */ v
> FROM VALUES (0D), (-0D) AS t(v)
> )
> SELECT approx_top_k(v, 2, 10) AS frequent_items
> FROM input;
> SELECT v, count(*) AS frequency
> FROM VALUES (0D), (-0D) AS t(v)
> GROUP BY v;
> {code}
> Execute:
> {code}
> spark-sql --master 'local[2]'
> --conf spark.ui.enabled=false -f tmp.sql
> {code}
> h2. Expected Result
> The first query should return a single zero-valued item with count 2:
> {code}
> [{"item":0.0,"count":2}]
> {code}
> The sign of the zero-valued item is not important.
> Spark SQL treats positive and negative zero as equal in grouping, as
> documented in the [SQL migration
> guide|https://spark.apache.org/docs/4.3.0/sql-migration-guide.html].
> With only two input rows and a tracking capacity of ten items, the
> discrepancy cannot be attributed to insufficient sketch capacity.
> h2. Actual Result
> {code}
> [{"item":0.0,"count":1},{"item":-0.0,"count":1}]
> {code}
> The two zero representations are counted separately. The independent {{GROUP
> BY}} query correctly returns one zero-valued group with frequency 2.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]