[
https://issues.apache.org/jira/browse/SPARK-58164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-58164:
-----------------------------------
Assignee: Eric Yang
> approx_top_k_estimate and approx_top_k_combine fail on sketches whose item
> type was widened by type coercion
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-58164
> URL: https://issues.apache.org/jira/browse/SPARK-58164
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Labels: pull-request-available
>
> A query that combines/estimates `approx_top_k` sketches whose item type is
> widened by type coercion (e.g. a `UNION ALL` of an INT and a BIGINT sketch
> column) fails at runtime.
> {code:java}
> SELECT approx_top_k_estimate(approx_top_k_combine(sketch, 5))
> FROM (SELECT approx_top_k_accumulate(c) AS sketch, 'int' AS tag
> FROM VALUES (1), (1), (2) AS t(c)
> UNION ALL
> SELECT approx_top_k_accumulate(c) AS sketch, 'long' AS tag
> FROM VALUES (CAST(9 AS BIGINT)) AS t(c))
> WHERE tag = 'int';{code}
> It throws:
> {code:java}
> 26/07/15 22:05:03 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 3)
> org.apache.datasketches.common.SketchesArgumentException: Bounds Violation:
> reqOffset: 0, reqLength: 16, (reqOff + reqLen): 16, allocSize: 10
> at org.apache.datasketches.common.Util.checkBounds(Util.java:659)
> at
> org.apache.datasketches.common.ArrayOfLongsSerDe.deserializeFromMemory(ArrayOfLongsSerDe.java:68)
> at
> org.apache.datasketches.common.ArrayOfLongsSerDe.deserializeFromMemory(ArrayOfLongsSerDe.java:34)
> at
> org.apache.datasketches.frequencies.ItemsSketch.getInstance(ItemsSketch.java:287)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
