[
https://issues.apache.org/jira/browse/SPARK-61239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61239:
-----------------------------------
Labels: correctness pull-request-available (was: correctness)
> Map subscript casts an integral key to a narrower map key type and returns
> the value of a different key
> -------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61239
> URL: https://issues.apache.org/jira/browse/SPARK-61239
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.3, 5.0.0, 4.4.0
> Reporter: sepuri sai krishna
> Priority: Major
> Labels: correctness, pull-request-available
>
> {{m[key]}} (also {{col("m")(key)}} and {{getItem}}) casts the key to the key
> type of the map. When the map key type is a narrower integral type than the
> key, this cast wraps around with ANSI mode off, so the lookup returns the
> value of a different key, and fails with ANSI mode on, although the key is
> simply not in the map. {{element_at}} widens the map keys instead and returns
> NULL.
> {code:sql}
> CREATE TABLE t USING parquet AS
> SELECT id, map(CAST(44 AS TINYINT), 'v') AS m, map(44, 'iv') AS mi FROM
> range(1);
> SET spark.sql.ansi.enabled=false;
> SELECT m[300], mi[4294967340L], element_at(m, 300), element_at(mi,
> 4294967340L) FROM t;
> -- v iv NULL NULL (expected NULL for all four)
> {code}
> 300 is cast to {{TINYINT}} and wraps to 44, and 4294967340 is cast to {{INT}}
> and wraps to 44. The same happens with a non-literal key, for example
> {{m[CAST(300 AS INT) + id * 0]}}. With {{spark.sql.ansi.enabled=true}} (the
> default), {{m[300]}} and {{mi[4294967340L]}} fail:
> {noformat}
> [CAST_OVERFLOW] The value 300 of the type "INT" cannot be cast to "TINYINT"
> due to an overflow. Use `try_cast` to tolerate overflow and return NULL
> instead.
> {noformat}
> Checked on 4.0.3 and master.
> {{GetMapValue.inputTypes}} is {{Seq(MapType, keyType)}}, so the implicit type
> coercion casts the key to the map key type. {{ElementAt.inputTypes}} uses
> {{TypeCoercion.findTightestCommonType(keyType, key type)}} and casts the map
> and the key to that type.
> A key of a fractional type is truncated the same way ({{mi[44.5D]}} returns
> {{iv}}, where {{element_at}} returns NULL), which may be worth looking at
> separately.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
