Aleksander Eskilson created SPARK-22826:
-------------------------------------------
Summary: [SQL] findWiderTypeForTwo Fails over StructField of Array
Key: SPARK-22826
URL: https://issues.apache.org/jira/browse/SPARK-22826
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.0
Reporter: Aleksander Eskilson
The {{findWiderTypeForTwo}} codepath in Catalyst {{TypeCoercion}} fails when
applied to to {{StructType}} having the following fields:
{noformat}
StructType(StructField("a", ArrayType(StringType, containsNull=true)) ::
Nil),
StructType(StructField("a", ArrayType(StringType, containsNull=false)) ::
Nil)
{noformat}
When in {{findTightestCommonType}}, the function attempts to recursively find
the tightest common type of two arrays. These two arrays are not equal types
(since one would admit null elements and the other would not), but
{{findTightestCommonType}} has no match case for {{ArrayType}} (or
{{MapType}}), so the {{get}}
[operation|https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L108]
operation on the dataType of the {{StructField}} throws a
{{NoSuchElementException}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
