Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22544#discussion_r220193922
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -2140,21 +2140,34 @@ case class ElementAt(left: Expression, right:
Expression) extends GetMapValueUti
}
override def inputTypes: Seq[AbstractDataType] = {
- Seq(TypeCollection(ArrayType, MapType),
- left.dataType match {
- case _: ArrayType => IntegerType
- case _: MapType => mapKeyType
- case _ => AnyDataType // no match for a wrong 'left' expression
type
- }
- )
+ (left.dataType, right.dataType) match {
+ case (ArrayType(e1, hasNull), e2: IntegralType) if (e2 != LongType)
=>
+ Seq(ArrayType(e1, hasNull), IntegerType)
+ case (MapType(keyType, valueType, hasNull), e2) =>
+ TypeCoercion.findTightestCommonType(keyType, e2) match {
+ case Some(dt) => Seq(MapType(dt, valueType, hasNull), dt)
+ case _ => Seq.empty
--- End diff --
Previously it will do implicitly conversion on the right expr to map's key
type. Map type is not converted at all.
This change finds the common type between right expr and map's key type and
will do conversion on map and right expr.
Can you provide an example showing why this is needed?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
