Github user gberger commented on the issue:
https://github.com/apache/spark/pull/19792
Hi all,
I have changed the error message to be like #18521. Here are some examples:
```
_merge_type(
StructType([StructField("f1", ArrayType(MapType(StringType(),
LongType())))]),
StructType([StructField("f1", ArrayType(MapType(DoubleType(),
LongType())))])
)
# TypeError: key of map element in array field f1: Can not merge type
<class 'pyspark.sql.types.StringType'> and <class
'pyspark.sql.types.DoubleType'>
_merge_type(
StructType([
StructField("f1", MapType(StringType(), LongType())),
StructField("f2", StringType())]),
StructType([
StructField("f1", MapType(StringType(), DoubleType())),
StructField("f2", StringType())]))
# TypeError: value of map field f1: Can not merge type <class
'pyspark.sql.types.LongType'> and <class 'pyspark.sql.types.DoubleType'>
_merge_type(
StructType([
StructField("f1", ArrayType(LongType())),
StructField("f2", StringType())]),
StructType([
StructField("f1", ArrayType(DoubleType())),
StructField("f2", StringType())]))
# TypeError: element in array field f1: Can not merge type <class
'pyspark.sql.types.LongType'> and <class 'pyspark.sql.types.DoubleType'>
_merge_type(
StructType([StructField("f1", LongType()), StructField("f2",
StringType())]),
StructType([StructField("f1", DoubleType()), StructField("f2",
StringType())]))
# TypeError: field f1: Can not merge type <class
'pyspark.sql.types.LongType'> and <class 'pyspark.sql.types.DoubleType'>
_merge_type(
StructType([StructField("f1", StructType([StructField("f2",
LongType())]))]),
StructType([StructField("f1", StructType([StructField("f2",
StringType())]))]))
# TypeError: field f2 in field f1: Can not merge type <class
'pyspark.sql.types.LongType'> and <class 'pyspark.sql.types.StringType'>
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
