[
https://issues.apache.org/jira/browse/SPARK-59678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59678:
-----------------------------------
Labels: pull-request-available (was: )
> pivot() on an array column fails when the column's element type is
> non-nullable
> -------------------------------------------------------------------------------
>
> Key: SPARK-59678
> URL: https://issues.apache.org/jira/browse/SPARK-59678
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Eric Yang
> Priority: Major
> Labels: pull-request-available
>
> {{pivot()}} on an array column fails at analysis with an error that prints
> the same type on both sides:
> {code:java}
> scala> Seq(1.0d).toDF("v").selectExpr("v", "array(v) AS
> a").groupBy("v").pivot("a").count()
> org.apache.spark.sql.AnalysisException: [PIVOT_VALUE_DATA_TYPE_MISMATCH]
> Invalid pivot value '[1.0]':
> value data type array<double> does not match pivot column data type
> array<double>
> {code}
> It only happens when the array's element type is non-nullable; the same query
> succeeds with {{{}Seq(Some(1.0d), None){}}}. There is no workaround, since
> the pivot values are collected by Spark rather than supplied by the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
