[
https://issues.apache.org/jira/browse/SPARK-41963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-41963.
----------------------------------
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 39960
[https://github.com/apache/spark/pull/39960]
> Different exception message in DataFrame.unpivot
> ------------------------------------------------
>
> Key: SPARK-41963
> URL: https://issues.apache.org/jira/browse/SPARK-41963
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
> Affects Versions: 3.4.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Fix For: 3.4.0
>
>
> Running {{test_parity_dataframe DataFrameParityTests.test_unpivot_negative}}
> fails as below:
> {code}
> with self.subTest(desc="with no value columns"):
> for values in [[], ()]:
> with self.subTest(values=values):
> with self.assertRaisesRegex(
> Exception, # (AnalysisException,
> SparkConnectException)
> r".*\[UNPIVOT_REQUIRES_VALUE_COLUMNS] At least one
> value column "
> r"needs to be specified for UNPIVOT, all columns
> specified as ids.*",
> ):
> > df.unpivot("id", values, "var", "val").collect()
> E AssertionError: ".*\[UNPIVOT_REQUIRES_VALUE_COLUMNS]
> At least one value column needs to be specified for UNPIVOT, all columns
> specified as ids.*" does not match "[UNPIVOT_VALUE_DATA_TYPE_MISMATCH]
> Unpivot value columns must share a least common type, some types do not:
> ["BIGINT" (`int`), "DOUBLE" (`double`), "STRING" (`str`)]
> E Plan: 'Unpivot ArraySeq(id#2947L),
> List(List(int#2948L), List(double#2949), List(str#2950)), var, [val]
> E +- Project [id#2939L AS id#2947L, int#2940L AS
> int#2948L, double#2941 AS double#2949, str#2942 AS str#2950]
> E +- LocalRelation [id#2939L, int#2940L,
> double#2941, str#2942]
> E "
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
