Enrico Minack created SPARK-38864:
-------------------------------------
Summary: Melt function for Dataset API
Key: SPARK-38864
URL: https://issues.apache.org/jira/browse/SPARK-38864
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 3.4.0
Reporter: Enrico Minack
As pointed out in SPARK-30273 and SPARK-37799, the Dataset API provides the
{{pivot}} method but not its reverse operation {{melt}}. It is only provided as
part of the PySpark Pandas API, implemented in Python. However, {{melt}} is not
available via the PySpark DataFrame API, nor through Spark Scala Dataset API.
As mentioned in
[#26912|https://github.com/apache/spark/pull/26912#pullrequestreview-332975715],
it should be implemented in Scala and re-uses from PySpark.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
