[
https://issues.apache.org/jira/browse/SPARK-22152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183585#comment-16183585
]
Drew Robb commented on SPARK-22152:
-----------------------------------
I personally use `Option` very frequently in datasets, and it is also idomatic
to use Option over null in scala if possible. Another use case would be for
`Dataset[Seq[T]] => Dataset[T]`:
{code:java}
scala> Seq(Seq(1,2,3)).toDS.flatMap{x => x}.show()
+-----+
|value|
+-----+
| 1|
| 2|
| 3|
+-----+
{code}
> Add Dataset flatten function
> ----------------------------
>
> Key: SPARK-22152
> URL: https://issues.apache.org/jira/browse/SPARK-22152
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Drew Robb
> Priority: Minor
>
> Currently you can use an identify flatMap to flatten a Dataset, for example
> to get from a Dataset[Option[T]] to a Dataset[T], but adding flatten directly
> would allow for a more similar API to scala collections.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
