Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15072
@HyukjinKwon can we move this into a set specific method within Dataset
itself, for example:
```scala
/** A convenient function to wrap a set based logical plan and produce a
Dataset. */
@inline private def withSetPlan[U : Encoder](logicalPlan: => LogicalPlan):
Dataset[U] = {
if (classTag.runtimeClass == classOf[Row]) {
// Set operators widen types (change the schema), so we cannot reuse
the row encoder.
Dataset.ofRows(sparkSession, logicalPlan).asInstanceOf[Dataset[U]]
} else {
Dataset(sparkSession, logicalPlan)
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
