[
https://issues.apache.org/jira/browse/SPARK-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust resolved SPARK-3295.
-------------------------------------
Resolution: Won't Fix
These are actually different operations. UnionAll is similar to the SQL
command and will fail if the two schema are different. union and ++ will not.
> [Spark SQL] schemaRdd1 ++ schemaRdd2 does not return another SchemaRdd
> -----------------------------------------------------------------------
>
> Key: SPARK-3295
> URL: https://issues.apache.org/jira/browse/SPARK-3295
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.0.2
> Reporter: Evan Chan
> Priority: Minor
>
> Right now,
> schemaRdd1.unionAll(schemaRdd2) returns a SchemaRdd.
> However,
> schemaRdd1 ++ schemaRdd2 returns an RDD[Row].
> Similarly,
> schemaRdd1.union(schemaRdd2) returns an RDD[Row].
> This is inconsistent. Let's make ++ and union have the same behavior as
> unionAll.
> Actually, not sure there needs to be both union and unionAll.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
