[
https://issues.apache.org/jira/browse/SPARK-17734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-17734.
----------------------------------
Resolution: Won't Fix
We have {{joinWith}} to return {{Dataset}}. Also, we have {{join}} and
{{Dataset}} and {{DataFrame}} are really interchangeable in the concept up to
my knowledge.
Even if the duplicated field names are problems, they can be just renamed
easily. It does not sound worth introducing another new API or changing
behaviour.
I am resolving this. Please reopen this if I misunderstood.
> inner equi-join shorthand that returns Datasets, like DataFrame already has
> ---------------------------------------------------------------------------
>
> Key: SPARK-17734
> URL: https://issues.apache.org/jira/browse/SPARK-17734
> Project: Spark
> Issue Type: Wish
> Reporter: Leif Warner
> Priority: Minor
>
> There's an existing ".join(right: Dataset[_], usingColumn: String):
> DataFrame" method on Dataset.
> Would appreciate it if a variant that returns typed Datasets would also
> available.
> If you write a join that contains the common column name name, you get an
> AnalysisException thrown because that's ambiguous, e.g:
> $"foo" === $"foo"
> So I wrote table1.toDF()("foo") === table2.toDF()("foo"), but that's a little
> error prone, and coworkers considered it a hack and didn't want to use it,
> because it "mixes DataFrame and Dataset api".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
