If you are using Spark-sql as opposed to spark-shell, then you can just use
UNION as in SQL for this. Pretty straight forward.
SELECT * from TABLE_A
UNION
SELECT * from TABLE_B
ORDER BY COLUMN_A, COLUMN_B;
Example
spark-sql> SELECT * FROM dummy where id = 1
> UNION
> SELECT * FROM dummy2 where id = 10
> ORDER by id;
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 2 March 2016 at 06:40, Mao, Wei <[email protected]> wrote:
> It should be a “union” operation instead of “join”.
>
>
>
> And besides from Ted’s answer, if you are working with DataSet API:
>
>
>
> *def **union(other: Dataset[**T**]): Dataset[**T**] = withPlan[**T**](other){
> (left, right) =>*
>
>
>
> Thanks,
>
> William
>
>
>
> *From:* Ted Yu [mailto:[email protected]]
> *Sent:* Wednesday, March 2, 2016 11:41 AM
> *To:* Angel Angel
> *Cc:* user
> *Subject:* Re: Connect the two tables in spark sql
>
>
>
> You only showed one record from each table.
>
>
>
> Have you looked at the following method in DataFrame ?
>
>
>
> def unionAll(other: DataFrame): DataFrame = withPlan {
>
>
>
> On Tue, Mar 1, 2016 at 7:13 PM, Angel Angel <[email protected]>
> wrote:
>
> Hello Sir/Madam,
>
>
>
> I am using the spark sql for the data operation.
>
>
>
> I have two tables with the same fields.
>
>
>
> Table 1
>
> name
>
> address
>
> phone Number
>
> sagar
>
> india
>
> 22222222
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Table 2
>
> name
>
> address
>
> phone Number
>
> jaya
>
> india
>
> 2222222
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I want to join this tables like the following way
>
>
> Result Table
>
> name
>
> address
>
> phone Number
>
> jaya
>
> india
>
> 2222222
>
> sagar
>
> india
>
> 22222222
>
>
>
>
>
> How can i join this table. I tried using the join command it add the table
> to left side.
>
>
>
> Please help me to solve this query.
>
>
>
> Thanks,
>
>
>
