iy...@gmail.com>>
Date: Friday, October 30, 2015 at 3:57 PM
To: Ted Yu mailto:yuzhih...@gmail.com>>
Cc: "user@spark.apache.org<mailto:user@spark.apache.org>"
mailto:user@spark.apache.org>>
Subject: Re: how to merge two dataframes
Not a bad idea I suspect but doesn
I see - you were trying to union a non-Cassandra DF with Cassandra DF :-(
On Fri, Oct 30, 2015 at 12:57 PM, Yana Kadiyska
wrote:
> Not a bad idea I suspect but doesn't help me. I dumbed down the repro to
> ask for help. In reality one of my dataframes is a cassandra DF.
> So cassDF.registerTempT
Not a bad idea I suspect but doesn't help me. I dumbed down the repro to
ask for help. In reality one of my dataframes is a cassandra DF.
So cassDF.registerTempTable("df1") registers the temp table in a different
SQL Context (new CassandraSQLContext(sc)).
scala> sql("select customer_id, uri, brow
How about the following ?
scala> df.registerTempTable("df")
scala> df1.registerTempTable("df1")
scala> sql("select customer_id, uri, browser, epoch from df union select
customer_id, uri, browser, epoch from df1").show()
+---+-+---+-+
|customer_id| uri|browser|e
Just use unionAll method:
df1.show()
nameid
a 1
b 2
df2.show()
nameid
c 3
d 4
df1.unionAll(df2).show()
nameid
a 1
b 2
c 3
d 4
Thanks,
Peter Rudneko
On 2015-04-22 16:38, bipin wrote:
I have looked into
