Re: insert Hive table with RDD

2015-03-04 Thread patcharee 

Hi,

I guess that toDF() api in spark 1.3 which is required build from source 
code?


Patcharee

On 03. mars 2015 13:42, Cheng, Hao wrote:

Using the SchemaRDD / DataFrame API via HiveContext

Assume you're using the latest code, something probably like:

val hc = new HiveContext(sc)
import hc.implicits._
existedRdd.toDF().insertInto("hivetable")
or

existedRdd.toDF().registerTempTable("mydata")
hc.sql("insert into hivetable as select xxx from mydata")



-Original Message-
From: patcharee [mailto:patcharee.thong...@uni.no]
Sent: Tuesday, March 3, 2015 7:09 PM
To: user@spark.apache.org
Subject: insert Hive table with RDD

Hi,

How can I insert an existing hive table with an RDD containing my data?
Any examples?

Best,
Patcharee

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional 
commands, e-mail: user-h...@spark.apache.org


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org




-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: insert Hive table with RDD

2015-03-03 Thread Jagat Singh 
Will this recognize the hive partitions as well.

Example

insert into specific partition of hive ?

On Tue, Mar 3, 2015 at 11:42 PM, Cheng, Hao  wrote:

> Using the SchemaRDD / DataFrame API via HiveContext
>
> Assume you're using the latest code, something probably like:
>
> val hc = new HiveContext(sc)
> import hc.implicits._
> existedRdd.toDF().insertInto("hivetable")
> or
>
> existedRdd.toDF().registerTempTable("mydata")
> hc.sql("insert into hivetable as select xxx from mydata")
>
>
>
> -Original Message-
> From: patcharee [mailto:patcharee.thong...@uni.no]
> Sent: Tuesday, March 3, 2015 7:09 PM
> To: user@spark.apache.org
> Subject: insert Hive table with RDD
>
> Hi,
>
> How can I insert an existing hive table with an RDD containing my data?
> Any examples?
>
> Best,
> Patcharee
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional
> commands, e-mail: user-h...@spark.apache.org
>
>

RE: insert Hive table with RDD

2015-03-03 Thread Cheng, Hao 
Using the SchemaRDD / DataFrame API via HiveContext

Assume you're using the latest code, something probably like:

val hc = new HiveContext(sc)
import hc.implicits._
existedRdd.toDF().insertInto("hivetable")
or

existedRdd.toDF().registerTempTable("mydata")
hc.sql("insert into hivetable as select xxx from mydata")



-Original Message-
From: patcharee [mailto:patcharee.thong...@uni.no] 
Sent: Tuesday, March 3, 2015 7:09 PM
To: user@spark.apache.org
Subject: insert Hive table with RDD

Hi,

How can I insert an existing hive table with an RDD containing my data? 
Any examples?

Best,
Patcharee

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional 
commands, e-mail: user-h...@spark.apache.org

insert Hive table with RDD

2015-03-03 Thread patcharee 

Hi,

How can I insert an existing hive table with an RDD containing my data? 
Any examples?


Best,
Patcharee

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org