Re: How to write data into Hive partitioned Parquet table?

2014-10-15 Thread Banias H
t implementation is much appreciated. Many thanks. Regards, BH On Tue, Oct 14, 2014 at 8:44 PM, Banias H wrote: > Hi, > > I am still new to Spark. Sorry if similar questions are asked here before. > I am trying to read a Hive table; then run a query and save the result into &g

How to write data into Hive partitioned Parquet table?

2014-10-14 Thread Banias H
Hi, I am still new to Spark. Sorry if similar questions are asked here before. I am trying to read a Hive table; then run a query and save the result into a Hive partitioned Parquet table. For example, I was able to run the following in Hive: INSERT INTO TABLE target_table PARTITION (partition_fi

Re: Getting table info from HiveContext

2014-10-02 Thread Banias
Thanks Michael. On Thursday, October 2, 2014 8:41 PM, Michael Armbrust wrote: We actually leave all the DDL commands up to hive, so there is no programatic way to access the things you are looking for. On Thu, Oct 2, 2014 at 5:17 PM, Banias wrote: Hi, > > >Would anybody kn

Getting table info from HiveContext

2014-10-02 Thread Banias
Hi, Would anybody know how to get the following information from HiveContext given a Hive table name? - partition key(s) - table directory - input/output format I am new to Spark. And I have a couple tables created using Parquet data like: CREATE EXTERNAL TABLE parquet_table ( COL1 string, COL