Re: UPDATE : Adding new columns to parquet based Hive table

2015-01-29 Thread Kumar V
I wanted to clarify something.  It works if the Hive-Parquet table is a plain vanilla table.  But if the table is a partitioned table, then the error occurs after adding new fields to the table.   Any ideas on how to handle this ? hive> create table nvctest_part(col1 string,col2 string, col3 int)

Re: Adding new columns to parquet based Hive table

2015-01-14 Thread Kumar V
Hi,    Thanks for your response.I can't do another insert as the data is already in the table. Also, since there is a lot of data in the table already, I am trying to find a way to avoid reprocessing/reloading. Thanks. On Wednesday, January 14, 2015 2:47 PM, Daniel Haviv wrote: Hi

Re: Adding new columns to parquet based Hive table

2015-01-14 Thread Daniel Haviv
Hi Kumar, Altering the table just update's Hive's metadata without updating parquet's schema. I believe that if you'll insert to your table (after adding the column) you'll be able to later on select all 3 columns. Daniel > On 14 בינו׳ 2015, at 21:34, Kumar V wrote: > > Hi, > > Any ideas

Re: Adding new columns to parquet based Hive table

2015-01-14 Thread Kumar V
Hi,     Any ideas on how to go about this ? Any insights you have would be helpful. I am kinda stuck here. Here are the steps I followed on hive 0.13 1) create table t (f1 String, f2 string) stored as Parquet;2) upload parquet files with 2 fields3) select * from t; < Works fine.4) alter table

Adding new columns to parquet based Hive table

2015-01-07 Thread Kumar V
Hi,    I have a Parquet format Hive table with a few columns.  I have loaded a lot of data to this table already and it seems to work.I have to add a few new columns to this table.  If I add new columns, queries don't work anymore since I have not reloaded the old data.Is there a way to add new