I was hoping there would be a configuration where I can set the outputformat
for
my query.
Regards,
Gaurav Jain
----- Original Message ----
From: Jacob R Rideout <apa...@jacobrideout.net>
To: hive-user@hadoop.apache.org
Sent: Wed, October 6, 2010 1:42:57 PM
Subject: Re: How to output SeqFile
On Wed, Oct 6, 2010 at 2:35 PM, gaurav jain <jainy_gau...@yahoo.com> wrote:
> I do have that.
>
> However I am not writing directly to the table partition. Instead, I first
>write
> my data in a tmp directory (eventually moved to the hdfs table partition) and
> then publish that partition using alter table statement in metastore.
>
> Something like this:
>
> -- create table x ... stored as SeqFile
> -- insert overwrite directory 'd' select * from table y
> -- distcp 'd' x/dateint=.../hour=...
> -- alter table x add partition ....
>
> In the second step above I need to produce SeqFile.
>
>
> Thanks for prompt reply.
> Gaurav Jain
>
>
> ----- Original Message ----
> From: Yang <teddyyyy...@gmail.com>
> To: jainy_gau...@yahoo.com
> Sent: Wed, October 6, 2010 1:28:42 PM
> Subject: Re: How to output SeqFile
>
> Gaurav:
>
> not sure if I understand your question correctly....
> when you create the output table, that has an option to set the
> output table SerDe
>
> Regards
> Yang
>
> On Wed, Oct 6, 2010 at 1:18 PM, gaurav jain <jainy_gau...@yahoo.com> wrote:
>>
>>
>>
>>
>> How can I produce a sequence file from query
>>
>> insert overwrite directory ....
>>
>>
>> I have set:
>>
>> SET io.seqfile.compression.type=BLOCK;
>> SET hive.exec.compress.output=true;
>> set mapred.output.compression.type=BLOCK;
>> set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;
>>
>>
>>
>> It seems to produce Text .gz format files.
>>
>>
>>
>> Regards,
>> Gaurav Jain
>>
>>
>>
>>
>
>
>
>
>
if you are inserting into the directory rather than the table, hive
won't know to look at the metadata description of the table
you need something like:
insert overwrite table x select * from table y
