Oh, I see.
just as the example we have:
FROM page_view_stg pvs
INSERT OVERWRITE TABLE page_view PARTITION(dt='2008-06-08', country)
SELECT pvs.viewTime, pvs.userid, pvs.page_url,
pvs.referrer_url, null, null, pvs.ip, pvs.country
The dynamic partition we have is on country, and the other partition is dt.
In this implementation, what if I want to import the data into page_view more
than 1 time ? Let us say, we import the data hourly, and with current
dynamic partition implementation , the existing country partition will be
overwritten!
Is there any other way to avoid this without telling me to import the data
once per day?
2011/4/15 Ning Zhang <nzh...@fb.com>
> The INSERT OVERWRITE command will not overwrite the whole table. If you
> specify a partition in that table, it will only overwrite that partition. If
> you specify dynamic partitions, it will only create/overwrite partitions
> that will be seen from the input query (pvs.country in the example).
>
>
> On Apr 15, 2011, at 12:31 AM, Erix Yao wrote:
>
> Does this mean if I want the type field as the partition key , I will have
> to split the raw data by myself and load the files into the target table?
>
> I see there's an example in tutorial:
>
> FROM page_view_stg pvs
> INSERT OVERWRITE TABLE page_view PARTITION(dt='2008-06-08', country)
> SELECT pvs.viewTime, pvs.userid, pvs.page_url, pvs.referrer_url,
> null, null, pvs.ip, pvs.country
>
> but insert must overwrite the table whole table partition.
> Can I insert without the overwrite key word?
>
>
> 2011/4/15 Ning Zhang <nzh...@fb.com>
>
>> The LOAD DATA command only copy the files to the destination directory. It
>> doesn't read the records of the input file, so it cannot do partitioning
>> based on record values.
>>
>> On Apr 14, 2011, at 10:52 PM, Erix Yao wrote:
>>
>> hi,all
>> The dynamic partition function is amazing ,but only works in insert
>> clause. Can I use it while loading data into table?
>>
>> For example: load data LOAD DATA LOCAL INPATH
>> `/tmp/pv_2008-06-08_us.txt` INTO TABLE page_view
>> PARTITION(date='2008-06-08', country='US', type);
>> type is the dynamic partition key in the raw data?
>>
>> This will be very cool! If is supported, I will not have to category the
>> raw data according to the type column.
>>
>>
>>
>> --
>> haitao.yao@Beijing
>>
>>
>>
>>
>>
>>
>
>
> --
> haitao.yao@Beijing
>
>
>
>
>
>
--
haitao.yao@Beijing
