It is a hadoop limitation. hdfs move operation is inexpensive. I am
assuming that is not an option to you because you want to save the path
structure (for some backward compatibility sake).
Something like symbolic links (i think its not supported in 0.20, not sure)
or path filter might help. But,
Hi Vince,
Hive partitioning can only exist by issueing new directories in HDFS. There
is no way to partition the data in a Hive table without adding extra
filepaths/dirs in HDFS.
For an external table you have to redistribute the data yourself in
corresponding filepaths and add the new partition
pache.org>"
mailto:user@hive.apache.org>>
Subject: RE: Partitioning EXTERNAL TABLE without copying or moving files
Hi Vince,
External tables shouldn’t issue copy or move commands to your data files. You
should define the base table location to ‘/logs’, and issue alter table
commands to ad
Hi Vince,
External tables shouldn't issue copy or move commands to your data files. You
should define the base table location to '/logs', and issue alter table
commands to add partitions for each date.
Example:
CREATE EXTERNAL TABLE logs (
Data STRING
) PARTITIONED BY (cal_date STRING)
ROW FO
