By having partition level metadata and not deriving it, quite a few things can be achieved
* partitions can be external (i.e. Location of a partitions need not be relative to the table) * partitions level schemas so that a table's schema can evolve * partitions can be disabled without deleting the actual data So that change was made to facilitate above things. Without that it will be difficult to achieve this. Instead of doing hadoop dfs -rmr <partition location>, you can use 'alter table <tbl_name> drop partition (<parition key value>)' and 'load data ... Into <tabl> <part>' to load data. I suppose we can give option to truncate partition, but for now dropping a partition or truncating it doesn't have much difference though. But if there is a great demand for automatically inferring partitions, we can put the old code back in but enable it only with a config param. Thanks, Prasad ________________________________ From: Frederick Oko <[email protected]> Reply-To: <[email protected]> Date: Sun, 26 Apr 2009 16:10:29 -0700 To: <[email protected]> Subject: Re: RE: Truncate Table Keep in mind that if your use case is to guarantee an empty table to load into then every 'insert overwrite table' does just that for you (it clobbers vs insert into). This and the workaround (which must be table specific) is complicated when you have partitions. If your usecase is reclaiming space, just data invalidation, or have partitions simply deleting the base data via hadoop -rmr /user/hive/warehouse/<table>/* (assuming default data mgmt) is decoupled but works and should not be harmful -- Hive is supposed to be just an overlay but there was a change to partition mgmt that I would like to hear whether people think raw delete is now undesirable. On Apr 26, 2009 12:34 AM, "Ashish Thusoo" <[email protected]> wrote: Suhail, Hive does not support truncate yet. I think Prasad already opened a JIRA for this https://issues.apache.org/jira/browse/HIVE-446 Matt, I must say that this is a very creative workaround :) Ashish ________________________________________ From: Matt Pestritto [[email protected]] Sent: Saturday, April 25, 2009 10:08 AM To: [email protected] Subject: Re: Truncate Table I've used the following: Insert overwrite tbl select * from tbl where col1 = 'fake value' On 4/25/...
