Hi Zheng
I have a hive version checked out from svn on Jan 28. (I thought it was
Feb 15)
Missed this change by a day!
I guess its time for me to move to the latest
Thanks,
Ashwin
On 5/11/10 3:54 PM, Zheng Shao wrote:
Hi Ashwin,
Which version of Hive are you using? The new directory structure
(yyyy-MM-dd) is introduced in
https://issues.apache.org/jira/browse/HIVE-1109
Zheng
On Tue, May 11, 2010 at 1:47 PM, Ashwin Agate<[email protected]> wrote:
I dont see yyyy-mm-dd directory names but rather series of numbers
/user/hive/tmp/hive-testuser/248737551<dir> 2010-04-11 16:36
rwxr-xr-x user supergroup
/user/hive/tmp/hive-testuser/334926967<dir> 2010-04-11 16:36
rwxr-xr-x user supergroup
/user/hive/tmp/hive-testuser/376386234<dir> 2010-04-11 16:36
rwxr-xr-x user supergroup
/user/hive/tmp/hive-testuser/773408034<dir> 2010-04-11 16:36
rwxr-xr-x user supergroup
/user/hive/tmp/hive-testuser/855926514<dir> 2010-04-11 16:36
rwxr-xr-x user supergroup
/user/hive/tmp/hive-testuser/plan.-1493385873<r 3> 52634 2010-04-11
16:36 rw-r--r-- testuser supergroup
/user/hive/tmp/hive-testuser/plan.-1975696128<r 3> 35389 2010-04-11
16:36 rw-r--r-- testuser supergroup
/user/hive/tmp/hive-testuser/plan.-508829937<r 3> 21375 2010-04-11
16:36 rw-r--r-- testuser supergroup
/user/hive/tmp/hive-testuser/plan.1732465416<r 3> 32861 2010-04-11
16:36 rw-r--r-- testuser supergroup
Thanks
Ashwin
On 5/11/10 1:34 PM, Namit Jain wrote:
The scratch dir has a format yyyy-mm-dd etc.
You can write a script to delete old data (say older than 2 days)
-----Original Message-----
From: Ashwin Agate [mailto:[email protected]]
Sent: Tuesday, May 11, 2010 1:31 PM
To: [email protected]
Subject: Re: Is hive.exec.scratchdir cleaned up ?
Hi Ning,
Thanks! Did verify that files do get deleted from the scratch directory
- I am probably seeing remnants from jobs that didnt finish normally
over the period of several weeks
Anyone knows of a smart technique of deleting those? (based on timestamp)
-A
On 5/11/10 1:01 PM, Ning Zhang wrote:
It should be cleaned up if the job finished normally or be killed by the
command printed out by CLI or through JT. If the CLI is killed by ^C or the
JT is killed etc the scratch dir won't be able to clean up.
On May 11, 2010, at 12:54 PM, Ashwin Agate wrote:
Hi
I see that hive.exec.scratchdir doesnt get cleaned up in hdfs.
Using hive checked out from svn built around feb 15.
Using Hive CLI to run a series of hive jobs and my test cluster doesnt
have much disk space to spare
Am I missing anything or has this been fixed in later versions of hive ?
Thanks,
A
