Hi Ashwin, Which version of Hive are you using? The new directory structure (yyyy-MM-dd) is introduced in https://issues.apache.org/jira/browse/HIVE-1109
Zheng On Tue, May 11, 2010 at 1:47 PM, Ashwin Agate <[email protected]> wrote: > I dont see yyyy-mm-dd directory names but rather series of numbers > > /user/hive/tmp/hive-testuser/248737551 <dir> 2010-04-11 16:36 > rwxr-xr-x user supergroup > /user/hive/tmp/hive-testuser/334926967 <dir> 2010-04-11 16:36 > rwxr-xr-x user supergroup > /user/hive/tmp/hive-testuser/376386234 <dir> 2010-04-11 16:36 > rwxr-xr-x user supergroup > /user/hive/tmp/hive-testuser/773408034 <dir> 2010-04-11 16:36 > rwxr-xr-x user supergroup > /user/hive/tmp/hive-testuser/855926514 <dir> 2010-04-11 16:36 > rwxr-xr-x user supergroup > /user/hive/tmp/hive-testuser/plan.-1493385873 <r 3> 52634 2010-04-11 > 16:36 rw-r--r-- testuser supergroup > /user/hive/tmp/hive-testuser/plan.-1975696128 <r 3> 35389 2010-04-11 > 16:36 rw-r--r-- testuser supergroup > /user/hive/tmp/hive-testuser/plan.-508829937 <r 3> 21375 2010-04-11 > 16:36 rw-r--r-- testuser supergroup > /user/hive/tmp/hive-testuser/plan.1732465416 <r 3> 32861 2010-04-11 > 16:36 rw-r--r-- testuser supergroup > > Thanks > Ashwin > > > On 5/11/10 1:34 PM, Namit Jain wrote: >> >> The scratch dir has a format yyyy-mm-dd etc. >> >> You can write a script to delete old data (say older than 2 days) >> >> >> -----Original Message----- >> From: Ashwin Agate [mailto:[email protected]] >> Sent: Tuesday, May 11, 2010 1:31 PM >> To: [email protected] >> Subject: Re: Is hive.exec.scratchdir cleaned up ? >> >> Hi Ning, >> >> Thanks! Did verify that files do get deleted from the scratch directory >> - I am probably seeing remnants from jobs that didnt finish normally >> over the period of several weeks >> >> Anyone knows of a smart technique of deleting those? (based on timestamp) >> >> -A >> >> On 5/11/10 1:01 PM, Ning Zhang wrote: >> >>> >>> It should be cleaned up if the job finished normally or be killed by the >>> command printed out by CLI or through JT. If the CLI is killed by ^C or the >>> JT is killed etc the scratch dir won't be able to clean up. >>> >>> On May 11, 2010, at 12:54 PM, Ashwin Agate wrote: >>> >>> >>> >>>> >>>> Hi >>>> >>>> I see that hive.exec.scratchdir doesnt get cleaned up in hdfs. >>>> >>>> Using hive checked out from svn built around feb 15. >>>> >>>> Using Hive CLI to run a series of hive jobs and my test cluster doesnt >>>> have much disk space to spare >>>> >>>> Am I missing anything or has this been fixed in later versions of hive ? >>>> >>>> Thanks, >>>> A >>>> >>>> >>> >>> >> >> > > -- Yours, Zheng http://www.linkedin.com/in/zshao
