insert overwrite directory leaves behind uncommitted/tmp files from failed tasks
--------------------------------------------------------------------------------
Key: HIVE-131
URL: https://issues.apache.org/jira/browse/HIVE-131
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Joydeep Sen Sarma
Priority: Critical
_tmp files are getting left behind on insert overwrite directory:
/user/jssarma/ctst1/40422_m_000195_0.deflate <r 3> 13285 2008-12-07 01:47
rw-r--r-- jssarma supergroup
/user/jssarma/ctst1/40422_m_000196_0.deflate <r 3> 3055 2008-12-07 01:46
rw-r--r-- jssarma supergroup
/user/jssarma/ctst1/_tmp.40422_m_000033_0 <r 3> 0 2008-12-07 01:53 rw-r--r--
jssarma supergroup
/user/jssarma/ctst1/_tmp.40422_m_000037_1 <r 3> 0 2008-12-07 01:53 rw-r--r--
jssarma supergroup
this happened with speculative execution. the code looks good (in fact in this
case many speculative tasks were launched - and only a couple caused problems).
Almost seems like these files did not appear in the namespace until after the
map-reduce job finished and the movetask did a listing of the output dir ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
