Friends, Hive seems to have introduced this notion of a hive.exec.stagingdir (for encryption) in one of the more recent versions and this defaults to .hive-staging under the table dir on HDFS. This results in the table dir (and the staging sub dir) being created much sooner during the query execution than in previous Hive versions. This has an unintended and problematic side-effect for Oozie in that the coordinator apps depending on this data-set could potentially kick-off much sooner than before with empty dir and data-sets. Has this caused issues for other users? Any thoughts or solutions in the works?
And Hive doesn't create a SUCCESS file or honor the "mapreduce.fileoutputcommitter.marksuccessfuljobs" setting as far as I can tell. So I am afraid a lot of coordinator apps are susceptible to this issue created by this new staging dir. Thanks,Aravind
