wanghualei opened a new pull request #2187: URL: https://github.com/apache/hive/pull/2187
after set Run as end user instead of Hive user , when execute insert overwrite , In MoveTask ,if source byte > HIVE_EXEC_COPYFILE_MAXSIZE and source file count> HIVE_EXEC_COPYFILE_MAXNUMFILES , HIve will use distcp method, it may cause tmp stats file lost. example: set hive.exec.copyfile.maxsize=0; set hive.exec.copyfile.maxnumfiles=0; insert overwrite table abc_new select * from abc; select count(1) from abc_new ; select * from abc_new ; then the count(1) result will be 0, but select * will display real data, because stats info lost. fix: distcp method different from FileUtil.copy, because distcp method ignore parent directory, then may overwrite stats files,cause stats info lost.? modify dst value, add parent directory to dst path, make distcp method run result as same as?FileUtil.copy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
