beliefer commented on issue #23950: [SPARK-27140][SQL]The feature is 'insert overwrite local directory' has an inconsistent behavior in different environment. URL: https://github.com/apache/spark/pull/23950#issuecomment-474656655 > Is there anything to review here? looks like you're trying to get it to fail, and there was a failure two weeks ago. We're having Jenkins trouble right now but you can retrigger it when it comes up. Thank a lot! I can always see your reply, as I expected. The whole process is below： First, the path `/tmp/noexistdir` is not exists and the following test cases occur in local mode. Second, when Maropu was running `insert overwrite local directory '/tmp/noexistdir/t' select * from t` in his environment, the not exists path would be created and result files and file `_SUCCESS` would be moved to the path. Third, when I was running `insert overwrite local directory '/tmp/noexistdir/t' select * from t` in my environment, the not exists path would be created as a single file and the single file is one of the result files. Second, I created this PR to execute the same test case in UT. When UT was running `insert overwrite local directory '/tmp/noexistdir/t' select * from t` in `jenkins` environment, the behavior is the same as the first test case. Why the feature 'insert overwrite local directory' has an inconsistent behavior in different environment? According to the source code of InsertIntoHiveDirCommand, there no chance to create the target path that doesn't exist yet. Could you help me, find out the reason. Thanks!
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
