Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18975
Even renaming staging location is not atomic, right? Hive users might still
see the extra data during the rename, right?
I am not sure how Hive works. If our underlying system is Linux, the
[rename of Linux doc](https://linux.die.net/man/3/rename) shows
> If one or more processes have the file open when the last link is
removed, the link shall be removed before rename() returns, but the removal of
the file contents shall be postponed until all references to the file are
closed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
