ehiggs commented on a change in pull request #575: HADOOP-13327 Output Stream Specification URL: https://github.com/apache/hadoop/pull/575#discussion_r264133266
########## File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md ########## @@ -598,8 +600,16 @@ The result is `FSDataOutputStream`, which through its operations may generate ne * S3A, Swift and potentially other Object Stores do not currently change the FS state until the output stream `close()` operation is completed. -This MAY be a bug, as it allows >1 client to create a file with `overwrite==false`, - and potentially confuse file/directory logic +This is a significant difference between the behavior of object stores +and that of filesystems, as it allows >1 client to create a file with `overwrite==false`, +and potentially confuse file/directory logic. In particular, using `create()` to acquire +an exclusive lock on a file (whoever creates the file without an error is considered +the holder of the lock) is not a valid algorithm when working with object stores. + +* Object stores may create an empty file as a marker when a file is created. +However, object stores whith overwrite=true semantics may not implement this atomically, Review comment: ```suggestion However, object stores with `overwrite==true` semantics may not implement this atomically, ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
