[
https://issues.apache.org/jira/browse/HADOOP-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275422#comment-16275422
]
Cheng Lian commented on HADOOP-15086:
-------------------------------------
To be more specific, when multiple threads rename files to the same target
path, more than 1 *but not all* threads can succeed. It's because check and
copy file in {{NativeAzureFileSystem#rename()}} is not atomic.
The problem here is that it's unclear what the expected semantics of
{{NativeAzureFileSystem#rename()}} is:
- If the semantics is "error if the destination file already exists", then only
1 thread can succeed.
- If the semantics is "overwrite if the destination file already exists", then
all threads should succeed.
> NativeAzureFileSystem.rename is not atomic
> ------------------------------------------
>
> Key: HADOOP-15086
> URL: https://issues.apache.org/jira/browse/HADOOP-15086
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 2.7.3
> Reporter: Shixiong Zhu
> Attachments: RenameReproducer.java
>
>
> When multiple threads rename files to the same target path, more than 1
> threads can succeed. It's because check and copy file in `rename` is not
> atomic.
> I would expect it's atomic just like HDFS.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
