[
https://issues.apache.org/jira/browse/HADOOP-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128044#comment-16128044
]
Esfandiar Manii commented on HADOOP-14583:
------------------------------------------
This is not related to if parent directory does not exist. The issue is
concurrency. If there are multiple threads trying to create and delete the same
file over and over, you will be having a scenario where:
ThreadA -> Creates File
ThreadB -> Looks up the file and it exists
ThreadA -> Removes file
ThreadB -> Looks up for metadata before creation and throws exception
The logic to make the metadata lookup safe was missed from the create function
where most of the other ones have this logic. I updated the code and added a
test to ensure this wont happen under over 100 threads.
Please take a look over the patch and let me know if you have comments.
> wasb throws an exception if you try to create a file and there's no parent
> directory
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-14583
> URL: https://issues.apache.org/jira/browse/HADOOP-14583
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-14583-001.patch
>
>
> It's a known defect of the Hadoop FS API (and one we don't explicitly test
> for enough), but you can create a file on a path which doesn't exist. In that
> situation, the create() logic is expectd to create the entries.
> Wasb appears to raise an exception if you try to call {{create(filepath)}}
> without calling {{mkdirs(filepath.getParent()}} first. That's the semantics
> expected of {{createNonRecursive()}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
