[
https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606335#comment-13606335
]
Matthew Farrellee commented on HADOOP-9371:
-------------------------------------------
[[email protected]] Does delete(path, true) need to be atomic?
My research suggests that only the HDFS implementation is atomic.
(Note: current = r2.0.3-alpha on 2013-02-15 19:41)
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html
FilterFileSystem - delegates w/o locking
ChecksumFileSystem - delegates w/o locking
LocalFileSystem - inherits, delegates to RawLocalFileSystem
HarFileSystem - not implemented
FTPFileSystem - FTPClient.removeDirectory w/o locking
KosmosFileSystem - (not on trunk) no locking
NativeS3FileSystem - no locking (even createParent()s to avoid errors, weird)
RawLocalFileSystem - uses File.delete (if isFile) and FileUtil.fullyDelete
w/o locking
S3FileSystem - no locking
ViewFileSystem - partial eval, no locking on top level
- ChRootFileSystem uses RawLocalFileSystem
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/AbstractFileSystem.html
AbstractFileSystem (uses FileContext.delete)
FilterFs - delegates w/o locking
ChecksumFs - delegates w/o locking
LocalFs - inherits, delegates to RawLocalFs
DelegateToFileSystem - delegates w/o locking
RawLocalFs - inherits, delegates to RawLocalFileSystem
FtpFs - inherits, delegates to FTPFileSystem
ViewFs - partial eval, no locking at top level
FileContext.delete - no hint of atomic requirement, delegates to
AbstractFileSystem
Side note - it's interesting to see how many FS implementations make their way
back to RawLocalFileSystem, sometimes through 3+ layers of indirection.
> Define Semantics of FileSystem and FileContext more rigorously
> --------------------------------------------------------------
>
> Key: HADOOP-9371
> URL: https://issues.apache.org/jira/browse/HADOOP-9371
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
> Affects Versions: 1.2.0, 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch,
> HadoopFilesystemContract.pdf
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The semantics of {{FileSystem}} and {{FileContext}} are not completely
> defined in terms of
> # core expectations of a filesystem
> # consistency requirements.
> # concurrency requirements.
> # minimum scale limits
> Furthermore, methods are not defined strictly enough in terms of their
> outcomes and failure modes.
> The requirements and method semantics should be defined more strictly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
