[
https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-9565:
-----------------------------------
Attachment: HADOOP-9565-001.patch
This is the -001 patch, which defines a new class, {{abstract class
ObjectStoreFileSystem extends FileSystem }}, which is a marker interface. It
allows the FS Semantics to be queried, and requirements placed on an FS & raise
an IOE if they aren't met. semantics include: consistency (create, delete,
list, update), atomicity (create, delete, rename...) and whether flush() does
anything.
This patch is just the class and tests, without any update to the FS
specification doc, and without reparenting the in-hadoop-codebase object
stores, which for swift:// and //s3? come with nearly no guarantees.
# Swift offers create consistency
# s3 create consistency varies a bit by region though.
# Azure does offer a consistent mode, so you can swap hdfs out for it.
# Netflix S3mper and AWS equivalent does improve s3 consistency, especially
metadata consistency
I don't expect apps to adjust their behaviour based on the guarantees, but
think some code could/should reject operations if the dest isn't suitable for
their actions (e.g. speculative operations committed by rename() require
atomic rename operations)
> Add a Blobstore interface to add to blobstore FileSystems
> ---------------------------------------------------------
>
> Key: HADOOP-9565
> URL: https://issues.apache.org/jira/browse/HADOOP-9565
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.0.4-alpha
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-9565-001.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really
> blobstores, with different atomicity and consistency guarantees, by adding a
> {{Blobstore}} interface to add to them.
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that
> all blobstores implement at server-side copy operation as a substitute for
> rename.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
