[
https://issues.apache.org/jira/browse/HADOOP-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15102230#comment-15102230
]
Chris Nauroth commented on HADOOP-12635:
----------------------------------------
It turns out I misinterpreted something in the design document. There is a
statement about supporting "a single writer". I interpreted this to mean that
the goal of the feature was to match HDFS single-writer enforcement semantics
exactly. Instead, the goal is to support append for applications that can
guarantee a single writer themselves. WASB will not provide this enforcement
internally. It becomes a responsibility of the application either to ensure
single-threaded handling for a particular file path, or rely on some external
locking mechanism of its own.
Exactly matching HDFS single-writer semantics might be a nice future goal, but
this would be a huge undertaking. It would nearly be a full rewrite with
reconsideration of locking strategies inside the entire API footprint. It also
will be challenging to achieve this without harming current performance.
Based on this, I think we can proceed with a patch similar to what has been
provided already, pending resolution of my code review feedback. Usage of the
API will carry some risk, so I propose that we guard it behind a configuration
flag, which would reject append calls with an error unless explicitly enabled.
Let's also call out the risks in documentation and make it clear to end users
that this differs from HDFS in that single-writer enforcement is an application
responsibility with WASB.
> Adding Append API support for WASB
> ----------------------------------
>
> Key: HADOOP-12635
> URL: https://issues.apache.org/jira/browse/HADOOP-12635
> Project: Hadoop Common
> Issue Type: Improvement
> Components: azure
> Affects Versions: 2.7.1
> Reporter: Dushyanth
> Assignee: Dushyanth
> Attachments: Append API.docx, HADOOP-12635-004.patch,
> HADOOP-12635.001.patch, HADOOP-12635.002.patch, HADOOP-12635.003.patch
>
>
> Currently the WASB implementation of the HDFS interface does not support
> Append API. This JIRA is added to design and implement the Append API support
> to WASB. The intended support for Append would only support a single writer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
