[
https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082848#comment-15082848
]
Dinesh S. Atreya commented on HDFS-9607:
----------------------------------------
Regarding below, we want to be clear from the outset that proposals are
different from standard POSIX hence use of different names such as
"writeInPlace". Later on it is up to the HDFS team to indicate the API naming
preferences.
{quote}
We have something called the Posix specification to tell us what to call
things. The write() operation is the one to consider here, simply having it
fail if an update condition cannot be met; possibly add a non-side-effecting
canWrite call which checks the conditions.
{quote}
I believe currently HDFS is not POSIX compliant. However, Use of
"canWriteInPlace" is a good suggestion instead of "checkWriteInPlace" and I
will use it.
I had also thought along the following lines:
{quote}
I know that the core HDFS write pipeline is one of the most critical parts of
HDFS, so leave it alone.
{quote}
The are a number of compelling reasons for *write-in-place* enhancements, the
detailed list of business capabilities in [parent/umbrella JIRA HADOOP-12620 |
https://issues.apache.org/jira/browse/HADOOP-12620] is one.
Also for those interested, you can check out
https://www.mapr.com/resources/videos/whiteboard-walkthrough-append-only-vs-read-write-file-system
My thought process has been, there should be an easier way to provide
equivalent capabilities while furthering HDFS without attempting to provide
full-fledged POSIX compliance. This JIRA and theme is an attempt towards that
end.
It is obvious that the whole process has to be very careful and deliberate. I
will add the implementation design in due course, covering how to tackle
reliable in-place updates, in HDFS, including in encrypted, and coping with
failures of 1+ node during that update process. (I have not been following
erasure-coding developments so far.)
> Advance Hadoop Architecture (AHA) - HDFS
> ----------------------------------------
>
> Key: HDFS-9607
> URL: https://issues.apache.org/jira/browse/HDFS-9607
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Dinesh S. Atreya
>
> Link to Umbrella JIRA
> https://issues.apache.org/jira/browse/HADOOP-12620
> Provide capability to carry out in-place writes/updates. Only writes in-place
> are supported where the existing length does not change.
> For example, "Hello World" can be replaced by "Hello HDFS!"
> See
> https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300
> for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
