[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233363#comment-15233363
]
stack commented on HDFS-3702:
-----------------------------
bq. I suggest adding a new create(..) method to DistributedFileSystem, either
with a new boolean or with the AddBlockFlag, in this JIRA so that the community
can try out the feature. We may add the CreateFlag.NO_LOCAL_WRITE once the
feature has been stabilized and we has decided that it is the right API.
Tell me more how this will process would work please [~szetszwo]? IIUC, a
downstream project, say HBase which already has an awful hack in place to try
and simulate a poor-man's version of this feature, would via reflection, look
first for the presence of this new create override IFF the implementation is
HDFS (don't look if LocalFS or S3, etc.)? If HDFS and if present, we'd drop our
hack and use the new method (via reflection). Later, after it is 'proven' that
a feature, one that hbase has wanted for years now, has 'merit', we would then
add a new path w/ more reflection (IFF the FS implementation is HDFS) that
would use the NO_LOCAL_WRITE when it becomes available? (Would we remove the
create override when the NO_LOCAL_WRITE FS hint gets added?) Are you suggesting
that downstream projects do this?
Regards favorednodes, thats an unfinished topic and of a different character to
what is being suggested here as it added overrides rather than a 'hint' flag as
this patch does here.
Thanks [~szetszwo]
> Add an option for NOT writing the blocks locally if there is a datanode on
> the same box as the client
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-3702
> URL: https://issues.apache.org/jira/browse/HDFS-3702
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.5.1
> Reporter: Nicolas Liochon
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HDFS-3702.000.patch, HDFS-3702.001.patch,
> HDFS-3702.002.patch, HDFS-3702.003.patch, HDFS-3702.004.patch,
> HDFS-3702.005.patch, HDFS-3702.006.patch, HDFS-3702.007.patch,
> HDFS-3702.008.patch, HDFS-3702.009.patch, HDFS-3702.010.patch,
> HDFS-3702.011.patch, HDFS-3702_Design.pdf
>
>
> This is useful for Write-Ahead-Logs: these files are writen for recovery
> only, and are not read when there are no failures.
> Taking HBase as an example, these files will be read only if the process that
> wrote them (the 'HBase regionserver') dies. This will likely come from a
> hardware failure, hence the corresponding datanode will be dead as well. So
> we're writing 3 replicas, but in reality only 2 of them are really useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
