[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237558#comment-15237558
]
stack commented on HDFS-3702:
-----------------------------
bq. I suggest HBase should do the same way of how it today is using
favoredNodes.
Thanks [~szetszwo] for the response, but you did not answer the question. The
question was if you thought the process of first staging a 'hidden' API is a
fair burden to put on your favorite downstream project (not to mention the mess
it makes inside HDFS -- see note above this one for graphic detail).
Lets back up. I think it will help us make some progress here again.
You say:
bq. So let add this flag later so that it allows us to test the feature and see
if it is good enough or we may actually need disfavoredNodes. Sound good?
No. It does not sound good. There is no need to stage a feature as hidden
first, one that is reasonable (see above discussion with the opinion of many),
and has an immediate need/user. If any concern that the feature is lacking or
does not work as advertised, lets do whatever proofing of the feature is needed
here as part of this issue and just get it done. If the bundled tests are
unsatisfactory or if you'd like me to try and report result of running this
facility at scale, just say... no problem. If the implementation has a bug,
lets fix in a follow-up. As we would do any other feature in HDFS.
On your concern that a new 'hint' to the create method exposes new API, an API
that by definition does not put a burden on any FS implementation that they
need implement the suggested operation -- i.e. the amount of API 'surface' is
miniscule -- it has been suggested above that we flag it
@InterfaceAudience.LimitedPrivate(HBase) for a probationary period. How about
we also add @InterfaceStability.Evolving on the flag so it can be yanked
anytime if for some unforeseen reason, it a total mistake. Would this assuage
your exposure concern [~szetszwo]? Thanks for your time.
> Add an option for NOT writing the blocks locally if there is a datanode on
> the same box as the client
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-3702
> URL: https://issues.apache.org/jira/browse/HDFS-3702
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.5.1
> Reporter: Nicolas Liochon
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HDFS-3702.000.patch, HDFS-3702.001.patch,
> HDFS-3702.002.patch, HDFS-3702.003.patch, HDFS-3702.004.patch,
> HDFS-3702.005.patch, HDFS-3702.006.patch, HDFS-3702.007.patch,
> HDFS-3702.008.patch, HDFS-3702.009.patch, HDFS-3702.010.patch,
> HDFS-3702.011.patch, HDFS-3702_Design.pdf
>
>
> This is useful for Write-Ahead-Logs: these files are writen for recovery
> only, and are not read when there are no failures.
> Taking HBase as an example, these files will be read only if the process that
> wrote them (the 'HBase regionserver') dies. This will likely come from a
> hardware failure, hence the corresponding datanode will be dead as well. So
> we're writing 3 replicas, but in reality only 2 of them are really useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
