[
https://issues.apache.org/jira/browse/HDFS-14352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793315#comment-16793315
]
Steven Rand commented on HDFS-14352:
------------------------------------
The above is actually incorrect. The 2.x NameNode doesn't have to send back a
{{LocatedBlocks}} with EC info; for client-side support we just need to add the
new constructor to {{LocatedBlocks}} that takes an {{ecPolicy}} variable, and
this lets us correctly handle responses from the 3.x NameNode. All responses
from the 2.x NameNode will (correctly) be interpreted as having a null
{{ecPolicy}}.
I'm still working on a patch for this locally. It currently fails because I
haven't correctly modified {{PBHelperClient}} to create a
{{LocatedStripedBlock}} when appropriate, but that should be fixable.
> backport client-side EC support to branch-2
> -------------------------------------------
>
> Key: HDFS-14352
> URL: https://issues.apache.org/jira/browse/HDFS-14352
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, hdfs-client
> Reporter: Steven Rand
> Priority: Major
>
> Currently, Hadoop 2.x clients can't read or write striped files from HDFS.
> This affects compatibility with 3.x clusters in two ways:
> * The obvious impact is that 2.x clients can't make use of the new erasure
> coding in feature in Hadoop 3.
> * For some use cases, clients built against Hadoop 3 won't be able to use
> erasure coding either. This is because if they write a striped file, then
> clients built against Hadoop 2 won't be able to read it.
> This ticket proposes backporting the client-side components of HDFS-7285 to
> branch-2 for improved compatibility between 2.x clients and 3.x clusters. I
> believe this can be done without also backporting the changes made to the
> NameNodes and the DataNodes. While many lines of code would need to be
> backported, most of it is new code that can be copy/pasted from trunk, which
> simplifies the process. The existing code in {{DFSClient}},
> {{DFSInputStream}}, {{DFSOutputStream}}, etc. that would need to be modified
> is still significant, but much smaller.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
