[
https://issues.apache.org/jira/browse/HDFS-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-3721:
------------------------------
Attachment: hdfs-3721.txt
This patch fixes the issue as follows:
- Refactors out the packet-reading code from BlockReceiver and
RemoteBlockReader2 into a new {{PacketReceiver}} class. This really simplified
BlockReceiver in particular, and has the nice side effect of getting us
significantly closer to HDFS-3529.
- All places where we used to assume a fixed-length packet header now support
variable length. In some cases this is achieved by allocating a
larger-than-necessary buffer and then, once we know the size for the header,
putting it at the right spot to make the header contiguous with the data. In
other cases, this is achieved by simply separating the buffer containing the
header from the buffer containing the data.
- Regarding the issue above with 2.1 clients writing to 2.0 servers, I fixed
the issue by having the PacketHeader class not set any value for the
{{syncBlock}} flag when it is false. That means that, so long as the new hsync
functionality isn't used, new clients can still talk to 2.0 servers. If hsync
is used, an error will occur. It's slightly unfortunate, but given that the 2.0
branch is pretty new, and this is a new feature. I think this is acceptable.
I manually tested a trunk client both reading and writing from a 2.0.0-alpha
cluster with this patch applied.
> hsync support broke wire compatibility
> --------------------------------------
>
> Key: HDFS-3721
> URL: https://issues.apache.org/jira/browse/HDFS-3721
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, hdfs client
> Affects Versions: 2.1.0-alpha
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: hdfs-3721.txt
>
>
> HDFS-744 added support for hsync to the data transfer wire protocol. However,
> it actually broke wire compatibility: if the client has hsync support but the
> server does not, the client cannot read or write data on the old cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
