[
https://issues.apache.org/jira/browse/HDFS-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Mace updated HDFS-5445:
--------------------------------
Description:
Summary: PacketReceiver reconstructs PacketHeaders with a packetLen 4 bytes
fewer than it should be. It doesn't cause any exceptions because the
reconstructed header is never reserialized, and the packetLen field is not used
in this part of the code.
In the BlockSender class, when a Packet is constructed it must be passed the
field packetLen, which is defined as the data length, checksum data length,
PLUS the length of the packetLen field itself (4 byte integer).
{code:title=BlockSender.java|borderStyle=solid}
484: private int sendPacket(ByteBuffer pkt, int maxChunks, OutputStream out,
485: boolean transferTo, DataTransferThrottler throttler) throws
IOException {
...
491: int packetLen = dataLen + checksumDataLen + 4;
...
504: int headerLen = writePacketHeader(pkt, dataLen, packetLen);
...
586: }
...
792: private int writePacketHeader(ByteBuffer pkt, int dataLen, int packetLen)
{
793: pkt.clear();
794: // both syncBlock and syncPacket are false
795: PacketHeader header = new PacketHeader(packetLen, offset, seqno,
796: (dataLen == 0), dataLen, false);
...
802: }
{code}
In the PacketReceiver class, the PacketHeader is reconstructed using the method
setFieldsFromData. However, the 4 bytes for the packetLen field length are
missing.
{code:title=PacketReceiver.java|borderStyle=solid}
112: private void doRead(ReadableByteChannel ch, InputStream in)
113: throws IOException {
...
136: int payloadLen = curPacketBuf.getInt();
...
144: int dataPlusChecksumLen = payloadLen - Ints.BYTES;
...
181: curHeader.setFieldsFromData(dataPlusChecksumLen, headerBuf);
...
192: }
{code}
The solution would be instead to do:
{code:title=PacketReceiver.java|borderStyle=solid}
181: curHeader.setFieldsFromData(payloadLen, headerBuf);
{code}
I found this because I was making small modifications to the code that exposed
this inconsistency.
was:
Summary: PacketReceiver reconstructs PacketHeaders with a packetLen 4 bytes
fewer than it should be. It doesn't cause any exceptions because the
reconstructed header is never reserialized, and the packetLen field is not used
in this part of the code.
In the BlockSender class, when a Packet is constructed it must be passed the
field packetLen, which is defined as the data length, checksum data length,
PLUS the length of the packetLen field itself (4 byte integer).
{code:title=BlockSender.java|borderStyle=solid}
484: private int sendPacket(ByteBuffer pkt, int maxChunks, OutputStream out,
485: boolean transferTo, DataTransferThrottler throttler) throws
IOException {
...
491: int packetLen = dataLen + checksumDataLen + 4;
...
504: int headerLen = writePacketHeader(pkt, dataLen, packetLen);
...
586: }
...
792: private int writePacketHeader(ByteBuffer pkt, int dataLen, int packetLen)
{
793: pkt.clear();
794: // both syncBlock and syncPacket are false
795: PacketHeader header = new PacketHeader(packetLen, offset, seqno,
796: (dataLen == 0), dataLen, false);
...
802: }
{code}
In the PacketReceiver class, the PacketHeader is reconstructed using the method
setFieldsFromData. However, the 4 bytes for the packetLen field length are
missing.
{code:title=PacketReceiver.java|borderStyle=solid}
112: private void doRead(ReadableByteChannel ch, InputStream in)
113: throws IOException {
...
136: int payloadLen = curPacketBuf.getInt();
...
144: int dataPlusChecksumLen = payloadLen - Ints.BYTES;
...
181: curHeader.setFieldsFromData(dataPlusChecksumLen, headerBuf);
...
192: }
{code}
I found this because I was making small modifications to the code that exposed
this inconsistency.
> PacketReceiver populates the packetLen field in PacketHeader incorrectly
> ------------------------------------------------------------------------
>
> Key: HDFS-5445
> URL: https://issues.apache.org/jira/browse/HDFS-5445
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.1.0-beta, 2.2.0
> Environment: Ubuntu 12.10, Hadoop 2.1.0-beta
> Reporter: Jonathan Mace
> Priority: Minor
> Labels: easyfix
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Summary: PacketReceiver reconstructs PacketHeaders with a packetLen 4 bytes
> fewer than it should be. It doesn't cause any exceptions because the
> reconstructed header is never reserialized, and the packetLen field is not
> used in this part of the code.
> In the BlockSender class, when a Packet is constructed it must be passed the
> field packetLen, which is defined as the data length, checksum data length,
> PLUS the length of the packetLen field itself (4 byte integer).
> {code:title=BlockSender.java|borderStyle=solid}
> 484: private int sendPacket(ByteBuffer pkt, int maxChunks, OutputStream out,
> 485: boolean transferTo, DataTransferThrottler throttler) throws
> IOException {
> ...
> 491: int packetLen = dataLen + checksumDataLen + 4;
> ...
> 504: int headerLen = writePacketHeader(pkt, dataLen, packetLen);
> ...
> 586: }
> ...
> 792: private int writePacketHeader(ByteBuffer pkt, int dataLen, int
> packetLen) {
> 793: pkt.clear();
> 794: // both syncBlock and syncPacket are false
> 795: PacketHeader header = new PacketHeader(packetLen, offset, seqno,
> 796: (dataLen == 0), dataLen, false);
> ...
> 802: }
> {code}
> In the PacketReceiver class, the PacketHeader is reconstructed using the
> method setFieldsFromData. However, the 4 bytes for the packetLen field
> length are missing.
> {code:title=PacketReceiver.java|borderStyle=solid}
> 112: private void doRead(ReadableByteChannel ch, InputStream in)
> 113: throws IOException {
> ...
> 136: int payloadLen = curPacketBuf.getInt();
> ...
> 144: int dataPlusChecksumLen = payloadLen - Ints.BYTES;
> ...
> 181: curHeader.setFieldsFromData(dataPlusChecksumLen, headerBuf);
> ...
> 192: }
> {code}
> The solution would be instead to do:
> {code:title=PacketReceiver.java|borderStyle=solid}
> 181: curHeader.setFieldsFromData(payloadLen, headerBuf);
> {code}
> I found this because I was making small modifications to the code that
> exposed this inconsistency.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
