[
https://issues.apache.org/jira/browse/HADOOP-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529679
]
Raghu Angadi commented on HADOOP-1908:
--------------------------------------
I checked {{writeBlock()}} to see if the first build failures could be
explained. I noticed the the following (not related to the build failure) :
# BlockReceiver : It invokes {{data.finalizeBlock()}} before closing the block
files. This will fail on Windows
# BlockReciever.receiveChunk() : {code}
//
// If stream-copy fails, continue
// writing to disk. We shouldn't
// interrupt client write.
//
mirrorOut = null;
{code} The above does not work the the way comments intends because mirrorOut
is just an arg and has no effect on mirrorOut up its stack (i.e. in
BlockReceiver.receiveBlock() and writeBlock().) This worked ebefore because all
of these were in single function.
# Minor : DFSClient.endBlock() : Couple of checks in '{{while (bytesLeft > 0)
{..}}' are not necessary because they were required when the loop was 'while
(bytesLeft >= 0){.. }}'.
# In BlockReceiver.receiveBlock() : the check '{{if (lastLen > 0 && lastLen !=
bytesPerChecksum) {..}}' was meant to be done for every chunk received. Now
this is checked only for the last-but-one chunk.
> Restructure data node code so that block sending/receiving is seperated from
> data transfer header handling
> ----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1908
> URL: https://issues.apache.org/jira/browse/HADOOP-1908
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: datanode.patch, datanodeRestructure.patch,
> datanodeRestructure.patch, datanodeRestructure1.patch
>
>
> This jira is intended for code sharing. I'd like to have a BlockSender which
> is resposible for reading a block from a disk and writing it to an output
> stream, and a BlockReceiver which receives a block from an input stream,
> writes it to a disk, and possibly writes it a mirror ouput stream to support
> pipeline writes. Block sender and receiver code is independent of any block
> IO protocols so it could be shared by block transfer, block write, block
> read, and future block replacement introduced by data node rebalancing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
