[
https://issues.apache.org/jira/browse/HDFS-142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12867146#action_12867146
]
sam rash commented on HDFS-142:
-------------------------------
this reminds me--in the test I posted on hdfs-1057, I had two tests that fail
doing concurrent reads and appends. The tests actually fail w/o the concurrent
read patch. I haven't had time to look into it. You can run the tests in
TestFileConcurrentReader. I can file a sep jira for this as well, but it seems
append-related.
// fails due to issue w/append, disable
public void _testUnfinishedBlockCRCErrorTransferToAppend() throws IOException
{
runTestUnfinishedBlockCRCError(true, SyncType.APPEND, DEFAULT_WRITE_SIZE);
}
// fails due to issue w/append, disable
public void _testUnfinishedBlockCRCErrorNormalTransferAppend()
throws IOException {
runTestUnfinishedBlockCRCError(false, SyncType.APPEND, DEFAULT_WRITE_SIZE);
}
> In 0.20, move blocks being written into a blocksBeingWritten directory
> ----------------------------------------------------------------------
>
> Key: HDFS-142
> URL: https://issues.apache.org/jira/browse/HDFS-142
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Raghu Angadi
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: appendQuestions.txt, deleteTmp.patch, deleteTmp2.patch,
> deleteTmp5_20.txt, deleteTmp5_20.txt, deleteTmp_0.18.patch, handleTmp1.patch,
> hdfs-142-commitBlockSynchronization-unknown-datanode.txt,
> HDFS-142-deaddn-fix.patch, HDFS-142-finalize-fix.txt,
> hdfs-142-minidfs-fix-from-409.txt,
> HDFS-142-multiple-blocks-datanode-exception.patch,
> hdfs-142-recovery-reassignment-and-bbw-cleanup.txt, hdfs-142-testcases.txt,
> hdfs-142-testleaserecovery-fix.txt, HDFS-142_20.patch,
> testfileappend4-deaddn.txt, validateBlockMetaData-synchronized.txt
>
>
> Before 0.18, when Datanode restarts, it deletes files under data-dir/tmp
> directory since these files are not valid anymore. But in 0.18 it moves these
> files to normal directory incorrectly making them valid blocks. One of the
> following would work :
> - remove the tmp files during upgrade, or
> - if the files under /tmp are in pre-18 format (i.e. no generation), delete
> them.
> Currently effect of this bug is that, these files end up failing block
> verification and eventually get deleted. But cause incorrect over-replication
> at the namenode before that.
> Also it looks like our policy regd treating files under tmp needs to be
> defined better. Right now there are probably one or two more bugs with it.
> Dhruba, please file them if you rememeber.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
