[
https://issues.apache.org/jira/browse/HDFS-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218995#comment-14218995
]
Yongjun Zhang commented on HDFS-7403:
-------------------------------------
I noticed the following code in BlockManager.java, which means even if a block
doesn't have enough replicas, it can still be marked as complete:
/**
* Force the given block in the given file to be marked as complete,
* regardless of whether enough replicas are present. This is necessary
* when tailing edit logs as a Standby.
*/
public BlockInfo forceCompleteBlock(final BlockCollection bc,
final BlockInfoUnderConstruction block) throws IOException {
block.commitBlock(block);
return completeBlock(bc, block, true);
}
that calls
{code}
/**
* Convert a specified block of the file to a complete block.
* @param bc file
* @param blkIndex block index in the file
* @throws IOException if the block does not have at least a minimal number
* of replicas reported from data-nodes.
*/
private BlockInfo completeBlock(final BlockCollection bc,
final int blkIndex, boolean force) throws IOException {
if(blkIndex < 0)
return null;
BlockInfo curBlock = bc.getBlocks()[blkIndex];
if(curBlock.isComplete())
return curBlock;
BlockInfoUnderConstruction ucBlock = (BlockInfoUnderConstruction)curBlock;
int numNodes = ucBlock.numNodes();
if (!force && numNodes < minReplication)
throw new IOException("Cannot complete block: " +
"block does not satisfy minimal replication requirement.");
if(!force && ucBlock.getBlockUCState() != BlockUCState.COMMITTED)
throw new IOException(
"Cannot complete block: block has not been COMMITTED by the client");
BlockInfo completeBlock = ucBlock.convertToCompleteBlock();
// replace penultimate block in file
bc.setBlock(blkIndex, completeBlock);
// Since safe-mode only counts complete blocks, and we now have
// one more complete block, we need to adjust the total up, and
// also count it as safe, if we have at least the minimum replica
// count. (We may not have the minimum replica count yet if this is
// a "forced" completion when a file is getting closed by an
// OP_CLOSE edit on the standby).
namesystem.adjustSafeModeBlockTotals(0, 1);
namesystem.incrementSafeBlockCount(
Math.min(numNodes, minReplication));
// replace block in the blocksMap
return blocksMap.replaceBlock(completeBlock);
}
{code}
Well, the javadoc of COMPLETE says it means "The block has at least one", thus
it is still confusing. I was thinking about changing the COMPLETE javadoc to be
something like
{code}
In common case, COMPLETE means that the block need to have >= minimal
replication replicas" (link to minimalReplica def). However, a block may be
forced to COMPLETE state even if it doesn't have any replica. (link to the
above method)
{code}
> Inaccurate javadoc of BlockUCState#COMPLETE state
> --------------------------------------------------
>
> Key: HDFS-7403
> URL: https://issues.apache.org/jira/browse/HDFS-7403
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Priority: Trivial
> Attachments: HDFS-7403.001.patch
>
>
> The current javadoc says
> {code}
> /**
> * States, which a block can go through while it is under construction.
> */
> static public enum BlockUCState {
> /**
> * Block construction completed.<br>
> * The block has at least one {@link ReplicaState#FINALIZED} replica,
> * and is not going to be modified.
> */
> COMPLETE,
> {code}
> However, COMPLETE blocks mean those that have reached minimal replication
> "dfs.namenode.replication.min", which could be different than one.
>
> Creating this jira to fix the javadoc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
