[jira] [Commented] (HDFS-7342) Lease Recovery doesn't happen some times

[Yongjun Zhang (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HDFS-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226765#comment-14226765
 ] 

Yongjun Zhang commented on HDFS-7342:
-------------------------------------

Hi [~vinayrpet],

Thanks for further explanation. I studied your latest two comments, and I agree 
with your analysis: the case that penultimate block being in COMMITTEED sate 
with minimal replication met may not exist. In other words, when penultimate 
block is in COMMITTED state, the minimal replication is not met.

Going back to the source that triggerd this discussion, it's the scenario#1 
described in:
 
https://issues.apache.org/jira/browse/HDFS-4882?focusedCommentId=14213992&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14213992

Here is what I think now:

scenario#1 exists for last block but not penultimate block
scenarion#2 exists for both penultimate and last block

My suggested solution for this jia is to share the same code that handles the 
case when last block is COMMITTEED, with some modifications:

* For scenarion#1 (last block), my suggested solution is to forceComplete the 
block before calling  {{finalizeINodeFileUnderConstruction}}:
{code}
        if (lastBlockState == BlockUCState.COMMITTED) {
          getBlockManager().forceCompleteBlock(pendingFile,
              (BlockInfoUnderConstruction) lastBlock);
        }
        finalizeINodeFileUnderConstruction(src, pendingFile,
            iip.getLatestSnapshotId());
{code}
thus avoiding the exception thrown from {{finalizeINodeFileUnderConstruction}} 
caused by
{code}
Preconditions.checkState(blocks[i].isComplete(), "Failed to finalize"
          + " %s %s since blocks[%s] is non-complete, where blocks=%s.",
          getClass().getSimpleName(), this, i, Arrays.asList(blocks));
{code}

* For scenarion#2, the same code can be shared to handle the case that when 
penultimate block is COMMITTED with minimal replication NOT met. the lease will 
be recovered, even if it's this kind of state.

On top of this suggested solution, we can add what Kihwal suggested.

What do you think?

Thanks.


> Lease Recovery doesn't happen some times
> ----------------------------------------
>
>                 Key: HDFS-7342
>                 URL: https://issues.apache.org/jira/browse/HDFS-7342
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
>         Attachments: HDFS-7342.1.patch, HDFS-7342.2.patch, HDFS-7342.3.patch
>
>
> In some cases, LeaseManager tries to recover a lease, but is not able to. 
> HDFS-4882 describes a possibility of that. We should fix this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)