[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-20 Thread Yiqun Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593466#comment-15593466
 ] 

Yiqun Lin commented on HDFS-11018:
--

Thanks [~jojochuang] for the commit!

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-20 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592572#comment-15592572
 ] 

Hudson commented on HDFS-11018:
---

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10646 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/10646/])
HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. (weichiu: 
rev 6d2da38d16cebe9b82f1048f87127eecee33664c)
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-20 Thread Wei-Chiu Chuang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592523#comment-15592523
 ] 

Wei-Chiu Chuang commented on HDFS-11018:


Committed this to trunk. I'll commit this to branch-2 and branch-2.8 later.

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-19 Thread Yiqun Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590359#comment-15590359
 ] 

Yiqun Lin commented on HDFS-11018:
--

Thanks [~jojochuang] for the review. One point that I want to say, the test 
{{TestDiskspaceQuotaUpdate}} fails in this JIRA. I found the reason is that 
there is some bind exception. I have filed the JIRA HDFS-11025 to make a quick 
fix. We have reach an agreement that back up on different port across 
restarting datanode will not influce the current logic. So I have one proposal 
that you can also make a commit for that after you have looked into for that, 
:). Thanks.

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-19 Thread Wei-Chiu Chuang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590094#comment-15590094
 ] 

Wei-Chiu Chuang commented on HDFS-11018:


LGTM +1 on the 003 patch.

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-18 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587786#comment-15587786
 ] 

Hadoop QA commented on HDFS-11018:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
 0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 59m 
55s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-11018 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12834099/HDFS-11018.003.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux d52e53d57a27 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 4bca385 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17212/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17212/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch, 
> HDFS-11018.003.patch
>
>
> The following error

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-18 Thread Wei-Chiu Chuang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585784#comment-15585784
 ] 

Wei-Chiu Chuang commented on HDFS-11018:


Hi [~linyiqun] thanks for your patch. I think it generally looks good to me. 
One nit: 
{code}
+ ": GenerationStamp not matched, info=" + infoByBlockId);
{code}
Instead of printing the whole ReplicaInfo which we do not need,
(it looks like this, if you are curious)
{noformat}
FinalizedReplica, blk_1073803461_74266, FINALIZED
  getNumBytes() = 54739015
  getBytesOnDisk()  = 54739015
  getVisibleLength()= 54739015
  getVolume()   = /data/4/dfs/dn/current
  getBlockFile()= 
/data/4/dfs/dn/current/BP-1092022411-10.0.0.55-1474407949037/current/finalized/subdir0/subdir240/blk_1073803461
{noformat}
 I think we just need to print the block:

so probably Block.toString(infoByBlockId)

Also, instead of printing "info=", use more self explanatory message, like 
"existing replica is "
What do you think?

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-18 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585163#comment-15585163
 ] 

Hadoop QA commented on HDFS-11018:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 61m 26s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 17s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-11018 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12833911/HDFS-11018.002.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux b09c6b75692e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / d26a1bb |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17200/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17200/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17200/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-15 Thread Yiqun Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577855#comment-15577855
 ] 

Yiqun Lin commented on HDFS-11018:
--

The findbug warning is related, will attach a new patch after others comments 
for this.

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch
>
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-15 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577811#comment-15577811
 ] 

Hadoop QA commented on HDFS-11018:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
48s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 75m 
18s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m  0s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-project/hadoop-hdfs |
|  |  Load of known null value in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(String,
 Block[])  At FsDatasetImpl.java:in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(String,
 Block[])  At FsDatasetImpl.java:[line 1893] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-11018 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12833517/HDFS-11018.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 8babee6ca381 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 391ce53 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| findbugs | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17172/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17172/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17172/console |
| Powered by | Apache Yetus 0.4.0-SNA

[jira] [Commented] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate

2016-10-14 Thread Yiqun Lin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577332#comment-15577332
 ] 

Yiqun Lin commented on HDFS-11018:
--

Good catch, [~jojochuang]! Will attach a patch soon.

> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
>
> The following error check and message is incorrect, because {{info}} is null 
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp 
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
>   // It is okay if the block is not found -- it may be deleted 
> earlier.
>   LOG.info("Failed to delete replica " + invalidBlks[i]
>   + ": ReplicaInfo not found.");
>   continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp()) 
> {
>   errors.add("Failed to delete replica " + invalidBlks[i]
>   + ": GenerationStamp not matched, info=" + info);
>   continue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org