[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-11018:
---
Release Note: Improves the error message when datanode removes a replica
which is not found.
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch,
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-11018:
---
Resolution: Fixed
Fix Version/s: 3.0.0-alpha2
2.8.0
Status: Resolved (was: Patch Available)
Committed this to trunk, branch-2.8 and branch-2. Thanks [~linyiqun] for the
patch!
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch,
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11018:
-
Attachment: HDFS-11018.003.patch
Thanks [~jojochuang] for the review. The comments look good to me. Attach a new
patch to address the comments.
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch,
> HDFS-11018.003.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11018:
-
Attachment: HDFS-11018.002.patch
Attach a new patch to fix findbug warning. Hi, [~jojochuang], could you make a
quick review, I think this is a minor change. Thanks!
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch, HDFS-11018.002.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11018:
-
Status: Patch Available (was: Open)
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11018) Incorrect check and message in FsDatasetImpl#invalidate
[
https://issues.apache.org/jira/browse/HDFS-11018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11018:
-
Attachment: HDFS-11018.001.patch
This issue will be happened when we use the method {{ReplicaMap#get(String
bpid, Block block)}} rather than {{ReplicaMap#get(String bpid, long blockId)}}.
In the former mehtod, the method returns null not always indicates that the
replica is not existed since it wll also do a generation stmap match.
I make a code glance in {{FSDatasetImpl}}, I found the method
{{FSDatasetImpl#getReplicaInfo}} also have the similar problem. I also make a
fix in my patch. Attach a initial patch to make a minor change.
> Incorrect check and message in FsDatasetImpl#invalidate
> ---
>
> Key: HDFS-11018
> URL: https://issues.apache.org/jira/browse/HDFS-11018
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-11018.001.patch
>
>
> The following error check and message is incorrect, because {{info}} is null
> if (1) the block id does not exist in ReplicaMap or (2) the generation stamp
> of block does not match the replica entry in ReplicaMap.
> {code:title=FsDatasetImpl#invalidate}
>final ReplicaInfo info = volumeMap.get(bpid, invalidBlks[i]);
> if (info == null) {
> // It is okay if the block is not found -- it may be deleted
> earlier.
> LOG.info("Failed to delete replica " + invalidBlks[i]
> + ": ReplicaInfo not found.");
> continue;
> }
> if (info.getGenerationStamp() != invalidBlks[i].getGenerationStamp())
> {
> errors.add("Failed to delete replica " + invalidBlks[i]
> + ": GenerationStamp not matched, info=" + info);
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
