[
https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289642#comment-14289642
]
Ming Ma commented on HDFS-7609:
---
Yeah, we also had this issue. It appears somehow an entry
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288505#comment-14288505
]
Ming Ma commented on HDFS-3519:
---
Thanks, Chris.
Checkpoint upload may interfere with a
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-3519:
--
Attachment: HDFS-3519-branch-2.patch
Thanks, Chris. Here is the patch for branch-2.
Checkpoint upload may
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-3519:
--
Attachment: HDFS-3519-3.patch
Thanks, Chris. Good point. Here is the updated patch with your suggestions.
[
https://issues.apache.org/jira/browse/HDFS-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278933#comment-14278933
]
Ming Ma commented on HDFS-7433:
---
Daryn, I agree with you, there is no need to complicate
[
https://issues.apache.org/jira/browse/HDFS-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279924#comment-14279924
]
Ming Ma commented on HDFS-6681:
---
Ratandeep, here is what you could do to make sure all events
[
https://issues.apache.org/jira/browse/HDFS-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14277639#comment-14277639
]
Ming Ma commented on HDFS-7433:
---
Daryn, I just reread the patch. You are right, that is not
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-3519:
--
Attachment: HDFS-3519-2.patch
The change of slowness parameter from 2 to 20 causes test case
[
https://issues.apache.org/jira/browse/HDFS-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274330#comment-14274330
]
Ming Ma commented on HDFS-6681:
---
Thanks, Ratandeep! I agree with your detailed analysis. For
[
https://issues.apache.org/jira/browse/HDFS-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272241#comment-14272241
]
Ming Ma commented on HDFS-7182:
---
Thanks, Jing and Akira!
JMX metrics aren't accessible when
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-3519:
--
Affects Version/s: (was: 2.0.0-alpha)
(was: 1.0.3)
Status: Patch
[
https://issues.apache.org/jira/browse/HDFS-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7182:
--
Attachment: HDFS-7182-3.patch
Good catch. Thanks, Jing. Here is the updated patch.
JMX metrics aren't
[
https://issues.apache.org/jira/browse/HDFS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-3519:
--
Attachment: HDFS-3519.patch
To follow up on this, https://issues.apache.org/jira/browse/HDFS-4811 discussed
[
https://issues.apache.org/jira/browse/HDFS-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-6184:
--
Attachment: HDFS-6184-2.patch
Rebase for trunk. Appreciate any input.
Capture NN's thread dump when it fails
[
https://issues.apache.org/jira/browse/HDFS-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14253681#comment-14253681
]
Ming Ma commented on HDFS-7182:
---
The test errors are known issues,
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254194#comment-14254194
]
Ming Ma commented on HDFS-7521:
---
[~zhz], good point. Yes, that seems to be the only case so
[
https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254213#comment-14254213
]
Ming Ma commented on HDFS-7411:
---
Couple more comments:
*
[
https://issues.apache.org/jira/browse/HDFS-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252063#comment-14252063
]
Ming Ma commented on HDFS-5535:
---
Opened https://issues.apache.org/jira/browse/HDFS-7541 to
[
https://issues.apache.org/jira/browse/HDFS-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252553#comment-14252553
]
Ming Ma commented on HDFS-7182:
---
Anyone else has suggestions on this? The patch has been
[
https://issues.apache.org/jira/browse/HDFS-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7182:
--
Attachment: HDFS-7182-2.patch
Rebased for trunk. Appreciate any input.
JMX metrics aren't accessible when NN
Ming Ma created HDFS-7541:
-
Summary: Support for fast HDFS datanode rolling upgrade
Key: HDFS-7541
URL: https://issues.apache.org/jira/browse/HDFS-7541
Project: Hadoop HDFS
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/HDFS-7541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7541:
--
Attachment: SupportforfastHDFSdatanoderollingupgrade.pdf
We ([~ctrezzo], [~jmeagher], [~lohit], [~l201514] and
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250713#comment-14250713
]
Ming Ma commented on HDFS-7521:
---
Folks, thanks for the comments.
[~wheat9], I agree with you
[
https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14248463#comment-14248463
]
Ming Ma commented on HDFS-6425:
---
Thanks [~kihwal] and [~arpitagarwal].
Large
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Attachment: HDFS-7521.patch
DNStateMachines.png
We will limit this jira to the first aspect, the
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Attachment: (was: HDFS-7521.patch)
Refactor DN state management
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Attachment: HDFS-7521.patch
Refactor DN state management
Key:
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249167#comment-14249167
]
Ming Ma commented on HDFS-7521:
---
Thanks [~wheat9], [~yzhangal] for the comments.
* Regarding
[
https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-6425:
--
Attachment: HDFS-6425-3.patch
Thanks, Kihwal.
Here is the updated patch for trunk based on a slightly different
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7400:
--
Attachment: (was: HDFS-7400.patch)
More reliable namenode health check to detect OS/HW issues
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7400:
--
Attachment: HDFS-7400.patch
More reliable namenode health check to detect OS/HW issues
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244648#comment-14244648
]
Ming Ma commented on HDFS-7400:
---
Thanks, Allen. If nobody raises any objection to providing
[
https://issues.apache.org/jira/browse/HDFS-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7491:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
Add incremental blockreport latency to DN metrics
Ming Ma created HDFS-7518:
-
Summary: Heartbeat processing doesn't have to take FSN readLock
Key: HDFS-7518
URL: https://issues.apache.org/jira/browse/HDFS-7518
Project: Hadoop HDFS
Issue Type:
Ming Ma created HDFS-7521:
-
Summary: Refactor DN state management
Key: HDFS-7521
URL: https://issues.apache.org/jira/browse/HDFS-7521
Project: Hadoop HDFS
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Description:
There are two aspects w.r.t. DN state management in NN.
* State machine management within active
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Description:
There are two aspects w.r.t. DN state management in NN.
* State machine management within active
[
https://issues.apache.org/jira/browse/HDFS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7521:
--
Description:
There are two aspects w.r.t. DN state management in NN.
* State machine management within active
[
https://issues.apache.org/jira/browse/HDFS-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238306#comment-14238306
]
Ming Ma commented on HDFS-7441:
---
Without this we use the following work around:
* Piggyback
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7400:
--
Attachment: HDFS-7400.patch
Thanks, [~ste...@apache.org], [~cmccabe] and [~aw] for the additional
suggestions
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7400:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
More reliable namenode health check to detect
Ming Ma created HDFS-7491:
-
Summary: Add incremental blockreport latency to DN metrics
Key: HDFS-7491
URL: https://issues.apache.org/jira/browse/HDFS-7491
Project: Hadoop HDFS
Issue Type:
[
https://issues.apache.org/jira/browse/HDFS-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7491:
--
Attachment: HDFS-7491.patch
Here is the patch to add IBR latency to DataNodeMetrics.
Add incremental
[
https://issues.apache.org/jira/browse/HDFS-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236420#comment-14236420
]
Ming Ma commented on HDFS-7396:
---
It might be nice if we can enforce the correctness via unit
[
https://issues.apache.org/jira/browse/HDFS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-5757:
--
Attachment: HDFS-5757.patch
Here is the rough estimate of how long dfsadmin -refreshNodes could take up
the
[
https://issues.apache.org/jira/browse/HDFS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-5757:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
Decommisson lots of nodes at the same time could
[
https://issues.apache.org/jira/browse/HDFS-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-5757:
--
Summary: refreshNodes with many nodes at the same time could slow down NN
(was: Decommisson lots of nodes at
[
https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236549#comment-14236549
]
Ming Ma commented on HDFS-7411:
---
Andrew, nice work. It appears I don't need to continue the
[
https://issues.apache.org/jira/browse/HDFS-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233895#comment-14233895
]
Ming Ma commented on HDFS-7433:
---
This is a good improvement.
Perhaps it is better call
Ming Ma created HDFS-7439:
-
Summary: Add BlockOpResponseProto's message to DFSClient's
exception message
Key: HDFS-7439
URL: https://issues.apache.org/jira/browse/HDFS-7439
Project: Hadoop HDFS
Ming Ma created HDFS-7441:
-
Summary: More accurate slow node detection in HDFS write pipeline
Key: HDFS-7441
URL: https://issues.apache.org/jira/browse/HDFS-7441
Project: Hadoop HDFS
Issue Type:
[
https://issues.apache.org/jira/browse/HDFS-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7441:
--
Summary: More accurate detection for slow node in HDFS write pipeline
(was: More accurate slow node detection
[
https://issues.apache.org/jira/browse/HDFS-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7441:
--
Description:
A DN could be slow due to OS or HW issues. HDFS write pipeline sometimes
couldn't detect the slow
[
https://issues.apache.org/jira/browse/HDFS-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7441:
--
Description:
A DN could be slow due to OS or HW issues. HDFS write pipeline sometimes
couldn't detect the slow
Ming Ma created HDFS-7442:
-
Summary: Optimization for decommission-in-progress check
Key: HDFS-7442
URL: https://issues.apache.org/jira/browse/HDFS-7442
Project: Hadoop HDFS
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219683#comment-14219683
]
Ming Ma commented on HDFS-7314:
---
Thanks, Colin.
1. There is an existing static method called
[
https://issues.apache.org/jira/browse/HDFS-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217230#comment-14217230
]
Ming Ma commented on HDFS-7409:
---
Thanks, [~andrew.wang]. The patch looks good. For the
[
https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212794#comment-14212794
]
Ming Ma commented on HDFS-7374:
---
[~andrew.wang], after a node is dead, all its blocks will be
Ming Ma created HDFS-7400:
-
Summary: More reliable namenode health check to detect OS/HW issues
Key: HDFS-7400
URL: https://issues.apache.org/jira/browse/HDFS-7400
Project: Hadoop HDFS
Issue Type:
Ming Ma created HDFS-7401:
-
Summary: Add block info to DFSInputStream' WARN message when it
adds node to deadNodes
Key: HDFS-7401
URL: https://issues.apache.org/jira/browse/HDFS-7401
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213244#comment-14213244
]
Ming Ma commented on HDFS-7374:
---
So maybe we can use if all blocks in the whole cluster are
[
https://issues.apache.org/jira/browse/HDFS-7400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213281#comment-14213281
]
Ming Ma commented on HDFS-7400:
---
Thanks, [~andrew.wang] and [~aw] for the comments. Here is
[
https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213355#comment-14213355
]
Ming Ma commented on HDFS-7374:
---
Yeah, that seems reasonable; How likely you get whole
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211584#comment-14211584
]
Ming Ma commented on HDFS-7314:
---
Thanks Colin for the good point. I also noticed that during
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205241#comment-14205241
]
Ming Ma commented on HDFS-7314:
---
Thanks, Colin. The reason to keep the thread running is to
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-6.patch
Thanks, Colin. Keeping the thread running shouldn't abort the same clients more
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-7.patch
Updated unit test TestDistributedFileSystem as the test has the assumption that
[
https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202371#comment-14202371
]
Ming Ma commented on HDFS-7374:
---
Yeah, the idea was to use {{DECOMMISSIONED}} node as the
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-5.patch
Thanks, Colin. Didn't know lease leak is a known issue.
Here is the updated
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-3.patch
Thanks, Colin. Here is the updated patch.
1. It turns out {{closeClient}} isn't
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-4.patch
It turns out a new bug not related to this was discovered by this change.
If
[
https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201646#comment-14201646
]
Ming Ma commented on HDFS-7374:
---
Zhe, thanks for reporting this.
At the high level, there is
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198998#comment-14198998
]
Ming Ma commented on HDFS-7314:
---
Thanks, Colin. Here are more explanations for the changes.
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314-2.patch
Thanks, [~cmccabe]. I have updated the patch based on your suggestion.
Aborted
[
https://issues.apache.org/jira/browse/HDFS-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196849#comment-14196849
]
Ming Ma commented on HDFS-7355:
---
Thanks, [~cnauroth]. The patch looks good. BTW, it seems
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
Aborted DFSClient's impact on long running service
[
https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7314:
--
Attachment: HDFS-7314.patch
Thanks [~kihwal] and [~cmccabe] for the good suggestions.
Here is the initial patch
Ming Ma created HDFS-7314:
-
Summary: Aborted DFSClient's impact on long running service like
YARN
Key: HDFS-7314
URL: https://issues.apache.org/jira/browse/HDFS-7314
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189241#comment-14189241
]
Ming Ma commented on HDFS-7281:
---
Thanks, Yongjun. HADOOP-11045 is useful. Both
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7281:
--
Attachment: (was: HDFS-7281-2.patch)
Missing block is marked as corrupted block
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7281:
--
Attachment: HDFS-7281-2.patch
Missing block is marked as corrupted block
[
https://issues.apache.org/jira/browse/HDFS-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183148#comment-14183148
]
Ming Ma commented on HDFS-5175:
---
Thanks, Chris. Really appreciate your input.
1. Each http
[
https://issues.apache.org/jira/browse/HDFS-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183373#comment-14183373
]
Ming Ma commented on HDFS-5175:
---
Yeah, we do have dependency on org.apache.httpcomponents'
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7281:
--
Attachment: HDFS-7281-2.patch
Thanks, Yongjun. Here is the updated patch to address your comment.
Missing
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7281:
--
Attachment: HDFS-7281.patch
Thanks, Yongjun. Besides missing block is marked as corrupted block, corrupted
[
https://issues.apache.org/jira/browse/HDFS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7281:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
Missing block is marked as corrupted block
Ming Ma created HDFS-7281:
-
Summary: Missing block is marked as corrupted block
Key: HDFS-7281
URL: https://issues.apache.org/jira/browse/HDFS-7281
Project: Hadoop HDFS
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HDFS-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177101#comment-14177101
]
Ming Ma commented on HDFS-7221:
---
Charles, thanks for the patch.
Maybe this config can be
[
https://issues.apache.org/jira/browse/HDFS-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177354#comment-14177354
]
Ming Ma commented on HDFS-7221:
---
Thanks, Charles. It shouldn't change the test result either
[
https://issues.apache.org/jira/browse/HDFS-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177375#comment-14177375
]
Ming Ma commented on HDFS-7221:
---
Thanks, Charles. The latest patch LGTM.
Ming Ma created HDFS-7269:
-
Summary: NN and DN don't check whether corrupted blocks reported
by clients are actually corrupted
Key: HDFS-7269
URL: https://issues.apache.org/jira/browse/HDFS-7269
Project:
[
https://issues.apache.org/jira/browse/HDFS-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177905#comment-14177905
]
Ming Ma commented on HDFS-7269:
---
Nicholas, in our case, the client only reported one replica
[
https://issues.apache.org/jira/browse/HDFS-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174703#comment-14174703
]
Ming Ma commented on HDFS-7221:
---
Thanks Yongjun and Charles for investigating this.
I agree
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7208:
--
Attachment: HDFS-7208-3.patch
Ha, thanks, Nicholas. Here is the new patch.
NN doesn't schedule replication
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173320#comment-14173320
]
Ming Ma commented on HDFS-7208:
---
Thanks Daryn for the input and Nicholas for the review and
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7208:
--
Attachment: HDFS-7208-2.patch
Thanks Nicholas for the review.
The latest patch addresses all your comments,
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7208:
--
Assignee: Ming Ma
Status: Patch Available (was: Open)
NN doesn't schedule replication when a DN storage
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-7208:
--
Attachment: HDFS-7208.patch
Here is the initial patch based on heartbeat notification approach, the
assumption
[
https://issues.apache.org/jira/browse/HDFS-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170203#comment-14170203
]
Ming Ma commented on HDFS-6745:
---
At RPC layer, we can add a new method similar to
[
https://issues.apache.org/jira/browse/HDFS-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167154#comment-14167154
]
Ming Ma commented on HDFS-7208:
---
Thanks, Daryn. We can do #3, but want to put the approaches
501 - 600 of 767 matches
Mail list logo
