[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[ https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073180#comment-17073180 ] Ahmed Hussein commented on HDFS-14968: -- I did not see much slowdown from enabling the feature. I opted to use a configuration since it is a new feature. > Add ability to know datanode staleness > -- > > Key: HDFS-14968 > URL: https://issues.apache.org/jira/browse/HDFS-14968 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1 > > Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch, > HDFS-14968.003.patch > > > There is no way to know whether a DataNode was marked stale or no longer > stale by the NameNode. > It will be good to have the option to enable logging the DataNode staleness > to figure out if the staleness was the reason behind remote reads. > Therefore, analyze performance and decision making of the local vs remote > reads. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[ https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071281#comment-17071281 ] Wei-Chiu Chuang commented on HDFS-14968: Why is it hidden behind. a configuration if it's just printing extra logs? Does it emit so much log that it slow down NN? Feels to me this configuration should be enabled all the time (at least that's what I would do in Cloudera) > Add ability to know datanode staleness > -- > > Key: HDFS-14968 > URL: https://issues.apache.org/jira/browse/HDFS-14968 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1 > > Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch, > HDFS-14968.003.patch > > > There is no way to know whether a DataNode was marked stale or no longer > stale by the NameNode. > It will be good to have the option to enable logging the DataNode staleness > to figure out if the staleness was the reason behind remote reads. > Therefore, analyze performance and decision making of the local vs remote > reads. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[ https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021201#comment-17021201 ] Kihwal Lee commented on HDFS-14968: --- Committed this to trunk, branch-3.2, branch-3.1 and branch-2.10. Thanks for the patch, Ahmed. > Add ability to know datanode staleness > -- > > Key: HDFS-14968 > URL: https://issues.apache.org/jira/browse/HDFS-14968 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1 > > Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch, > HDFS-14968.003.patch > > > There is no way to know whether a DataNode was marked stale or no longer > stale by the NameNode. > It will be good to have the option to enable logging the DataNode staleness > to figure out if the staleness was the reason behind remote reads. > Therefore, analyze performance and decision making of the local vs remote > reads. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[ https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021200#comment-17021200 ] Hudson commented on HDFS-14968: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17891 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17891/]) HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed (kihwal: rev bd03053ea2f32ef982e37fbf2ffd679cb7dda797) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java > Add ability to know datanode staleness > -- > > Key: HDFS-14968 > URL: https://issues.apache.org/jira/browse/HDFS-14968 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch, > HDFS-14968.003.patch > > > There is no way to know whether a DataNode was marked stale or no longer > stale by the NameNode. > It will be good to have the option to enable logging the DataNode staleness > to figure out if the staleness was the reason behind remote reads. > Therefore, analyze performance and decision making of the local vs remote > reads. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[ https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021165#comment-17021165 ] Kihwal Lee commented on HDFS-14968: --- +1 we have been running production with this. > Add ability to know datanode staleness > -- > > Key: HDFS-14968 > URL: https://issues.apache.org/jira/browse/HDFS-14968 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch, > HDFS-14968.003.patch > > > There is no way to know whether a DataNode was marked stale or no longer > stale by the NameNode. > It will be good to have the option to enable logging the DataNode staleness > to figure out if the staleness was the reason behind remote reads. > Therefore, analyze performance and decision making of the local vs remote > reads. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[
https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016082#comment-17016082
]
Ahmed Hussein commented on HDFS-14968:
--
JUnit tests in the report seem to fail intermittently:
# {{TestRedudantBlocks}}: HDFS-15092
# {{TestBalancerRPCDelay}}: HDFS-14757
# {{TestDecommission.testAllocAndIBRWhileDecommission}}: I wil file a new Jira
to fix it. It is more likely to be a race condition reading the block state
without waiting for it to be come {{COMPLETE}}.
[~kihwal], Can you please take a look at the patch?
> Add ability to know datanode staleness
> --
>
> Key: HDFS-14968
> URL: https://issues.apache.org/jira/browse/HDFS-14968
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, logging, namenode
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-14968.001.patch, HDFS-14968.002.patch,
> HDFS-14968.003.patch
>
>
> There is no way to know whether a DataNode was marked stale or no longer
> stale by the NameNode.
> It will be good to have the option to enable logging the DataNode staleness
> to figure out if the staleness was the reason behind remote reads.
> Therefore, analyze performance and decision making of the local vs remote
> reads.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[
https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015471#comment-17015471
]
Hadoop QA commented on HDFS-14968:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 460 unchanged - 2 fixed = 460 total (was 462) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 58s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}114m 44s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestRedudantBlocks |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.balancer.TestBalancerRPCDelay |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-14968 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12990912/HDFS-14968.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux f10fe5a819e2 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1c51f36 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| unit |
[jira] [Commented] (HDFS-14968) Add ability to know datanode staleness
[
https://issues.apache.org/jira/browse/HDFS-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015305#comment-17015305
]
Hadoop QA commented on HDFS-14968:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 460 unchanged - 2 fixed = 460 total (was 462) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 3s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.tools.TestHdfsConfigFields |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.TestRedudantBlocks |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-14968 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12990888/HDFS-14968.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 3663678bc898 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1c51f36 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| unit |
