[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=565654=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-565654 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 13/Mar/21 07:38 Start Date: 13/Mar/21 07:38 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-797884140 Close this PR because the associated patch has been merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 565654) Time Spent: 6h 20m (was: 6h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Fix For: 3.3.1, 3.4.0, 3.1.5, 2.10.2, 3.2.3 > > Attachments: ExpiredHeartbeat.png, HDFS-15808-branch-3.3.001.patch, > lockLongHoldCount > > Time Spent: 6h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=565655=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-565655 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 13/Mar/21 07:38 Start Date: 13/Mar/21 07:38 Worklog Time Spent: 10m Work Description: tomscut closed pull request #2668: URL: https://github.com/apache/hadoop/pull/2668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 565655) Time Spent: 6.5h (was: 6h 20m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Fix For: 3.3.1, 3.4.0, 3.1.5, 2.10.2, 3.2.3 > > Attachments: ExpiredHeartbeat.png, HDFS-15808-branch-3.3.001.patch, > lockLongHoldCount > > Time Spent: 6.5h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=560791=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-560791 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 04/Mar/21 02:28 Start Date: 04/Mar/21 02:28 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-790235902 Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA, please help to review it. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 560791) Time Spent: 6h 10m (was: 6h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, HDFS-15808-branch-3.3.001.patch, > lockLongHoldCount > > Time Spent: 6h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=560313=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-560313 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 03/Mar/21 06:55 Start Date: 03/Mar/21 06:55 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-789484964 Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA. > +1 LGTM Thanks @dineshchitlangia for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 560313) Time Spent: 6h (was: 5h 50m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, HDFS-15808-branch-3.3.001.patch, > lockLongHoldCount > > Time Spent: 6h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=559433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-559433 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 01/Mar/21 15:58 Start Date: 01/Mar/21 15:58 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-788061149 Hi @arp7 , could you take a quick look at this? Let's see if there are any other questions. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 559433) Time Spent: 5h 50m (was: 5h 40m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, lockLongHoldCount > > Time Spent: 5h 50m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=559168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-559168 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 01/Mar/21 03:26 Start Date: 01/Mar/21 03:26 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-787609646 Hi @anuengineer , could you please review again? Let's see if there are any other questions. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 559168) Time Spent: 5h 40m (was: 5.5h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, lockLongHoldCount > > Time Spent: 5h 40m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockLongHoldCount/WriteLockLongHoldCount), which are exposed in > JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=556122=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-556122 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 23/Feb/21 01:26 Start Date: 23/Feb/21 01:26 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-783798764 Failed junit tests hadoop.hdfs.server.namenode.ha.TestHAAppend hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.datanode.TestBlockScanner Those failed unit tests were unrelated to the change. And they worked fine locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 556122) Time Spent: 5.5h (was: 5h 20m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, lockLongHoldCount > > Time Spent: 5.5h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=556116=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-556116 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 23/Feb/21 01:23 Start Date: 23/Feb/21 01:23 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with it? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 556116) Time Spent: 5h 10m (was: 5h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, lockLongHoldCount > > Time Spent: 5h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=556117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-556117 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 23/Feb/21 01:23 Start Date: 23/Feb/21 01:23 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780595016 > This message was automatically generated. Hi @anuengineer, the failed check looks unrelated to the change. May I ask how I should deal with it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 556117) Time Spent: 5h 20m (was: 5h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: ExpiredHeartbeat.png, lockLongHoldCount > > Time Spent: 5h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=67=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-67
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 22/Feb/21 02:19
Start Date: 22/Feb/21 02:19
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r579912802
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs)
{
+ numReadLockLongHold.addAndGet(1);
Review comment:
> nit: you could use incrementAndGet. Don't have to fix it for me to
commit it, just flagging for your consideration.
>
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
Hi @anuengineer , I fixed the problem. And I invited you to join the
discussion on JIRA.
https://issues.apache.org/jira/browse/HDFS-15808
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 67)
Time Spent: 5h (was: 4h 50m)
> Add metrics for FSNamesystem read/write lock hold long time
> ---
>
> Key: HDFS-15808
> URL: https://issues.apache.org/jira/browse/HDFS-15808
> Project: Hadoop HDFS
> Issue Type: Wish
> Components: hdfs
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: hdfs, lock, metrics, pull-request-available
> Attachments: ExpiredHeartbeat.png, lockLongHoldCount
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> To monitor how often read/write locks exceed thresholds, we can add two
> metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=66=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-66
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 22/Feb/21 02:18
Start Date: 22/Feb/21 02:18
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r579912802
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs)
{
+ numReadLockLongHold.addAndGet(1);
Review comment:
> nit: you could use incrementAndGet. Don't have to fix it for me to
commit it, just flagging for your consideration.
>
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
Hi @anuengineer , I fixed the problem. And I invited you to join the
discussion on JIRA.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 66)
Time Spent: 4h 50m (was: 4h 40m)
> Add metrics for FSNamesystem read/write lock hold long time
> ---
>
> Key: HDFS-15808
> URL: https://issues.apache.org/jira/browse/HDFS-15808
> Project: Hadoop HDFS
> Issue Type: Wish
> Components: hdfs
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: hdfs, lock, metrics, pull-request-available
> Attachments: ExpiredHeartbeat.png, lockLongHoldCount
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> To monitor how often read/write locks exceed thresholds, we can add two
> metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=555096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555096
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 20/Feb/21 07:25
Start Date: 20/Feb/21 07:25
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-782578219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 18s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 17s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 7s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 5s | | trunk passed |
| -0 :warning: | patch | 3m 24s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 55s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 47s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 57s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 204m 49s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 312m 18s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6ac55fde21d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554555 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 19/Feb/21 01:57 Start Date: 19/Feb/21 01:57 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781760195 > Just reposting my comment from the jira for visibility. > > The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often. > See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful. > I assume you need something like a rate. Hey @shvachko , thank you for your comments and suggestions. I replied to you in JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554555) Time Spent: 4.5h (was: 4h 20m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Attachments: lockLongHoldCount > > Time Spent: 4.5h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554448=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554448 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 20:18 Start Date: 18/Feb/21 20:18 Worklog Time Spent: 10m Work Description: shvachko edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781609671 Just reposting my comment from the jira for visibility. The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often. See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful. I assume you need something like a rate. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554448) Time Spent: 4h 20m (was: 4h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554447 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 20:18 Start Date: 18/Feb/21 20:18 Worklog Time Spent: 10m Work Description: shvachko commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781609671 Just reposting my comment on the jira for visibility. The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often. See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful. I assume you need something like a rate. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554447) Time Spent: 4h 10m (was: 4h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554034=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554034 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 02:20 Start Date: 18/Feb/21 02:20 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with it? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554034) Time Spent: 4h (was: 3h 50m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554033 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 02:20 Start Date: 18/Feb/21 02:20 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554033) Time Spent: 3h 50m (was: 3h 40m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554032=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554032 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 02:18 Start Date: 18/Feb/21 02:18 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554032) Time Spent: 3h 40m (was: 3.5h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554031=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554031 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 02:13 Start Date: 18/Feb/21 02:13 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554031) Time Spent: 3.5h (was: 3h 20m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=554030=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554030 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 18/Feb/21 02:12 Start Date: 18/Feb/21 02:12 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248 I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870 Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you. `[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:03.887Z] Already on 'trunk' [2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:04.412Z] Already up to date. [2021-02-17T01:59:04.412Z] Current branch trunk is up to date. [2021-02-17T01:59:04.412Z] Already on 'trunk' [2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'. [2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 554030) Time Spent: 3h 20m (was: 3h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=553632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-553632 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 17/Feb/21 14:31 Start Date: 17/Feb/21 14:31 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780595016 > This message was automatically generated. Hi @anuengineer, the failed check looks unrelated to the change. May I ask how I should deal with it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 553632) Time Spent: 3h 10m (was: 3h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=553322=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-553322 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 17/Feb/21 01:59 Start Date: 17/Feb/21 01:59 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780242371 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 19m 14s | | Docker mode activated. | | -1 :x: | yetus | 0m 6s | | Unprocessed flag(s): --findbugs-strict-precheck | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2668 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/console | | versions | git=2.25.1 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 553322) Time Spent: 3h (was: 2h 50m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=553311=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-553311
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 17/Feb/21 01:43
Start Date: 17/Feb/21 01:43
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r577266767
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs)
{
+ numReadLockLongHold.addAndGet(1);
Review comment:
Thank you for your review and suggestion. Using incrementAndGet is a
better way. I fixed it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 553311)
Time Spent: 2h 50m (was: 2h 40m)
> Add metrics for FSNamesystem read/write lock hold long time
> ---
>
> Key: HDFS-15808
> URL: https://issues.apache.org/jira/browse/HDFS-15808
> Project: Hadoop HDFS
> Issue Type: Wish
> Components: hdfs
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: hdfs, lock, metrics, pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> To monitor how often read/write locks exceed thresholds, we can add two
> metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=553030=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-553030
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 15:51
Start Date: 16/Feb/21 15:51
Worklog Time Spent: 10m
Work Description: anuengineer commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r576929632
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs)
{
+ numReadLockLongHold.addAndGet(1);
Review comment:
+1, LGTM. if you are making any changes, please let me know. Otherwise,
I will leave for a little while for @arp7 or @xiaoyuyao to take a quick look.
Thank you for your work and contribution.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 553030)
Time Spent: 2h 40m (was: 2.5h)
> Add metrics for FSNamesystem read/write lock hold long time
> ---
>
> Key: HDFS-15808
> URL: https://issues.apache.org/jira/browse/HDFS-15808
> Project: Hadoop HDFS
> Issue Type: Wish
> Components: hdfs
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: hdfs, lock, metrics, pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> To monitor how often read/write locks exceed thresholds, we can add two
> metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=553028=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-553028
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 15:50
Start Date: 16/Feb/21 15:50
Worklog Time Spent: 10m
Work Description: anuengineer commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r576928327
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs)
{
+ numReadLockLongHold.addAndGet(1);
Review comment:
nit: you could use incrementAndGet. Don't have to fix it for me to
commit it, just flagging for your consideration.
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 553028)
Time Spent: 2.5h (was: 2h 20m)
> Add metrics for FSNamesystem read/write lock hold long time
> ---
>
> Key: HDFS-15808
> URL: https://issues.apache.org/jira/browse/HDFS-15808
> Project: Hadoop HDFS
> Issue Type: Wish
> Components: hdfs
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: hdfs, lock, metrics, pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> To monitor how often read/write locks exceed thresholds, we can add two
> metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552788=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552788 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 16/Feb/21 02:54 Start Date: 16/Feb/21 02:54 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap Those failed unit tests are unrelated to the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552788) Time Spent: 2h 20m (was: 2h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552615=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552615 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 15/Feb/21 16:54 Start Date: 15/Feb/21 16:54 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap Those failed unit tests were unrelated to the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552615) Time Spent: 2h 10m (was: 2h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552608
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 16:28
Start Date: 15/Feb/21 16:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779330421
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 39s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 56s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 6s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 4s | | trunk passed |
| -0 :warning: | patch | 3m 22s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 4s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 9s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 194m 59s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 281m 25s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0c1c25c99215 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c3134ab3a99 |
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552072=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552072 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 13/Feb/21 00:25 Start Date: 13/Feb/21 00:25 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778526792 Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks Sorry. I didn't update this unit tests, and they worked fine locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552072) Time Spent: 1h 50m (was: 1h 40m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=551855=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-551855
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 12/Feb/21 15:18
Start Date: 12/Feb/21 15:18
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778256569
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 14m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 7s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 32s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 5s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 3s | | trunk passed |
| -0 :warning: | patch | 3m 21s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| -0 :warning: | checkstyle | 0m 57s |
[/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged
- 0 fixed = 175 total (was 167) |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 12m 55s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 2s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 190m 34s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 289m 36s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 15073fe6c019 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=551844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-551844
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 12/Feb/21 15:04
Start Date: 12/Feb/21 15:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778248179
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 16s | | trunk passed |
| +1 :green_heart: | compile | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 35s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 8s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 6s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| -0 :warning: | checkstyle | 0m 57s |
[/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged
- 0 fixed = 175 total (was 167) |
| +1 :green_heart: | mvnsite | 1m 12s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 7s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 12s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 192m 53s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 280m 44s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4e47b5746c6d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
