[jira] [Updated] (HDFS-15821) Add metrics for in-service datanodes
[ https://issues.apache.org/jira/browse/HDFS-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated HDFS-15821: --- Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1 Resolution: Fixed Status: Resolved (was: Patch Available) I have committed this to trunk - branch-3.1 > Add metrics for in-service datanodes > > > Key: HDFS-15821 > URL: https://issues.apache.org/jira/browse/HDFS-15821 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Zehao Chen >Assignee: Zehao Chen >Priority: Minor > Labels: pull-request-available > Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > We currently have metrics for live datanodes but some of the datanodes may be > in decommissioning state or maintenance state. Adding this metric allows us > to know how many nodes are currently in service, where NumInServiceDatanodes > = NumLiveDataNodes - NumDecomLiveDataNodes - NumInMaintenanceLiveDataNodes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15423) RBF: WebHDFS create shouldn't choose DN from all sub-clusters
[
https://issues.apache.org/jira/browse/HDFS-15423?focusedWorklogId=552659=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552659
]
ASF GitHub Bot logged work on HDFS-15423:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 19:12
Start Date: 15/Feb/21 19:12
Worklog Time Spent: 10m
Work Description: fengnanli commented on pull request #2605:
URL: https://github.com/apache/hadoop/pull/2605#issuecomment-779405236
Rebase on latest trunk and force push.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552659)
Time Spent: 4h 50m (was: 4h 40m)
> RBF: WebHDFS create shouldn't choose DN from all sub-clusters
> -
>
> Key: HDFS-15423
> URL: https://issues.apache.org/jira/browse/HDFS-15423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf, webhdfs
>Reporter: Chao Sun
>Assignee: Fengnan Li
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> In {{RouterWebHdfsMethods}} and for a {{CREATE}} call, {{chooseDatanode}}
> first gets all DNs via {{getDatanodeReport}}, and then randomly pick one from
> the list via {{getRandomDatanode}}. This logic doesn't seem correct as it
> should pick a DN for the specific cluster(s) of the input {{path}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15423) RBF: WebHDFS create shouldn't choose DN from all sub-clusters
[
https://issues.apache.org/jira/browse/HDFS-15423?focusedWorklogId=552715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552715
]
ASF GitHub Bot logged work on HDFS-15423:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 20:50
Start Date: 15/Feb/21 20:50
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2605:
URL: https://github.com/apache/hadoop/pull/2605#issuecomment-779441844
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 25s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 31s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 1m 14s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 12s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 33s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 1m 18s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 17m 13s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2605/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 97m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.fs.contract.router.TestRouterHDFSContractCreate |
| | hadoop.fs.contract.router.TestRouterHDFSContractCreateSecure |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2605/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2605 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c72969321a27 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23
18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 07a4220cd27 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552788=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552788 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 16/Feb/21 02:54 Start Date: 16/Feb/21 02:54 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap Those failed unit tests are unrelated to the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552788) Time Spent: 2h 20m (was: 2h 10m) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15836) RBF: Fix contract tests after HADOOP-13327
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15836:
-
Summary: RBF: Fix contract tests after HADOOP-13327 (was: RBF: Fix
TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure)
> RBF: Fix contract tests after HADOOP-13327
> --
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?focusedWorklogId=552824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552824
]
ASF GitHub Bot logged work on HDFS-15836:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 05:51
Start Date: 16/Feb/21 05:51
Worklog Time Spent: 10m
Work Description: aajisaka merged pull request #2702:
URL: https://github.com/apache/hadoop/pull/2702
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552824)
Time Spent: 40m (was: 0.5h)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?focusedWorklogId=552825=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552825
]
ASF GitHub Bot logged work on HDFS-15836:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 05:51
Start Date: 16/Feb/21 05:51
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2702:
URL: https://github.com/apache/hadoop/pull/2702#issuecomment-779599419
Merged. Thank you @ayushtkn
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552825)
Time Spent: 50m (was: 40m)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15423) RBF: WebHDFS create shouldn't choose DN from all sub-clusters
[
https://issues.apache.org/jira/browse/HDFS-15423?focusedWorklogId=552781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552781
]
ASF GitHub Bot logged work on HDFS-15423:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 01:12
Start Date: 16/Feb/21 01:12
Worklog Time Spent: 10m
Work Description: fengnanli commented on pull request #2605:
URL: https://github.com/apache/hadoop/pull/2605#issuecomment-779521144
The test failures are related with the change in
[HADOOP-13327](https://issues.apache.org/jira/browse/HADOOP-13327) and is under
fix in [HDFS-15836](https://issues.apache.org/jira/browse/HDFS-15836)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552781)
Time Spent: 5h 10m (was: 5h)
> RBF: WebHDFS create shouldn't choose DN from all sub-clusters
> -
>
> Key: HDFS-15423
> URL: https://issues.apache.org/jira/browse/HDFS-15423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf, webhdfs
>Reporter: Chao Sun
>Assignee: Fengnan Li
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> In {{RouterWebHdfsMethods}} and for a {{CREATE}} call, {{chooseDatanode}}
> first gets all DNs via {{getDatanodeReport}}, and then randomly pick one from
> the list via {{getRandomDatanode}}. This logic doesn't seem correct as it
> should pick a DN for the specific cluster(s) of the input {{path}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284992#comment-17284992
]
Fengnan Li commented on HDFS-15836:
---
+1 for the fix. Thanks [~aajisaka]
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li reassigned HDFS-15836:
-
Assignee: Akira Ajisaka
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?focusedWorklogId=552810=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552810
]
ASF GitHub Bot logged work on HDFS-15836:
-
Author: ASF GitHub Bot
Created on: 16/Feb/21 04:39
Start Date: 16/Feb/21 04:39
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2702:
URL: https://github.com/apache/hadoop/pull/2702#issuecomment-779576047
Thank you @fengnanli
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552810)
Time Spent: 0.5h (was: 20m)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15836) RBF: Fix contract tests after HADOOP-13327
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HDFS-15836.
--
Fix Version/s: 3.4.0
3.3.1
Resolution: Fixed
Committed to trunk and branch-3.3. Thanks [~fengnanli] and [~ayushtkn] for
reviewing this.
> RBF: Fix contract tests after HADOOP-13327
> --
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285041#comment-17285041 ] Arpit Agarwal commented on HDFS-15808: -- Hi [~tomscut], I probably won't be able to look at this soon. > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15781) Add metrics for how blocks are moved in replaceBlock
[ https://issues.apache.org/jira/browse/HDFS-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-15781: -- Labels: pull-request-available (was: ) > Add metrics for how blocks are moved in replaceBlock > > > Key: HDFS-15781 > URL: https://issues.apache.org/jira/browse/HDFS-15781 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We can add some metrics for to track how the blocks are being moved, to get > a sense of the locality of movements. > * How many blocks copied to local host? > * How many blocks moved to local disk thru hardlink? > * How many blocks are copied out of the host > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15781) Add metrics for how blocks are moved in replaceBlock
[ https://issues.apache.org/jira/browse/HDFS-15781?focusedWorklogId=552844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552844 ] ASF GitHub Bot logged work on HDFS-15781: - Author: ASF GitHub Bot Created on: 16/Feb/21 07:46 Start Date: 16/Feb/21 07:46 Worklog Time Spent: 10m Work Description: LeonGao91 opened a new pull request #2704: URL: https://github.com/apache/hadoop/pull/2704 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552844) Remaining Estimate: 0h Time Spent: 10m > Add metrics for how blocks are moved in replaceBlock > > > Key: HDFS-15781 > URL: https://issues.apache.org/jira/browse/HDFS-15781 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > We can add some metrics for to track how the blocks are being moved, to get > a sense of the locality of movements. > * How many blocks copied to local host? > * How many blocks moved to local disk thru hardlink? > * How many blocks are copied out of the host > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15834) Remove the usage of org.apache.log4j.Level
[
https://issues.apache.org/jira/browse/HDFS-15834?focusedWorklogId=552469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552469
]
ASF GitHub Bot logged work on HDFS-15834:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 08:10
Start Date: 15/Feb/21 08:10
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2696:
URL: https://github.com/apache/hadoop/pull/2696#issuecomment-779036029
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 76 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 6s | | trunk passed |
| +1 :green_heart: | compile | 4m 49s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 4m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 39s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 21s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 1m 14s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 22s | |
branch/hadoop-hdfs-project/hadoop-hdfs-native-client no findbugs output file
(findbugsXml.xml) |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 50s | | the patch passed |
| +1 :green_heart: | compile | 4m 41s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 4m 41s | |
hadoop-hdfs-project-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 0 new + 692 unchanged - 79
fixed = 692 total (was 771) |
| +1 :green_heart: | compile | 4m 21s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 4m 21s | |
hadoop-hdfs-project-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with
JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 0 new + 669
unchanged - 79 fixed = 669 total (was 748) |
| +1 :green_heart: | checkstyle | 1m 49s | | hadoop-hdfs-project: The
patch generated 0 new + 1630 unchanged - 4 fixed = 1630 total (was 1634) |
| +1 :green_heart: | mvnsite | 2m 48s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 1s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 12m 44s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 7s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | findbugs | 0m 19s | |
hadoop-hdfs-project/hadoop-hdfs-native-client has no data from findbugs |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 20s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 193m 15s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 8m 15s | | hadoop-hdfs-native-client in
the patch passed. |
| -1 :x: | unit | 16m 50s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 344m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate |
| | hadoop.fs.contract.router.TestRouterHDFSContractCreate |
| |
[jira] [Created] (HDFS-15835) Erasure coding: Add/remove logs for the better readability/debugging
Bhavik Patel created HDFS-15835: --- Summary: Erasure coding: Add/remove logs for the better readability/debugging Key: HDFS-15835 URL: https://issues.apache.org/jira/browse/HDFS-15835 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: Bhavik Patel * Unnecessary Namenode logs displaying for Disabling EC policies which are already disabled. * There is no info/debug are present for addPolicy, unsetPolicy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15835) Erasure coding: Add/remove logs for the better readability/debugging
[ https://issues.apache.org/jira/browse/HDFS-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated HDFS-15835: Component/s: erasure-coding > Erasure coding: Add/remove logs for the better readability/debugging > > > Key: HDFS-15835 > URL: https://issues.apache.org/jira/browse/HDFS-15835 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding, hdfs >Reporter: Bhavik Patel >Priority: Minor > > * Unnecessary Namenode logs displaying for Disabling EC policies which are > already disabled. > * There is no info/debug are present for addPolicy, unsetPolicy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15835) Erasure coding: Add/remove logs for the better readability/debugging
[ https://issues.apache.org/jira/browse/HDFS-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated HDFS-15835: Attachment: HDFS-15835.001.patch > Erasure coding: Add/remove logs for the better readability/debugging > > > Key: HDFS-15835 > URL: https://issues.apache.org/jira/browse/HDFS-15835 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding, hdfs >Reporter: Bhavik Patel >Priority: Minor > Attachments: HDFS-15835.001.patch > > > * Unnecessary Namenode logs displaying for Disabling EC policies which are > already disabled. > * There is no info/debug are present for addPolicy, unsetPolicy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15835) Erasure coding: Add/remove logs for the better readability/debugging
[ https://issues.apache.org/jira/browse/HDFS-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavik Patel updated HDFS-15835: Status: Patch Available (was: Open) > Erasure coding: Add/remove logs for the better readability/debugging > > > Key: HDFS-15835 > URL: https://issues.apache.org/jira/browse/HDFS-15835 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding, hdfs >Reporter: Bhavik Patel >Priority: Minor > Attachments: HDFS-15835.001.patch > > > * Unnecessary Namenode logs displaying for Disabling EC policies which are > already disabled. > * There is no info/debug are present for addPolicy, unsetPolicy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15836) Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
Akira Ajisaka created HDFS-15836:
Summary: Fix TestRouterHDFSContractCreate and
TestRouterHDFSContractCreateSecure
Key: HDFS-15836
URL: https://issues.apache.org/jira/browse/HDFS-15836
Project: Hadoop HDFS
Issue Type: Sub-task
Components: rbf
Reporter: Akira Ajisaka
{noformat}
[ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 19.094
s <<< FAILURE! - in
org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
[ERROR]
testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
Time elapsed: 0.102 s <<< FAILURE!
java.lang.AssertionError: Should not have capability: hflush in
FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{noformat}
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15836:
-
Summary: RBF: Fix TestRouterHDFSContractCreate and
TestRouterHDFSContractCreateSecure (was: Fix TestRouterHDFSContractCreate and
TestRouterHDFSContractCreateSecure)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15814) Make some parameters configurable for DataNodeDiskMetrics
[ https://issues.apache.org/jira/browse/HDFS-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284666#comment-17284666 ] tomscut commented on HDFS-15814: Thanks [~arp] and [~dineshchitlangia] . > Make some parameters configurable for DataNodeDiskMetrics > - > > Key: HDFS-15814 > URL: https://issues.apache.org/jira/browse/HDFS-15814 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Fix For: 3.3.1 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > For ease of use, especially for small clusters, we can change some > parameters(MIN_OUTLIER_DETECTION_DISKS, SLOW_DISK_LOW_THRESHOLD_MS) > configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-15836:
--
Labels: pull-request-available (was: )
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?focusedWorklogId=552530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552530
]
ASF GitHub Bot logged work on HDFS-15836:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 11:33
Start Date: 15/Feb/21 11:33
Worklog Time Spent: 10m
Work Description: aajisaka opened a new pull request #2702:
URL: https://github.com/apache/hadoop/pull/2702
JIRA: https://issues.apache.org/jira/browse/HDFS-15836
Fix the following tests:
- TestRouterHDFSContractCreate
- TestRouterHDFSContractCreateSecure
- TestRouterWebHDFSContractCreate
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552530)
Remaining Estimate: 0h
Time Spent: 10m
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284668#comment-17284668 ] tomscut commented on HDFS-15808: Hi [~arp], could you please help review the code? Thank you. https://github.com/apache/hadoop/pull/2668 > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15830) Support to make dfs.image.parallel.load reconfigurable
[ https://issues.apache.org/jira/browse/HDFS-15830?focusedWorklogId=552482=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552482 ] ASF GitHub Bot logged work on HDFS-15830: - Author: ASF GitHub Bot Created on: 15/Feb/21 09:00 Start Date: 15/Feb/21 09:00 Worklog Time Spent: 10m Work Description: ferhui commented on pull request #2694: URL: https://github.com/apache/hadoop/pull/2694#issuecomment-779069487 @dineshchitlangia Thanks for review! Wait until @sodonnel take another look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552482) Time Spent: 40m (was: 0.5h) > Support to make dfs.image.parallel.load reconfigurable > -- > > Key: HDFS-15830 > URL: https://issues.apache.org/jira/browse/HDFS-15830 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Hui Fei >Assignee: Hui Fei >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > After HDFS-14617, loading fsimage improve a lot. > If something unexpected happens, we have to load old image to restart > namenode. > So advise that we make dfs.image.parallel.load reconfigurable, then we can > save new fsimage. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15834) Remove the usage of org.apache.log4j.Level
[ https://issues.apache.org/jira/browse/HDFS-15834?focusedWorklogId=552498=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552498 ] ASF GitHub Bot logged work on HDFS-15834: - Author: ASF GitHub Bot Created on: 15/Feb/21 09:44 Start Date: 15/Feb/21 09:44 Worklog Time Spent: 10m Work Description: aajisaka commented on pull request #2696: URL: https://github.com/apache/hadoop/pull/2696#issuecomment-779098100 Filed https://issues.apache.org/jira/browse/HDFS-15836 to fix the test failures. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552498) Time Spent: 40m (was: 0.5h) > Remove the usage of org.apache.log4j.Level > -- > > Key: HDFS-15834 > URL: https://issues.apache.org/jira/browse/HDFS-15834 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Replace org.apache.log4j.Level with org.slf4j.event.Level in hadoop-hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15836:
-
Target Version/s: 3.3.1, 3.4.0 (was: 3.4.0)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.testSyncable(AbstractContractCreateTest.java:459)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2696/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-15808: --- Comment: was deleted (was: Hi [~arp], could you please help review the code? Thank you. https://github.com/apache/hadoop/pull/2668) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15835) Erasure coding: Add/remove logs for the better readability/debugging
[
https://issues.apache.org/jira/browse/HDFS-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284737#comment-17284737
]
Hadoop QA commented on HDFS-15835:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
34s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 20s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
5s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config;
considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
3s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 57s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/474/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt{color}
| {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new +
219 unchanged - 0 fixed = 220 total (was 219) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 15s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts.
[jira] [Work logged] (HDFS-15836) RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
[
https://issues.apache.org/jira/browse/HDFS-15836?focusedWorklogId=552548=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552548
]
ASF GitHub Bot logged work on HDFS-15836:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 12:41
Start Date: 15/Feb/21 12:41
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2702:
URL: https://github.com/apache/hadoop/pull/2702#issuecomment-779198376
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 47m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 13m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 37s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 66m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2702/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2702 |
| Optional Tests | dupname asflicense unit xml |
| uname | Linux 87c21ab0f2b7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c3134ab3a99 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2702/1/testReport/ |
| Max. process+thread count | 667 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2702/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 552548)
Time Spent: 20m (was: 10m)
> RBF: Fix TestRouterHDFSContractCreate and TestRouterHDFSContractCreateSecure
>
>
> Key: HDFS-15836
> URL: https://issues.apache.org/jira/browse/HDFS-15836
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 16, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 19.094 s <<< FAILURE! - in
> org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate
> [ERROR]
> testSyncable(org.apache.hadoop.fs.contract.router.TestRouterHDFSContractCreate)
> Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Should not have capability: hflush in
> FSDataOutputStream{wrappedStream=DFSOutputStream:block==null}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.assertCapabilities(ContractTestUtils.java:1553)
> at
> org.apache.hadoop.fs.contract.AbstractContractCreateTest.validateSyncableSemantics(AbstractContractCreateTest.java:497)
> at
>
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[
https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552608
]
ASF GitHub Bot logged work on HDFS-15808:
-
Author: ASF GitHub Bot
Created on: 15/Feb/21 16:28
Start Date: 15/Feb/21 16:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779330421
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 39s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 56s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 6s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 4s | | trunk passed |
| -0 :warning: | patch | 3m 22s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 4s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 9s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 194m 59s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 281m 25s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0c1c25c99215 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c3134ab3a99 |
[jira] [Commented] (HDFS-15566) NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0
[
https://issues.apache.org/jira/browse/HDFS-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284790#comment-17284790
]
Sunil G commented on HDFS-15566:
[~weichiu] [~brahmareddy]
I could see the +1 in the Jira, but this was not merged.
Could we land this?
> NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0
>
>
> Key: HDFS-15566
> URL: https://issues.apache.org/jira/browse/HDFS-15566
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: HDFS-15566-001.patch, HDFS-15566-002.patch,
> HDFS-15566-003.patch
>
>
> * After rollingUpgrade NN from 3.1.3/3.2.1 to 3.3.0, if the NN is restarted,
> it fails while replaying edit logs.
> * HDFS-14922, HDFS-14924, and HDFS-15054 introduced the *modification time*
> bits to the editLog transactions.
> * When NN is restarted and the edit logs are replayed, the NN reads the old
> layout version from the editLog file. When parsing the transactions, it
> assumes that the transactions are also from the previous layout and hence
> skips parsing the *modification time* bits.
> * This cascades into reading the wrong set of bits for other fields and
> leads to NN shutting down.
> {noformat}
> 2020-09-07 19:34:42,085 | DEBUG | main | Stopping client | Client.java:1361
> 2020-09-07 19:34:42,087 | ERROR | main | Failed to start namenode. |
> NameNode.java:1751
> java.lang.IllegalArgumentException
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
> at org.apache.hadoop.ipc.ClientId.toString(ClientId.java:56)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogOp.appendRpcIdsToString(FSEditLogOp.java:318)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogOp.access$700(FSEditLogOp.java:153)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogOp$DeleteSnapshotOp.toString(FSEditLogOp.java:3606)
> at java.lang.String.valueOf(String.java:2994)
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:305)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:188)
> at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:932)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:779)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1136)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:742)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:654)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:716)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:959)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:932)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1674)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1744){noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?focusedWorklogId=552615=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552615 ] ASF GitHub Bot logged work on HDFS-15808: - Author: ASF GitHub Bot Created on: 15/Feb/21 16:54 Start Date: 15/Feb/21 16:54 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap Those failed unit tests were unrelated to the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552615) Time Spent: 2h 10m (was: 2h) > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15821) Add metrics for in-service datanodes
[ https://issues.apache.org/jira/browse/HDFS-15821?focusedWorklogId=552622=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552622 ] ASF GitHub Bot logged work on HDFS-15821: - Author: ASF GitHub Bot Created on: 15/Feb/21 17:13 Start Date: 15/Feb/21 17:13 Worklog Time Spent: 10m Work Description: jbrennan333 commented on pull request #2690: URL: https://github.com/apache/hadoop/pull/2690#issuecomment-779355481 @zehaoc2 verified that if he builds hadoop before running the failed hadoop-hdfs-rbf tests, it does not fail. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552622) Time Spent: 1h 10m (was: 1h) > Add metrics for in-service datanodes > > > Key: HDFS-15821 > URL: https://issues.apache.org/jira/browse/HDFS-15821 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Zehao Chen >Assignee: Zehao Chen >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > We currently have metrics for live datanodes but some of the datanodes may be > in decommissioning state or maintenance state. Adding this metric allows us > to know how many nodes are currently in service, where NumInServiceDatanodes > = NumLiveDataNodes - NumDecomLiveDataNodes - NumInMaintenanceLiveDataNodes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15821) Add metrics for in-service datanodes
[ https://issues.apache.org/jira/browse/HDFS-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284867#comment-17284867 ] Jim Brennan commented on HDFS-15821: Thanks for the contribution [~zehao.chen]! I have committed this to trunk and will cherry-pick to earlier branches. > Add metrics for in-service datanodes > > > Key: HDFS-15821 > URL: https://issues.apache.org/jira/browse/HDFS-15821 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Zehao Chen >Assignee: Zehao Chen >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > We currently have metrics for live datanodes but some of the datanodes may be > in decommissioning state or maintenance state. Adding this metric allows us > to know how many nodes are currently in service, where NumInServiceDatanodes > = NumLiveDataNodes - NumDecomLiveDataNodes - NumInMaintenanceLiveDataNodes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284837#comment-17284837 ] tomscut commented on HDFS-15808: Hi [~arp], could you please help review the code? Thank you.:) https://github.com/apache/hadoop/pull/2668 > Add metrics for FSNamesystem read/write lock hold long time > --- > > Key: HDFS-15808 > URL: https://issues.apache.org/jira/browse/HDFS-15808 > Project: Hadoop HDFS > Issue Type: Wish > Components: hdfs >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: hdfs, lock, metrics, pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > To monitor how often read/write locks exceed thresholds, we can add two > metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15821) Add metrics for in-service datanodes
[ https://issues.apache.org/jira/browse/HDFS-15821?focusedWorklogId=552624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-552624 ] ASF GitHub Bot logged work on HDFS-15821: - Author: ASF GitHub Bot Created on: 15/Feb/21 17:14 Start Date: 15/Feb/21 17:14 Worklog Time Spent: 10m Work Description: jbrennan333 merged pull request #2690: URL: https://github.com/apache/hadoop/pull/2690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 552624) Time Spent: 1h 20m (was: 1h 10m) > Add metrics for in-service datanodes > > > Key: HDFS-15821 > URL: https://issues.apache.org/jira/browse/HDFS-15821 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Zehao Chen >Assignee: Zehao Chen >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > We currently have metrics for live datanodes but some of the datanodes may be > in decommissioning state or maintenance state. Adding this metric allows us > to know how many nodes are currently in service, where NumInServiceDatanodes > = NumLiveDataNodes - NumDecomLiveDataNodes - NumInMaintenanceLiveDataNodes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
