date:20220725



 [ 
https://issues.apache.org/jira/browse/HADOOP-18362?focusedWorklogId=795125=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795125
 ]

ASF GitHub Bot logged work on HADOOP-18362:
---

Author: ASF GitHub Bot
Created on: 26/Jul/22 05:05
Start Date: 26/Jul/22 05:05
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4625:
URL: https://github.com/apache/hadoop/pull/4625#issuecomment-1195007128

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  12m 48s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ branch-2.10.2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  16m 17s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  compile  |  13m 30s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  11m 19s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  mvnsite  |   1m 46s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  javadoc  |   1m 39s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | -1 :x: |  spotbugs  |   2m 50s | 
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
 |  hadoop-common-project/hadoop-common in branch-2.10.2 has 2 extant spotbugs 
warnings.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  12m 25s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  12m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 16s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  javac  |  11m 16s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 39s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 36s |  |  the patch passed  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  10m 18s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 114m 41s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.ha.TestZKFailoverControllerStress |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4625 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 30a70fbf0c0e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10.2 / 4fe152fb54c3c6ea21aa6fb627362bdef1870f06 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   |  Test Results |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4625: HADOOP-18362. Solve ZKFailoverController throw ambiguous exception



hadoop-yetus commented on PR #4625:
URL: https://github.com/apache/hadoop/pull/4625#issuecomment-1195007128

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  12m 48s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ branch-2.10.2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  16m 17s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  compile  |  13m 30s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  11m 19s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  mvnsite  |   1m 46s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  javadoc  |   1m 39s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | -1 :x: |  spotbugs  |   2m 50s | 
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
 |  hadoop-common-project/hadoop-common in branch-2.10.2 has 2 extant spotbugs 
warnings.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  12m 25s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  12m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 16s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  javac  |  11m 16s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 39s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 36s |  |  the patch passed  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  10m 18s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 114m 41s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.ha.TestZKFailoverControllerStress |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4625 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 30a70fbf0c0e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10.2 / 4fe152fb54c3c6ea21aa6fb627362bdef1870f06 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/testReport/ |
   | Max. process+thread count | 1384 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/3/console |
   | versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
   | Powered by | Apache Yetus

[GitHub] [hadoop] jojochuang commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



jojochuang commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r929524721


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -201,6 +201,7 @@ private void check() {
 iterkey).iterator();
 final List toRemove = new ArrayList<>();
 final List unhealthyDns = new ArrayList<>();
+boolean inValidState = false;

Review Comment:
   IMO the variable name is confusing in the first glance. Is it "valid" or 
"invalid" when it's true? How about renaming it as "isValid"?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18362) Running org.apache.hadoop.ha.TestZKFailoverController when "hadoop.security.groups.cache.secs" is zero or negative numbers will throw ambiguous exception



 [ 
https://issues.apache.org/jira/browse/HADOOP-18362?focusedWorklogId=795122=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795122
 ]

ASF GitHub Bot logged work on HADOOP-18362:
---

Author: ASF GitHub Bot
Created on: 26/Jul/22 04:56
Start Date: 26/Jul/22 04:56
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4625:
URL: https://github.com/apache/hadoop/pull/4625#issuecomment-1195002259

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   8m 31s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ branch-2.10.2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  16m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  compile  |  13m 27s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  11m 17s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  mvnsite  |   1m 45s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  javadoc  |   1m 46s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | -1 :x: |  spotbugs  |   2m 49s | 
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
 |  hadoop-common-project/hadoop-common in branch-2.10.2 has 2 extant spotbugs 
warnings.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  12m 32s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  12m 32s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 22s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  javac  |  11m 22s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  7s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 41s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 24s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 32s |  |  the patch passed  |
    _ Other Tests _ |
   | -1 :x: |  unit  |   8m 59s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 18s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 109m  4s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.ha.TestZKFailoverControllerStress |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4625 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 499dbf9cc29b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10.2 / 4fe152fb54c3c6ea21aa6fb627362bdef1870f06 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   |  Test Results |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4625: HADOOP-18362. Solve ZKFailoverController throw ambiguous exception



hadoop-yetus commented on PR #4625:
URL: https://github.com/apache/hadoop/pull/4625#issuecomment-1195002259

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   8m 31s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ branch-2.10.2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  16m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  compile  |  13m 27s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  11m 17s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  mvnsite  |   1m 45s |  |  branch-2.10.2 passed  |
   | +1 :green_heart: |  javadoc  |   1m 46s |  |  branch-2.10.2 passed with 
JDK Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  branch-2.10.2 passed with 
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | -1 :x: |  spotbugs  |   2m 49s | 
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
 |  hadoop-common-project/hadoop-common in branch-2.10.2 has 2 extant spotbugs 
warnings.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  12m 32s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  12m 32s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 22s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  javac  |  11m 22s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  7s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 41s |  |  the patch passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   1m 24s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 32s |  |  the patch passed  |
    _ Other Tests _ |
   | -1 :x: |  unit  |   8m 59s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 18s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 109m  4s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.ha.TestZKFailoverControllerStress |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4625 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 499dbf9cc29b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10.2 / 4fe152fb54c3c6ea21aa6fb627362bdef1870f06 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/testReport/ |
   | Max. process+thread count | 1370 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4625/2/console |
   | versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
   | Powered by | Apache Yetus

[GitHub] [hadoop] PrabhuJoseph commented on pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



PrabhuJoseph commented on PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#issuecomment-1194967117

   Thanks @ashutoshcipher for the patch. Can you include a test case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] PrabhuJoseph merged pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…



PrabhuJoseph merged PR #4563:
URL: https://github.com/apache/hadoop/pull/4563


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] PrabhuJoseph commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…



PrabhuJoseph commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194965682

   Thanks @KevinWikant for the patch and @ashutoshcipher for additional review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18362) Running org.apache.hadoop.ha.TestZKFailoverController when "hadoop.security.groups.cache.secs" is zero or negative numbers will throw ambiguous exception



 [ 
https://issues.apache.org/jira/browse/HADOOP-18362?focusedWorklogId=795108=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795108
 ]

ASF GitHub Bot logged work on HADOOP-18362:
---

Author: ASF GitHub Bot
Created on: 26/Jul/22 03:06
Start Date: 26/Jul/22 03:06
Worklog Time Spent: 10m 
  Work Description: MEILIDEKCL opened a new pull request, #4625:
URL: https://github.com/apache/hadoop/pull/4625

   
   
   ### Description of PR
   After modifying the ZKFailoverController.java, we will get clear information.
   Running org.apache.hadoop.ha.TestZKFailoverController when 
hadoop.security.groups.cache.secs is zero or negative number, it will throw 
clear Exception like below.
   
   [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
0.904 s <<< FAILURE! - in org.apache.hadoop.ha.TestZKFailoverController
   [ERROR] 
testGracefulFailoverMultipleZKfcs(org.apache.hadoop.ha.TestZKFailoverController)
 Time elapsed: 0.724 s <<< ERROR!
   java.lang.IllegalArgumentException: duration must be positive: 0 MILLISECONDS
   at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
   at 
com.google.common.cache.CacheBuilder.refreshAfterWrite(CacheBuilder.java:633)
   
   ### How was this patch tested?
   
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id 
(e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the 
endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, 
`NOTICE-binary` files?
   
   




Issue Time Tracking
---

Worklog Id: (was: 795108)
Time Spent: 0.5h  (was: 20m)

> Running org.apache.hadoop.ha.TestZKFailoverController when 
> "hadoop.security.groups.cache.secs" is zero or negative numbers will throw 
> ambiguous exception
> -
>
> Key: HADOOP-18362
> URL: https://issues.apache.org/jira/browse/HADOOP-18362
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.10.2
> Environment: Linux version 4.15.0-142-generic 
> (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu 
> 5.4.0-6ubuntu1~16.04.12))
>Reporter: Jingxuan Fu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> {quote}
>   hadoop.security.groups.cache.secs
>   300
>   
>     This is the config controlling the validity of the entries in the cache
>     containing the user->group mapping. When this duration has expired,
>     then the implementation of the group mapping provider is invoked to get
>     the groups of the user and then cached back.
>   
> 
> {quote}
> As we see  in core-default.xml of hadoop.security.groups.cache.secs,  the 
> default value is 300. But when we set it to zero or negative number and  then 
> run 
> org.apache.hadoop.ha.TestZKFailoverController#testGracefulFailoverMultipleZKfcs,
>  it will throw NullPointerException as below：
> {quote}{{[INFO] Running org.apache.hadoop.ha.TestZKFailoverController}}
> {{[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 0.932 s <<< FAILURE! - in org.apache.hadoop.ha.TestZKFailoverController}}
> {{[ERROR] 
> testGracefulFailoverMultipleZKfcs(org.apache.hadoop.ha.TestZKFailoverController)
>   Time elapsed: 0.799 s  <<< ERROR!}}
> {{java.lang.NullPointerException}}
> {{        at 
> org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:188)}}
> {{        at 
> org.apache.hadoop.ha.MiniZKFCCluster.start(MiniZKFCCluster.java:116)}}
> {{        at 
> org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverMultipleZKfcs(TestZKFailoverController.java:581)}}
> {{        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{        at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{        at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{        at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{        at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)}}
> {{        at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{        at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)}}
> {{        at 
>

[jira] [Work logged] (HADOOP-18362) Running org.apache.hadoop.ha.TestZKFailoverController when "hadoop.security.groups.cache.secs" is zero or negative numbers will throw ambiguous exception



 [ 
https://issues.apache.org/jira/browse/HADOOP-18362?focusedWorklogId=795106=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795106
 ]

ASF GitHub Bot logged work on HADOOP-18362:
---

Author: ASF GitHub Bot
Created on: 26/Jul/22 03:00
Start Date: 26/Jul/22 03:00
Worklog Time Spent: 10m 
  Work Description: MEILIDEKCL closed pull request #4625: HADOOP-18362. 
Solve ZKFailoverController throw ambiguous exception 
URL: https://github.com/apache/hadoop/pull/4625




Issue Time Tracking
---

Worklog Id: (was: 795106)
Time Spent: 20m  (was: 10m)

> Running org.apache.hadoop.ha.TestZKFailoverController when 
> "hadoop.security.groups.cache.secs" is zero or negative numbers will throw 
> ambiguous exception
> -
>
> Key: HADOOP-18362
> URL: https://issues.apache.org/jira/browse/HADOOP-18362
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.10.2
> Environment: Linux version 4.15.0-142-generic 
> (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu 
> 5.4.0-6ubuntu1~16.04.12))
>Reporter: Jingxuan Fu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {quote}
>   hadoop.security.groups.cache.secs
>   300
>   
>     This is the config controlling the validity of the entries in the cache
>     containing the user->group mapping. When this duration has expired,
>     then the implementation of the group mapping provider is invoked to get
>     the groups of the user and then cached back.
>   
> 
> {quote}
> As we see  in core-default.xml of hadoop.security.groups.cache.secs,  the 
> default value is 300. But when we set it to zero or negative number and  then 
> run 
> org.apache.hadoop.ha.TestZKFailoverController#testGracefulFailoverMultipleZKfcs,
>  it will throw NullPointerException as below：
> {quote}{{[INFO] Running org.apache.hadoop.ha.TestZKFailoverController}}
> {{[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 0.932 s <<< FAILURE! - in org.apache.hadoop.ha.TestZKFailoverController}}
> {{[ERROR] 
> testGracefulFailoverMultipleZKfcs(org.apache.hadoop.ha.TestZKFailoverController)
>   Time elapsed: 0.799 s  <<< ERROR!}}
> {{java.lang.NullPointerException}}
> {{        at 
> org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:188)}}
> {{        at 
> org.apache.hadoop.ha.MiniZKFCCluster.start(MiniZKFCCluster.java:116)}}
> {{        at 
> org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverMultipleZKfcs(TestZKFailoverController.java:581)}}
> {{        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{        at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{        at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{        at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{        at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)}}
> {{        at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{        at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)}}
> {{        at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{        at 
> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)}}
> {{        at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)}}
> {{        at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)}}
> {{        at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:53)}}
> {{        at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)}}
> {{        at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)}}
> {{        at java.util.concurrent.FutureTask.run(FutureTask.java:266)}}
> {{        at java.lang.Thread.run(Thread.java:748)}}
> {quote}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] MEILIDEKCL closed pull request #4625: HADOOP-18362. Solve ZKFailoverController throw ambiguous exception



MEILIDEKCL closed pull request #4625: HADOOP-18362. Solve ZKFailoverController 
throw ambiguous exception 
URL: https://github.com/apache/hadoop/pull/4625


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ZanderXu commented on pull request #4155: HDFS-16533. COMPOSITE_CRC failed between replicated file and striped file due to invalid requested length



ZanderXu commented on PR #4155:
URL: https://github.com/apache/hadoop/pull/4155#issuecomment-1194902292

   @jojochuang Thank you very much for your review and suggestions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18345) Enhance client protocol to propagate last seen state IDs for multiple nameservices.



 [ 
https://issues.apache.org/jira/browse/HADOOP-18345?focusedWorklogId=795096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795096
 ]

ASF GitHub Bot logged work on HADOOP-18345:
---

Author: ASF GitHub Bot
Created on: 26/Jul/22 02:15
Start Date: 26/Jul/22 02:15
Worklog Time Spent: 10m 
  Work Description: simbadzina commented on PR #4584:
URL: https://github.com/apache/hadoop/pull/4584#issuecomment-1194894453

   Yes, it should be optional. Thanks for spotting the error. I'll fix that.
   I'm considering making the field a byte array in the client protocol. Only
   routers need to parse it.
   
   On Mon, Jul 25, 2022, 16:51 Erik Krogen ***@***.***> wrote:
   
   > ***@***. commented on this pull request.
   > --
   >
   > In hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
   > :
   >
   > > @@ -157,6 +158,7 @@ message RpcResponseHeaderProto {
   >optional bytes clientId = 7; // Globally unique client ID
   >optional sint32 retryCount = 8 [default = -1];
   >optional int64 stateId = 9; // The last written Global State ID
   > +  map nameserviceStateIds = 10; // Last seen state IDs for 
multiple nameservices.
   >
   > Doesn't this field need to be optional for backwards compatibility?
   >
   > —
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   




Issue Time Tracking
---

Worklog Id: (was: 795096)
Time Spent: 1h 40m  (was: 1.5h)

> Enhance client protocol to propagate last seen state IDs for multiple 
> nameservices.
> ---
>
> Key: HADOOP-18345
> URL: https://issues.apache.org/jira/browse/HADOOP-18345
> Project: Hadoop Common
>  Issue Type: New Feature
>Reporter: Simbarashe Dzinamarira
>Assignee: Simbarashe Dzinamarira
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> The RPCHeader in the client protocol currently contains a single value to 
> indicate the last seen state ID for a namenode.
> {noformat}
> optional int64 stateId = 8; // The last seen Global State ID
> {noformat}
> When there are multiple namenodes, such as in router based federation, the 
> headers need to carry the state IDs for each of these nameservices that are 
> part of the federation.
> This change is a prerequisite for HDFS-13522: RBF: Support observer node from 
> Router-Based Federation



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] simbadzina commented on pull request #4584: HADOOP-18345: Enhance client protocol to propagate last seen state IDs for multiple nameservices.



simbadzina commented on PR #4584:
URL: https://github.com/apache/hadoop/pull/4584#issuecomment-1194894453

   Yes, it should be optional. Thanks for spotting the error. I'll fix that.
   I'm considering making the field a byte array in the client protocol. Only
   routers need to parse it.
   
   On Mon, Jul 25, 2022, 16:51 Erik Krogen ***@***.***> wrote:
   
   > ***@***. commented on this pull request.
   > --
   >
   > In hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto
   > :
   >
   > > @@ -157,6 +158,7 @@ message RpcResponseHeaderProto {
   >optional bytes clientId = 7; // Globally unique client ID
   >optional sint32 retryCount = 8 [default = -1];
   >optional int64 stateId = 9; // The last written Global State ID
   > +  map nameserviceStateIds = 10; // Last seen state IDs for 
multiple nameservices.
   >
   > Doesn't this field need to be optional for backwards compatibility?
   >
   > —
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18345) Enhance client protocol to propagate last seen state IDs for multiple nameservices.



 [ 
https://issues.apache.org/jira/browse/HADOOP-18345?focusedWorklogId=795084=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795084
 ]

ASF GitHub Bot logged work on HADOOP-18345:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 23:51
Start Date: 25/Jul/22 23:51
Worklog Time Spent: 10m 
  Work Description: xkrogen commented on code in PR #4584:
URL: https://github.com/apache/hadoop/pull/4584#discussion_r929398786


##
hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto:
##
@@ -157,6 +158,7 @@ message RpcResponseHeaderProto {
   optional bytes clientId = 7; // Globally unique client ID
   optional sint32 retryCount = 8 [default = -1];
   optional int64 stateId = 9; // The last written Global State ID
+  map nameserviceStateIds = 10; // Last seen state IDs for 
multiple nameservices.

Review Comment:
   Doesn't this field need to be `optional` for backwards compatibility?





Issue Time Tracking
---

Worklog Id: (was: 795084)
Time Spent: 1.5h  (was: 1h 20m)

> Enhance client protocol to propagate last seen state IDs for multiple 
> nameservices.
> ---
>
> Key: HADOOP-18345
> URL: https://issues.apache.org/jira/browse/HADOOP-18345
> Project: Hadoop Common
>  Issue Type: New Feature
>Reporter: Simbarashe Dzinamarira
>Assignee: Simbarashe Dzinamarira
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The RPCHeader in the client protocol currently contains a single value to 
> indicate the last seen state ID for a namenode.
> {noformat}
> optional int64 stateId = 8; // The last seen Global State ID
> {noformat}
> When there are multiple namenodes, such as in router based federation, the 
> headers need to carry the state IDs for each of these nameservices that are 
> part of the federation.
> This change is a prerequisite for HDFS-13522: RBF: Support observer node from 
> Router-Based Federation



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] xkrogen commented on a diff in pull request #4584: HADOOP-18345: Enhance client protocol to propagate last seen state IDs for multiple nameservices.



xkrogen commented on code in PR #4584:
URL: https://github.com/apache/hadoop/pull/4584#discussion_r929398786


##
hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto:
##
@@ -157,6 +158,7 @@ message RpcResponseHeaderProto {
   optional bytes clientId = 7; // Globally unique client ID
   optional sint32 retryCount = 8 [default = -1];
   optional int64 stateId = 9; // The last written Global State ID
+  map nameserviceStateIds = 10; // Last seen state IDs for 
multiple nameservices.

Review Comment:
   Doesn't this field need to be `optional` for backwards compatibility?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #4628: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer.



hadoop-yetus commented on PR #4628:
URL: https://github.com/apache/hadoop/pull/4628#issuecomment-1194775595

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 57s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  42m 15s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 52s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   1m 35s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 22s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 47s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 26s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 44s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m  2s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  26m 19s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 24s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 28s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | -1 :x: |  javac  |   1m 28s | 
[/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4628/1/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
 |  
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 1 new + 
937 unchanged - 0 fixed = 938 total (was 937)  |
   | +1 :green_heart: |  compile  |   1m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | -1 :x: |  javac  |   1m 19s | 
[/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4628/1/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
 |  
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new 
+ 916 unchanged - 0 fixed = 917 total (was 916)  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  2s |  |  
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 155 unchanged - 1 
fixed = 155 total (was 156)  |
   | +1 :green_heart: |  mvnsite  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 28s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 34s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  25m 39s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 334m 34s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4628/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 59s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 454m 18s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hdfs.server.mover.TestMover |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4628/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4628 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle

[jira] [Work logged] (HADOOP-18364) All method metrics related to the rpc protocol should be initialized



 [ 
https://issues.apache.org/jira/browse/HADOOP-18364?focusedWorklogId=795082=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795082
 ]

ASF GitHub Bot logged work on HADOOP-18364:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 23:34
Start Date: 25/Jul/22 23:34
Worklog Time Spent: 10m 
  Work Description: xkrogen commented on code in PR #4624:
URL: https://github.com/apache/hadoop/pull/4624#discussion_r929389912


##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java:
##
@@ -71,11 +74,20 @@ public void init(Class protocol) {
 if (protocolCache.contains(protocol)) {
   return;
 }
-protocolCache.add(protocol);
-for (Method method : protocol.getDeclaredMethods()) {
-  String name = method.getName();
-  LOG.debug(name);
-  addMetricIfNotExists(name);
+
+List> protocols = new ArrayList<>();
+protocols.add(protocol);
+if (protocol.getDeclaredMethods().length == 0) {
+  protocols.addAll(Arrays.asList(protocol.getInterfaces()));
+}
+
+for (Class pClass : protocols) {
+  protocolCache.add(pClass);
+  for (Method method : pClass.getDeclaredMethods()) {
+String name = method.getName();
+LOG.debug(name);
+addMetricIfNotExists(name);
+  }

Review Comment:
   Seems we can fix this much more simply by just using 
[getMethods()](https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getMethods--)
 which includes inherited methods. The main difference is that we'll only get 
public methods, not private/package-private, but I don't think we have any use 
cases which need those methods anyway. This is focused on protocol classes for 
which we only need the public methods.
   
   ```suggestion
   protocolCache.add(protocol);
   for (Method method : protocol.getMethods()) {
 String name = method.getName();
 LOG.debug(name);
 addMetricIfNotExists(name);
   ```





Issue Time Tracking
---

Worklog Id: (was: 795082)
Time Spent: 0.5h  (was: 20m)

> All method metrics related to the rpc protocol should be initialized
> 
>
> Key: HADOOP-18364
> URL: https://issues.apache.org/jira/browse/HADOOP-18364
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Shuyan Zhang
>Assignee: Shuyan Zhang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When an RPC protocol is used, the metric of protocol-related methods should 
> be initialized; otherwise, metric information will be incomplete. For 
> example, when we call HAServiceProtocol#monitorHealth(), only the metric of 
> monitorHealth() are initialized, and the metric of transitionToStandby() are 
> still not reported. This incompleteness caused a little trouble for our 
> monitoring system.
> The root cause is that the parameter passed by RpcEngine to 
> MutableRatesWithAggregation#init(java.lang.Class)  is always 
> XXXProtocolPB, which is inherited from BlockingInterface and does not 
> implement any methods. We should fix this bug.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] xkrogen commented on a diff in pull request #4624: HADOOP-18364. All method metrics related to the RPC protocol should be …



xkrogen commented on code in PR #4624:
URL: https://github.com/apache/hadoop/pull/4624#discussion_r929389912


##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java:
##
@@ -71,11 +74,20 @@ public void init(Class protocol) {
 if (protocolCache.contains(protocol)) {
   return;
 }
-protocolCache.add(protocol);
-for (Method method : protocol.getDeclaredMethods()) {
-  String name = method.getName();
-  LOG.debug(name);
-  addMetricIfNotExists(name);
+
+List> protocols = new ArrayList<>();
+protocols.add(protocol);
+if (protocol.getDeclaredMethods().length == 0) {
+  protocols.addAll(Arrays.asList(protocol.getInterfaces()));
+}
+
+for (Class pClass : protocols) {
+  protocolCache.add(pClass);
+  for (Method method : pClass.getDeclaredMethods()) {
+String name = method.getName();
+LOG.debug(name);
+addMetricIfNotExists(name);
+  }

Review Comment:
   Seems we can fix this much more simply by just using 
[getMethods()](https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getMethods--)
 which includes inherited methods. The main difference is that we'll only get 
public methods, not private/package-private, but I don't think we have any use 
cases which need those methods anyway. This is focused on protocol classes for 
which we only need the public methods.
   
   ```suggestion
   protocolCache.add(protocol);
   for (Method method : protocol.getMethods()) {
 String name = method.getName();
 LOG.debug(name);
 addMetricIfNotExists(name);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18190) s3a prefetching streams to collect iostats on prefetching operations



 [ 
https://issues.apache.org/jira/browse/HADOOP-18190?focusedWorklogId=795080=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795080
 ]

ASF GitHub Bot logged work on HADOOP-18190:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 23:31
Start Date: 25/Jul/22 23:31
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4458:
URL: https://github.com/apache/hadoop/pull/4458#issuecomment-1194768803

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 43s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ feature-HADOOP-18028-s3a-prefetch Compile Tests _ |
   | +0 :ok: |  mvndep  |  14m 54s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  26m 24s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  compile  |  24m 57s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  22m  0s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 38s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 33s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  javadoc  |   3m  2s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 33s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  24m  6s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 47s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 18s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 18s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 57s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 57s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   4m 15s |  |  root: The patch generated 
0 new + 0 unchanged - 1 fixed = 0 total (was 1)  |
   | +1 :green_heart: |  mvnsite  |   3m 40s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 42s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 18s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  25m 13s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 58s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 19s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 32s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 252m 12s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/13/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4458 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 24427805232f 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4458: HADOOP-18190. Adds iostats for prefetching



hadoop-yetus commented on PR #4458:
URL: https://github.com/apache/hadoop/pull/4458#issuecomment-1194768803

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 43s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ feature-HADOOP-18028-s3a-prefetch Compile Tests _ |
   | +0 :ok: |  mvndep  |  14m 54s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  26m 24s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  compile  |  24m 57s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  22m  0s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 38s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 33s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  javadoc  |   3m  2s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 33s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  24m  6s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 47s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 18s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 18s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 57s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 57s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   4m 15s |  |  root: The patch generated 
0 new + 0 unchanged - 1 fixed = 0 total (was 1)  |
   | +1 :green_heart: |  mvnsite  |   3m 40s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 42s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 18s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  25m 13s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 58s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 19s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 32s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 252m 12s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/13/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4458 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 24427805232f 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | feature-HADOOP-18028-s3a-prefetch / 
021c98ae94e09a1650111dbb4a81d65f93579b6c |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/13/testReport/ |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4618: YARN-11180. Refactor some code of getNewApplication, submitApplication etc.



slfan1989 commented on code in PR #4618:
URL: https://github.com/apache/hadoop/pull/4618#discussion_r929382545


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -489,13 +488,12 @@ public SubmitApplicationResponse submitApplication(
 }
 
 routerMetrics.incrAppsFailedSubmitted();
-String errMsg = "Application "
-+ request.getApplicationSubmissionContext().getApplicationName()
-+ " with appId " + applicationId + " failed to be submitted.";
-RouterAuditLogger.logFailure(user.getShortUserName(),
-RouterAuditLogger.AuditConstants.SUBMIT_NEW_APP, "UNKNOWN",
-"RouterClientRMService", errMsg, applicationId);
-throw new YarnException(errMsg);
+String msg = String.format("Application %s with appId %s failed to be 
submitted.",
+request.getApplicationSubmissionContext().getApplicationName(), 
applicationId);
+RouterAuditLogger.logFailure(user.getShortUserName(), SUBMIT_NEW_APP, 
UNKNOWN,
+TARGET_CLIENT_RM_SERVICE, msg, applicationId);
+RouterServerUtil.logAndThrowException(msg, null);
+return response;

Review Comment:
   I will modify the code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4618: YARN-11180. Refactor some code of getNewApplication, submitApplication etc.



slfan1989 commented on code in PR #4618:
URL: https://github.com/apache/hadoop/pull/4618#discussion_r929382239


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestFederationClientInterceptor.java:
##
@@ -293,17 +288,15 @@ public void testSubmitApplicationEmptyRequest()
   interceptor.submitApplication(null);
   Assert.fail();

Review Comment:
   Thank you for helping to review the code, I will fix it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17461) Add thread-level IOStatistics Context



 [ 
https://issues.apache.org/jira/browse/HADOOP-17461?focusedWorklogId=795074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795074
 ]

ASF GitHub Bot logged work on HADOOP-17461:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 23:02
Start Date: 25/Jul/22 23:02
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4566:
URL: https://github.com/apache/hadoop/pull/4566#issuecomment-1194744613

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  6s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 20s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  26m  1s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  22m 20s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 18s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  6s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 47s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m  7s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 37s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 25s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 43s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  25m  2s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  25m  2s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m 35s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m 35s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   4m 29s |  |  root: The patch generated 
0 new + 97 unchanged - 1 fixed = 97 total (was 98)  |
   | +1 :green_heart: |  mvnsite  |   3m 13s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 17s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 54s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  24m 37s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 33s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m  2s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 20s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 254m 31s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4566/6/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4566 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 950ed0434155 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 9dd221dda28fd9e7d7ffb5d4605e6b3d89774fef |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private

[GitHub] [hadoop] hadoop-yetus commented on pull request #4566: HADOOP-17461. IOStatisticsContext + committer integration



hadoop-yetus commented on PR #4566:
URL: https://github.com/apache/hadoop/pull/4566#issuecomment-1194744613

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  6s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 20s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  26m  1s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  22m 20s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 18s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  6s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 47s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m  7s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 37s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 25s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 43s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  25m  2s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  25m  2s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m 35s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m 35s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   4m 29s |  |  root: The patch generated 
0 new + 97 unchanged - 1 fixed = 97 total (was 98)  |
   | +1 :green_heart: |  mvnsite  |   3m 13s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 17s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 54s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  24m 37s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 33s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m  2s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 20s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 254m 31s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4566/6/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4566 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 950ed0434155 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 9dd221dda28fd9e7d7ffb5d4605e6b3d89774fef |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4566/6/testReport/ |
   | Max. process+thread count | 1448 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws 
U: . |
   | Console output |

[jira] [Work logged] (HADOOP-18364) All method metrics related to the rpc protocol should be initialized



 [ 
https://issues.apache.org/jira/browse/HADOOP-18364?focusedWorklogId=795069=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795069
 ]

ASF GitHub Bot logged work on HADOOP-18364:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 22:36
Start Date: 25/Jul/22 22:36
Worklog Time Spent: 10m 
  Work Description: sunchao commented on code in PR #4624:
URL: https://github.com/apache/hadoop/pull/4624#discussion_r929362325


##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java:
##
@@ -71,11 +74,20 @@ public void init(Class protocol) {
 if (protocolCache.contains(protocol)) {
   return;
 }
-protocolCache.add(protocol);
-for (Method method : protocol.getDeclaredMethods()) {
-  String name = method.getName();
-  LOG.debug(name);
-  addMetricIfNotExists(name);
+
+List> protocols = new ArrayList<>();
+protocols.add(protocol);
+if (protocol.getDeclaredMethods().length == 0) {

Review Comment:
   make add some comments for this since it's not very obvious why we do it.





Issue Time Tracking
---

Worklog Id: (was: 795069)
Time Spent: 20m  (was: 10m)

> All method metrics related to the rpc protocol should be initialized
> 
>
> Key: HADOOP-18364
> URL: https://issues.apache.org/jira/browse/HADOOP-18364
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Shuyan Zhang
>Assignee: Shuyan Zhang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> When an RPC protocol is used, the metric of protocol-related methods should 
> be initialized; otherwise, metric information will be incomplete. For 
> example, when we call HAServiceProtocol#monitorHealth(), only the metric of 
> monitorHealth() are initialized, and the metric of transitionToStandby() are 
> still not reported. This incompleteness caused a little trouble for our 
> monitoring system.
> The root cause is that the parameter passed by RpcEngine to 
> MutableRatesWithAggregation#init(java.lang.Class)  is always 
> XXXProtocolPB, which is inherited from BlockingInterface and does not 
> implement any methods. We should fix this bug.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] sunchao commented on a diff in pull request #4624: HADOOP-18364. All method metrics related to the RPC protocol should be …



sunchao commented on code in PR #4624:
URL: https://github.com/apache/hadoop/pull/4624#discussion_r929362325


##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java:
##
@@ -71,11 +74,20 @@ public void init(Class protocol) {
 if (protocolCache.contains(protocol)) {
   return;
 }
-protocolCache.add(protocol);
-for (Method method : protocol.getDeclaredMethods()) {
-  String name = method.getName();
-  LOG.debug(name);
-  addMetricIfNotExists(name);
+
+List> protocols = new ArrayList<>();
+protocols.add(protocol);
+if (protocol.getDeclaredMethods().length == 0) {

Review Comment:
   make add some comments for this since it's not very obvious why we do it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18363) Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers.



 [ 
https://issues.apache.org/jira/browse/HADOOP-18363?focusedWorklogId=795064=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795064
 ]

ASF GitHub Bot logged work on HADOOP-18363:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 22:09
Start Date: 25/Jul/22 22:09
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4627:
URL: https://github.com/apache/hadoop/pull/4627#issuecomment-1194693092

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 58s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  41m  6s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 44s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 34s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  4s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 23s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 56s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 56s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 56s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  7s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m 56s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 21s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 228m 14s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4627 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 9966908f556f 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8 
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 55052eb08cbb826d53ee072f48c1baa485085d6b |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/2/testReport/ |
   | Max. process+thread count | 3137 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U:

[jira] [Commented] (HADOOP-18354) Upgrade reload4j due to XXE vulnerability

2022-07-25 Thread Wei-Chiu Chuang (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571116#comment-17571116
 ] 

Wei-Chiu Chuang commented on HADOOP-18354:
--

I don't think we'll have 3.3.9 any time soon :) updated fix version to 3.3.4.

> Upgrade reload4j due to XXE vulnerability
> -
>
> Key: HADOOP-18354
> URL: https://issues.apache.org/jira/browse/HADOOP-18354
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.4
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> https://github.com/qos-ch/reload4j/issues/53 fixed in reload4j 1.2.22



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #4627: HADOOP-18363. Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers



hadoop-yetus commented on PR #4627:
URL: https://github.com/apache/hadoop/pull/4627#issuecomment-1194693092

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 58s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  41m  6s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 44s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 34s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  4s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 23s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 56s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 56s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 56s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  7s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m 56s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 21s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 228m 14s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4627 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 9966908f556f 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8 
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 55052eb08cbb826d53ee072f48c1baa485085d6b |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/2/testReport/ |
   | Max. process+thread count | 3137 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/2/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and

[jira] [Updated] (HADOOP-18354) Upgrade reload4j due to XXE vulnerability

2022-07-25 Thread Wei-Chiu Chuang (Jira)



 [ 
https://issues.apache.org/jira/browse/HADOOP-18354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HADOOP-18354:
-
Fix Version/s: 3.3.4
   (was: 3.3.9)

> Upgrade reload4j due to XXE vulnerability
> -
>
> Key: HADOOP-18354
> URL: https://issues.apache.org/jira/browse/HADOOP-18354
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.4
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> https://github.com/qos-ch/reload4j/issues/53 fixed in reload4j 1.2.22



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.77.Final



 [ 
https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=795058=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795058
 ]

ASF GitHub Bot logged work on HADOOP-18079:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 21:54
Start Date: 25/Jul/22 21:54
Worklog Time Spent: 10m 
  Work Description: jojochuang commented on PR #4592:
URL: https://github.com/apache/hadoop/pull/4592#issuecomment-1194679018

   The shadeclient error does not repro locally.




Issue Time Tracking
---

Worklog Id: (was: 795058)
Time Spent: 5h  (was: 4h 50m)

> Upgrade Netty to 4.1.77.Final
> -
>
> Key: HADOOP-18079
> URL: https://issues.apache.org/jira/browse/HADOOP-18079
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.3.3
>Reporter: Renukaprasad C
>Assignee: Wei-Chiu Chuang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> h4. Netty version - 4.1.71 has fix some CVEs.
> CVE-2019-20444,
> CVE-2019-20445
> CVE-2022-24823
> Upgrade to latest version.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] jojochuang commented on pull request #4592: HADOOP-18079. Upgrade Netty to 4.1.77. (#3977)



jojochuang commented on PR #4592:
URL: https://github.com/apache/hadoop/pull/4592#issuecomment-1194679018

   The shadeclient error does not repro locally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18227) Add input stream IOstats for vectored IO api in S3A.

2022-07-25 Thread Mukund Thakur (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571096#comment-17571096
 ] 

Mukund Thakur commented on HADOOP-18227:


Adding these stats. Any other we need [~ste...@apache.org]  [~mehakmeetSingh] 
 # Add for vectored IO operation started.
 # Add for bytes drained during vectored read.
 # Add for number of http get requests made during vectored reads. This the 
most important one as the optimized vectored read will have less number of http 
get requests. Although this is already captured. 

> Add input stream IOstats for vectored IO api in S3A.
> 
>
> Key: HADOOP-18227
> URL: https://issues.apache.org/jira/browse/HADOOP-18227
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-18227) Add input stream IOstats for vectored IO api in S3A.

2022-07-25 Thread Mukund Thakur (Jira)



 [ 
https://issues.apache.org/jira/browse/HADOOP-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukund Thakur updated HADOOP-18227:
---
Component/s: fs/s3

> Add input stream IOstats for vectored IO api in S3A.
> 
>
> Key: HADOOP-18227
> URL: https://issues.apache.org/jira/browse/HADOOP-18227
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-18227) Add input stream IOstats for vectored IO api in S3A.

2022-07-25 Thread Mukund Thakur (Jira)



 [ 
https://issues.apache.org/jira/browse/HADOOP-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukund Thakur updated HADOOP-18227:
---
Summary: Add input stream IOstats for vectored IO api in S3A.  (was: Add 
IOstats and auditing for vectored IO api.)

> Add input stream IOstats for vectored IO api in S3A.
> 
>
> Key: HADOOP-18227
> URL: https://issues.apache.org/jira/browse/HADOOP-18227
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] jojochuang merged pull request #4155: HDFS-16533. COMPOSITE_CRC failed between replicated file and striped file due to invalid requested length



jojochuang merged PR #4155:
URL: https://github.com/apache/hadoop/pull/4155


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] KevinWikant commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…



KevinWikant commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194540951

   @PrabhuJoseph I have addressed all the checkstyle issues except for the 
SaslRpcClient Indentation issue which, as I mentioned, would result in 
inconsistent indentation if addressed:
   
   ```
 case KERBEROS: {
   if (ugi.getRealAuthenticationMethod().getAuthMethod() !=
   AuthMethod.KERBEROS) {
 LOG.debug("client isn't using kerberos");
 return null;
   }
 final String serverPrincipal;
 try {
   serverPrincipal = getServerPrincipal(authType);
 } catch (IllegalArgumentException ex) {
   // YARN-11210: getServerPrincipal can throw IllegalArgumentException 
if Kerberos
   // configuration is bad, this is surfaced as a non-retryable 
SaslException
   throw new SaslException("Bad Kerberos server principal 
configuration", ex);
 }
   if (serverPrincipal == null) {
 LOG.debug("protocol doesn't use kerberos");
 return null;
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #4623: HDFS-16682. [SBN Read] make estimated transactions configurable



hadoop-yetus commented on PR #4623:
URL: https://github.com/apache/hadoop/pull/4623#issuecomment-1194508990

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 59s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  xmllint  |   0m  1s |  |  xmllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  39m 23s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 44s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   1m 35s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 18s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 36s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 17s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 38s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 40s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  23m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 19s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 22s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |   1m 22s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   1m 23s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 27s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 54s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 38s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 50s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 52s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 432m 28s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4623/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 15s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 545m  6s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4623/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4623 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
   | uname | Linux 5012e0b506cc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / ccf0404c74fad34fdcdd5e98544f5bc93d49e2f1 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4623/1/testReport/ |
   | Max. process+thread count | 2961 (vs. ulimit of 5500) |
   | modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
   | Console output |

[jira] [Commented] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out

2022-07-25 Thread Steve Loughran (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571061#comment-17571061
 ] 

Steve Loughran commented on HADOOP-18366:
-

i haven't seen this BTW

> ITestS3Select.testSelectSeekFullLandsat is timing out
> -
>
> Key: HADOOP-18366
> URL: https://issues.apache.org/jira/browse/HADOOP-18366
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Ahmar Suhail
>Priority: Minor
>
> ITestS3Select.testSelectSeekFullLandsat is timing out. When I run it 
> separately on my IDE, I get the following output repeatedly till I stop the 
> test manually:
>  
> {code:java}
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
> When running the entire test suite on my EC2, I get:
> {code:java}
> Time elapsed: 600.01 s  <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 60 
> milliseconds {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out

2022-07-25 Thread Steve Loughran (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571060#comment-17571060
 ] 

Steve Loughran commented on HADOOP-18366:
-

i blame the networking :)

actually, it could well be, if there is something stopping your VM talking to 
the relevant region

> ITestS3Select.testSelectSeekFullLandsat is timing out
> -
>
> Key: HADOOP-18366
> URL: https://issues.apache.org/jira/browse/HADOOP-18366
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Ahmar Suhail
>Priority: Minor
>
> ITestS3Select.testSelectSeekFullLandsat is timing out. When I run it 
> separately on my IDE, I get the following output repeatedly till I stop the 
> test manually:
>  
> {code:java}
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
> When running the entire test suite on my EC2, I get:
> {code:java}
> Time elapsed: 600.01 s  <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 60 
> milliseconds {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18313) AliyunOSS: AliyunOSSBlockOutputStream should not mark the temporary file for deletion

2022-07-25 Thread Steve Loughran (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571058#comment-17571058
 ] 

Steve Loughran commented on HADOOP-18313:
-

yes, add it

> AliyunOSS: AliyunOSSBlockOutputStream should not mark the temporary file for 
> deletion
> -
>
> Key: HADOOP-18313
> URL: https://issues.apache.org/jira/browse/HADOOP-18313
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/oss
>Affects Versions: 2.10.2, 3.2.4, 3.3.3
>Reporter: wujinhu
>Assignee: wujinhu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> AliyunOSSBlockOutputStream buffers data in local directory before uploading 
> to OSS. It uses LocalDirAllocator.createTmpFileForWrite which will create a 
> temp file on this directory and invoke DeleteOnExitHook.add to add the path 
> to a set and will be deleted on VM exit through a shutdown hook. However, the 
> size of set will be growing if the VM does not exit.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17461) Add thread-level IOStatistics Context



 [ 
https://issues.apache.org/jira/browse/HADOOP-17461?focusedWorklogId=795019=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795019
 ]

ASF GitHub Bot logged work on HADOOP-17461:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 19:00
Start Date: 25/Jul/22 19:00
Worklog Time Spent: 10m 
  Work Description: steveloughran commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1194488631

   ok, one more change, as I write up something on integration. The RawLocal 
streams should support IOContext too.
   
   why so? makes testing integration easier, e.g. for distcp, spark etc. no 
need to wait until object store tests.
   
   This also lines up for moving ITestS3AIOStatisticsContext into hadoop common 
unit tests as a contract test.
   
   I'm not going to make that a requirement of this PR, but for adding abfs in 
we should do that




Issue Time Tracking
---

Worklog Id: (was: 795019)
Time Spent: 9h 10m  (was: 9h)

> Add thread-level IOStatistics Context
> -
>
> Key: HADOOP-17461
> URL: https://issues.apache.org/jira/browse/HADOOP-17461
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs, fs/azure, fs/s3
>Affects Versions: 3.3.1
>Reporter: Steve Loughran
>Assignee: Mehakmeet Singh
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> For effective reporting of the iostatistics of individual worker threads, we 
> need a thread-level context which IO components update.
> * this contact needs to be passed in two background thread forming work on 
> behalf of a task.
> * IO Components (streams, iterators, filesystems) need to update this context 
> statistics as they perform work
> * Without double counting anything.
> I imagine a ThreadLocal IOStatisticContext which will be updated in the 
> FileSystem API Calls. This context MUST be passed into the background threads 
> used by a task, so that IO is correctly aggregated.
> I don't want streams, listIterators  to do the updating as there is more 
> risk of double counting. However, we need to see their statistics if we want 
> to know things like "bytes discarded in backwards seeks". And I don't want to 
> be updating a shared context object on every read() call.
> If all we want is store IO (HEAD, GET, DELETE, list performance etc) then the 
> FS is sufficient. 
> If we do want the stream-specific detail, then I propose
> * caching the context in the constructor
> * updating it only in close() or unbuffer() (as we do from S3AInputStream to 
> S3AInstrumenation)
> * excluding those we know the FS already collects.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #4352: HADOOP-17461. Thread-level IOStatistics in S3A



steveloughran commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1194488631

   ok, one more change, as I write up something on integration. The RawLocal 
streams should support IOContext too.
   
   why so? makes testing integration easier, e.g. for distcp, spark etc. no 
need to wait until object store tests.
   
   This also lines up for moving ITestS3AIOStatisticsContext into hadoop common 
unit tests as a contract test.
   
   I'm not going to make that a requirement of this PR, but for adding abfs in 
we should do that


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] goiri merged pull request #4426: YARN-10883. [Router] Router Audit Log Add Client IP Address.



goiri merged PR #4426:
URL: https://github.com/apache/hadoop/pull/4426


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr…



hadoop-yetus commented on PR #4622:
URL: https://github.com/apache/hadoop/pull/4622#issuecomment-1194469649

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  6s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  1s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  39m 18s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 19s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   1m 14s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m  9s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 20s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 13s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 18s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  6s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 59s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   0m 59s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  1s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 54s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m  2s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m  6s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m  8s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  | 103m 56s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 50s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 205m 48s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4622 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 440845d1b226 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 68691a23a3f5a1c279364e8d489fee8c0734ca76 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/3/testReport/ |
   | Max. process+thread count | 974 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/3/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr…



hadoop-yetus commented on PR #4622:
URL: https://github.com/apache/hadoop/pull/4622#issuecomment-1194465848

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  3s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  39m 28s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   1m  6s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 18s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 12s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  0s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 22s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 59s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 56s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  1s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 56s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   0m 56s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 48s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 13s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 35s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  | 103m 11s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 59s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 205m 23s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4622 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 51f0020eb3df 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 68691a23a3f5a1c279364e8d489fee8c0734ca76 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/2/testReport/ |
   | Max. process+thread count | 939 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4622/2/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] cnauroth merged pull request #4026: MAPREDUCE-7372 MapReduce set permission too late in copyJar method



cnauroth merged PR #4026:
URL: https://github.com/apache/hadoop/pull/4026


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] cnauroth commented on pull request #4026: MAPREDUCE-7372 MapReduce set permission too late in copyJar method



cnauroth commented on PR #4026:
URL: https://github.com/apache/hadoop/pull/4026#issuecomment-1194427833

   I can commit this today.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18190) s3a prefetching streams to collect iostats on prefetching operations



 [ 
https://issues.apache.org/jira/browse/HADOOP-18190?focusedWorklogId=795005=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795005
 ]

ASF GitHub Bot logged work on HADOOP-18190:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 18:02
Start Date: 25/Jul/22 18:02
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4458:
URL: https://github.com/apache/hadoop/pull/4458#issuecomment-1194423651

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 47s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ feature-HADOOP-18028-s3a-prefetch Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m  7s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 27s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  compile  |  26m 39s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  23m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 29s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 21s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  shadedclient  |  25m  0s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 32s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 46s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 31s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 31s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m 23s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   4m 12s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/12/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1)  |
   | +1 :green_heart: |  mvnsite  |   3m 33s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 29s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 25s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 18s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 49s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 11s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 24s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 255m 42s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/12/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4458 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux feb2707df24c 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 
10:50:38 UTC 2022 x86_64 x86_64 x86_64

[GitHub] [hadoop] hadoop-yetus commented on pull request #4458: HADOOP-18190. Adds iostats for prefetching



hadoop-yetus commented on PR #4458:
URL: https://github.com/apache/hadoop/pull/4458#issuecomment-1194423651

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 47s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ feature-HADOOP-18028-s3a-prefetch Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m  7s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 27s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  compile  |  26m 39s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  23m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 29s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 21s |  |  
feature-HADOOP-18028-s3a-prefetch passed with JDK Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m  7s |  |  
feature-HADOOP-18028-s3a-prefetch passed  |
   | +1 :green_heart: |  shadedclient  |  25m  0s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 32s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 46s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 31s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 31s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m 23s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   4m 12s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/12/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1)  |
   | +1 :green_heart: |  mvnsite  |   3m 33s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 29s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 25s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 18s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 49s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 11s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 24s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 255m 42s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4458/12/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4458 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux feb2707df24c 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | feature-HADOOP-18028-s3a-prefetch / 
492b2ef499cbc1778d879a8bd4b27d2dc1a44109 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private

[GitHub] [hadoop] hadoop-yetus commented on pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



hadoop-yetus commented on PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#issuecomment-1194417858

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 38s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  38m 12s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 43s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   1m 40s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 23s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 46s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 25s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 49s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 39s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  23m  6s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 24s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 27s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |   1m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 20s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   1m 20s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  1s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 57s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 30s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 18s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m 35s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  | 243m 29s |  |  hadoop-hdfs in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 14s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 353m 28s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4626/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4626 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 31b1acdfdf9e 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 28c374cf535405254f4183528864014ea5776fc8 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4626/1/testReport/ |
   | Max. process+thread count | 3405 (vs. ulimit of 5500) |
   | modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4626/1/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019



 [ 
https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=794997=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794997
 ]

ASF GitHub Bot logged work on HADOOP-18357:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 17:48
Start Date: 25/Jul/22 17:48
Worklog Time Spent: 10m 
  Work Description: goiri commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929146402


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -39,12 +37,14 @@
 false
 true
 Unicode
+v142

Review Comment:
   Can we check if it was inherited from somewhere else?
   This may trigger inconsistencies.





Issue Time Tracking
---

Worklog Id: (was: 794997)
Time Spent: 1h 10m  (was: 1h)

> Retarget solution file to VS2019
> 
>
> Key: HADOOP-18357
> URL: https://issues.apache.org/jira/browse/HADOOP-18357
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.4.0
> Environment: Windows 10
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
>  Labels: libhdfscpp, pull-request-available
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The Visual Studio version used by winutils and native components in Hadoop 
> common are quite old. We need to retarget the solution and vcxproj files to 
> use the latest version (Visual Studio 2019 as of this writing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] goiri commented on a diff in pull request #4616: HADOOP-18357. Retarget solution file to VS2019



goiri commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929146402


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -39,12 +37,14 @@
 false
 true
 Unicode
+v142

Review Comment:
   Can we check if it was inherited from somewhere else?
   This may trigger inconsistencies.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] GauthamBanasandra merged pull request #4615: HDFS-16681. Do not pass GCC flags for MSVC in libhdfspp



GauthamBanasandra merged PR #4615:
URL: https://github.com/apache/hadoop/pull/4615


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019



 [ 
https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=794988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794988
 ]

ASF GitHub Bot logged work on HADOOP-18357:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 17:24
Start Date: 25/Jul/22 17:24
Worklog Time Spent: 10m 
  Work Description: GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929126701


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj.filters:
##
@@ -48,19 +46,16 @@
 
   Source Files
 
-
-  Source Files
-
-
+
   Source Files
 
-
+

Review Comment:
   These changes were automatically made by Visual Studio when I ran the 
retarget command.
   
   The `lz4` library was removed in https://github.com/apache/hadoop/pull/2576. 
The author didn't remove them from Visual Studio, so I removed them - 
https://github.com/apache/hadoop/commit/44910b53f383f8ab092b7f9889fbf353e057d3f7.
 But I forgot to remove them from `native.vcxproj.filters` file (the current 
file). When I retargeted the native project, Visual Studio was smart enough to 
detect that these files don't exist and has removed them for us.





Issue Time Tracking
---

Worklog Id: (was: 794988)
Time Spent: 1h  (was: 50m)

> Retarget solution file to VS2019
> 
>
> Key: HADOOP-18357
> URL: https://issues.apache.org/jira/browse/HADOOP-18357
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.4.0
> Environment: Windows 10
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
>  Labels: libhdfscpp, pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The Visual Studio version used by winutils and native components in Hadoop 
> common are quite old. We need to retarget the solution and vcxproj files to 
> use the latest version (Visual Studio 2019 as of this writing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4616: HADOOP-18357. Retarget solution file to VS2019



GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929126701


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj.filters:
##
@@ -48,19 +46,16 @@
 
   Source Files
 
-
-  Source Files
-
-
+
   Source Files
 
-
+

Review Comment:
   These changes were automatically made by Visual Studio when I ran the 
retarget command.
   
   The `lz4` library was removed in https://github.com/apache/hadoop/pull/2576. 
The author didn't remove them from Visual Studio, so I removed them - 
https://github.com/apache/hadoop/commit/44910b53f383f8ab092b7f9889fbf353e057d3f7.
 But I forgot to remove them from `native.vcxproj.filters` file (the current 
file). When I retargeted the native project, Visual Studio was smart enough to 
detect that these files don't exist and has removed them for us.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019



 [ 
https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=794987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794987
 ]

ASF GitHub Bot logged work on HADOOP-18357:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 17:18
Start Date: 25/Jul/22 17:18
Worklog Time Spent: 10m 
  Work Description: GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929121380


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -142,16 +140,16 @@
 
   /D 
HADOOP_ISAL_LIBRARY=L\"isa-l.dll\"
 
-
-
-
-
-
-
-
-
-
-
+

Review Comment:
   Sure, I'll create a separate PR for formatting changes.





Issue Time Tracking
---

Worklog Id: (was: 794987)
Time Spent: 50m  (was: 40m)

> Retarget solution file to VS2019
> 
>
> Key: HADOOP-18357
> URL: https://issues.apache.org/jira/browse/HADOOP-18357
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.4.0
> Environment: Windows 10
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
>  Labels: libhdfscpp, pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The Visual Studio version used by winutils and native components in Hadoop 
> common are quite old. We need to retarget the solution and vcxproj files to 
> use the latest version (Visual Studio 2019 as of this writing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4616: HADOOP-18357. Retarget solution file to VS2019



GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929121380


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -142,16 +140,16 @@
 
   /D 
HADOOP_ISAL_LIBRARY=L\"isa-l.dll\"
 
-
-
-
-
-
-
-
-
-
-
+

Review Comment:
   Sure, I'll create a separate PR for formatting changes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019



 [ 
https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=794986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794986
 ]

ASF GitHub Bot logged work on HADOOP-18357:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 17:17
Start Date: 25/Jul/22 17:17
Worklog Time Spent: 10m 
  Work Description: GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929120970


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -39,12 +37,14 @@
 false
 true
 Unicode
+v142

Review Comment:
   I think the old number should've been `v100`. I'm not sure why it wasn't 
defined. Visual Studio automatically added this when I ran the command to 
retarget the project files.





Issue Time Tracking
---

Worklog Id: (was: 794986)
Time Spent: 40m  (was: 0.5h)

> Retarget solution file to VS2019
> 
>
> Key: HADOOP-18357
> URL: https://issues.apache.org/jira/browse/HADOOP-18357
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.4.0
> Environment: Windows 10
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
>  Labels: libhdfscpp, pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The Visual Studio version used by winutils and native components in Hadoop 
> common are quite old. We need to retarget the solution and vcxproj files to 
> use the latest version (Visual Studio 2019 as of this writing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4616: HADOOP-18357. Retarget solution file to VS2019



GauthamBanasandra commented on code in PR #4616:
URL: https://github.com/apache/hadoop/pull/4616#discussion_r929120970


##
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj:
##
@@ -39,12 +37,14 @@
 false
 true
 Unicode
+v142

Review Comment:
   I think the old number should've been `v100`. I'm not sure why it wasn't 
defined. Visual Studio automatically added this when I ran the command to 
retarget the project files.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18363) Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers.



 [ 
https://issues.apache.org/jira/browse/HADOOP-18363?focusedWorklogId=794985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794985
 ]

ASF GitHub Bot logged work on HADOOP-18363:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 17:09
Start Date: 25/Jul/22 17:09
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4627:
URL: https://github.com/apache/hadoop/pull/4627#issuecomment-1194365408

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  0s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  41m  4s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 18s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 52s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 31s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 30s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  2s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  26m  4s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  7s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 20s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 20s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m  3s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m  3s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 57s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  4s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  8s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m  3s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  18m 20s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 227m 15s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.metrics2.impl.TestGangliaMetrics |
   |   | hadoop.metrics2.sink.ganglia.TestGangliaSink |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4627 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 2ef55287e20f 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8 
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / ac10b5cc62306e82e50dfbec8bf03c5e8eeda7de |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private

[GitHub] [hadoop] hadoop-yetus commented on pull request #4627: HADOOP-18363. Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers



hadoop-yetus commented on PR #4627:
URL: https://github.com/apache/hadoop/pull/4627#issuecomment-1194365408

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  0s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  41m  4s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 18s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 52s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 31s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 30s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  2s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  26m  4s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  7s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 20s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 20s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m  3s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  22m  3s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 57s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m  4s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  8s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m  3s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  18m 20s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 17s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 227m 15s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.metrics2.impl.TestGangliaMetrics |
   |   | hadoop.metrics2.sink.ganglia.TestGangliaSink |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4627 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 2ef55287e20f 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8 
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / ac10b5cc62306e82e50dfbec8bf03c5e8eeda7de |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4627/1/testReport/ |
   | Max. process+thread count | 2264 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
   | Console output |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…



hadoop-yetus commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194363279

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 50s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  14m 54s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 11s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m  9s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 47s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   4m 39s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   3m 59s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   3m 24s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   7m 40s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m 14s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 33s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 11s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 11s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 53s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 53s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   4m 22s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 6 new + 229 unchanged - 3 fixed = 235 total (was 
232)  |
   | +1 :green_heart: |  mvnsite  |   4m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   3m 49s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   3m 16s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   8m 18s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m 22s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 55s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   1m 31s |  |  hadoop-yarn-api in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 20s |  |  hadoop-yarn-common in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 12s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 269m 59s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4563 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 3b4ee9e513bd 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 28b2c47600cc36e20a866efbf5127228f7582735 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/testReport/ |
   | Max. process+thread count | 3134 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common

[GitHub] [hadoop] goiri merged pull request #4610: YARN-11161. Support getAttributesToNodes, getClusterNodeAttributes, getNodesToAttributes API's for Federation



goiri merged PR #4610:
URL: https://github.com/apache/hadoop/pull/4610


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] goiri commented on a diff in pull request #4594: YARN-6572. Refactoring Router services to use common util classes for pipeline creations.



goiri commented on code in PR #4594:
URL: https://github.com/apache/hadoop/pull/4594#discussion_r929108932


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/RouterWebServices.java:
##
@@ -206,50 +179,20 @@ protected Map 
getPipelines() {
*/
   @VisibleForTesting
   protected RESTRequestInterceptor createRequestInterceptorChain() {
-
-List interceptorClassNames = getInterceptorClassNames(conf);
-
-RESTRequestInterceptor pipeline = null;
-RESTRequestInterceptor current = null;
-for (String interceptorClassName : interceptorClassNames) {
-  try {
-Class interceptorClass = conf.getClassByName(interceptorClassName);
-if (RESTRequestInterceptor.class.isAssignableFrom(interceptorClass)) {
-  RESTRequestInterceptor interceptorInstance =
-  (RESTRequestInterceptor) ReflectionUtils
-  .newInstance(interceptorClass, conf);
-  if (pipeline == null) {
-pipeline = interceptorInstance;
-current = interceptorInstance;
-continue;
-  } else {
-current.setNextInterceptor(interceptorInstance);
-current = interceptorInstance;
-  }
-} else {
-  throw new YarnRuntimeException(
-  "Class: " + interceptorClassName + " not instance of "
-  + RESTRequestInterceptor.class.getCanonicalName());
-}
-  } catch (ClassNotFoundException e) {
-throw new YarnRuntimeException(
-"Could not instantiate RESTRequestInterceptor: "
-+ interceptorClassName,
-e);
-  }
-}
-
-if (pipeline == null) {
-  throw new YarnRuntimeException(
-  "RequestInterceptor pipeline is not configured in the system");
+try {
+  return RouterServerUtil.createRequestInterceptorChain(conf,
+  YarnConfiguration.ROUTER_WEBAPP_INTERCEPTOR_CLASS_PIPELINE,
+  YarnConfiguration.DEFAULT_ROUTER_WEBAPP_INTERCEPTOR_CLASS,
+  RESTRequestInterceptor.class);
+} catch (YarnRuntimeException ex) {
+  throw new YarnRuntimeException("Create RequestInterceptor Chain error.", 
ex);

Review Comment:
   We catch the exception to throw it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] goiri commented on a diff in pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.



goiri commented on code in PR #4614:
URL: https://github.com/apache/hadoop/pull/4614#discussion_r929104887


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -958,7 +956,7 @@ public NodesInfo getNodes(String states) {
 try {
   subClustersActive = getActiveSubclusters();
 } catch (Exception e) {
-  LOG.error("Cannot get nodes: {}", e.getMessage());
+  LOG.error("Cannot get nodes: {}", e.getMessage(), e);

Review Comment:
   Do we need the full stack trace?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] goiri commented on a diff in pull request #4618: YARN-11180. Refactor some code of getNewApplication, submitApplication etc.



goiri commented on code in PR #4618:
URL: https://github.com/apache/hadoop/pull/4618#discussion_r929101762


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -280,47 +281,47 @@ private SubClusterId getRandomActiveSubCluster(
   public GetNewApplicationResponse getNewApplication(
   GetNewApplicationRequest request) throws YarnException, IOException {
 
-long startTime = clock.getTime();
+if (request == null) {
+  routerMetrics.incrAppsFailedCreated();
+  String errMsg = "Missing getNewApplication request.";
+  RouterAuditLogger.logFailure(user.getShortUserName(), GET_NEW_APP, 
UNKNOWN,
+  TARGET_CLIENT_RM_SERVICE, errMsg);
+  RouterServerUtil.logAndThrowException(errMsg, null);
+}
 
+long startTime = clock.getTime();
 Map subClustersActive =
 federationFacade.getSubClusters(true);
 
+GetNewApplicationResponse response = null;

Review Comment:
   Why not leave the initialization where it was?



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -227,35 +234,29 @@ protected ApplicationClientProtocol 
getClientRMProxyForSubCluster(
 ApplicationClientProtocol clientRMProxy = null;
 try {
   boolean serviceAuthEnabled = getConf().getBoolean(
-  CommonConfigurationKeys.HADOOP_SECURITY_AUTHORIZATION, false);
+  CommonConfigurationKeys.HADOOP_SECURITY_AUTHORIZATION, false);
   UserGroupInformation realUser = user;
   if (serviceAuthEnabled) {
 realUser = UserGroupInformation.createProxyUser(
-user.getShortUserName(), UserGroupInformation.getLoginUser());
+user.getShortUserName(), UserGroupInformation.getLoginUser());
   }
   clientRMProxy = FederationProxyProviderUtil.createRMProxy(getConf(),
   ApplicationClientProtocol.class, subClusterId, realUser);
 } catch (Exception e) {
   RouterServerUtil.logAndThrowException(
-  "Unable to create the interface to reach the SubCluster "
-  + subClusterId,
-  e);
+  "Unable to create the interface to reach the SubCluster " + 
subClusterId, e);
 }
-
 clientRMProxies.put(subClusterId, clientRMProxy);
 return clientRMProxy;
   }
 
   private SubClusterId getRandomActiveSubCluster(
-  Map activeSubclusters)
-  throws YarnException {
-
-if (activeSubclusters == null || activeSubclusters.size() < 1) {
+  Map activeSubClusters) throws 
YarnException {
+if (activeSubClusters.isEmpty()) {

Review Comment:
   Shouldn't we check for null?



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestFederationClientInterceptor.java:
##
@@ -293,17 +288,15 @@ public void testSubmitApplicationEmptyRequest()
   interceptor.submitApplication(null);
   Assert.fail();

Review Comment:
   LambdaTestUtils#intercep



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -489,13 +488,12 @@ public SubmitApplicationResponse submitApplication(
 }
 
 routerMetrics.incrAppsFailedSubmitted();
-String errMsg = "Application "
-+ request.getApplicationSubmissionContext().getApplicationName()
-+ " with appId " + applicationId + " failed to be submitted.";
-RouterAuditLogger.logFailure(user.getShortUserName(),
-RouterAuditLogger.AuditConstants.SUBMIT_NEW_APP, "UNKNOWN",
-"RouterClientRMService", errMsg, applicationId);
-throw new YarnException(errMsg);
+String msg = String.format("Application %s with appId %s failed to be 
submitted.",
+request.getApplicationSubmissionContext().getApplicationName(), 
applicationId);
+RouterAuditLogger.logFailure(user.getShortUserName(), SUBMIT_NEW_APP, 
UNKNOWN,
+TARGET_CLIENT_RM_SERVICE, msg, applicationId);
+RouterServerUtil.logAndThrowException(msg, null);
+return response;

Review Comment:
   This pattern is very weird.
   There is no way to let know that we always throw the exception?



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -280,47 +274,50 @@ private SubClusterId getRandomActiveSubCluster(
   public GetNewApplicationResponse getNewApplication(
   GetNewApplicationRequest request) throws YarnException, IOException {
 
-long startTime = clock.getTime();
+if(request == null) {
+  routerMetrics.incrAppsFailedCreated();
+  String errMsg = "Missing

[GitHub] [hadoop] goiri commented on pull request #4426: YARN-10883. [Router] Router Audit Log Add Client IP Address.



goiri commented on PR #4426:
URL: https://github.com/apache/hadoop/pull/4426#issuecomment-1194345556

   @ayushtkn do you mind taking a look at this too?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18342) Upgrade to Avro 1.11.0



 [ 
https://issues.apache.org/jira/browse/HADOOP-18342?focusedWorklogId=794961=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794961
 ]

ASF GitHub Bot logged work on HADOOP-18342:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 16:20
Start Date: 25/Jul/22 16:20
Worklog Time Spent: 10m 
  Work Description: pjfanning closed pull request #4581: HADOOP-18342: 
upgrade to avro 1.11.0
URL: https://github.com/apache/hadoop/pull/4581




Issue Time Tracking
---

Worklog Id: (was: 794961)
Time Spent: 40m  (was: 0.5h)

> Upgrade to Avro 1.11.0
> --
>
> Key: HADOOP-18342
> URL: https://issues.apache.org/jira/browse/HADOOP-18342
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Latest version of Avro. Aimed only at trunk as there is no security concern 
> addressed here.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18342) Upgrade to Avro 1.11.0



 [ 
https://issues.apache.org/jira/browse/HADOOP-18342?focusedWorklogId=794965=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794965
 ]

ASF GitHub Bot logged work on HADOOP-18342:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 16:22
Start Date: 25/Jul/22 16:22
Worklog Time Spent: 10m 
  Work Description: xkrogen commented on PR #4581:
URL: https://github.com/apache/hadoop/pull/4581#issuecomment-1194312076

   Just to be clear, I am not generally -1 on Avro upgrades. #4580 would have 
bumped the Avro major version in a maintenance release of Hadoop; a major Avro 
version bump is way too big of a change to go into a maintenance release. I 
would not take the same stack for an upgrade in trunk, which would go into a 
minor release.
   
   Though notably Avro 1.11.0 has at least one major performance issue, which 
we have found to be severe enough internally at LinkedIn to specifically 
disallow use of this version: 
[AVRO-3243](https://issues.apache.org/jira/browse/AVRO-3243). So I would 
recommend that we avoid any Avro 1.11 adoption until 1.11.1 is available.




Issue Time Tracking
---

Worklog Id: (was: 794965)
Time Spent: 50m  (was: 40m)

> Upgrade to Avro 1.11.0
> --
>
> Key: HADOOP-18342
> URL: https://issues.apache.org/jira/browse/HADOOP-18342
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Latest version of Avro. Aimed only at trunk as there is no security concern 
> addressed here.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] xkrogen commented on pull request #4581: HADOOP-18342: upgrade to avro 1.11.0



xkrogen commented on PR #4581:
URL: https://github.com/apache/hadoop/pull/4581#issuecomment-1194312076

   Just to be clear, I am not generally -1 on Avro upgrades. #4580 would have 
bumped the Avro major version in a maintenance release of Hadoop; a major Avro 
version bump is way too big of a change to go into a maintenance release. I 
would not take the same stack for an upgrade in trunk, which would go into a 
minor release.
   
   Though notably Avro 1.11.0 has at least one major performance issue, which 
we have found to be severe enough internally at LinkedIn to specifically 
disallow use of this version: 
[AVRO-3243](https://issues.apache.org/jira/browse/AVRO-3243). So I would 
recommend that we avoid any Avro 1.11 adoption until 1.11.1 is available.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-18342) Upgrade to Avro 1.11.0



 [ 
https://issues.apache.org/jira/browse/HADOOP-18342?focusedWorklogId=794960=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794960
 ]

ASF GitHub Bot logged work on HADOOP-18342:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 16:20
Start Date: 25/Jul/22 16:20
Worklog Time Spent: 10m 
  Work Description: pjfanning commented on PR #4581:
URL: https://github.com/apache/hadoop/pull/4581#issuecomment-1194309623

   I'll close this then




Issue Time Tracking
---

Worklog Id: (was: 794960)
Time Spent: 0.5h  (was: 20m)

> Upgrade to Avro 1.11.0
> --
>
> Key: HADOOP-18342
> URL: https://issues.apache.org/jira/browse/HADOOP-18342
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Latest version of Avro. Aimed only at trunk as there is no security concern 
> addressed here.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] pjfanning closed pull request #4581: HADOOP-18342: upgrade to avro 1.11.0



pjfanning closed pull request #4581: HADOOP-18342: upgrade to avro 1.11.0
URL: https://github.com/apache/hadoop/pull/4581


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] pjfanning commented on pull request #4581: HADOOP-18342: upgrade to avro 1.11.0



pjfanning commented on PR #4581:
URL: https://github.com/apache/hadoop/pull/4581#issuecomment-1194309623

   I'll close this then


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ZanderXu opened a new pull request, #4628: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer.

ZanderXu opened a new pull request, #4628:
URL: https://github.com/apache/hadoop/pull/4628

### Description of PR
Standby NameNode crashes when transitioning to Active with a in-progress
tailer.
And the error message as blew:
```java
Caused by: java.lang.IllegalStateException: Cannot start writing at txid X
when there is a stream available for read: ByteStringEditLog[X, Y],
ByteStringEditLog[X, 0]
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.openForWrite(FSEditLog.java:344)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.openForWrite(FSEditLogAsync.java:113)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:1423)
at
org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:2132)
... 36 more
```

After tracing and found there is a critical bug in
`EditlogTailer#catchupDuringFailover()` when `DFS_HA_TAILEDITS_INPROGRESS_KEY`
is true. Because `catchupDuringFailover()` try to replay all edits with
`onlyDurableTxns` is true. It may cannot replay any edits when they are some
abnormal JournalNodes.

Reproduce method, suppose:

- There are 2 namenode, namely NN0 and NN1, and the status of echo namenode
is Active, Standby respectively. And there are 3 JournalNodes, namely JN0, JN1
and JN2.
- NN0 try to sync 3 edits to JNs with started txid 3, but only successfully
synced them to JN1 and JN3. And JN0 is abnormal, such as GC, bad network or
restarted.
- NN1's lastAppliedTxId is 2, and at the moment, we are trying failover
active from NN0 to NN1.
- NN1 only got two responses from JN0 and JN1 when it try to selecting
inputStreams with `fromTxnId=3` and `onlyDurableTxns=true`, and the count txid
of response is 0, 3 respectively. JN2 is abnormal, such as GC, bad network or
restarted.
- NN1 will cannot replay any Edits with `fromTxnId=3` from JournalNodes
because the `maxAllowedTxns` is 0.

So I think Standby NameNode should `catchupDuringFailover()` with
`onlyDurableTxns=true` , so that it can replay all missed edits from
JournalNode.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4314: YARN-11153. Make proxy server support YARN federation.



slfan1989 commented on code in PR #4314:
URL: https://github.com/apache/hadoop/pull/4314#discussion_r928791799


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestWebAppProxyServletFed.java:
##
@@ -0,0 +1,366 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.webproxy;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.HttpURLConnection;
+import java.net.URI;
+import java.net.URL;
+
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.http.HttpServer2;
+import org.apache.hadoop.security.authorize.AccessControlList;
+import org.apache.hadoop.service.CompositeService;
+import org.apache.hadoop.util.StringUtils;
+import org.apache.hadoop.yarn.api.ApplicationClientProtocol;
+import org.apache.hadoop.yarn.api.ApplicationHistoryProtocol;
+import org.apache.hadoop.yarn.api.protocolrecords.GetApplicationReportRequest;
+import org.apache.hadoop.yarn.api.protocolrecords.GetApplicationReportResponse;
+import org.apache.hadoop.yarn.api.records.ApplicationId;
+import org.apache.hadoop.yarn.api.records.ApplicationReport;
+import org.apache.hadoop.yarn.api.records.YarnApplicationState;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
+import org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException;
+import 
org.apache.hadoop.yarn.server.federation.store.records.ApplicationHomeSubCluster;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterInfo;
+import 
org.apache.hadoop.yarn.server.federation.store.records.SubClusterRegisterRequest;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterState;
+import 
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade;
+import org.apache.hadoop.yarn.webapp.util.WebAppUtils;
+import org.eclipse.jetty.server.Server;
+import org.eclipse.jetty.server.ServerConnector;
+import org.eclipse.jetty.servlet.ServletContextHandler;
+import org.eclipse.jetty.servlet.ServletHolder;
+import org.eclipse.jetty.util.thread.QueuedThreadPool;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Test the WebAppProxyServlet and WebAppProxy. For back end use simple web 
server.
+ */
+public class TestWebAppProxyServletFed {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(TestWebAppProxyServletFed.class);
+
+  public static final String AM_PREFIX = "AM";
+  public static final String RM_PREFIX = "RM";
+  public static final String AHS_PREFIX = "AHS";
+
+  /*
+  * Mocked Server is used for simulating the web of AppMaster, 
ResourceMamanger or TimelineServer.
+  * */
+  private static Server mockServer;
+  private static int mockServerPort = 0;
+
+  /**
+   * Simple http server. Server should send answer with status 200
+   */
+  @BeforeClass
+  public static void setUp() throws Exception {
+mockServer = new Server(0);
+((QueuedThreadPool) mockServer.getThreadPool()).setMaxThreads(20);
+ServletContextHandler context = new ServletContextHandler();
+context.setContextPath("/");
+context.addServlet(new ServletHolder(new MockWebServlet(AM_PREFIX)), 
"/amweb/*");
+context.addServlet(new ServletHolder(new MockWebServlet(RM_PREFIX)), 
"/cluster/app/*");
+context.addServlet(new ServletHolder(new MockWebServlet(AHS_PREFIX)),
+"/applicationhistory/app/*");
+mockServer.setHandler(context);
+
+((ServerConnector) mockServer.getConnectors()[0]).setHost("localhost");
+mockServer.start();
+mockServerPort = ((ServerConnector) 
mockServer.getConnectors()[0]).getLocalPort();
+LOG.info("Running embedded servlet container at: http://localhost:; + 
mockServerPort);
+  }
+
+  @AfterClass
+  public static void tearDown() throws

[jira] [Work logged] (HADOOP-17461) Add thread-level IOStatistics Context



 [ 
https://issues.apache.org/jira/browse/HADOOP-17461?focusedWorklogId=794943=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794943
 ]

ASF GitHub Bot logged work on HADOOP-17461:
---

Author: ASF GitHub Bot
Created on: 25/Jul/22 15:16
Start Date: 25/Jul/22 15:16
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1194187132

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 53s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  14m 48s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 47s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 18s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 13s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 23s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 39s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  24m 44s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 11s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 26s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 25s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 51s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 51s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   4m 24s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4352/15/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 1 new + 77 unchanged - 0 fixed = 78 total (was 
77)  |
   | +1 :green_heart: |  mvnsite  |   3m 11s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 15s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  8s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 25s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  27m  1s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 56s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m  8s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 18s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 253m 20s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4352/15/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4352 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 76bd37db2c7b 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / cdc9915d9c52644d35057844c101653858ed10eb |
   |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4352: HADOOP-17461. Thread-level IOStatistics in S3A



hadoop-yetus commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1194187132

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 53s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 6 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  14m 48s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 47s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  25m 18s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |  21m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   4m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 13s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 23s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   4m 39s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  24m 44s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  25m 11s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 26s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 42s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 25s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |  24m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 51s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  21m 51s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   4m 24s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4352/15/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 1 new + 77 unchanged - 0 fixed = 78 total (was 
77)  |
   | +1 :green_heart: |  mvnsite  |   3m 11s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 15s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   2m  8s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 25s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  27m  1s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  18m 56s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m  8s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m 18s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 253m 20s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4352/15/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4352 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 76bd37db2c7b 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / cdc9915d9c52644d35057844c101653858ed10eb |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4352/15/testReport/ |
   | Max. process+thread count | 1263 (vs. ulimit of

[GitHub] [hadoop] ashutoshcipher commented on pull request #4618: YARN-11180. Refactor some code of getNewApplication, submitApplication etc.



ashutoshcipher commented on PR #4618:
URL: https://github.com/apache/hadoop/pull/4618#issuecomment-1194146730

   LGTM +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#issuecomment-1194143412

   Thanks @slfan1989 for your review and approval :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928961283


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -201,6 +201,7 @@ private void check() {
 iterkey).iterator();
 final List toRemove = new ArrayList<>();
 final List unhealthyDns = new ArrayList<>();
+boolean inValidState = false;
 
 while (it.hasNext() && !exceededNumBlocksPerCheck() && namesystem

Review Comment:
   This change hasnt made my me. For fixing any such existing `Single Line` 
issues - new JIRA can be created to clean or modify such issues on module 
level. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928963626


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -265,6 +266,7 @@ private void check() {
   // to track maintenance expiration.
   dnAdmin.setInMaintenance(dn);
 } else {
+  inValidState  = true;
   Preconditions.checkState(false,

Review Comment:
   >"Node %s is in an invalid state! " + "Invalid state: %s %s blocks are on 
this dn."
   
   I am not sure if there is really an indentation issue here. I have not added 
this change here. To correct existing indentations, a separate JIRA can be 
created to track and correct this on module level if required.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928961283


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -201,6 +201,7 @@ private void check() {
 iterkey).iterator();
 final List toRemove = new ArrayList<>();
 final List unhealthyDns = new ArrayList<>();
+boolean inValidState = false;
 
 while (it.hasNext() && !exceededNumBlocksPerCheck() && namesystem

Review Comment:
   This change hasnt made my me. For fixing any such issues - new JIRA can be 
created to clean or modify such issues on module level. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.



slfan1989 commented on PR #4614:
URL: https://github.com/apache/hadoop/pull/4614#issuecomment-1194129033

   @goiri please help me review the code again, thank you very much!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on pull request #4618: YARN-11180. Refactor some code of getNewApplication, submitApplication etc.



slfan1989 commented on PR #4618:
URL: https://github.com/apache/hadoop/pull/4618#issuecomment-1194126724

   @goiri Please help me review the code again, thank you very much.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



slfan1989 commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928949967


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -201,6 +201,7 @@ private void check() {
 iterkey).iterator();
 final List toRemove = new ArrayList<>();
 final List unhealthyDns = new ArrayList<>();
+boolean inValidState = false;
 
 while (it.hasNext() && !exceededNumBlocksPerCheck() && namesystem

Review Comment:
   `while (it.hasNext() && !exceededNumBlocksPerCheck() && 
namesystem.isRunning())` 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



slfan1989 commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928949038


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -265,6 +266,7 @@ private void check() {
   // to track maintenance expiration.
   dnAdmin.setInMaintenance(dn);
 } else {
+  inValidState  = true;
   Preconditions.checkState(false,

Review Comment:
   `"Node %s is in an invalid state! "
 + "Invalid state: %s %s blocks are on this dn.",`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out



[ 
https://issues.apache.org/jira/browse/HADOOP-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570930#comment-17570930
 ] 

Ahmar Suhail commented on HADOOP-18366:
---

I'm not sure if this is a known issue, I run scale tests quite often have been 
seeing this since the past week. I haven't looked into this too much yet, but 
wondering if anyone else is also experiencing this.

> ITestS3Select.testSelectSeekFullLandsat is timing out
> -
>
> Key: HADOOP-18366
> URL: https://issues.apache.org/jira/browse/HADOOP-18366
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Ahmar Suhail
>Priority: Minor
>
> ITestS3Select.testSelectSeekFullLandsat is timing out. When I run it 
> separately on my IDE, I get the following output repeatedly till I stop the 
> test manually:
>  
> {code:java}
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
> When running the entire test suite on my EC2, I get:
> {code:java}
> Time elapsed: 600.01 s  <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 60 
> milliseconds {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out



 [ 
https://issues.apache.org/jira/browse/HADOOP-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmar Suhail updated HADOOP-18366:
--
Environment: (was: ITestS3Select.testSelectSeekFullLandsat is timing 
out. When I run it separately on my IDE, I get the following output repeatedly 
till I stop the test manually:

 
{code:java}
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
When running the entire test suite on my EC2, I get:
{code:java}
Time elapsed: 600.01 s  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 60 
milliseconds {code})

> ITestS3Select.testSelectSeekFullLandsat is timing out
> -
>
> Key: HADOOP-18366
> URL: https://issues.apache.org/jira/browse/HADOOP-18366
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Ahmar Suhail
>Priority: Minor
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Created] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out

Ahmar Suhail created HADOOP-18366:
-

 Summary: ITestS3Select.testSelectSeekFullLandsat is timing out
 Key: HADOOP-18366
 URL: https://issues.apache.org/jira/browse/HADOOP-18366
 Project: Hadoop Common
  Issue Type: Sub-task
 Environment: ITestS3Select.testSelectSeekFullLandsat is timing out. 
When I run it separately on my IDE, I get the following output repeatedly till 
I stop the test manually:

 
{code:java}
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
When running the entire test suite on my EC2, I get:
{code:java}
Time elapsed: 600.01 s  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 60 
milliseconds {code}
Reporter: Ahmar Suhail






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-18366) ITestS3Select.testSelectSeekFullLandsat is timing out



 [ 
https://issues.apache.org/jira/browse/HADOOP-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmar Suhail updated HADOOP-18366:
--
Description: 
ITestS3Select.testSelectSeekFullLandsat is timing out. When I run it separately 
on my IDE, I get the following output repeatedly till I stop the test manually:

 
{code:java}
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:(80)) - Starting: read()
2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
(DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
When running the entire test suite on my EC2, I get:
{code:java}
Time elapsed: 600.01 s  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 60 
milliseconds {code}

> ITestS3Select.testSelectSeekFullLandsat is timing out
> -
>
> Key: HADOOP-18366
> URL: https://issues.apache.org/jira/browse/HADOOP-18366
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Ahmar Suhail
>Priority: Minor
>
> ITestS3Select.testSelectSeekFullLandsat is timing out. When I run it 
> separately on my IDE, I get the following output repeatedly till I stop the 
> test manually:
>  
> {code:java}
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:(80)) - Starting: read()
> 2022-07-25 15:15:37,463 [JUnit-testSelectSeekFullLandsat] DEBUG s3a.Invoker 
> (DurationInfo.java:close(101)) - read(): duration 0:00.000s {code}
> When running the entire test suite on my EC2, I get:
> {code:java}
> Time elapsed: 600.01 s  <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 60 
> milliseconds {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] 9uapaw commented on pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr…



9uapaw commented on PR #4622:
URL: https://github.com/apache/hadoop/pull/4622#issuecomment-1194109589

   Hi @K0K0V0K, thank you for taking this task. I have some comments on the 
approach:
   - Wildcarding should not be exposed to ConfigurationProperties (I have not 
fully checked whether something breaks there, but we should keep it as simple 
as possible)
   - In theory it is possible to come up with an algorithm that handles 
wildcarding with explicit queue parts in between (like root.*.a.*), but I think 
it is done with backtracking and it is not a very efficient one.
   - That being said I believe we should encourage templating for dynamic 
queues only, which makes *.a.* kind paths a bad practice. My opinion is to 
narrow the scope down to support full wildcard queue paths (eg. root.*.* or 
root.*.*.*) only. We can revisit it later if there is a demand for a more 
intelligent approach.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…



ashutoshcipher commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194107879

   LGTM +1(non-binding)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.



hadoop-yetus commented on PR #4614:
URL: https://github.com/apache/hadoop/pull/4614#issuecomment-1194076399

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 45s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 10s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m  7s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   4m 23s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  compile  |   3m 30s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 34s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   2m  2s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 54s |  |  trunk passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 33s |  |  trunk passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 27s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 44s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 50s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javac  |   3m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 28s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 28s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  6s |  |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new + 
2 unchanged - 1 fixed = 2 total (was 3)  |
   | +1 :green_heart: |  mvnsite  |   1m 40s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 20s |  |  the patch passed with JDK 
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1  |
   | +1 :green_heart: |  javadoc  |   1m 15s |  |  the patch passed with JDK 
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 26s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 33s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  | 101m 10s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   3m 20s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 51s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 231m 12s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4614/8/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4614 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux fd36db27c930 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / c996ef6b2e9f681b8b887cbad4afe707eafe81d6 |
   | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private 
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4614/8/testReport/ |
   | Max. process+thread count | 1474 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router 
U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928899669


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -288,7 +290,11 @@ private void check() {
 // an invalid state.
 LOG.warn("DatanodeAdminMonitor caught exception when processing node "
 + "{}.", dn, e);

Review Comment:
   I havent made this change. To correct Log Single Line issues is any and 
required - A separate JIRA(to check and fix in different modules) can be 
created I think. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Created] (HADOOP-18365) Updated addresses are still accessed using the old IP address

2022-07-25 Thread Steve Vaughan (Jira)

Steve Vaughan created HADOOP-18365:
--

 Summary: Updated addresses are still accessed using the old IP 
address
 Key: HADOOP-18365
 URL: https://issues.apache.org/jira/browse/HADOOP-18365
 Project: Hadoop Common
  Issue Type: Improvement
  Components: common
 Environment: Demonstrated in a Kubernetes environment running Java 11.
Reporter: Steve Vaughan


When the IPC Client recognizes that an IP address has changed, it updates the 
server field and logs a message:
Address change detected. Old: 
journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.178:8485 New: 
journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.182:8485
Although the change is detected, the client will continue to connect to the old 
IP address, resulting in repeated log messages.  This is seen in managed 
environments when JournalNode syncing is enabled and a JournalNode is 
restarted, with the remaining nodes in the set repeatedly logging this message 
when syncing to the restarted JournalNode.

The source of the problem is that the remoteId.address is not updated.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928900049


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -201,6 +201,7 @@ private void check() {
 iterkey).iterator();
 final List toRemove = new ArrayList<>();
 final List unhealthyDns = new ArrayList<>();
+boolean inValidState = false;
 
 while (it.hasNext() && !exceededNumBlocksPerCheck() && namesystem

Review Comment:
   What do you mean by Single Line?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously



ashutoshcipher commented on code in PR #4626:
URL: https://github.com/apache/hadoop/pull/4626#discussion_r928899669


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:
##
@@ -288,7 +290,11 @@ private void check() {
 // an invalid state.
 LOG.warn("DatanodeAdminMonitor caught exception when processing node "
 + "{}.", dn, e);

Review Comment:
   I havent made this change. To correct Log Single Line issues is any and 
required - A separate JIRA can be created I think. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4626: HDFS-16676. DatanodeAdminManager$Monitor reports a node as invalid continuously