[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[
https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160578#comment-17160578
]
Prabhu Joseph commented on HADOOP-15518:
[~jnp] Have not faced this issue while testing Hadoop Daemons after
HADOOP-16314. I think this will happen only if user has configured multiple
AuthenticationFilterInitializer. But it is safer to have this fix as
AuthenticationFilter is used by many other projects like Ranger, Knox, Oozie
and it is difficult to find the reason for "Request is a replay attack" on some
systems.
As per previous comments from [~sunilg], the patch has to use *getRemoteUser*
instead of *getUserPrincipal*.
> Authentication filter calling handler after request already authenticated
> -
>
> Key: HADOOP-15518
> URL: https://issues.apache.org/jira/browse/HADOOP-15518
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.7.1
>Reporter: Kevin Minder
>Assignee: Kevin Minder
>Priority: Major
> Attachments: HADOOP-15518-001.patch, HADOOP-15518.002.patch
>
>
> The hadoop-auth AuthenticationFilter will invoke its handler even if a prior
> successful authentication has occurred in the current request. This
> primarily affects situations where multiple authentication mechanism has been
> configured. For example when core-site.xml's has
> hadoop.http.authentication.type=kerberos and yarn-site.xml has
> yarn.timeline-service.http-authentication.type=kerberos the result is an
> attempt to perform two Kerberos authentications for the same request. This
> in turn results in Kerberos triggering a replay attack detection. The
> javadocs for AuthenticationHandler
> ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)]
> indicate for the authenticate method that
> {quote}This method is invoked by the AuthenticationFilter only if the HTTP
> client request is not yet authenticated.
> {quote}
> This does not appear to be the case in practice.
> I've create a patch and tested on a limited number of functional use cases
> (e.g. the timeline-service issue noted above). If there is general agreement
> that the change is valid I'll add unit tests to the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2145: HADOOP-17133. Implement HttpServer2 metrics
hadoop-yetus commented on pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#issuecomment-660552051 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 5s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 52s | trunk passed | | +1 :green_heart: | compile | 22m 54s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 19m 50s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 52s | trunk passed | | +1 :green_heart: | mvnsite | 2m 37s | trunk passed | | +1 :green_heart: | shadedclient | 21m 17s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 28s | hadoop-kms in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 30s | hadoop-hdfs-httpfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 50s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 0m 54s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 2m 18s | hadoop-common-project/hadoop-common in trunk has 2 extant findbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 36s | the patch passed | | +1 :green_heart: | compile | 21m 41s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 21m 41s | the patch passed | | +1 :green_heart: | compile | 19m 29s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 19m 29s | the patch passed | | -0 :warning: | checkstyle | 3m 2s | root: The patch generated 1 new + 164 unchanged - 0 fixed = 165 total (was 164) | | +1 :green_heart: | mvnsite | 2m 50s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 16m 45s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 37s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 31s | hadoop-kms in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 33s | hadoop-hdfs-httpfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 58s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 4m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 9m 23s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 3m 36s | hadoop-kms in the patch passed. | | -1 :x: | unit | 4m 53s | hadoop-hdfs-httpfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | The patch does not generate ASF License warnings. | | | | 192m 47s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.security.TestRaceWhenRelogin | | | hadoop.fs.http.server.TestHttpFSServerWebServer | | | hadoop.fs.http.server.TestHttpFSServerWebServerWithRandomSecret | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2145/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2145 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 01596f2d0d20 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build
[GitHub] [hadoop] hadoop-yetus commented on pull request #2145: HADOOP-17133. Implement HttpServer2 metrics
hadoop-yetus commented on pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#issuecomment-660551254 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 4s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 42s | trunk passed | | +1 :green_heart: | compile | 20m 40s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 17m 30s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 53s | trunk passed | | +1 :green_heart: | mvnsite | 2m 37s | trunk passed | | +1 :green_heart: | shadedclient | 21m 25s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 30s | hadoop-kms in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 31s | hadoop-hdfs-httpfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 52s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 0m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 2m 12s | hadoop-common-project/hadoop-common in trunk has 2 extant findbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 33s | the patch passed | | +1 :green_heart: | compile | 20m 3s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 20m 3s | the patch passed | | +1 :green_heart: | compile | 17m 26s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 17m 26s | the patch passed | | -0 :warning: | checkstyle | 2m 52s | root: The patch generated 1 new + 164 unchanged - 0 fixed = 165 total (was 164) | | +1 :green_heart: | mvnsite | 2m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 18s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 29s | hadoop-kms in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 32s | hadoop-hdfs-httpfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 52s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 4m 7s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 9m 24s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 3m 35s | hadoop-kms in the patch passed. | | -1 :x: | unit | 4m 52s | hadoop-hdfs-httpfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 45s | The patch does not generate ASF License warnings. | | | | 181m 50s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.security.TestFixKerberosTicketOrder | | | hadoop.fs.http.server.TestHttpFSServerWebServerWithRandomSecret | | | hadoop.fs.http.server.TestHttpFSServerWebServer | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2145/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2145 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 03c0fb495a0e 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[
https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160548#comment-17160548
]
Jitendra Nath Pandey commented on HADOOP-15518:
---
[~prabhujoseph]
Is this jira still relevant given bunch of changes have gone in with
HADOOP-16095 and its subtasks, particularly HADOOP-16314?
> Authentication filter calling handler after request already authenticated
> -
>
> Key: HADOOP-15518
> URL: https://issues.apache.org/jira/browse/HADOOP-15518
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.7.1
>Reporter: Kevin Minder
>Assignee: Kevin Minder
>Priority: Major
> Attachments: HADOOP-15518-001.patch, HADOOP-15518.002.patch
>
>
> The hadoop-auth AuthenticationFilter will invoke its handler even if a prior
> successful authentication has occurred in the current request. This
> primarily affects situations where multiple authentication mechanism has been
> configured. For example when core-site.xml's has
> hadoop.http.authentication.type=kerberos and yarn-site.xml has
> yarn.timeline-service.http-authentication.type=kerberos the result is an
> attempt to perform two Kerberos authentications for the same request. This
> in turn results in Kerberos triggering a replay attack detection. The
> javadocs for AuthenticationHandler
> ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)]
> indicate for the authenticate method that
> {quote}This method is invoked by the AuthenticationFilter only if the HTTP
> client request is not yet authenticated.
> {quote}
> This does not appear to be the case in practice.
> I've create a patch and tested on a limited number of functional use cases
> (e.g. the timeline-service issue noted above). If there is general agreement
> that the change is valid I'll add unit tests to the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2145: HADOOP-17133. Implement HttpServer2 metrics
hadoop-yetus commented on pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#issuecomment-660543807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 36s | trunk passed | | +1 :green_heart: | compile | 20m 18s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 17m 15s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 3m 6s | trunk passed | | +1 :green_heart: | mvnsite | 3m 10s | trunk passed | | +1 :green_heart: | shadedclient | 21m 45s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 46s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 39s | hadoop-kms in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 40s | hadoop-hdfs-httpfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 2m 22s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 2m 12s | hadoop-common-project/hadoop-common in trunk has 2 extant findbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 37s | the patch passed | | +1 :green_heart: | compile | 18m 50s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 18m 50s | the patch passed | | +1 :green_heart: | compile | 16m 38s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 16m 38s | the patch passed | | -0 :warning: | checkstyle | 2m 43s | root: The patch generated 1 new + 164 unchanged - 0 fixed = 165 total (was 164) | | +1 :green_heart: | mvnsite | 3m 0s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 13m 54s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 45s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 39s | hadoop-kms in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 41s | hadoop-hdfs-httpfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 2m 22s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 4m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 29s | hadoop-common in the patch passed. | | -1 :x: | unit | 2m 28s | hadoop-kms in the patch passed. | | +1 :green_heart: | unit | 5m 15s | hadoop-hdfs-httpfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 55s | The patch does not generate ASF License warnings. | | | | 180m 11s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.crypto.key.kms.server.TestKMSWithZK | | | hadoop.crypto.key.kms.server.TestKMS | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2145/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2145 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux acddea6382c4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git
[GitHub] [hadoop] hadoop-yetus commented on pull request #2140: HDFS-15463. Add a tool to validate FsImage.
hadoop-yetus commented on pull request #2140: URL: https://github.com/apache/hadoop/pull/2140#issuecomment-660537832 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 18m 55s | trunk passed | | +1 :green_heart: | compile | 1m 22s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 1m 15s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 0m 57s | trunk passed | | +1 :green_heart: | mvnsite | 1m 21s | trunk passed | | +1 :green_heart: | shadedclient | 14m 16s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 41s | hadoop-hdfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 0m 51s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 3m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 2m 59s | hadoop-hdfs-project/hadoop-hdfs in trunk has 4 extant findbugs warnings. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 11s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 1m 4s | the patch passed | | -0 :warning: | checkstyle | 0m 45s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 214 unchanged - 0 fixed = 216 total (was 214) | | +1 :green_heart: | mvnsite | 1m 9s | the patch passed | | +1 :green_heart: | shellcheck | 0m 30s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 13m 57s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 0m 46s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 3m 1s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 95m 55s | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 168m 43s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader | | | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2140/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2140 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs compile javac javadoc mvninstall shadedclient findbugs checkstyle | | uname | Linux 607fa303d998 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 99655167f30 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2140/10/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2140/10/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html | | checkstyle |
[jira] [Updated] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated HADOOP-17100:
---
Attachment: HADOOP-17100.branch-3.1.006.patch
> Replace Guava Supplier with Java8+ Supplier in Hadoop
> -
>
> Key: HADOOP-17100
> URL: https://issues.apache.org/jira/browse/HADOOP-17100
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch,
> HADOOP-17100.003.patch, HADOOP-17100.006.patch,
> HADOOP-17100.branch-3.1.006.patch, HADOOP-17100.branch-3.2.006.patch,
> HADOOP-17100.branch-3.3.006.patch
>
>
> Replacing Usage of {{guava.Supplier<>}} are in Unit tests
> {{GenereicTestUtils.waitFor()}} in Hadoop project.
> * To make things more convenient for reviewers, I decided:
> ** Not to replace Object instantiation by lambda expressions because this
> will increase the patch size significantly and require code adjustments to
> pass the checkstyle scripts.
> ** Not to refactor the imports because this will make reading the patch more
> difficult.
> * Merge should be done to the following branches: trunk, branch-3.3,
> branch-3.2, branch-3.1
> The task is straightforward because {{java.util.Supplier}} has the same API
> as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units.
> Therefore, we need only to do the following a "one-line" change in all 147
> files.
> {code:bash}
>
> -import com.google.common.base.Supplier;
> +import java.util.function.Supplier;
> {code}
> The code change needs to be applied to the following list of files:
> {code:java}
>
> Targets
> Occurrences of 'com.google.common.base.Supplier' in project with mask
> '*.java'
> Found Occurrences (146 usages found)
> org.apache.hadoop.conf (1 usage found)
> TestReconfiguration.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> TestKMS.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> org.apache.hadoop.fs (2 usages found)
> FCStatisticsBaseTest.java (1 usage found)
> 40 import com.google.common.base.Supplier;
> TestEnhancedByteBufferAccess.java (1 usage found)
> 75 import com.google.common.base.Supplier;
> org.apache.hadoop.fs.viewfs (1 usage found)
> TestViewFileSystemWithTruncate.java (1 usage found)
> 23 import com.google.common.base.Supplier;
> org.apache.hadoop.ha (1 usage found)
> TestZKFailoverController.java (1 usage found)
> 25 import com.google.common.base.Supplier;
> org.apache.hadoop.hdfs (20 usages found)
> DFSTestUtil.java (1 usage found)
> 79 import com.google.common.base.Supplier;
> MiniDFSCluster.java (1 usage found)
> 78 import com.google.common.base.Supplier;
> TestBalancerBandwidth.java (1 usage found)
> 29 import com.google.common.base.Supplier;
> TestClientProtocolForPipelineRecovery.java (1 usage found)
> 30 import com.google.common.base.Supplier;
> TestDatanodeRegistration.java (1 usage found)
> 44 import com.google.common.base.Supplier;
> TestDataTransferKeepalive.java (1 usage found)
> 47 import com.google.common.base.Supplier;
> TestDeadNodeDetection.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestDecommission.java (1 usage found)
> 41 import com.google.common.base.Supplier;
> TestDFSShell.java (1 usage found)
> 37 import com.google.common.base.Supplier;
> TestEncryptedTransfer.java (1 usage found)
> 35 import com.google.common.base.Supplier;
> TestEncryptionZonesWithKMS.java (1 usage found)
> 22 import com.google.common.base.Supplier;
> TestFileCorruption.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestLeaseRecovery2.java (1 usage found)
> 32 import com.google.common.base.Supplier;
> TestLeaseRecoveryStriped.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestMaintenanceState.java (1 usage found)
> 63 import com.google.common.base.Supplier;
> TestPread.java (1 usage found)
> 61 import com.google.common.base.Supplier;
> TestQuota.java (1 usage found)
> 39 import com.google.common.base.Supplier;
> TestReplaceDatanodeOnFailure.java (1 usage found)
> 20 import com.google.common.base.Supplier;
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #2151: HDFS-15476 Make AsyncStream executor private
hadoop-yetus commented on pull request #2151: URL: https://github.com/apache/hadoop/pull/2151#issuecomment-660535309 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 24m 8s | trunk passed | | +1 :green_heart: | compile | 2m 13s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 2m 6s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | mvnsite | 0m 20s | trunk passed | | +1 :green_heart: | shadedclient | 44m 44s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 15s | the patch passed | | -1 :x: | compile | 1m 28s | hadoop-hdfs-native-client in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | cc | 1m 28s | hadoop-hdfs-native-client in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | golang | 1m 28s | hadoop-hdfs-native-client in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javac | 1m 28s | hadoop-hdfs-native-client in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | compile | 1m 31s | hadoop-hdfs-native-client in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | cc | 1m 31s | hadoop-hdfs-native-client in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | golang | 1m 31s | hadoop-hdfs-native-client in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javac | 1m 31s | hadoop-hdfs-native-client in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | +1 :green_heart: | mvnsite | 0m 14s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 16m 34s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | -1 :x: | unit | 1m 58s | hadoop-hdfs-native-client in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 71m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2151 | | Optional Tests | dupname asflicense compile cc mvnsite javac unit golang | | uname | Linux f0eb833aec8f 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 99655167f30 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | golang | https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-2151/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09.txt | | cc |
[jira] [Updated] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated HADOOP-17100:
---
Attachment: HADOOP-17100.branch-3.2.006.patch
> Replace Guava Supplier with Java8+ Supplier in Hadoop
> -
>
> Key: HADOOP-17100
> URL: https://issues.apache.org/jira/browse/HADOOP-17100
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch,
> HADOOP-17100.003.patch, HADOOP-17100.006.patch,
> HADOOP-17100.branch-3.2.006.patch, HADOOP-17100.branch-3.3.006.patch
>
>
> Replacing Usage of {{guava.Supplier<>}} are in Unit tests
> {{GenereicTestUtils.waitFor()}} in Hadoop project.
> * To make things more convenient for reviewers, I decided:
> ** Not to replace Object instantiation by lambda expressions because this
> will increase the patch size significantly and require code adjustments to
> pass the checkstyle scripts.
> ** Not to refactor the imports because this will make reading the patch more
> difficult.
> * Merge should be done to the following branches: trunk, branch-3.3,
> branch-3.2, branch-3.1
> The task is straightforward because {{java.util.Supplier}} has the same API
> as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units.
> Therefore, we need only to do the following a "one-line" change in all 147
> files.
> {code:bash}
>
> -import com.google.common.base.Supplier;
> +import java.util.function.Supplier;
> {code}
> The code change needs to be applied to the following list of files:
> {code:java}
>
> Targets
> Occurrences of 'com.google.common.base.Supplier' in project with mask
> '*.java'
> Found Occurrences (146 usages found)
> org.apache.hadoop.conf (1 usage found)
> TestReconfiguration.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> TestKMS.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> org.apache.hadoop.fs (2 usages found)
> FCStatisticsBaseTest.java (1 usage found)
> 40 import com.google.common.base.Supplier;
> TestEnhancedByteBufferAccess.java (1 usage found)
> 75 import com.google.common.base.Supplier;
> org.apache.hadoop.fs.viewfs (1 usage found)
> TestViewFileSystemWithTruncate.java (1 usage found)
> 23 import com.google.common.base.Supplier;
> org.apache.hadoop.ha (1 usage found)
> TestZKFailoverController.java (1 usage found)
> 25 import com.google.common.base.Supplier;
> org.apache.hadoop.hdfs (20 usages found)
> DFSTestUtil.java (1 usage found)
> 79 import com.google.common.base.Supplier;
> MiniDFSCluster.java (1 usage found)
> 78 import com.google.common.base.Supplier;
> TestBalancerBandwidth.java (1 usage found)
> 29 import com.google.common.base.Supplier;
> TestClientProtocolForPipelineRecovery.java (1 usage found)
> 30 import com.google.common.base.Supplier;
> TestDatanodeRegistration.java (1 usage found)
> 44 import com.google.common.base.Supplier;
> TestDataTransferKeepalive.java (1 usage found)
> 47 import com.google.common.base.Supplier;
> TestDeadNodeDetection.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestDecommission.java (1 usage found)
> 41 import com.google.common.base.Supplier;
> TestDFSShell.java (1 usage found)
> 37 import com.google.common.base.Supplier;
> TestEncryptedTransfer.java (1 usage found)
> 35 import com.google.common.base.Supplier;
> TestEncryptionZonesWithKMS.java (1 usage found)
> 22 import com.google.common.base.Supplier;
> TestFileCorruption.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestLeaseRecovery2.java (1 usage found)
> 32 import com.google.common.base.Supplier;
> TestLeaseRecoveryStriped.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestMaintenanceState.java (1 usage found)
> 63 import com.google.common.base.Supplier;
> TestPread.java (1 usage found)
> 61 import com.google.common.base.Supplier;
> TestQuota.java (1 usage found)
> 39 import com.google.common.base.Supplier;
> TestReplaceDatanodeOnFailure.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestReplication.java (1 usage found)
[jira] [Updated] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated HADOOP-17100:
---
Attachment: HADOOP-17100.branch-3.3.006.patch
> Replace Guava Supplier with Java8+ Supplier in Hadoop
> -
>
> Key: HADOOP-17100
> URL: https://issues.apache.org/jira/browse/HADOOP-17100
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch,
> HADOOP-17100.003.patch, HADOOP-17100.006.patch,
> HADOOP-17100.branch-3.3.006.patch
>
>
> Replacing Usage of {{guava.Supplier<>}} are in Unit tests
> {{GenereicTestUtils.waitFor()}} in Hadoop project.
> * To make things more convenient for reviewers, I decided:
> ** Not to replace Object instantiation by lambda expressions because this
> will increase the patch size significantly and require code adjustments to
> pass the checkstyle scripts.
> ** Not to refactor the imports because this will make reading the patch more
> difficult.
> * Merge should be done to the following branches: trunk, branch-3.3,
> branch-3.2, branch-3.1
> The task is straightforward because {{java.util.Supplier}} has the same API
> as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units.
> Therefore, we need only to do the following a "one-line" change in all 147
> files.
> {code:bash}
>
> -import com.google.common.base.Supplier;
> +import java.util.function.Supplier;
> {code}
> The code change needs to be applied to the following list of files:
> {code:java}
>
> Targets
> Occurrences of 'com.google.common.base.Supplier' in project with mask
> '*.java'
> Found Occurrences (146 usages found)
> org.apache.hadoop.conf (1 usage found)
> TestReconfiguration.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> TestKMS.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> org.apache.hadoop.fs (2 usages found)
> FCStatisticsBaseTest.java (1 usage found)
> 40 import com.google.common.base.Supplier;
> TestEnhancedByteBufferAccess.java (1 usage found)
> 75 import com.google.common.base.Supplier;
> org.apache.hadoop.fs.viewfs (1 usage found)
> TestViewFileSystemWithTruncate.java (1 usage found)
> 23 import com.google.common.base.Supplier;
> org.apache.hadoop.ha (1 usage found)
> TestZKFailoverController.java (1 usage found)
> 25 import com.google.common.base.Supplier;
> org.apache.hadoop.hdfs (20 usages found)
> DFSTestUtil.java (1 usage found)
> 79 import com.google.common.base.Supplier;
> MiniDFSCluster.java (1 usage found)
> 78 import com.google.common.base.Supplier;
> TestBalancerBandwidth.java (1 usage found)
> 29 import com.google.common.base.Supplier;
> TestClientProtocolForPipelineRecovery.java (1 usage found)
> 30 import com.google.common.base.Supplier;
> TestDatanodeRegistration.java (1 usage found)
> 44 import com.google.common.base.Supplier;
> TestDataTransferKeepalive.java (1 usage found)
> 47 import com.google.common.base.Supplier;
> TestDeadNodeDetection.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestDecommission.java (1 usage found)
> 41 import com.google.common.base.Supplier;
> TestDFSShell.java (1 usage found)
> 37 import com.google.common.base.Supplier;
> TestEncryptedTransfer.java (1 usage found)
> 35 import com.google.common.base.Supplier;
> TestEncryptionZonesWithKMS.java (1 usage found)
> 22 import com.google.common.base.Supplier;
> TestFileCorruption.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestLeaseRecovery2.java (1 usage found)
> 32 import com.google.common.base.Supplier;
> TestLeaseRecoveryStriped.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestMaintenanceState.java (1 usage found)
> 63 import com.google.common.base.Supplier;
> TestPread.java (1 usage found)
> 61 import com.google.common.base.Supplier;
> TestQuota.java (1 usage found)
> 39 import com.google.common.base.Supplier;
> TestReplaceDatanodeOnFailure.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestReplication.java (1 usage found)
> 27 import
[GitHub] [hadoop] hadoop-yetus commented on pull request #2145: HADOOP-17133. Implement HttpServer2 metrics
hadoop-yetus commented on pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#issuecomment-660525195 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 0m 56s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 18m 59s | trunk passed | | +1 :green_heart: | compile | 19m 15s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 16m 37s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 41s | trunk passed | | +1 :green_heart: | mvnsite | 2m 59s | trunk passed | | +1 :green_heart: | shadedclient | 20m 19s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 45s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 39s | hadoop-kms in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 40s | hadoop-hdfs-httpfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 2m 21s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 0m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 2m 13s | hadoop-common-project/hadoop-common in trunk has 2 extant findbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 38s | the patch passed | | +1 :green_heart: | compile | 18m 42s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 18m 42s | the patch passed | | +1 :green_heart: | compile | 16m 37s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 16m 37s | the patch passed | | -0 :warning: | checkstyle | 2m 41s | root: The patch generated 1 new + 164 unchanged - 0 fixed = 165 total (was 164) | | +1 :green_heart: | mvnsite | 2m 57s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 31s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 39s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 32s | hadoop-kms in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 34s | hadoop-hdfs-httpfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 2m 7s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 4m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 31s | hadoop-common in the patch passed. | | -1 :x: | unit | 3m 37s | hadoop-kms in the patch passed. | | +1 :green_heart: | unit | 5m 29s | hadoop-hdfs-httpfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 175m 27s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.crypto.key.kms.server.TestKMS | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2145/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2145 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux dd010b9fcb77 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 8a9a674ef10 | | Default Java |
[GitHub] [hadoop] surajj-naik opened a new pull request #2151: HDFS-15476: Make AsyncStream executor private
surajj-naik opened a new pull request #2151: URL: https://github.com/apache/hadoop/pull/2151 The executor_ member of the AsyncStream class is public, but there is a getter get_executor() that returns this executor_ member. This member should be private. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16682) Remove unnecessary toString() invocations
[ https://issues.apache.org/jira/browse/HADOOP-16682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160507#comment-17160507 ] Hudson commented on HADOOP-16682: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18449 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18449/]) HADOOP-16682. ABFS: Removing unnecessary toString() invocations (github: rev 99655167f308b9c59e66b1b5d0d1fd5741cd75de) * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java > Remove unnecessary toString() invocations > - > > Key: HADOOP-16682 > URL: https://issues.apache.org/jira/browse/HADOOP-16682 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Jeetesh Mangwani >Assignee: Bilahari T H >Priority: Minor > Labels: abfsactive > > Remove unnecessary toString() invocations from the hadoop-azure module > For example: > permission.toString() in the line here: > https://github.com/apache/hadoop/blob/04a6c095cf6d09b6ad417f1f7b7c64fbfdc9d5e4/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java#L386 > path.toString() in the line here: > https://github.com/apache/hadoop/blob/04a6c095cf6d09b6ad417f1f7b7c64fbfdc9d5e4/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java#L795 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] DadanielZ merged pull request #2144: HADOOP-16682. ABFS: Removing unnecessary toString() invocations
DadanielZ merged pull request #2144: URL: https://github.com/apache/hadoop/pull/2144 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17106) Replace Guava Joiner with Java8 String Join
[
https://issues.apache.org/jira/browse/HADOOP-17106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160387#comment-17160387
]
Ayush Saxena commented on HADOOP-17106:
---
Thanx [~ahussein] for sharing.
For the
[DFClient|https://github.com/apache/hadoop/blob/696a663cd0a1deb818f6b44ddc8ca20571bac980/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java#L386]
I changed to {{StringUtils.join(',', Arrays.asList(localInterfaceAddrs)) +
"]");}}
and that gave me the same result. For maps {{withKeyValueSeparator}} I think
some wrapper would be required, maybe can add one in {{StringUtils}} only.
Note : The StringUtils used is the one present in
[Hadoop-Common|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java]
> Replace Guava Joiner with Java8 String Join
> ---
>
> Key: HADOOP-17106
> URL: https://issues.apache.org/jira/browse/HADOOP-17106
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17106.001.patch
>
>
> Replace \{{com.google.common.base.Joiner}} with String.join.
>
> {code:java}
> Targets
> Occurrences of 'com.google.common.base.Joiner' in project with mask
> '*.java'
> Found Occurrences (103 usages found)
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> SimpleKMSAuditLogger.java (1 usage found)
> 26 import com.google.common.base.Joiner;
> org.apache.hadoop.fs (1 usage found)
> TestPath.java (1 usage found)
> 37 import com.google.common.base.Joiner;
> org.apache.hadoop.fs.s3a (1 usage found)
> StorageStatisticsTracker.java (1 usage found)
> 25 import com.google.common.base.Joiner;
> org.apache.hadoop.ha (1 usage found)
> TestHAAdmin.java (1 usage found)
> 34 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs (8 usages found)
> DFSClient.java (1 usage found)
> 196 import com.google.common.base.Joiner;
> DFSTestUtil.java (1 usage found)
> 76 import com.google.common.base.Joiner;
> DFSUtil.java (1 usage found)
> 108 import com.google.common.base.Joiner;
> DFSUtilClient.java (1 usage found)
> 20 import com.google.common.base.Joiner;
> HAUtil.java (1 usage found)
> 59 import com.google.common.base.Joiner;
> MiniDFSCluster.java (1 usage found)
> 145 import com.google.common.base.Joiner;
> StripedFileTestUtil.java (1 usage found)
> 20 import com.google.common.base.Joiner;
> TestDFSUpgrade.java (1 usage found)
> 53 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.protocol (1 usage found)
> LayoutFlags.java (1 usage found)
> 26 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.protocolPB (1 usage found)
> TestPBHelper.java (1 usage found)
> 118 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.qjournal (1 usage found)
> MiniJournalCluster.java (1 usage found)
> 43 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.qjournal.client (5 usages found)
> AsyncLoggerSet.java (1 usage found)
> 38 import com.google.common.base.Joiner;
> QuorumCall.java (1 usage found)
> 32 import com.google.common.base.Joiner;
> QuorumException.java (1 usage found)
> 25 import com.google.common.base.Joiner;
> QuorumJournalManager.java (1 usage found)
> 62 import com.google.common.base.Joiner;
> TestQuorumCall.java (1 usage found)
> 29 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.server.blockmanagement (4 usages found)
> HostSet.java (1 usage found)
> 21 import com.google.common.base.Joiner;
> TestBlockManager.java (1 usage found)
> 20 import com.google.common.base.Joiner;
> TestBlockReportRateLimiting.java (1 usage found)
> 24 import com.google.common.base.Joiner;
> TestPendingDataNodeMessages.java (1 usage found)
> 41 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.server.common (1 usage found)
> StorageInfo.java (1 usage found)
> 37 import com.google.common.base.Joiner;
> org.apache.hadoop.hdfs.server.datanode (7 usages found)
> BlockPoolManager.java (1 usage found)
> 32 import com.google.common.base.Joiner;
> BlockRecoveryWorker.java (1 usage found)
> 21 import
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2083: HADOOP-17077. S3A delegation token binding to support secondary binding list
hadoop-yetus removed a comment on pull request #2083: URL: https://github.com/apache/hadoop/pull/2083#issuecomment-657880743 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2046: HADOOP-16202 Enhance S3A openFile()
hadoop-yetus removed a comment on pull request #2046: URL: https://github.com/apache/hadoop/pull/2046#issuecomment-638468900 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 20m 41s | trunk passed | | +1 :green_heart: | compile | 17m 31s | trunk passed | | +1 :green_heart: | checkstyle | 2m 48s | trunk passed | | +1 :green_heart: | mvnsite | 2m 5s | trunk passed | | +1 :green_heart: | shadedclient | 20m 26s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 27s | trunk passed | | +0 :ok: | spotbugs | 1m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 9s | trunk passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 21s | the patch passed | | +1 :green_heart: | compile | 16m 52s | the patch passed | | +1 :green_heart: | javac | 16m 52s | the patch passed | | +1 :green_heart: | checkstyle | 2m 49s | the patch passed | | +1 :green_heart: | mvnsite | 2m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 24s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 29s | the patch passed | | +1 :green_heart: | findbugs | 3m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 19s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 28s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | | | | 123m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2046 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint | | uname | Linux 643ca18e831c 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 40d63e02f04 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/2/testReport/ | | Max. process+thread count | 1332 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/2/console | | versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2083: HADOOP-17077. S3A delegation token binding to support secondary binding list
hadoop-yetus removed a comment on pull request #2083: URL: https://github.com/apache/hadoop/pull/2083#issuecomment-652089254 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 18m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 3s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 19m 34s | trunk passed | | +1 :green_heart: | compile | 19m 13s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 16m 49s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 45s | trunk passed | | +1 :green_heart: | mvnsite | 2m 11s | trunk passed | | +1 :green_heart: | shadedclient | 19m 17s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 46s | hadoop-hdfs in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 41s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 32s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 1m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 23s | trunk passed | | -0 :warning: | patch | 1m 43s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 18m 55s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 18m 55s | the patch passed | | +1 :green_heart: | compile | 16m 41s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 16m 41s | the patch passed | | -0 :warning: | checkstyle | 2m 42s | root: The patch generated 12 new + 31 unchanged - 3 fixed = 43 total (was 34) | | +1 :green_heart: | mvnsite | 2m 21s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedclient | 14m 7s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 50s | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 45s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 42s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 4m 30s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 93m 38s | hadoop-hdfs in the patch passed. | | +1 :green_heart: | unit | 1m 48s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 58s | The patch does not generate ASF License warnings. | | | | 269m 34s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.namenode.TestFSImageWithAcl | | | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks | | | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier | | | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2083/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2083 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint | | uname | Linux 83e9172cbc90 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e8dc862d385 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
[jira] [Commented] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160383#comment-17160383 ] Hudson commented on HADOOP-17100: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18447 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18447/]) HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. (ayushsaxena: rev 6bcb24d26930b3a2abfdd533f4aea0ce670c78a1) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/checker/TestThrottledAsyncChecker.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNodeSync.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FCStatisticsBaseTest.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestTaskHeartbeatHandler.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestGenericTestUtils.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManagerSafeMode.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferKeepalive.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/HATestUtil.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilter.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/TestZKFailoverController.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestContinuousScheduling.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCorruptMetadataFile.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceState.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestPendingInvalidateBlock.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRandomOpsWithSnapshots.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCorruption.java * (edit)
[jira] [Commented] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160376#comment-17160376
]
Ayush Saxena commented on HADOOP-17100:
---
Committed to trunk.
Thanx [~ahussein] for the contribution, Can you upload patch for lower
branches, the cherry-pick has conflicts.
> Replace Guava Supplier with Java8+ Supplier in Hadoop
> -
>
> Key: HADOOP-17100
> URL: https://issues.apache.org/jira/browse/HADOOP-17100
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch,
> HADOOP-17100.003.patch, HADOOP-17100.006.patch
>
>
> Replacing Usage of {{guava.Supplier<>}} are in Unit tests
> {{GenereicTestUtils.waitFor()}} in Hadoop project.
> * To make things more convenient for reviewers, I decided:
> ** Not to replace Object instantiation by lambda expressions because this
> will increase the patch size significantly and require code adjustments to
> pass the checkstyle scripts.
> ** Not to refactor the imports because this will make reading the patch more
> difficult.
> * Merge should be done to the following branches: trunk, branch-3.3,
> branch-3.2, branch-3.1
> The task is straightforward because {{java.util.Supplier}} has the same API
> as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units.
> Therefore, we need only to do the following a "one-line" change in all 147
> files.
> {code:bash}
>
> -import com.google.common.base.Supplier;
> +import java.util.function.Supplier;
> {code}
> The code change needs to be applied to the following list of files:
> {code:java}
>
> Targets
> Occurrences of 'com.google.common.base.Supplier' in project with mask
> '*.java'
> Found Occurrences (146 usages found)
> org.apache.hadoop.conf (1 usage found)
> TestReconfiguration.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> TestKMS.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> org.apache.hadoop.fs (2 usages found)
> FCStatisticsBaseTest.java (1 usage found)
> 40 import com.google.common.base.Supplier;
> TestEnhancedByteBufferAccess.java (1 usage found)
> 75 import com.google.common.base.Supplier;
> org.apache.hadoop.fs.viewfs (1 usage found)
> TestViewFileSystemWithTruncate.java (1 usage found)
> 23 import com.google.common.base.Supplier;
> org.apache.hadoop.ha (1 usage found)
> TestZKFailoverController.java (1 usage found)
> 25 import com.google.common.base.Supplier;
> org.apache.hadoop.hdfs (20 usages found)
> DFSTestUtil.java (1 usage found)
> 79 import com.google.common.base.Supplier;
> MiniDFSCluster.java (1 usage found)
> 78 import com.google.common.base.Supplier;
> TestBalancerBandwidth.java (1 usage found)
> 29 import com.google.common.base.Supplier;
> TestClientProtocolForPipelineRecovery.java (1 usage found)
> 30 import com.google.common.base.Supplier;
> TestDatanodeRegistration.java (1 usage found)
> 44 import com.google.common.base.Supplier;
> TestDataTransferKeepalive.java (1 usage found)
> 47 import com.google.common.base.Supplier;
> TestDeadNodeDetection.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestDecommission.java (1 usage found)
> 41 import com.google.common.base.Supplier;
> TestDFSShell.java (1 usage found)
> 37 import com.google.common.base.Supplier;
> TestEncryptedTransfer.java (1 usage found)
> 35 import com.google.common.base.Supplier;
> TestEncryptionZonesWithKMS.java (1 usage found)
> 22 import com.google.common.base.Supplier;
> TestFileCorruption.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestLeaseRecovery2.java (1 usage found)
> 32 import com.google.common.base.Supplier;
> TestLeaseRecoveryStriped.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestMaintenanceState.java (1 usage found)
> 63 import com.google.common.base.Supplier;
> TestPread.java (1 usage found)
> 61 import com.google.common.base.Supplier;
> TestQuota.java (1 usage found)
> 39 import com.google.common.base.Supplier;
> TestReplaceDatanodeOnFailure.java (1 usage found)
> 20 import com.google.common.base.Supplier;
[jira] [Commented] (HADOOP-17100) Replace Guava Supplier with Java8+ Supplier in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160374#comment-17160374
]
Ayush Saxena commented on HADOOP-17100:
---
v006 LGTM +1
> Replace Guava Supplier with Java8+ Supplier in Hadoop
> -
>
> Key: HADOOP-17100
> URL: https://issues.apache.org/jira/browse/HADOOP-17100
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch,
> HADOOP-17100.003.patch, HADOOP-17100.006.patch
>
>
> Replacing Usage of {{guava.Supplier<>}} are in Unit tests
> {{GenereicTestUtils.waitFor()}} in Hadoop project.
> * To make things more convenient for reviewers, I decided:
> ** Not to replace Object instantiation by lambda expressions because this
> will increase the patch size significantly and require code adjustments to
> pass the checkstyle scripts.
> ** Not to refactor the imports because this will make reading the patch more
> difficult.
> * Merge should be done to the following branches: trunk, branch-3.3,
> branch-3.2, branch-3.1
> The task is straightforward because {{java.util.Supplier}} has the same API
> as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units.
> Therefore, we need only to do the following a "one-line" change in all 147
> files.
> {code:bash}
>
> -import com.google.common.base.Supplier;
> +import java.util.function.Supplier;
> {code}
> The code change needs to be applied to the following list of files:
> {code:java}
>
> Targets
> Occurrences of 'com.google.common.base.Supplier' in project with mask
> '*.java'
> Found Occurrences (146 usages found)
> org.apache.hadoop.conf (1 usage found)
> TestReconfiguration.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> org.apache.hadoop.crypto.key.kms.server (1 usage found)
> TestKMS.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> org.apache.hadoop.fs (2 usages found)
> FCStatisticsBaseTest.java (1 usage found)
> 40 import com.google.common.base.Supplier;
> TestEnhancedByteBufferAccess.java (1 usage found)
> 75 import com.google.common.base.Supplier;
> org.apache.hadoop.fs.viewfs (1 usage found)
> TestViewFileSystemWithTruncate.java (1 usage found)
> 23 import com.google.common.base.Supplier;
> org.apache.hadoop.ha (1 usage found)
> TestZKFailoverController.java (1 usage found)
> 25 import com.google.common.base.Supplier;
> org.apache.hadoop.hdfs (20 usages found)
> DFSTestUtil.java (1 usage found)
> 79 import com.google.common.base.Supplier;
> MiniDFSCluster.java (1 usage found)
> 78 import com.google.common.base.Supplier;
> TestBalancerBandwidth.java (1 usage found)
> 29 import com.google.common.base.Supplier;
> TestClientProtocolForPipelineRecovery.java (1 usage found)
> 30 import com.google.common.base.Supplier;
> TestDatanodeRegistration.java (1 usage found)
> 44 import com.google.common.base.Supplier;
> TestDataTransferKeepalive.java (1 usage found)
> 47 import com.google.common.base.Supplier;
> TestDeadNodeDetection.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestDecommission.java (1 usage found)
> 41 import com.google.common.base.Supplier;
> TestDFSShell.java (1 usage found)
> 37 import com.google.common.base.Supplier;
> TestEncryptedTransfer.java (1 usage found)
> 35 import com.google.common.base.Supplier;
> TestEncryptionZonesWithKMS.java (1 usage found)
> 22 import com.google.common.base.Supplier;
> TestFileCorruption.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestLeaseRecovery2.java (1 usage found)
> 32 import com.google.common.base.Supplier;
> TestLeaseRecoveryStriped.java (1 usage found)
> 21 import com.google.common.base.Supplier;
> TestMaintenanceState.java (1 usage found)
> 63 import com.google.common.base.Supplier;
> TestPread.java (1 usage found)
> 61 import com.google.common.base.Supplier;
> TestQuota.java (1 usage found)
> 39 import com.google.common.base.Supplier;
> TestReplaceDatanodeOnFailure.java (1 usage found)
> 20 import com.google.common.base.Supplier;
> TestReplication.java (1 usage found)
> 27 import com.google.common.base.Supplier;
>
