[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=322331=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322331 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 22:41 Start Date: 02/Oct/19 22:41 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 322331) Time Spent: 5h 40m (was: 5.5h) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 5h 40m > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=322330=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322330 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 22:41 Start Date: 02/Oct/19 22:41 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-537711587 Thank You @xiaoyuyao for the review. I will commit this to the trunk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 322330) Time Spent: 5.5h (was: 5h 20m) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 5.5h > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=322322=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322322 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 22:36 Start Date: 02/Oct/19 22:36 Worklog Time Spent: 10m Work Description: xiaoyuyao commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-537710142 LGTM, +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 322322) Time Spent: 5h 20m (was: 5h 10m) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 5h 20m > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321691=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321691 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:30 Start Date: 02/Oct/19 05:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533725569 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 75 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | -1 | compile | 19 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 59 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 108 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 174 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 20 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 28 | hadoop-ozone in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 25 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | -1 | shadedclient | 28 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 276 | hadoop-hdds in the patch passed. | | -1 | unit | 22 | hadoop-ozone in the patch failed. | | +1 | asflicense | 24 | The patch does not generate ASF License warnings. | | | | 1735 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux addcf29dbc03 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/testReport/ | | Max. process+thread count | 429 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321689=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321689 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:30 Start Date: 02/Oct/19 05:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533766921 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 31 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 65 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 109 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 49 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 179 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 24 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 35 | hadoop-ozone in the patch failed. | | -1 | compile | 23 | hadoop-ozone in the patch failed. | | -1 | javac | 23 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 28 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedclient | 32 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 53 | hadoop-ozone in the patch failed. | | -1 | findbugs | 25 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 258 | hadoop-hdds in the patch passed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 1819 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux a444e8fa4173 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/testReport/ | | Max. process+thread count | 542 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321690 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:30 Start Date: 02/Oct/19 05:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533728099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 46 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 35 | hadoop-ozone in trunk failed. | | -1 | compile | 24 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 82 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 145 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 56 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 226 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 25 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 33 | hadoop-ozone in the patch failed. | | -1 | compile | 22 | hadoop-ozone in the patch failed. | | -1 | javac | 22 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 30 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedclient | 32 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 56 | hadoop-ozone in the patch failed. | | -1 | findbugs | 26 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 266 | hadoop-hdds in the patch passed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 2058 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 1395f9604b97 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/testReport/ | | Max. process+thread count | 478 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321687=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321687 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:29 Start Date: 02/Oct/19 05:29 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-536120276 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 37 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 36 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 44 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-hdds in trunk failed. | | -1 | compile | 16 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 63 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 865 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 25 | hadoop-hdds in trunk failed. | | -1 | javadoc | 20 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 983 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 47 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 34 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 37 | hadoop-ozone in the patch failed. | | -1 | compile | 26 | hadoop-hdds in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 26 | hadoop-hdds in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 59 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 730 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in the patch failed. | | -1 | javadoc | 21 | hadoop-ozone in the patch failed. | | -1 | findbugs | 32 | hadoop-hdds in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 28 | hadoop-hdds in the patch failed. | | -1 | unit | 27 | hadoop-ozone in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 2416 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux edb0fc8744f3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 13b427f | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321686=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321686 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:29 Start Date: 02/Oct/19 05:29 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-537216356 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 31 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 37 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 50 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 852 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in trunk failed. | | -1 | javadoc | 17 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 951 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 34 | hadoop-hdds in trunk failed. | | -1 | findbugs | 20 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 35 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 36 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-hdds in the patch failed. | | -1 | compile | 18 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-hdds in the patch failed. | | -1 | javac | 18 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 55 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 718 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in the patch failed. | | -1 | javadoc | 19 | hadoop-ozone in the patch failed. | | -1 | findbugs | 32 | hadoop-hdds in the patch failed. | | -1 | findbugs | 20 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 26 | hadoop-hdds in the patch failed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 2348 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6ea0fc34e081 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d947ded | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321688=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321688 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 02/Oct/19 05:30 Start Date: 02/Oct/19 05:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-534799294 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 41 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | -1 | compile | 22 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 63 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 856 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in trunk failed. | | -1 | javadoc | 25 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 963 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 33 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 36 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 29 | hadoop-ozone in the patch failed. | | -1 | compile | 24 | hadoop-hdds in the patch failed. | | -1 | compile | 20 | hadoop-ozone in the patch failed. | | -1 | javac | 24 | hadoop-hdds in the patch failed. | | -1 | javac | 20 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 41 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 708 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in the patch failed. | | -1 | javadoc | 19 | hadoop-ozone in the patch failed. | | -1 | findbugs | 31 | hadoop-hdds in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 29 | hadoop-hdds in the patch failed. | | -1 | unit | 24 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 2355 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e8ede916717d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / a346381 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321504=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321504 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 01/Oct/19 20:30 Start Date: 01/Oct/19 20:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-537216356 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 31 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 37 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 50 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 852 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in trunk failed. | | -1 | javadoc | 17 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 951 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 34 | hadoop-hdds in trunk failed. | | -1 | findbugs | 20 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 35 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 36 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-hdds in the patch failed. | | -1 | compile | 18 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-hdds in the patch failed. | | -1 | javac | 18 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 55 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 718 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in the patch failed. | | -1 | javadoc | 19 | hadoop-ozone in the patch failed. | | -1 | findbugs | 32 | hadoop-hdds in the patch failed. | | -1 | findbugs | 20 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 26 | hadoop-hdds in the patch failed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 2348 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6ea0fc34e081 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d947ded | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/4/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321480 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 01/Oct/19 19:51 Start Date: 01/Oct/19 19:51 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-537200400 Thank You @xiaoyuyao for the review. I have addressed the review comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 321480) Time Spent: 4h (was: 3h 50m) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321479
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 01/Oct/19 19:50
Start Date: 01/Oct/19 19:50
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r330243464
##
File path:
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
##
@@ -0,0 +1,129 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.s3.util;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.security.SecurityUtil;
+import org.apache.hadoop.test.GenericTestUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Collection;
+
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP;
+import static org.junit.Assert.fail;
+
+/**
+ * Class used to test OzoneS3Util.
+ */
+public class TestOzoneS3Util {
+
+
+ private OzoneConfiguration configuration;
+
+ @Before
+ public void setConf() {
+configuration = new OzoneConfiguration();
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+ }
+
+ @Test
+ public void testBuildServiceNameForToken() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+String expectedOmServiceAddress = buildOMNodeAddresses(nodeIDList,
+serviceID);
+
+SecurityUtil.setConfiguration(configuration);
+String omserviceAddr = OzoneS3Util.buildServiceNameForToken(configuration,
+serviceID, nodeIDList);
+
+Assert.assertEquals(expectedOmServiceAddress, omserviceAddr);
+ }
+
+ @Test
+ public void testBuildServiceNameForTokenIncorrectConfig() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+// Don't set om3 node rpc address.
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om1"), "om1:9862");
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om2"), "om2:9862");
+
+
+SecurityUtil.setConfiguration(configuration);
+
+try {
+ OzoneS3Util.buildServiceNameForToken(configuration,
+ serviceID, nodeIDList);
+ fail("testBuildServiceNameForTokenIncorrectConfig failed");
+} catch (IllegalArgumentException ex) {
+ GenericTestUtils.assertExceptionContains("Could not find rpcAddress " +
+ "for", ex);
+}
+
+
+ }
+
+
+ private String buildOMNodeAddresses(Collection nodeIDList,
+ String serviceID) {
+StringBuilder omServiceAddrBuilder = new StringBuilder();
+int port = 9862;
+int nodesLength = nodeIDList.size();
+int counter = 0;
+for (String nodeID : nodeIDList) {
+ counter++;
+ String addr = nodeID + ":" + port++;
+ configuration.set(OmUtils.addKeySuffixes(
+
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321478=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321478
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 01/Oct/19 19:50
Start Date: 01/Oct/19 19:50
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r330243429
##
File path:
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
##
@@ -0,0 +1,129 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.s3.util;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.security.SecurityUtil;
+import org.apache.hadoop.test.GenericTestUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Collection;
+
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP;
+import static org.junit.Assert.fail;
+
+/**
+ * Class used to test OzoneS3Util.
+ */
+public class TestOzoneS3Util {
+
+
+ private OzoneConfiguration configuration;
+
+ @Before
+ public void setConf() {
+configuration = new OzoneConfiguration();
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+ }
+
+ @Test
+ public void testBuildServiceNameForToken() {
+String serviceID = "omService";
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 321478)
Time Spent: 3h 40m (was: 3.5h)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=321467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321467
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 01/Oct/19 19:26
Start Date: 01/Oct/19 19:26
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r330233292
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -20,33 +20,75 @@
import org.apache.hadoop.hdds.conf.OzoneConfiguration;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.s3.util.OzoneS3Util;
import org.apache.hadoop.security.SecurityUtil;
import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;
import javax.inject.Inject;
+
+import java.util.Arrays;
+import java.util.Collection;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
/**
* This class creates the OM service .
*/
@ApplicationScoped
public class OzoneServiceProvider {
- private Text omServiceAdd;
+ private Text omServiceAddr;
+
+ private String omserviceID;
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAddr = SecurityUtil.buildTokenService(OmUtils.
+ getOmAddressForClients(conf));
+} else {
+ // HA cluster.
+ //For now if multiple service id's are configured we throw exception.
Review comment:
Right now we don't have DNS discovery. Discussed offline for now, we are
going with current approach. When serviceName is configured with DNS will
re-look into this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 321467)
Time Spent: 3.5h (was: 3h 20m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=320795=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320795
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 30/Sep/19 19:29
Start Date: 30/Sep/19 19:29
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r329748308
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -20,33 +20,75 @@
import org.apache.hadoop.hdds.conf.OzoneConfiguration;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.s3.util.OzoneS3Util;
import org.apache.hadoop.security.SecurityUtil;
import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;
import javax.inject.Inject;
+
+import java.util.Arrays;
+import java.util.Collection;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
/**
* This class creates the OM service .
*/
@ApplicationScoped
public class OzoneServiceProvider {
- private Text omServiceAdd;
+ private Text omServiceAddr;
+
+ private String omserviceID;
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAddr = SecurityUtil.buildTokenService(OmUtils.
+ getOmAddressForClients(conf));
+} else {
+ // HA cluster.
+ //For now if multiple service id's are configured we throw exception.
Review comment:
This is very similar to what HDFS HA does. Since OM support service
discovery, can we get the internal service id via service discovery?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 320795)
Time Spent: 3h 20m (was: 3h 10m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=320788=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320788
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 30/Sep/19 19:25
Start Date: 30/Sep/19 19:25
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r329746422
##
File path:
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
##
@@ -0,0 +1,129 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.s3.util;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.security.SecurityUtil;
+import org.apache.hadoop.test.GenericTestUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Collection;
+
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP;
+import static org.junit.Assert.fail;
+
+/**
+ * Class used to test OzoneS3Util.
+ */
+public class TestOzoneS3Util {
+
+
+ private OzoneConfiguration configuration;
+
+ @Before
+ public void setConf() {
+configuration = new OzoneConfiguration();
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+ }
+
+ @Test
+ public void testBuildServiceNameForToken() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+String expectedOmServiceAddress = buildOMNodeAddresses(nodeIDList,
+serviceID);
+
+SecurityUtil.setConfiguration(configuration);
+String omserviceAddr = OzoneS3Util.buildServiceNameForToken(configuration,
+serviceID, nodeIDList);
+
+Assert.assertEquals(expectedOmServiceAddress, omserviceAddr);
+ }
+
+ @Test
+ public void testBuildServiceNameForTokenIncorrectConfig() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+// Don't set om3 node rpc address.
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om1"), "om1:9862");
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om2"), "om2:9862");
+
+
+SecurityUtil.setConfiguration(configuration);
+
+try {
+ OzoneS3Util.buildServiceNameForToken(configuration,
+ serviceID, nodeIDList);
+ fail("testBuildServiceNameForTokenIncorrectConfig failed");
+} catch (IllegalArgumentException ex) {
+ GenericTestUtils.assertExceptionContains("Could not find rpcAddress " +
+ "for", ex);
+}
+
+
+ }
+
+
+ private String buildOMNodeAddresses(Collection nodeIDList,
+ String serviceID) {
+StringBuilder omServiceAddrBuilder = new StringBuilder();
+int port = 9862;
+int nodesLength = nodeIDList.size();
+int counter = 0;
+for (String nodeID : nodeIDList) {
+ counter++;
+ String addr = nodeID + ":" + port++;
+ configuration.set(OmUtils.addKeySuffixes(
+
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=320787=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320787
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 30/Sep/19 19:24
Start Date: 30/Sep/19 19:24
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r329746422
##
File path:
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
##
@@ -0,0 +1,129 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.s3.util;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.security.SecurityUtil;
+import org.apache.hadoop.test.GenericTestUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Collection;
+
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP;
+import static org.junit.Assert.fail;
+
+/**
+ * Class used to test OzoneS3Util.
+ */
+public class TestOzoneS3Util {
+
+
+ private OzoneConfiguration configuration;
+
+ @Before
+ public void setConf() {
+configuration = new OzoneConfiguration();
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+ }
+
+ @Test
+ public void testBuildServiceNameForToken() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+String expectedOmServiceAddress = buildOMNodeAddresses(nodeIDList,
+serviceID);
+
+SecurityUtil.setConfiguration(configuration);
+String omserviceAddr = OzoneS3Util.buildServiceNameForToken(configuration,
+serviceID, nodeIDList);
+
+Assert.assertEquals(expectedOmServiceAddress, omserviceAddr);
+ }
+
+ @Test
+ public void testBuildServiceNameForTokenIncorrectConfig() {
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+
+Collection nodeIDList = configuration.getStringCollection(
+OMConfigKeys.OZONE_OM_NODE_ID_KEY);
+
+// Don't set om3 node rpc address.
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om1"), "om1:9862");
+configuration.set(OmUtils.addKeySuffixes(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+serviceID, "om2"), "om2:9862");
+
+
+SecurityUtil.setConfiguration(configuration);
+
+try {
+ OzoneS3Util.buildServiceNameForToken(configuration,
+ serviceID, nodeIDList);
+ fail("testBuildServiceNameForTokenIncorrectConfig failed");
+} catch (IllegalArgumentException ex) {
+ GenericTestUtils.assertExceptionContains("Could not find rpcAddress " +
+ "for", ex);
+}
+
+
+ }
+
+
+ private String buildOMNodeAddresses(Collection nodeIDList,
+ String serviceID) {
+StringBuilder omServiceAddrBuilder = new StringBuilder();
+int port = 9862;
+int nodesLength = nodeIDList.size();
+int counter = 0;
+for (String nodeID : nodeIDList) {
+ counter++;
+ String addr = nodeID + ":" + port++;
+ configuration.set(OmUtils.addKeySuffixes(
+
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=320780=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320780
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 30/Sep/19 19:09
Start Date: 30/Sep/19 19:09
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r329740302
##
File path:
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
##
@@ -0,0 +1,129 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.s3.util;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.apache.hadoop.security.SecurityUtil;
+import org.apache.hadoop.test.GenericTestUtils;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Collection;
+
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP;
+import static org.junit.Assert.fail;
+
+/**
+ * Class used to test OzoneS3Util.
+ */
+public class TestOzoneS3Util {
+
+
+ private OzoneConfiguration configuration;
+
+ @Before
+ public void setConf() {
+configuration = new OzoneConfiguration();
+String serviceID = "omService";
+String nodeIDs = "om1,om2,om3";
+configuration.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+configuration.set(OMConfigKeys.OZONE_OM_NODE_ID_KEY, nodeIDs);
+configuration.setBoolean(HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, false);
+ }
+
+ @Test
+ public void testBuildServiceNameForToken() {
+String serviceID = "omService";
Review comment:
Line 55-59 is redundant since you have them in setConf before each test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 320780)
Time Spent: 2h 50m (was: 2h 40m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=319879=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319879 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 27/Sep/19 22:53 Start Date: 27/Sep/19 22:53 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-536120276 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 37 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 36 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 44 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-hdds in trunk failed. | | -1 | compile | 16 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 63 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 865 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 25 | hadoop-hdds in trunk failed. | | -1 | javadoc | 20 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 983 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 47 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 34 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 37 | hadoop-ozone in the patch failed. | | -1 | compile | 26 | hadoop-hdds in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 26 | hadoop-hdds in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 59 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 730 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in the patch failed. | | -1 | javadoc | 21 | hadoop-ozone in the patch failed. | | -1 | findbugs | 32 | hadoop-hdds in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 28 | hadoop-hdds in the patch failed. | | -1 | unit | 27 | hadoop-ozone in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 2416 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux edb0fc8744f3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 13b427f | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/3/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=319834=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319834 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 27/Sep/19 21:06 Start Date: 27/Sep/19 21:06 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-536096732 Ping @xiaoyuyao for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 319834) Time Spent: 2.5h (was: 2h 20m) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317925=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317925 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 25/Sep/19 00:28 Start Date: 25/Sep/19 00:28 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-534799294 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 41 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | -1 | compile | 22 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 63 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 856 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 23 | hadoop-hdds in trunk failed. | | -1 | javadoc | 25 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 963 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 33 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 36 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 29 | hadoop-ozone in the patch failed. | | -1 | compile | 24 | hadoop-hdds in the patch failed. | | -1 | compile | 20 | hadoop-ozone in the patch failed. | | -1 | javac | 24 | hadoop-hdds in the patch failed. | | -1 | javac | 20 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 41 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 708 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in the patch failed. | | -1 | javadoc | 19 | hadoop-ozone in the patch failed. | | -1 | findbugs | 31 | hadoop-hdds in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 29 | hadoop-hdds in the patch failed. | | -1 | unit | 24 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 2355 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e8ede916717d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / a346381 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317908=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317908 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 24/Sep/19 23:48 Start Date: 24/Sep/19 23:48 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-534790961 Thank You @xiaoyuyao for the review. I have addressed review comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 317908) Time Spent: 1.5h (was: 1h 20m) > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317909
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 24/Sep/19 23:48
Start Date: 24/Sep/19 23:48
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327880432
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/OzoneS3Util.java
##
@@ -33,4 +42,29 @@ public static String getVolumeName(String userName) {
Objects.requireNonNull(userName);
return DigestUtils.md5Hex(userName);
}
+
+ /**
+ * Generate service Name for token.
+ * @param configuration
+ * @param serviceId - ozone manager service ID
+ * @param omNodeIds - list of node ids for the given OM service.
+ * @return service Name.
+ */
+ public static String buildServiceNameForToken(
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317909)
Time Spent: 1h 40m (was: 1.5h)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317910=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317910
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 24/Sep/19 23:48
Start Date: 24/Sep/19 23:48
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327880447
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317910)
Time Spent: 1h 50m (was: 1h 40m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317912=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317912
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 24/Sep/19 23:48
Start Date: 24/Sep/19 23:48
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327880447
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
Review comment:
Existing one. Fixed in this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317912)
Time Spent: 2h (was: 1h 50m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=317913=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317913
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 24/Sep/19 23:48
Start Date: 24/Sep/19 23:48
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1489:
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327880512
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
+ getOmAddressForClients(conf));
+} else {
+ // HA cluster
+ Collection serviceIds =
Review comment:
fixed it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317913)
Time Spent: 2h 10m (was: 2h)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=316889=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316889
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 23/Sep/19 18:11
Start Date: 23/Sep/19 18:11
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327256562
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/OzoneS3Util.java
##
@@ -33,4 +42,29 @@ public static String getVolumeName(String userName) {
Objects.requireNonNull(userName);
return DigestUtils.md5Hex(userName);
}
+
+ /**
+ * Generate service Name for token.
+ * @param configuration
+ * @param serviceId - ozone manager service ID
+ * @param omNodeIds - list of node ids for the given OM service.
+ * @return service Name.
+ */
+ public static String buildServiceNameForToken(
Review comment:
Can we add some unit tests for this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 316889)
Time Spent: 1h 20m (was: 1h 10m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=316881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316881
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 23/Sep/19 18:03
Start Date: 23/Sep/19 18:03
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327252888
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
Review comment:
NIT: typo: you mean omServiceAddr?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 316881)
Time Spent: 1h 10m (was: 1h)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=316879=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316879
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 23/Sep/19 18:01
Start Date: 23/Sep/19 18:01
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327251983
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
+ getOmAddressForClients(conf));
+} else {
+ // HA cluster
+ Collection serviceIds =
Review comment:
We can reuse the serviceIDList above to avoid unnecessary call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 316879)
Time Spent: 50m (was: 40m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[
https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=316880=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316880
]
ASF GitHub Bot logged work on HDDS-2019:
Author: ASF GitHub Bot
Created on: 23/Sep/19 18:01
Start Date: 23/Sep/19 18:01
Worklog Time Spent: 10m
Work Description: xiaoyuyao commented on pull request #1489: HDDS-2019.
Handle Set DtService of token in S3Gateway for OM HA.
URL: https://github.com/apache/hadoop/pull/1489#discussion_r327251983
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
##
@@ -34,13 +41,45 @@
private Text omServiceAdd;
+ private String omserviceID;
+
@Inject
private OzoneConfiguration conf;
@PostConstruct
public void init() {
-omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
-getOmAddressForClients(conf));
+Collection serviceIdList =
+conf.getTrimmedStringCollection(OZONE_OM_SERVICE_IDS_KEY);
+if (serviceIdList.size() == 0) {
+ // Non-HA cluster
+ omServiceAdd = SecurityUtil.buildTokenService(OmUtils.
+ getOmAddressForClients(conf));
+} else {
+ // HA cluster
+ Collection serviceIds =
Review comment:
We can reuse the serviceIdList above to avoid unnecessary call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 316880)
Time Spent: 1h (was: 50m)
> Handle Set DtService of token in S3Gateway for OM HA
>
>
> Key: HDDS-2019
> URL: https://issues.apache.org/jira/browse/HDDS-2019
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> When OM HA is enabled, when tokens are generated, the service name should be
> set with address of all OM's.
>
> Current without HA, it is set with Om RpcAddress string. This Jira is to
> handle:
> # Set dtService with all OM address. Right now in OMClientProducer, UGI is
> created with S3 token, and serviceName of token is set with OMAddress, for HA
> case, this should be set with all OM RPC addresses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=316054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316054 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 21/Sep/19 04:35 Start Date: 21/Sep/19 04:35 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533766921 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 31 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 65 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 109 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 49 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 179 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 24 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 35 | hadoop-ozone in the patch failed. | | -1 | compile | 23 | hadoop-ozone in the patch failed. | | -1 | javac | 23 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 28 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedclient | 32 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 53 | hadoop-ozone in the patch failed. | | -1 | findbugs | 25 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 258 | hadoop-hdds in the patch passed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 1819 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux a444e8fa4173 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/testReport/ | | Max. process+thread count | 542 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=315962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315962 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 20/Sep/19 22:15 Start Date: 20/Sep/19 22:15 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533728099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 46 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 35 | hadoop-ozone in trunk failed. | | -1 | compile | 24 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 82 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 145 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 56 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 226 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 25 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 33 | hadoop-ozone in the patch failed. | | -1 | compile | 22 | hadoop-ozone in the patch failed. | | -1 | javac | 22 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 30 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedclient | 32 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 56 | hadoop-ozone in the patch failed. | | -1 | findbugs | 26 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 266 | hadoop-hdds in the patch passed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 2058 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 1395f9604b97 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/2/testReport/ | | Max. process+thread count | 478 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=315955=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315955 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 20/Sep/19 22:05 Start Date: 20/Sep/19 22:05 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489#issuecomment-533725569 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 75 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | -1 | compile | 19 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 59 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | -1 | shadedclient | 108 | branch has errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 174 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 20 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 28 | hadoop-ozone in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 25 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | -1 | shadedclient | 28 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 276 | hadoop-hdds in the patch passed. | | -1 | unit | 22 | hadoop-ozone in the patch failed. | | +1 | asflicense | 24 | The patch does not generate ASF License warnings. | | | | 1735 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1489 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux addcf29dbc03 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d7d6ec8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1489/1/testReport/ | | Max. process+thread count | 429 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/s3gateway U: hadoop-ozone/s3gateway | | Console output |
[jira] [Work logged] (HDDS-2019) Handle Set DtService of token in S3Gateway for OM HA
[ https://issues.apache.org/jira/browse/HDDS-2019?focusedWorklogId=315946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315946 ] ASF GitHub Bot logged work on HDDS-2019: Author: ASF GitHub Bot Created on: 20/Sep/19 21:34 Start Date: 20/Sep/19 21:34 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1489: HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. URL: https://github.com/apache/hadoop/pull/1489 Need some work on server-side for adding secure-om-s3-ha cluster, as some fields like keytab are still not supporting parameter with serviceid and node id. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 315946) Remaining Estimate: 0h Time Spent: 10m > Handle Set DtService of token in S3Gateway for OM HA > > > Key: HDDS-2019 > URL: https://issues.apache.org/jira/browse/HDDS-2019 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Critical > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When OM HA is enabled, when tokens are generated, the service name should be > set with address of all OM's. > > Current without HA, it is set with Om RpcAddress string. This Jira is to > handle: > # Set dtService with all OM address. Right now in OMClientProducer, UGI is > created with S3 token, and serviceName of token is set with OMAddress, for HA > case, this should be set with all OM RPC addresses. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
