[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563964=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563964 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Mar/21 18:25 Start Date: 10/Mar/21 18:25 Worklog Time Spent: 10m Work Description: steveloughran merged pull request #2597: URL: https://github.com/apache/hadoop/pull/2597 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 563964) Time Spent: 14h (was: 13h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 14h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563956=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563956 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Mar/21 18:21 Start Date: 10/Mar/21 18:21 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-759410406 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 32m 21s | branch-3.3 passed | | +1 :green_heart: | mvnsite | 0m 42s | branch-3.3 passed | | +1 :green_heart: | shadedclient | 15m 39s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | -1 :x: | shellcheck | 0m 1s | The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 14s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 34s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 69m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 715a6a471575 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 75f34a1 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/artifact/out/diff-patch-shellcheck.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/testReport/ | | Max. process+thread count | 535 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 563956) Time Spent: 13h 50m (was: 13h 40m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature.
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563306=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563306 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Mar/21 19:54 Start Date: 09/Mar/21 19:54 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-794374377 I'm not worried about shellcheck if its some version mismatch there. Is the change actually enabling a check which is disabled on the later branch? if so: interesting. Do you think this is ready to merge? If so, do a rebase, run the tests, push up the rebased PR and I'll do the merge This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 563306) Time Spent: 13h 40m (was: 13.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563305=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563305 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Mar/21 19:51 Start Date: 09/Mar/21 19:51 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-757514175 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 563305) Time Spent: 13.5h (was: 13h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563304=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563304 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Mar/21 19:51 Start Date: 09/Mar/21 19:51 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-755239267 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 563304) Time Spent: 13h 20m (was: 13h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=559175=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-559175 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 01/Mar/21 03:48 Start Date: 01/Mar/21 03:48 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-787615880 Hi @steveloughran, request you to review this for backporting to branch-3.3 [The yetus warnings are possibly due to an older version of shellcheck used for this branch (0.3.7), while trunk had 0.4.6. Let me know if you have any concerns about it, or if we should suppress these warnings in the code.] This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 559175) Time Spent: 13h 10m (was: 13h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=537225=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-537225 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 18/Jan/21 04:29 Start Date: 18/Jan/21 04:29 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-761972135 Yetus shellcheck warnings about unused variables in `runtests.sh` are not valid. Variables declared in the runtests script are used in `testsupport.sh`; this is evident from test results that contain each of the three combinations (HNS-OAuth, HNS-SharedKey, NonHNS-SharedKey) specified in the runtests script. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 537225) Time Spent: 13h (was: 12h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 13h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=535333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-535333 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 13/Jan/21 12:15 Start Date: 13/Jan/21 12:15 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-759410406 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 32m 21s | branch-3.3 passed | | +1 :green_heart: | mvnsite | 0m 42s | branch-3.3 passed | | +1 :green_heart: | shadedclient | 15m 39s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | -1 :x: | shellcheck | 0m 1s | The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 14s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 34s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 69m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 715a6a471575 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 75f34a1 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/artifact/out/diff-patch-shellcheck.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/testReport/ | | Max. process+thread count | 535 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/7/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 535333) Time Spent: 12h 50m (was: 12h 40m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 12h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=535267=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-535267 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 13/Jan/21 09:35 Start Date: 13/Jan/21 09:35 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-759326481 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 431, Failures: 0, Errors: 0, Skipped: 68 [WARNING] Tests run: 204, Failures: 0, Errors: 0, Skipped: 24 HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 24 [WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 16 Non-HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 456, Failures: 0, Errors: 0, Skipped: 247 [WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 16 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 535267) Time Spent: 12h 40m (was: 12.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 12h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=534813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534813 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 12/Jan/21 12:33 Start Date: 12/Jan/21 12:33 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-758626503 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 27m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 40m 0s | branch-3.3 passed | | +1 :green_heart: | mvnsite | 0m 49s | branch-3.3 passed | | +1 :green_heart: | shadedclient | 19m 11s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | the patch passed | | +1 :green_heart: | mvnsite | 0m 32s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 16s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 19m 12s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 30s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 111m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 2ebe87519afd 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / b3f8557 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/6/testReport/ | | Max. process+thread count | 555 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/6/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534813) Time Spent: 12.5h (was: 12h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 12.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. >
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=534787=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534787 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 12/Jan/21 10:51 Start Date: 12/Jan/21 10:51 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-758574414 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | -1 :x: | mvninstall | 29m 7s | root in branch-3.3 failed. | | -1 :x: | mvnsite | 0m 24s | hadoop-azure in branch-3.3 failed. | | +1 :green_heart: | shadedclient | 0m 31s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 38s | the patch passed | | +1 :green_heart: | mvnsite | 0m 45s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 24s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 9m 58s | patch has errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 48s | hadoop-azure in the patch passed. | | -1 :x: | asflicense | 0m 44s | The patch generated 1 ASF License warnings. | | | | 47m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux a128c0e5f9ef 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / b3f8557 | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/artifact/out/branch-mvninstall-root.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/testReport/ | | asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 94 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/5/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534787) Time Spent: 12h 20m (was: 12h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent:
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=534712=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534712 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 12/Jan/21 07:13 Start Date: 12/Jan/21 07:13 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-758457060 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 59s | branch-3.3 passed | | +1 :green_heart: | mvnsite | 0m 46s | branch-3.3 passed | | +1 :green_heart: | shadedclient | 15m 23s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | -1 :x: | shellcheck | 0m 1s | The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | shelldocs | 0m 18s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 9s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 34s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 68m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 956815ef4bb9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / b74d642 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/4/artifact/out/diff-patch-shellcheck.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/4/testReport/ | | Max. process+thread count | 535 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/4/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534712) Time Spent: 12h 10m (was: 12h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 12h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=534035=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534035 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Jan/21 17:40 Start Date: 10/Jan/21 17:40 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-757514175 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 29m 55s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 32m 14s | branch-3.3 passed | | +1 :green_heart: | mvnsite | 0m 44s | branch-3.3 passed | | +1 :green_heart: | shadedclient | 15m 21s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | -1 :x: | shellcheck | 0m 0s | The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | | +1 :green_heart: | shelldocs | 0m 19s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 2s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 34s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 97m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 2d19777d6ef5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 9869013 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/3/artifact/out/diff-patch-shellcheck.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/3/testReport/ | | Max. process+thread count | 662 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/3/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534035) Time Spent: 12h (was: 11h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 12h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=533544=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-533544 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Jan/21 03:53 Start Date: 10/Jan/21 03:53 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-756625767 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 68 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24 HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 24 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 Non-HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 249 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 533544) Time Spent: 11h 50m (was: 11h 40m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 11h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=532904=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532904 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 08/Jan/21 08:37 Start Date: 08/Jan/21 08:37 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-756625767 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 68 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24 HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 24 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 Non-HNS SharedKey [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 461, Failures: 0, Errors: 0, Skipped: 249 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532904) Time Spent: 11h 40m (was: 11.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 11h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=532885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532885 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 08/Jan/21 07:55 Start Date: 08/Jan/21 07:55 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-756608457 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 25m 58s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | -1 :x: | mvninstall | 6m 11s | root in branch-3.3 failed. | | -1 :x: | mvnsite | 0m 27s | hadoop-azure in branch-3.3 failed. | | -1 :x: | shadedclient | 1m 5s | branch has errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 15s | hadoop-azure in the patch failed. | | -1 :x: | mvnsite | 0m 14s | hadoop-azure in the patch failed. | | -1 :x: | shellcheck | 0m 1s | The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | | +1 :green_heart: | shelldocs | 0m 16s | There were no new shelldocs issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 0m 58s | patch has errors when building and testing our client artifacts. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 14s | hadoop-azure in the patch failed. | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 38m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 50a82dd00090 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 3748c25 | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/branch-mvninstall-root.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/diff-patch-shellcheck.txt | | unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/testReport/ | | Max. process+thread count | 93 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/2/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532885) Time Spent: 11.5h (was: 11h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > >
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531832=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531832 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 11:33 Start Date: 06/Jan/21 11:33 Worklog Time Spent: 10m Work Description: sumangala-patki edited a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-755247313 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 450, Failures: 0, Errors: 0, Skipped: 73 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24 ``` ``` HNS SharedKey [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [ERROR] ITestAzureBlobFileSystemDelegationSAS.testCheckAccess:94 » AbfsRestOperation O... [ERROR] ITestAzureBlobFileSystemDelegationSAS.testRootPath:343 » AbfsRestOperation Ope... [ERROR] Tests run: 450, Failures: 0, Errors: 2, Skipped: 33 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` ``` Non-HNS SharedKey [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 450, Failures: 0, Errors: 0, Skipped: 245 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` Errors tracked in JIRA [Delegation SAS](https://issues.apache.org/jira/browse/HADOOP-17397) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531832) Time Spent: 11h 20m (was: 11h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 11h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531831=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531831 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 11:31 Start Date: 06/Jan/21 11:31 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-755247313 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US HNS OAuth ``` [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 450, Failures: 0, Errors: 0, Skipped: 73 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24 ``` HNS SharedKey ``` [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [ERROR] ITestAzureBlobFileSystemDelegationSAS.testCheckAccess:94 » AbfsRestOperation O... [ERROR] ITestAzureBlobFileSystemDelegationSAS.testRootPath:343 » AbfsRestOperation Ope... [ERROR] Tests run: 450, Failures: 0, Errors: 2, Skipped: 33 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` Non-HNS SharedKey ``` [INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 450, Failures: 0, Errors: 0, Skipped: 245 [WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16 ``` Errors tracked in JIRA [Delegation SAS](https://issues.apache.org/jira/browse/HADOOP-17397) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531831) Time Spent: 11h 10m (was: 11h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 11h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531821=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531821 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 11:12 Start Date: 06/Jan/21 11:12 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-755239267 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 25m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | -1 :x: | mvninstall | 6m 10s | root in branch-3.3 failed. | | -1 :x: | mvnsite | 0m 24s | hadoop-azure in branch-3.3 failed. | | -1 :x: | shadedclient | 0m 55s | branch has errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 13s | hadoop-azure in the patch failed. | | -1 :x: | mvnsite | 0m 13s | hadoop-azure in the patch failed. | | -1 :x: | shellcheck | 0m 0s | The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | | +1 :green_heart: | shelldocs | 0m 15s | There were no new shelldocs issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 0m 45s | patch has errors when building and testing our client artifacts. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 12s | hadoop-azure in the patch failed. | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 36m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2597 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux f51821d66288 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 3748c25 | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/branch-mvninstall-root.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/diff-patch-shellcheck.txt | | unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/testReport/ | | Max. process+thread count | 93 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2597/1/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531821) Time Spent: 11h (was: 10h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > >
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531801=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531801 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 10:34 Start Date: 06/Jan/21 10:34 Worklog Time Spent: 10m Work Description: sumangala-patki opened a new pull request #2597: URL: https://github.com/apache/hadoop/pull/2597 - Contributed by Bilahari T H (cherry picked from commit [4c033ba](https://github.com/apache/hadoop/commit/4c033bafa02855722a901def4773a6a15b214318#diff-3750721475f40454079559d1809fd98fc156fcdeffb1fe1556e69786f1b5166f)) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531801) Time Spent: 10h 50m (was: 10h 40m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 10h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531744 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 08:02 Start Date: 06/Jan/21 08:02 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2596: URL: https://github.com/apache/hadoop/pull/2596#issuecomment-755148171 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 24m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-3.3 Compile Tests _ | | -1 :x: | mvninstall | 6m 7s | root in branch-3.3 failed. | | -1 :x: | mvnsite | 0m 26s | hadoop-azure in branch-3.3 failed. | | -1 :x: | shadedclient | 0m 52s | branch has errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 14s | hadoop-azure in the patch failed. | | -1 :x: | mvnsite | 0m 12s | hadoop-azure in the patch failed. | | -1 :x: | shellcheck | 0m 0s | The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | | +1 :green_heart: | shelldocs | 0m 14s | There were no new shelldocs issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 0m 45s | patch has errors when building and testing our client artifacts. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 15s | hadoop-azure in the patch failed. | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 35m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2596 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint xml | | uname | Linux 668102721f2d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 3748c25 | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/branch-mvninstall-root.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt | | mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt | | mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/diff-patch-shellcheck.txt | | unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/testReport/ | | Max. process+thread count | 93 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2596/1/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.3.7 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531744) Time Spent: 10h 40m (was: 10.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > >
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531732 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 07:42 Start Date: 06/Jan/21 07:42 Worklog Time Spent: 10m Work Description: sumangala-patki closed pull request #2596: URL: https://github.com/apache/hadoop/pull/2596 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531732) Time Spent: 10.5h (was: 10h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 10.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531727=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531727 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Jan/21 07:25 Start Date: 06/Jan/21 07:25 Worklog Time Spent: 10m Work Description: sumangala-patki opened a new pull request #2596: URL: https://github.com/apache/hadoop/pull/2596 - Contributed by Bilahari T H (cherry picked from commit [4c033ba](https://github.com/apache/hadoop/commit/4c033bafa02855722a901def4773a6a15b214318#diff-3750721475f40454079559d1809fd98fc156fcdeffb1fe1556e69786f1b5166f)) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531727) Time Spent: 10h 20m (was: 10h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 10h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=525194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-525194 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 16/Dec/20 18:35 Start Date: 16/Dec/20 18:35 Worklog Time Spent: 10m Work Description: DadanielZ merged pull request #2417: URL: https://github.com/apache/hadoop/pull/2417 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 525194) Time Spent: 10h 10m (was: 10h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 10h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=524258=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524258
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 15/Dec/20 05:30
Start Date: 15/Dec/20 05:30
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r543057835
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,230 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combinationConfigFiles
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combinationConfigFiles/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >>
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=524257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524257
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 15/Dec/20 05:29
Start Date: 15/Dec/20 05:29
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r543057671
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -592,6 +592,61 @@ with the Hadoop Distributed File System permissions model
when hierarchical
namespace is enabled for the storage account. Furthermore, the metadata and
data
produced by ADLS Gen 2 REST API can be consumed by Blob REST API, and vice
versa.
+## Generating test run configurations and test triggers over various config
combinations
+
+To simplify the testing across various authentication and features combinations
+that are mandatory for a PR, script `dev-support/testrun-scripts/runtests.sh`
+should be used. Once the script is updated with relevant config settings for
+various test combinations, it will:
+1. Auto-generate configs specific to each test combinations
+2. Run tests for all combinations
+3. Summarize results across all the test combination runs.
+
+As a pre-requiste step, fill config values for test accounts and credentials
+needed for authentication in `src/test/resources/azure-auth-keys.xml.template`
+and rename as `src/test/resources/azure-auth-keys.xml`.
+
+**To add a new test combination:** Templates for mandatory test combinations
+for PR validation are present in `dev-support/testrun-scripts/runtests.sh`.
+If a new one needs to be added, add a combination set within
+`dev-support/testrun-scripts/runtests.sh` similar to the ones already defined
+and
+1. Provide a new combination name
+2. Update properties and values array which need to be effective for the test
+combination
+3. Call generateconfigs
+
+**To run PR validation:** Running command
+* `dev-support/testrun-scripts/runtests.sh` will generate configurations for
+each of the combinations defined and run tests for all the combinations.
+* `dev-support/testrun-scripts/runtests.sh -c {combinationname}` Specific
+combinations can be provided with -c option. If combinations are provided
+with -c option, tests for only those combinations will be run.
+
+**Test logs:** Test runs will create a folder within dev-support/testlogs to
+save the test logs. Folder name will be the test start timestamp. The mvn
verify
+command line logs for each combination will be saved into a file as
+Test-Logs-$combination.txt into this folder. In case of any failures, this file
+will have the failure exception stack. At the end of the test run, the
+consolidated results of all the combination runs will be saved into a file as
+Test-Results.log in the same folder. When run for PR validation, the
+consolidated test results needs to be pasted into the PR comment section.
+
+**To generate config for use in IDE:** Running command with -a (activate)
option
+`dev-support/testrun-scripts/runtests.sh -a {combination name}` will update
+the effective config relevant for the specific test combination. Hence the same
+config files used by the mvn test runs can be used for IDE without any manual
+updates needed within config file.
+
+**Other command line options:**
+* -a Specify the combination name which needs to be
Review comment:
Also user is expected to know the combnations from the runtests.sh file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 524257)
Time Spent: 9h 50m (was: 9h 40m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=524256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524256
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 15/Dec/20 05:28
Start Date: 15/Dec/20 05:28
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r543057354
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -592,6 +592,61 @@ with the Hadoop Distributed File System permissions model
when hierarchical
namespace is enabled for the storage account. Furthermore, the metadata and
data
produced by ADLS Gen 2 REST API can be consumed by Blob REST API, and vice
versa.
+## Generating test run configurations and test triggers over various config
combinations
+
+To simplify the testing across various authentication and features combinations
+that are mandatory for a PR, script `dev-support/testrun-scripts/runtests.sh`
+should be used. Once the script is updated with relevant config settings for
+various test combinations, it will:
+1. Auto-generate configs specific to each test combinations
+2. Run tests for all combinations
+3. Summarize results across all the test combination runs.
+
+As a pre-requiste step, fill config values for test accounts and credentials
+needed for authentication in `src/test/resources/azure-auth-keys.xml.template`
+and rename as `src/test/resources/azure-auth-keys.xml`.
+
+**To add a new test combination:** Templates for mandatory test combinations
+for PR validation are present in `dev-support/testrun-scripts/runtests.sh`.
+If a new one needs to be added, add a combination set within
+`dev-support/testrun-scripts/runtests.sh` similar to the ones already defined
+and
+1. Provide a new combination name
+2. Update properties and values array which need to be effective for the test
+combination
+3. Call generateconfigs
+
+**To run PR validation:** Running command
+* `dev-support/testrun-scripts/runtests.sh` will generate configurations for
+each of the combinations defined and run tests for all the combinations.
+* `dev-support/testrun-scripts/runtests.sh -c {combinationname}` Specific
+combinations can be provided with -c option. If combinations are provided
+with -c option, tests for only those combinations will be run.
+
+**Test logs:** Test runs will create a folder within dev-support/testlogs to
+save the test logs. Folder name will be the test start timestamp. The mvn
verify
+command line logs for each combination will be saved into a file as
+Test-Logs-$combination.txt into this folder. In case of any failures, this file
+will have the failure exception stack. At the end of the test run, the
+consolidated results of all the combination runs will be saved into a file as
+Test-Results.log in the same folder. When run for PR validation, the
+consolidated test results needs to be pasted into the PR comment section.
+
+**To generate config for use in IDE:** Running command with -a (activate)
option
+`dev-support/testrun-scripts/runtests.sh -a {combination name}` will update
+the effective config relevant for the specific test combination. Hence the same
+config files used by the mvn test runs can be used for IDE without any manual
+updates needed within config file.
+
+**Other command line options:**
+* -a Specify the combination name which needs to be
Review comment:
I have done it in the script. It won't be right to add here, as the user
of the script can add, change the combinations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 524256)
Time Spent: 9h 40m (was: 9.5h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth,
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=524197=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524197
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 15/Dec/20 01:13
Start Date: 15/Dec/20 01:13
Worklog Time Spent: 10m
Work Description: DadanielZ commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r542968091
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,230 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combinationConfigFiles
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combinationConfigFiles/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >> "$testresultsfilename"
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=519367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-519367 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 03/Dec/20 04:41 Start Date: 03/Dec/20 04:41 Worklog Time Spent: 10m Work Description: bilaharith closed pull request #2278: URL: https://github.com/apache/hadoop/pull/2278 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 519367) Time Spent: 9h 20m (was: 9h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 9h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=508013=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-508013
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 05/Nov/20 07:57
Start Date: 05/Nov/20 07:57
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#issuecomment-722209212
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 18s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 69m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2417 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux b742b1883cfc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7e52c0975f7 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/5/testReport/ |
| Max. process+thread count | 404 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/5/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 508013)
Time Spent: 9h 10m (was: 9h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507975=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507975 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 05/Nov/20 06:53 Start Date: 05/Nov/20 06:53 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2417: URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825936 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to handle large files —t Tests at this scale are slow: they are best executed from hosts running in the cloud infrastructure where the storage endpoint is based. +##No test no review: Run different combinations of tests using the runtests.sh script Review comment: Done ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to handle large files —t Tests at this scale are slow: they are best executed from hosts running in the cloud infrastructure where the storage endpoint is based. +##No test no review: Run different combinations of tests using the runtests.sh script + +This is the expected way in which the tests have to be ran before raising a PR. Review comment: Done ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to handle large files —t Tests at this scale are slow: they are best executed from hosts running in the cloud infrastructure where the storage endpoint is based. +##No test no review: Run different combinations of tests using the runtests.sh script + +This is the expected way in which the tests have to be ran before raising a PR. +The script `runtests.sh` contain template for 3 combinations of tests. Ensure +the auth configs for all the accounts used for testing are provided in +azure-auth-keys.xml. In case any new flags or properties are introduced +with the code change, add the combinations with the possible configurations +into the `runtests.sh`. The thread count can be specified as the command line +argument for the script. By default the same will be 8. -n option can be +specified if build is not required prior to the tests. + +Adding a combination of tests involves setting the variable combination (ex: HNS +-OAuth) and specifying the specific configurations for the particular +combination with 2 arrays namely properties and values. Specify the property +names within the array properties and corresponding values in the values +array. The property and value is determined by the array index. The value for +the property mentioned at index 1 of array properties should be specified at +index 1 of the array values. Call the function generateconfigs once the 3 +values mentioned are set. Now the script `runtests.sh` is ready to be ran. + +Once the tests are completed, logs will be present in the directory +dev-support/testrun-scripts/testlogs/$startname, $startname will be the start Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 507975) Time Spent: 9h (was: 8h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 9h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc..
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507974=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507974
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 05/Nov/20 06:52
Start Date: 05/Nov/20 06:52
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825878
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,224 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combconfs
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+shouldbuild=1
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combconfs/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >> "$testresultsfilename"
+
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507973=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507973 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 05/Nov/20 06:52 Start Date: 05/Nov/20 06:52 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2417: URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825795 ## File path: hadoop-tools/hadoop-azure/.gitignore ## @@ -1,2 +1,5 @@ .checkstyle -bin/ \ No newline at end of file +bin/ +src/test/resources/combconfs Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 507973) Time Spent: 8h 40m (was: 8.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 8h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507971
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 05/Nov/20 06:51
Start Date: 05/Nov/20 06:51
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825422
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,224 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combconfs
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+shouldbuild=1
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combconfs/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >> "$testresultsfilename"
+
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507972=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507972
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 05/Nov/20 06:51
Start Date: 05/Nov/20 06:51
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825492
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,224 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combconfs
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+shouldbuild=1
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combconfs/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >> "$testresultsfilename"
+
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507970=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507970
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 05/Nov/20 06:50
Start Date: 05/Nov/20 06:50
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r517825112
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,224 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+testresourcesdir=src/test/resources
+combconfsdir=$testresourcesdir/combconfs
+combtestfile=$testresourcesdir/abfs-combination-test-configs.xml
+
+shouldbuild=1
+
+logdir=dev-support/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+properties=
+values=
+
+validate() {
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$combination" ]; then
+ echo "Exiting. combination cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for combination $combination"
+exit -1
+ fi
+
+ for filename in "${combinations[@]}"; do
+if [[ ! -f "$combconfsdir/$filename.xml" ]]; then
+ echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+ exit -1
+fi
+ done
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+cleancombinationconfigs() {
+ rm -rf $combconfsdir
+ mkdir -p $combconfsdir
+}
+
+generateconfigs() {
+ combconffile="$combconfsdir/$combination.xml"
+ rm -rf "$combconffile"
+ cat > "$combconffile" << ENDOFFILE
+
+
+
+ENDOFFILE
+
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $combination"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ formatxml "$combconffile"
+}
+
+formatxml() {
+ xmlstarlet fo -s 2 "$1" > "$1.tmp"
+ mv "$1.tmp" "$1"
+}
+
+setactiveconf() {
+ if [[ ! -f "$combconfsdir/$combination.xml" ]]; then
+echo "Exiting. Combination config file ($combconfsdir/$combination.xml)
does not exist."
+exit -1
+ fi
+ rm -rf $combtestfile
+ cat > $combtestfile << ENDOFFILE
+
+
+
+ENDOFFILE
+ xmlstarlet ed -P -L -s /configuration -t elem -n include -v "" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n href -v
"combconfs/$combination.xml" $combtestfile
+ xmlstarlet ed -P -L -i /configuration/include -t attr -n xmlns -v
"http://www.w3.org/2001/XInclude; $combtestfile
+ formatxml $combtestfile
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" "$combconffile"
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property "$combconffile"
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" "$combconffile"
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+summary() {
+ {
+echo ""
+echo "$combination"
+echo ""
+pcregrep -M "$testresultsregex" "$testlogfilename"
+ } >> "$testresultsfilename"
+
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=507624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507624
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 04/Nov/20 12:41
Start Date: 04/Nov/20 12:41
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#issuecomment-721709084
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 28m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 4m 37s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/4/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | mvnsite | 2m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 16s | | There were no new
shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 34s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 72m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2417 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux af63183c3787 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3067a25fa12 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/4/testReport/ |
| Max. process+thread count | 423 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/4/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 507624)
Time Spent: 8h (was: 7h 50m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=506990=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506990
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 03/Nov/20 14:08
Start Date: 03/Nov/20 14:08
Worklog Time Spent: 10m
Work Description: snvijaya commented on a change in pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#discussion_r515741344
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to
handle large files —t
Tests at this scale are slow: they are best executed from hosts running in
the cloud infrastructure where the storage endpoint is based.
+##No test no review: Run different combinations of tests using the runtests.sh
script
+
+This is the expected way in which the tests have to be ran before raising a PR.
Review comment:
Change to:
To simplify the testing across various authentication and features
combinations that are mandatory for a PR, script
`dev-support/testrun-scripts/runTest.sh` should be used. Once the script is
updated with relevant config settings for various test combinations, it will:
1. Auto-generate configs specific to each test combinations
2. Run tests for all combinations
3. Summarize results across all the test combination runs.
As a pre-requiste step, fill config values for test accounts and credentials
needed for authentication in `src/test/resources/azure-auth-keys.xml.template`
and rename as `src/test/resources/azure-auth-keys.xml`.
**To add a new test combination**: Mandatory test combinations for PR
validation are already pre-filled in `dev-support/testrun-scripts/runTest.sh`.
If a new one needs to be added, add a combination set within
`dev-support/testrun-scripts/runTest.sh` similar to the ones already defined and
1. Provide a new combination name
2. Update properties and values array which need to be effective for the
test combination
3. Call generateConfigs
**To run PR validation**: Running command
- `runTest.sh -testCombination #combinationname#` or `runTest.sh -tc
#combinationname#` : will generate configurations effective for that test
combination and also run the test.
- `runTest.sh -testCombination all` or `runTest.sh -tc all` : will generate
configurations for each of the combinations defined and run tests.
**Test logs**:
**To generate config for use in IDE**: Running command
`runTest.sh -generateConfig #combinationname#` or `runTest.sh -gc
#combinationname#`
will update the effective config relevant for the specific test combination.
Hence the same config files used by the mvn test runs can be used for IDE
without any manual updates needed within config file.
**Other command line options**:
1. Thread count : ABFS mvn tests are run in parallel mode. Tests by default
are run with 8 thread count. It can be changed by providing -t #ThreadCount#
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to
handle large files —t
Tests at this scale are slow: they are best executed from hosts running in
the cloud infrastructure where the storage endpoint is based.
+##No test no review: Run different combinations of tests using the runtests.sh
script
Review comment:
Change to
Generating test run configurations and test triggers over various
config combinations
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -357,6 +357,34 @@ The Huge File tests validate Azure storages's ability to
handle large files —t
Tests at this scale are slow: they are best executed from hosts running in
the cloud infrastructure where the storage endpoint is based.
+##No test no review: Run different combinations of tests using the runtests.sh
script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script `runtests.sh` contain template for 3 combinations of tests. Ensure
+the auth configs for all the accounts used for testing are provided in
+azure-auth-keys.xml. In case any new flags or properties are introduced
+with the code change, add the combinations with the possible configurations
+into the `runtests.sh`. The thread count can be specified as the command line
+argument for the script. By default the same will be 8. -n option can be
+specified if build is not required prior to the tests.
+
+Adding a combination of tests involves setting the variable combination (ex:
HNS
+-OAuth) and specifying the specific configurations for the particular
+combination with 2 arrays namely properties and values. Specify the property
+names
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=506971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506971
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 03/Nov/20 14:06
Start Date: 03/Nov/20 14:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#issuecomment-721103960
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 15s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 30s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 74m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2417 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux f8583c3523b7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e580280a8b0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/3/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/3/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 506971)
Time Spent: 7h 40m (was: 7.5h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=506555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506555
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 30/Oct/20 07:10
Start Date: 30/Oct/20 07:10
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#issuecomment-719329552
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 9s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 71m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2417 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux cc410d793fce 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8ee6bc2518b |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/2/testReport/ |
| Max. process+thread count | 359 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/2/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 506555)
Time Spent: 7.5h (was: 7h 20m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=505240=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-505240 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 27/Oct/20 14:11 Start Date: 27/Oct/20 14:11 Worklog Time Spent: 10m Work Description: bilaharith commented on pull request #2417: URL: https://github.com/apache/hadoop/pull/2417#issuecomment-717271366 **Driver test results using accounts in Central India** mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify HNS-OAuth [INFO] Results: [INFO] [INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0 [INFO] Results: [INFO] [WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 64 [INFO] Results: [INFO] [WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 24 HNS-SharedKey [INFO] Results: [INFO] [INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0 [INFO] Results: [INFO] [WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 24 [INFO] Results: [INFO] [WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 16 NonHNS-SharedKey [INFO] Results: [INFO] [INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0 [INFO] Results: [INFO] [WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 245 [INFO] Results: [INFO] [WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 16 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 505240) Time Spent: 7h 20m (was: 7h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 7h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=505226=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-505226
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 27/Oct/20 13:52
Start Date: 27/Oct/20 13:52
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2417:
URL: https://github.com/apache/hadoop/pull/2417#issuecomment-717258351
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 33m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 103m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2417 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 98e08ebb010d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae74407ac43 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/1/testReport/ |
| Max. process+thread count | 310 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2417/1/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 505226)
Time Spent: 7h 10m (was: 7h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=505176=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-505176 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 27/Oct/20 12:07 Start Date: 27/Oct/20 12:07 Worklog Time Spent: 10m Work Description: bilaharith opened a new pull request #2417: URL: https://github.com/apache/hadoop/pull/2417 ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. The PR introduces a shell script with which the developer can specify the configuration variants and get different combinations of tests executed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 505176) Time Spent: 7h (was: 6h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 7h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=502381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502381
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 18:49
Start Date: 19/Oct/20 18:49
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r507929259
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
Review comment:
set -eu for error reporting and rejection of uninited vars
##
File path: hadoop-tools/hadoop-azure/dev-support/testrun-scripts/testsupport.sh
##
@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/azure-auth-keys.xml
+bkpconffile=src/test/resources/azure-auth-keys_BKP.xml
+logdir=target/testlogs
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+threadcount=
+defaultthreadcount=8
+
+validate() {
+ if ! [ -s "$conffile" ] ; then
+echo "Exiting. conf file is empty"
+exit -1
+ fi
+ if ! xmlstarlet val --well-formed "$conffile" ; then
+echo "Exiting. conf file is not well formed xml"
+exit -1
+ fi
+ if [ -z "$threadcount" ] ; then
+threadcount=$defaultthreadcount
+ fi
+ numberegex='^[0-9]+$'
+ if ! [[ $threadcount =~ $numberegex ]] ; then
+echo "Exiting. The script param (threadcount) should be a number"
+exit -1
+ fi
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit -1
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit -1
+ fi
+
+ if ! [ -f "$conffile" ]; then
+echo "Exiting. $conffile missing"
+exit -1
+ fi
+}
+
+checkdependencies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit -1
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit -1
+ fi
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" $conffile
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property $conffile
+ if ! xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n
value -v "$2" $conffile
+ then
+echo "Exiting. Changing config property failed."
+exit -1
+ fi
+}
+
+removeproperties() {
+ propertiessize=${#properties[@]}
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+xmlstarlet ed -P -L -d "/configuration/property[name='$key']" $conffile
+ done
+}
+
+testwithconfs() {
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $scenario"
+exit -1
+ fi
+ for ((i = 0; i < propertiessize;
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=499352=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499352
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 12/Oct/20 11:59
Start Date: 12/Oct/20 11:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-707076328
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 32m 8s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 10s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 46s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 16m 6s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 2s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 100m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/17/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 678d2c73b453 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 69ef9b1ee88 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/17/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/17/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499352)
Time Spent: 6h 40m (was: 6.5h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=499315=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499315 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 12/Oct/20 10:20 Start Date: 12/Oct/20 10:20 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r503193523 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -80,7 +80,7 @@ tests against mocked storage, which is an in-memory emulation of Azure Storage. The integration tests are designed to test directly against an Azure storage service, and require an account and credentials in order to run. -This is done by creating the file to `src/test/resources/azure-auth-keys.xml` +This is done by creating the file to `src/test/resources/abfs-testrun-configs.xml` Review comment: Basically the script runs the following command mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=$threadcount verify >> "$testlogfilename" So this is intended for ABFS This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 499315) Time Spent: 6.5h (was: 6h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 6.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=499314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499314 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 12/Oct/20 10:18 Start Date: 12/Oct/20 10:18 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r503192587 ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,181 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 6h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496146
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 19:56
Start Date: 06/Oct/20 19:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704518289
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 28m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 17s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 92m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 743a91cf2e00 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/testReport/ |
| Max. process+thread count | 475 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496146)
Time Spent: 6h 10m (was: 6h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496129
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 19:27
Start Date: 06/Oct/20 19:27
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704504047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 51s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 16m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 68m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 822fade534f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496129)
Time Spent: 6h (was: 5h 50m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496092=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496092
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 18:22
Start Date: 06/Oct/20 18:22
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r500506248
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
+the auth configs for all the accounts used for testing are provided in
+abfs-testrun-configs.xml. In case any new flags or properties are introduced
+with the code change, add the combinations with the possible configurations
+into the runtests.sh. The thread count can be specified as the command line
+argument for the script. By default the same will be 8.
+
+Adding a combination of tests involves setting the variable scenario (ex: HNS
+-OAuth) and specifying the specific configurations for the particular
+combination with 2 arrays namely properties and values. Specify the property
+names within the array properties and corresponding values in the values
+array. The property and value is determined by the array index. The value for
+the property mentioned at index 1 of array properties should be specified at
+index 1 of the array values. Call the function runtestwithconfs once the 3
+values mentioned are set. Now the script runtests.sh is ready to be ran.
+
+Once the tests are completed, logs will be present in the directory
+target/testlogs. A consolidated test results will be present in the file
+Test-$starttime-Results.log, $startname will be the start time of the test.
+Similarly, the full test report can be found in individual log files, for each
+of the scenarios with the file name Test-$starttime-Logs-$scenario. Please
+attach the consolidates test results from the file Test-$starttime-Results.log
Review comment:
Done
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496092)
Time Spent: 5h 50m (was: 5h 40m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without hierarchical namespace support using various
> authorization mechanisms.
> Currently the developer has to manually run the tests with different
> combinations of configurations ex: HNS account with SharedKey and OAuth,
> NonHNS account with SharedKey etc..
> The expectation is to automate these runs with different combinations. This
> will help the developer to run the integration tests with different variants
> of configurations automatically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495954=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495954 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:26 Start Date: 06/Oct/20 15:26 Worklog Time Spent: 10m Work Description: steveloughran commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r500379235 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -80,7 +80,7 @@ tests against mocked storage, which is an in-memory emulation of Azure Storage. The integration tests are designed to test directly against an Azure storage service, and require an account and credentials in order to run. -This is done by creating the file to `src/test/resources/azure-auth-keys.xml` +This is done by creating the file to `src/test/resources/abfs-testrun-configs.xml` Review comment: how will this work with the wasb tests? They don't get run through these shell scripts, do they? ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,181 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 5h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495948=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495948 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:11 Start Date: 06/Oct/20 15:11 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-689816045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 30m 1s | trunk passed | | +1 :green_heart: | compile | 0m 40s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 31s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 14m 27s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 3s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 18s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 25s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 74m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux b09effa10fbb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/testReport/ | | Max. process+thread count |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495945 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:09 Start Date: 06/Oct/20 15:09 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495945) Time Spent: 5h 20m (was: 5h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 5h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495878
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 12:38
Start Date: 06/Oct/20 12:38
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704239734
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 0s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 59s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 57s | | trunk passed |
| -0 :warning: | patch | 1m 15s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 41s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 0s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 77m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux dcb5b341b388 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495801=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495801
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 09:09
Start Date: 06/Oct/20 09:09
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 37s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 59s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 5s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 3s | | trunk passed |
| -0 :warning: | patch | 1m 23s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| -1 :x: | shellcheck | 0m 0s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 13s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 5s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 0m 59s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 79m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 54956957feae 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495799
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 09:05
Start Date: 06/Oct/20 09:05
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704135023
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 19s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 42s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 1s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 59s | | trunk passed |
| -0 :warning: | patch | 1m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| -1 :x: | shellcheck | 0m 2s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 59s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 1s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 79m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 11d2b02cf7bb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495047 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 04/Oct/20 18:17 Start Date: 04/Oct/20 18:17 Worklog Time Spent: 10m Work Description: bilaharith commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-703294541 > -1 as is > > I would like azure-auth-keys.xml to continue to be supported unless there is a very good reason not to > > I am permanently switching to different branches with different hadoop versions on, and if different releases change their test run config/process, then things stop working. > > If you have a good reason to change the name of the test file other than "I didn't think the name was sufficient", then please state it. > > Thanks > > Also : move the scripts to dev-support We planned this rename so that the same is known to the hadoop-azure developers that it is required to follow the template file and add all those configs as part of the configs file. This ensures uniformity among the test results across all the developers. This ensures better code coverage and helps bubbling up in case any regression is introduced as part of the code change. Currently the configs differ for most of us. Recently some test case failure got caught with a developer who was running the tests with sharedkey. The same was missed for those who used OAuth. We also would like to make it madatory to publish the test results for 3 main scenarios which is added in the runtests.sh so that the aforementioned situations can be avoided in the future. I have moved the script under dev-support. Let me know if you have more comments on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495047) Time Spent: 4h 40m (was: 4.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=493112=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-493112 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 30/Sep/20 18:44 Start Date: 30/Sep/20 18:44 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-701573884 I should add that I did checkout and do a test run to confirm: yes, all my tests were skipped. I didn't want that This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 493112) Time Spent: 4.5h (was: 4h 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481430 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Sep/20 12:37 Start Date: 10/Sep/20 12:37 Worklog Time Spent: 10m Work Description: bilaharith commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-690253346 > Happy with all the changes. Still unsure about the full implications. Can I still run "mvn verify" at the command line and individual tests in the IDE? > > * @mehakmeet Yes you can run mvn verify like before. But I would reccommand using this shellscript and incorporate as many combinations of configurations as part of runtests.sh. And publish the consolidated testresults with the PR. This would help in better coverage and identifying regressions if any. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481430) Time Spent: 4h 20m (was: 4h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481427=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481427 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Sep/20 12:34 Start Date: 10/Sep/20 12:34 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r486299349 ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,156 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481386=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481386 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Sep/20 11:28 Start Date: 10/Sep/20 11:28 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-690188077 Happy with all the changes. Still unsure about the full implications. Can I still run "mvn verify" at the command line and individual tests in the IDE? + @mehakmeet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481386) Time Spent: 4h (was: 3h 50m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481384 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Sep/20 11:26 Start Date: 10/Sep/20 11:26 Worklog Time Spent: 10m Work Description: steveloughran commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r486262209 ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,156 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481382 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 10/Sep/20 11:23 Start Date: 10/Sep/20 11:23 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-687718737 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481382) Time Spent: 3h 40m (was: 3.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481066=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481066 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Sep/20 21:27 Start Date: 09/Sep/20 21:27 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-689831046 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 33m 12s | trunk passed | | +1 :green_heart: | compile | 0m 43s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 39s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 29s | trunk passed | | +1 :green_heart: | mvnsite | 0m 45s | trunk passed | | +1 :green_heart: | shadedclient | 17m 33s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 32s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 6s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | the patch passed | | +1 :green_heart: | compile | 0m 35s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 35s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | checkstyle | 0m 19s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 16m 48s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 38s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 83m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux 454ccf5dccf6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/9/testReport/ | | Max. process+thread count | 414
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=481054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481054 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Sep/20 20:53 Start Date: 09/Sep/20 20:53 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-689816045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 30m 1s | trunk passed | | +1 :green_heart: | compile | 0m 40s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 31s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 14m 27s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 3s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 18s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 25s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 74m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux b09effa10fbb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/testReport/ | | Max. process+thread count | 401 (vs.
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480980 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Sep/20 18:31 Start Date: 09/Sep/20 18:31 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r485828996 ## File path: hadoop-tools/hadoop-azure/.gitignore ## @@ -1,2 +1,4 @@ .checkstyle -bin/ \ No newline at end of file +bin/ +testlogs Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 480980) Time Spent: 3h 10m (was: 3h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480979=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480979
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 09/Sep/20 18:30
Start Date: 09/Sep/20 18:30
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r485828835
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit
+ fi
+}
+
+checkdependancies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit
+ fi
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" $conffile
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property $conffile
+ xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n value
-v "$2" $conffile
Review comment:
Could not find the same documented with the xmlstarlet documentation or
produce anything returning non zero. But still added a check for the safer side.
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit
+ fi
+}
+
+checkdependancies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit
+ fi
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" $conffile
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property $conffile
+
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480975=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480975
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 09/Sep/20 18:29
Start Date: 09/Sep/20 18:29
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r485827879
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit
+ fi
+}
+
+checkdependancies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 480975)
Time Spent: 2h 50m (was: 2h 40m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without hierarchical namespace support using various
> authorization mechanisms.
> Currently the developer has to manually run the tests with different
> combinations of configurations ex: HNS account with SharedKey and OAuth,
> NonHNS account with SharedKey etc..
> The expectation is to automate these runs with different combinations. This
> will help the developer to run the integration tests with different variants
> of configurations automatically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480974=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480974
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 09/Sep/20 18:27
Start Date: 09/Sep/20 18:27
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r485827246
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit
+ fi
+}
+
+checkdependancies() {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 480974)
Time Spent: 2h 40m (was: 2.5h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without hierarchical namespace support using various
> authorization mechanisms.
> Currently the developer has to manually run the tests with different
> combinations of configurations ex: HNS account with SharedKey and OAuth,
> NonHNS account with SharedKey etc..
> The expectation is to automate these runs with different combinations. This
> will help the developer to run the integration tests with different variants
> of configurations automatically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480973=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480973 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Sep/20 18:21 Start Date: 09/Sep/20 18:21 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r485823482 ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,156 @@ + + + + + + + + + + +fs.azure.account.auth.type +SharedKey + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480966=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480966 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 09/Sep/20 18:12 Start Date: 09/Sep/20 18:12 Worklog Time Spent: 10m Work Description: bilaharith commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r485818694 ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,156 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480964=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480964
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 09/Sep/20 18:08
Start Date: 09/Sep/20 18:08
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r485816830
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelegationSAS.java
##
@@ -64,7 +64,7 @@
LoggerFactory.getLogger(ITestAzureBlobFileSystemDelegationSAS.class);
public ITestAzureBlobFileSystemDelegationSAS() throws Exception {
-// These tests rely on specific settings in azure-auth-keys.xml:
+// These tests rely on specific settings in abfs-testrun-configs.xml:
Review comment:
Basically we are renaming azure-auth-keys to abfs-testrun-configs.
Because we expect the same to contain non auth related configs too.
In the main config file (azure-test.xml the xinclude is now made to
abfs-testrun-configs)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 480964)
Time Spent: 2h 10m (was: 2h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without hierarchical namespace support using various
> authorization mechanisms.
> Currently the developer has to manually run the tests with different
> combinations of configurations ex: HNS account with SharedKey and OAuth,
> NonHNS account with SharedKey etc..
> The expectation is to automate these runs with different combinations. This
> will help the developer to run the integration tests with different variants
> of configurations automatically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480421
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 08/Sep/20 20:09
Start Date: 08/Sep/20 20:09
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r485164407
##
File path: hadoop-tools/hadoop-azure/.gitignore
##
@@ -1,2 +1,4 @@
.checkstyle
-bin/
\ No newline at end of file
+bin/
+testlogs
Review comment:
test results should go into target so a "mvn clean" will delete them
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size. Please check for scenario $scenario"
+exit
+ fi
+}
+
+checkdependancies() {
+ if ! [ "$(command -v pcregrep)" ]; then
+echo "Exiting. pcregrep is required to run the script."
+exit
+ fi
+ if ! [ "$(command -v xmlstarlet)" ]; then
+echo "Exiting. xmlstarlet is required to run the script."
+exit
+ fi
+}
+
+changeconf() {
+ xmlstarlet ed -P -L -d "/configuration/property[name='$1']" $conffile
+ xmlstarlet ed -P -L -s /configuration -t elem -n propertyTMP -v "" -s
/configuration/propertyTMP -t elem -n name -v "$1" -r
/configuration/propertyTMP -v property $conffile
+ xmlstarlet ed -P -L -s "/configuration/property[name='$1']" -t elem -n value
-v "$2" $conffile
+}
+
+testwithconfs() {
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Number of properties and values differ for $scenario"
+exit
+ fi
+ for ((i = 0; i < propertiessize; i++)); do
+key=${properties[$i]}
+val=${values[$i]}
+changeconf "$key" "$val"
+ done
+ mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify >>
"$testlogfilename"
Review comment:
should thread count be configurable?
##
File path: hadoop-tools/hadoop-azure/testsupport.sh
##
@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+conffile=src/test/resources/abfs-testrun-configs.xml
+bkpconffile=src/test/resources/abfs-testrun-configs_BKP.xml
+testresultsregex="Results:(\n|.)*?Tests run:"
+testresultsfilename=
+starttime=
+
+validate() {
+ if [ -z "$scenario" ]; then
+ echo "Exiting. scenario cannot be empty"
+ exit
+ fi
+ propertiessize=${#properties[@]}
+ valuessize=${#values[@]}
+ if [ "$propertiessize" -lt 1 ] || [ "$valuessize" -lt 1 ] || [
"$propertiessize" -ne "$valuessize" ]; then
+echo "Exiting. Both properties and values arrays has to be populated and
of same size.
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480414=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480414 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 08/Sep/20 20:00 Start Date: 08/Sep/20 20:00 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688480976 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 480414) Time Spent: 1h 40m (was: 1.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480415=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480415 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 08/Sep/20 20:00 Start Date: 08/Sep/20 20:00 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688262400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 480415) Time Spent: 1h 50m (was: 1h 40m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=480203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-480203 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 08/Sep/20 15:10 Start Date: 08/Sep/20 15:10 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688943301 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 32m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 26s | trunk passed | | +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 32s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 26s | trunk passed | | +1 :green_heart: | mvnsite | 0m 37s | trunk passed | | +1 :green_heart: | shadedclient | 15m 53s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 26s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 0s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 30s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 16s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 14s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 54s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 22s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 17s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 108m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux 476360760f4a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 0d855159f09 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/7/testReport/ | | Max. process+thread count | 309
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479758 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 07/Sep/20 19:17 Start Date: 07/Sep/20 19:17 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688480976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 29m 35s | trunk passed | | +1 :green_heart: | compile | 0m 42s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 32s | trunk passed | | +1 :green_heart: | mvnsite | 0m 44s | trunk passed | | +1 :green_heart: | shadedclient | 14m 18s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 36s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 1s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | checkstyle | 0m 19s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | -1 :x: | shellcheck | 0m 0s | The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 13m 53s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hadoop-azure in the patch passed. | | -1 :x: | asflicense | 0m 35s | The patch generated 1 ASF License warnings. | | | | 73m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux d00bb145dfad 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 34fe74da0e9 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/6/artifact/out/diff-patch-shellcheck.txt | | whitespace |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479749 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 07/Sep/20 18:48 Start Date: 07/Sep/20 18:48 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688472439 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 29m 31s | trunk passed | | +1 :green_heart: | compile | 0m 40s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 32s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 14m 24s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 35s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 2s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 59s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 19s | the patch passed | | +1 :green_heart: | mvnsite | 0m 32s | the patch passed | | -1 :x: | shellcheck | 0m 0s | The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 19s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hadoop-azure in the patch passed. | | -1 :x: | asflicense | 0m 36s | The patch generated 1 ASF License warnings. | | | | 73m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux 5dc09c5795e2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 34fe74da0e9 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/5/artifact/out/diff-patch-shellcheck.txt | | whitespace |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479744 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 07/Sep/20 18:31 Start Date: 07/Sep/20 18:31 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688466988 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 59s | trunk passed | | +1 :green_heart: | compile | 0m 41s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 32s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 26s | trunk passed | | +1 :green_heart: | mvnsite | 0m 38s | trunk passed | | +1 :green_heart: | shadedclient | 15m 58s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 26s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 0m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 56s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 30s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 17s | the patch passed | | +1 :green_heart: | mvnsite | 0m 29s | the patch passed | | -1 :x: | shellcheck | 0m 1s | The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shelldocs | 0m 13s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 54s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 22s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 17s | hadoop-azure in the patch passed. | | -1 :x: | asflicense | 0m 32s | The patch generated 1 ASF License warnings. | | | | 78m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux f42098b5f563 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 34fe74da0e9 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/4/artifact/out/diff-patch-shellcheck.txt | | whitespace |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479594=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479594 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 07/Sep/20 11:25 Start Date: 07/Sep/20 11:25 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-688262400 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 39s | trunk passed | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 32s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 25s | trunk passed | | +1 :green_heart: | mvnsite | 0m 35s | trunk passed | | +1 :green_heart: | shadedclient | 15m 54s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 28s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 6s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 33s | the patch passed | | +1 :green_heart: | compile | 0m 33s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 33s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | checkstyle | 0m 16s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | -1 :x: | shellcheck | 0m 0s | The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) | | +1 :green_heart: | shelldocs | 0m 13s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 17m 9s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 27s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 80m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux 6368d9d69464 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 34fe74da0e9 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | shellcheck |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479421 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Sep/20 18:25 Start Date: 06/Sep/20 18:25 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-687862577 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 36m 48s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 18m 31s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 39s | the patch passed | | +1 :green_heart: | mvnsite | 0m 35s | the patch passed | | -1 :x: | shellcheck | 0m 0s | The patch generated 23 new + 0 unchanged - 0 fixed = 23 total (was 0) | | +1 :green_heart: | shelldocs | 0m 15s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 17m 52s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 80m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint | | uname | Linux c962aaf34582 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 1841a5bb03f | | shellcheck | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/2/artifact/out/diff-patch-shellcheck.txt | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/2/testReport/ | | Max. process+thread count | 308 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/2/console | | versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 479421) Time Spent: 40m (was: 0.5h) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=479393=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479393 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Sep/20 07:48 Start Date: 06/Sep/20 07:48 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-687718737 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 40s | trunk passed | | +1 :green_heart: | mvnsite | 0m 38s | trunk passed | | +1 :green_heart: | shadedclient | 15m 50s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 29s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 14s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 36s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 29s | hadoop-azure in the patch passed. | | -1 :x: | asflicense | 0m 32s | The patch generated 2 ASF License warnings. | | | | 68m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint | | uname | Linux 59e40bddadd5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 1841a5bb03f | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/1/testReport/ | | asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/1/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 310 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/1/console | | versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 479393) Time Spent: 0.5h (was: 20m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests
