[GitHub] [hadoop] hadoop-yetus commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts.
hadoop-yetus commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts. URL: https://github.com/apache/hadoop/pull/1738#issuecomment-562471325 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/3/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts.
aajisaka commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts. URL: https://github.com/apache/hadoop/pull/1738#issuecomment-562471004 Python2 is still required in Dockerfile because releasedocmaker.py in Apache Yetus only supports Python2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts.
hadoop-yetus commented on issue #1738: HADOOP-16748. Support Python 3 in dev-support scripts. URL: https://github.com/apache/hadoop/pull/1738#issuecomment-562467708 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 9s | Maven dependency ordering for branch | | +1 :green_heart: | shadedclient | 13m 21s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch | | -1 :x: | pylint | 0m 6s | The patch generated 1 new + 43 unchanged - 338 fixed = 44 total (was 381) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 13m 9s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 31m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1738 | | Optional Tests | dupname asflicense pylint | | uname | Linux cc891ba47269 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 18059ac | | pylint | https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/2/artifact/out/diff-patch-pylint.txt | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: U: | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/2/console | | versions | git=2.7.4 maven=3.3.9 pylint=1.9.2 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16744) Fix building instruction to enable zstd
[ https://issues.apache.org/jira/browse/HADOOP-16744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989444#comment-16989444 ] Hudson commented on HADOOP-16744: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17730 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17730/]) HADOOP-16744. Fix building instruction to enable zstd. (#1736) (github: rev 18059acb6ae16e72a6cdd08795f6281cda122bff) * (edit) BUILDING.txt > Fix building instruction to enable zstd > > > Key: HADOOP-16744 > URL: https://issues.apache.org/jira/browse/HADOOP-16744 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2 > > > The instruction in the BUILDING.txt lacks libzstd-dev required on Ubuntu. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16744) Fix building instruction to enable zstd
[ https://issues.apache.org/jira/browse/HADOOP-16744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16744: -- Fix Version/s: 3.2.2 3.1.4 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch-3.1 and above. Thanks for reviewing, [~aajisaka]. > Fix building instruction to enable zstd > > > Key: HADOOP-16744 > URL: https://issues.apache.org/jira/browse/HADOOP-16744 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2 > > > The instruction in the BUILDING.txt lacks libzstd-dev required on Ubuntu. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd.
iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736#issuecomment-562449738 Thanks, @aajisaka . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims merged pull request #1736: HADOOP-16744. Fix building instruction to enable zstd.
iwasakims merged pull request #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1737: HDFS-14522. Allow compact property description in xml in httpfs.
hadoop-yetus commented on issue #1737: HDFS-14522. Allow compact property description in xml in httpfs. URL: https://github.com/apache/hadoop/pull/1737#issuecomment-562448781 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 27m 18s | trunk passed | | +1 :green_heart: | compile | 0m 26s | trunk passed | | +1 :green_heart: | checkstyle | 0m 21s | trunk passed | | +1 :green_heart: | mvnsite | 0m 35s | trunk passed | | +1 :green_heart: | shadedclient | 17m 38s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 25s | trunk passed | | +0 :ok: | spotbugs | 0m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 48s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 26s | the patch passed | | +1 :green_heart: | compile | 0m 21s | the patch passed | | +1 :green_heart: | javac | 0m 21s | the patch passed | | -0 :warning: | checkstyle | 0m 15s | hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 2 new + 8 unchanged - 5 fixed = 10 total (was 13) | | +1 :green_heart: | mvnsite | 0m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 17m 17s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | | +1 :green_heart: | findbugs | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 5m 54s | hadoop-hdfs-httpfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 77m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1737/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1737 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux b7ef34bafb31 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 19512b2 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1737/3/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1737/3/testReport/ | | Max. process+thread count | 594 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1737/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd.
aajisaka commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736#issuecomment-562436774 +1, make sense. Thank you @iwasakims This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on a change in pull request #1737: HDFS-14522. Allow compact property description in xml in httpfs.
iwasakims commented on a change in pull request #1737: HDFS-14522. Allow
compact property description in xml in httpfs.
URL: https://github.com/apache/hadoop/pull/1737#discussion_r354667174
##
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/lib/util/TestConfigurationUtils.java
##
@@ -124,4 +127,22 @@ public void testVarResolutionAndSysProps() {
assertEquals(conf.get("user.name"), "foo");
}
+ @Test
+ public void testCompactFormatProperty() throws IOException {
Review comment:
I updated the code based on your suggestion.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on issue #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN
liuml07 commented on issue #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN URL: https://github.com/apache/hadoop/pull/1733#issuecomment-562404299 @steveloughran I don't have write access to the wiki page at https://cwiki.apache.org/confluence/display/HADOOP2, do you have admin access to add me there? Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16735) Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN
[
https://issues.apache.org/jira/browse/HADOOP-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-16735:
---
Fix Version/s: 2.11.0
3.2.2
3.1.4
3.3.0
3.0.4
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to all fixed versions. Thanks Steve for review.
> Make it clearer in config default that EnvironmentVariableCredentialsProvider
> supports AWS_SESSION_TOKEN
>
>
> Key: HADOOP-16735
> URL: https://issues.apache.org/jira/browse/HADOOP-16735
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, fs/s3
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>Priority: Minor
> Fix For: 3.0.4, 3.3.0, 3.1.4, 3.2.2, 2.11.0
>
>
> In the great doc {{hadoop-aws/tools/hadoop-aws/index.html}}, user can find
> that authenticating via the AWS Environment Variables supports session token.
> However, the config description in core-default.xml does not make it clear.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on issue #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN
liuml07 commented on issue #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN URL: https://github.com/apache/hadoop/pull/1733#issuecomment-562399310 Thanks @steveloughran . I don't have write access to this GitHub repo, and I have sent an email to ask about this. Not sure why. At the meantime, I have committed via command line using the tips you provided. One nit is that, I enabled the `git config commit.gpgsign true` so I don't have to remember using the `git commit -S ` option when committing. It's now showing the signature is "verified". I will update the wiki page with the above tips, and maybe you can review that later. Will let you know. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16735) Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN
[
https://issues.apache.org/jira/browse/HADOOP-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989313#comment-16989313
]
Hudson commented on HADOOP-16735:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17729 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17729/])
HADOOP-16735. Make it clearer in config default that (liuml07: rev
19512b21e36f2571fb5a67623635fa0988aeee94)
* (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
* (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
> Make it clearer in config default that EnvironmentVariableCredentialsProvider
> supports AWS_SESSION_TOKEN
>
>
> Key: HADOOP-16735
> URL: https://issues.apache.org/jira/browse/HADOOP-16735
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, fs/s3
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>Priority: Minor
>
> In the great doc {{hadoop-aws/tools/hadoop-aws/index.html}}, user can find
> that authenticating via the AWS Environment Variables supports session token.
> However, the config description in core-default.xml does not make it clear.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] asfgit closed pull request #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN
asfgit closed pull request #1733: HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN URL: https://github.com/apache/hadoop/pull/1733 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16727) KMS Jetty server does not startup if trust store password is null
[
https://issues.apache.org/jira/browse/HADOOP-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989279#comment-16989279
]
Hanisha Koneru commented on HADOOP-16727:
-
Thank you [~smeng].
The javac issues are not introduced by this patch.
> KMS Jetty server does not startup if trust store password is null
> -
>
> Key: HADOOP-16727
> URL: https://issues.apache.org/jira/browse/HADOOP-16727
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
>Priority: Major
> Attachments: HADOOP-16727.003.patch, HDFS-14951.001.patch,
> HDFS-14951.002.patch
>
>
> In HttpServe2, if the trustStore is set but the trust store password is not,
> then we set the TrustStorePassword of SSLContextFactory to null. This results
> in the Jetty server not starting up.
> {code:java}
> In HttpServer2#createHttpsChannelConnector(),
> if (trustStore != null) {
> sslContextFactory.setTrustStorePath(trustStore);
> sslContextFactory.setTrustStoreType(trustStoreType);
> sslContextFactory.setTrustStorePassword(trustStorePassword);
> }
> {code}
> Before setting the trust store password, we should check that it is not null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16734) Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989258#comment-16989258 ] Virajith Jalaparti commented on HADOOP-16734: - Thanks for posting this [~bilahari.th]. Can you also create a PR for branch-2.10? I can do the cherry-pick but I am not sure if other changes need to go in first, > Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to > branch-2 > --- > > Key: HADOOP-16734 > URL: https://issues.apache.org/jira/browse/HADOOP-16734 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 2.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 2.11.0 > > > Backport https://issues.apache.org/jira/browse/HADOOP-16455 to branch-2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajith commented on a change in pull request #1735: HADOOP-16734. ABFS: Implement FileSystem.access() method
virajith commented on a change in pull request #1735: HADOOP-16734. ABFS:
Implement FileSystem.access() method
URL: https://github.com/apache/hadoop/pull/1735#discussion_r354605336
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -884,6 +884,26 @@ public AclStatus getAclStatus(final Path path) throws
IOException {
return aclStatusBuilder.build();
}
+ public void access(final Path path, final FsAction mode)
+ throws AzureBlobFileSystemException {
+LOG.debug("access for filesystem: {}, path: {}, mode: {}",
+this.client.getFileSystem(), path, mode);
+if (!this.abfsConfiguration.isCheckAccessEnabled()
+|| !getIsNamespaceEnabled()) {
+ LOG.debug("Returning; either check access is not enabled or the account"
+ + " used is not namespace enabled");
+ return;
+}
+// Uncomment when cherry-picking HADOOP-16612
+// try (AbfsPerfInfo perfInfo = startTracking("access", "checkAccess")) {
Review comment:
Either HADOOP-16612 needs to go in first or add this after it is in.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajith commented on a change in pull request #1735: HADOOP-16734. ABFS: Implement FileSystem.access() method
virajith commented on a change in pull request #1735: HADOOP-16734. ABFS: Implement FileSystem.access() method URL: https://github.com/apache/hadoop/pull/1735#discussion_r354605572 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java ## @@ -67,6 +67,8 @@ public static final boolean DEFAULT_ENABLE_HTTPS = true; public static final boolean DEFAULT_USE_UPN = false; + public static final boolean DEFAULT_ENABLE_CHECK_ACCESS = false; + public static final boolean DEFAULT_ABFS_LATENCY_TRACK = false; Review comment: Isn't this part of HADOOP-16612? Remove this change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16750) Backport HADOOP-16548 - ABFS: Config to enable/disable flush operation issue to branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mandar Inamdar updated HADOOP-16750: Fix Version/s: (was: 3.3.0) 3.2 3.2.2 > Backport HADOOP-16548 - ABFS: Config to enable/disable flush operation issue > to branch-3.2 > -- > > Key: HADOOP-16750 > URL: https://issues.apache.org/jira/browse/HADOOP-16750 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Mandar Inamdar >Assignee: Sneha Vijayarajan >Priority: Minor > Fix For: 3.2.2, 3.2 > > > Make flush operation enabled/disabled through configuration. This is part of > performance improvements for ABFS driver. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16750) Backport HADOOP-16548 - ABFS: Config to enable/disable flush operation issue to branch-3.2
Mandar Inamdar created HADOOP-16750: --- Summary: Backport HADOOP-16548 - ABFS: Config to enable/disable flush operation issue to branch-3.2 Key: HADOOP-16750 URL: https://issues.apache.org/jira/browse/HADOOP-16750 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Mandar Inamdar Assignee: Sneha Vijayarajan Fix For: 3.3.0 Make flush operation enabled/disabled through configuration. This is part of performance improvements for ABFS driver. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #1737: HDFS-14522. Allow compact property description in xml in httpfs.
goiri commented on a change in pull request #1737: HDFS-14522. Allow compact
property description in xml in httpfs.
URL: https://github.com/apache/hadoop/pull/1737#discussion_r354455494
##
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/lib/util/TestConfigurationUtils.java
##
@@ -124,4 +127,22 @@ public void testVarResolutionAndSysProps() {
assertEquals(conf.get("user.name"), "foo");
}
+ @Test
+ public void testCompactFormatProperty() throws IOException {
Review comment:
Having a file makes it cleaner than putting the whole string in code.
That being said, I think both should be there.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16749) Configuration parsing of CDATA values are blank
[
https://issues.apache.org/jira/browse/HADOOP-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988911#comment-16988911
]
Hadoop QA commented on HADOOP-16749:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HADOOP-16749 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-16749 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12987612/HADOOP-16749.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16688/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Configuration parsing of CDATA values are blank
> ---
>
> Key: HADOOP-16749
> URL: https://issues.apache.org/jira/browse/HADOOP-16749
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Jonathan Turner Eagles
>Assignee: Daryn Sharp
>Priority: Major
> Attachments: HADOOP-16749.patch
>
>
> When using CDATA. CDATA elements are skipped by the parser.
> In fact someone on stack overflow was asking this same question a few months
> ago.
> https://stackoverflow.com/questions/57829034/why-is-apache-hadoop-configuration-module-ignores-cdata
> {code}
>
> test.cdata
> hello
>
> {code}
> conf.get("test.cdata") parses as 'hello ' instead of 'hello world'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16749) Configuration parsing of CDATA values are blank
[
https://issues.apache.org/jira/browse/HADOOP-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-16749:
-
Status: Patch Available (was: Open)
> Configuration parsing of CDATA values are blank
> ---
>
> Key: HADOOP-16749
> URL: https://issues.apache.org/jira/browse/HADOOP-16749
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Jonathan Turner Eagles
>Assignee: Daryn Sharp
>Priority: Major
> Attachments: HADOOP-16749.patch
>
>
> When using CDATA. CDATA elements are skipped by the parser.
> In fact someone on stack overflow was asking this same question a few months
> ago.
> https://stackoverflow.com/questions/57829034/why-is-apache-hadoop-configuration-module-ignores-cdata
> {code}
>
> test.cdata
> hello
>
> {code}
> conf.get("test.cdata") parses as 'hello ' instead of 'hello world'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16749) Configuration parsing of CDATA values are blank
[
https://issues.apache.org/jira/browse/HADOOP-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-16749:
-
Attachment: HADOOP-16749.patch
> Configuration parsing of CDATA values are blank
> ---
>
> Key: HADOOP-16749
> URL: https://issues.apache.org/jira/browse/HADOOP-16749
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Jonathan Turner Eagles
>Assignee: Daryn Sharp
>Priority: Major
> Attachments: HADOOP-16749.patch
>
>
> When using CDATA. CDATA elements are skipped by the parser.
> In fact someone on stack overflow was asking this same question a few months
> ago.
> https://stackoverflow.com/questions/57829034/why-is-apache-hadoop-configuration-module-ignores-cdata
> {code}
>
> test.cdata
> hello
>
> {code}
> conf.get("test.cdata") parses as 'hello ' instead of 'hello world'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode
hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-562176322 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 19s | trunk passed | | +1 :green_heart: | compile | 18m 31s | trunk passed | | +1 :green_heart: | checkstyle | 2m 53s | trunk passed | | +1 :green_heart: | mvnsite | 2m 7s | trunk passed | | +1 :green_heart: | shadedclient | 20m 3s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 55s | trunk passed | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 18s | trunk passed | | -0 :warning: | patch | 1m 37s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 26s | the patch passed | | +1 :green_heart: | compile | 17m 57s | the patch passed | | +1 :green_heart: | javac | 17m 57s | the patch passed | | -0 :warning: | checkstyle | 2m 55s | root: The patch generated 21 new + 95 unchanged - 0 fixed = 116 total (was 95) | | +1 :green_heart: | mvnsite | 2m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 6s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 57s | the patch passed | | +1 :green_heart: | findbugs | 3m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 24s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 27s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | The patch does not generate ASF License warnings. | | | | 128m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1707 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml markdownlint | | uname | Linux c26b3187d0a3 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 682e6fd | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/12/artifact/out/diff-checkstyle-root.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/12/testReport/ | | Max. process+thread count | 1347 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/12/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode
hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-562175265 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 4s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 20m 25s | trunk passed | | +1 :green_heart: | compile | 18m 6s | trunk passed | | +1 :green_heart: | checkstyle | 2m 56s | trunk passed | | +1 :green_heart: | mvnsite | 2m 3s | trunk passed | | +1 :green_heart: | shadedclient | 20m 25s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 56s | trunk passed | | +0 :ok: | spotbugs | 1m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 9s | trunk passed | | -0 :warning: | patch | 1m 27s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 20s | the patch passed | | +1 :green_heart: | compile | 17m 9s | the patch passed | | +1 :green_heart: | javac | 17m 9s | the patch passed | | -0 :warning: | checkstyle | 2m 55s | root: The patch generated 21 new + 95 unchanged - 0 fixed = 116 total (was 95) | | +1 :green_heart: | mvnsite | 2m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 13m 58s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 0s | the patch passed | | +1 :green_heart: | findbugs | 3m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 56s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 17s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 126m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1707 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml markdownlint | | uname | Linux 256d3b967a67 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 682e6fd | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/13/artifact/out/diff-checkstyle-root.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/13/testReport/ | | Max. process+thread count | 1376 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/13/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16749) Configuration parsing of CDATA values are blank
[
https://issues.apache.org/jira/browse/HADOOP-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Turner Eagles reassigned HADOOP-16749:
---
Assignee: Daryn Sharp
> Configuration parsing of CDATA values are blank
> ---
>
> Key: HADOOP-16749
> URL: https://issues.apache.org/jira/browse/HADOOP-16749
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Jonathan Turner Eagles
>Assignee: Daryn Sharp
>Priority: Major
>
> When using CDATA. CDATA elements are skipped by the parser.
> In fact someone on stack overflow was asking this same question a few months
> ago.
> https://stackoverflow.com/questions/57829034/why-is-apache-hadoop-configuration-module-ignores-cdata
> {code}
>
> test.cdata
> hello
>
> {code}
> conf.get("test.cdata") parses as 'hello ' instead of 'hello world'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16749) Configuration parsing of CDATA values are blank
Jonathan Turner Eagles created HADOOP-16749:
---
Summary: Configuration parsing of CDATA values are blank
Key: HADOOP-16749
URL: https://issues.apache.org/jira/browse/HADOOP-16749
Project: Hadoop Common
Issue Type: Bug
Components: conf
Reporter: Jonathan Turner Eagles
When using CDATA. CDATA elements are skipped by the parser.
In fact someone on stack overflow was asking this same question a few months
ago.
https://stackoverflow.com/questions/57829034/why-is-apache-hadoop-configuration-module-ignores-cdata
{code}
test.cdata
hello
{code}
conf.get("test.cdata") parses as 'hello ' instead of 'hello world'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd.
hadoop-yetus commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736#issuecomment-562134111 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | shadedclient | 14m 58s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 15s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 33m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1736/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1736 | | Optional Tests | dupname asflicense | | uname | Linux c473327d73f5 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 682e6fd | | Max. process+thread count | 306 (vs. ulimit of 5500) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1736/2/console | | versions | git=2.7.4 maven=3.3.9 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on a change in pull request #1737: HDFS-14522. Allow compact property description in xml in httpfs.
iwasakims commented on a change in pull request #1737: HDFS-14522. Allow
compact property description in xml in httpfs.
URL: https://github.com/apache/hadoop/pull/1737#discussion_r354309758
##
File path:
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/lib/util/TestConfigurationUtils.java
##
@@ -124,4 +127,22 @@ public void testVarResolutionAndSysProps() {
assertEquals(conf.get("user.name"), "foo");
}
+ @Test
+ public void testCompactFormatProperty() throws IOException {
Review comment:
The `ConfigurationUtils#load` takes only InputStream as input and does not
care about where the data come from. I think loading configs from file is
covered by test cases in o.a.h.lib.server.TestServer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1668: HADOOP-16645. S3A Delegation Token extension point to use StoreContext.
steveloughran commented on issue #1668: HADOOP-16645. S3A Delegation Token extension point to use StoreContext. URL: https://github.com/apache/hadoop/pull/1668#issuecomment-562124245 Can I get some reviewing here? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd.
iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736#issuecomment-562119741 The Dockerfile is based on xenial (16.04) in which libzstd-dev points to older version (0.5.1). We whould use libzstd1-dev pointing to newer version. Since libzstd1-dev is still available on bioinic (18.04) as transitional package, libzstd1-dev would be better in BUILDING.txt too. I will update the code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran edited a comment on issue #1707: HADOOP-16697. Tune/audit auth mode
steveloughran edited a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-562104318 findbugs > Unchecked/unconfirmed cast from org.apache.hadoop.fs.s3a.s3guard.BulkOperationState to org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$AncestorState in org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.markAsAuthoritative(Path, BulkOperationState) At DynamoDBMetadataStore.java:org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$AncestorState in org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.markAsAuthoritative(Path, BulkOperationState) At DynamoDBMetadataStore.java:[line 2104] > Switch statement found in org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(String[], PrintStream) where one case falls through to the next case At S3GuardTool.java:PrintStream) where one case falls through to the next case At S3GuardTool.java:[lines 1276-1283] This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd.
iwasakims commented on issue #1736: HADOOP-16744. Fix building instruction to enable zstd. URL: https://github.com/apache/hadoop/pull/1736#issuecomment-562115779 > In the Dockerfile (dev-support/docker/Dockerfile), should we change `libzstd1-dev` to `libzstd-dev` (virtual package) as well? Sure. While we have the same result by using transitional package name (ibzstd1-dev), it would be better that the Dockerfile and documentation are consistent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant commented on issue #1530: HDFS-14869 Copy renamed files which are not excluded anymore by filter
bshashikant commented on issue #1530: HDFS-14869 Copy renamed files which are not excluded anymore by filter URL: https://github.com/apache/hadoop/pull/1530#issuecomment-562110153 @aasha , can you please address the checkstyle issues? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-562104318 findbugs ``` Unchecked/unconfirmed cast from org.apache.hadoop.fs.s3a.s3guard.BulkOperationState to org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$AncestorState in org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.markAsAuthoritative(Path, BulkOperationState) At DynamoDBMetadataStore.java:org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$AncestorState in org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.markAsAuthoritative(Path, BulkOperationState) At DynamoDBMetadataStore.java:[line 2104] -- | Switch statement found in org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(String[], PrintStream) where one case falls through to the next case At S3GuardTool.java:PrintStream) where one case falls through to the next case At S3GuardTool.java:[lines 1276-1283] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354277577
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStoreAuthoritativeMode.java
##
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.s3guard;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.concurrent.Callable;
+import java.util.concurrent.TimeUnit;
+
+import org.assertj.core.api.Assertions;
+import org.junit.AfterClass;
+import org.junit.Ignore;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.fs.contract.ContractTestUtils;
+import org.apache.hadoop.fs.s3a.AbstractS3ATestBase;
+import org.apache.hadoop.fs.s3a.S3AFileStatus;
+import org.apache.hadoop.fs.s3a.S3AFileSystem;
+import org.apache.hadoop.fs.s3a.S3ATestUtils;
+import org.apache.hadoop.fs.s3a.Tristate;
+import org.apache.hadoop.fs.s3a.impl.StoreContext;
+import org.apache.hadoop.io.IOUtils;
+
+import static org.apache.hadoop.fs.contract.ContractTestUtils.rm;
+import static org.apache.hadoop.fs.contract.ContractTestUtils.writeTextFile;
+import static org.apache.hadoop.fs.s3a.Constants.AUTHORITATIVE_PATH;
+import static org.apache.hadoop.fs.s3a.Constants.METADATASTORE_AUTHORITATIVE;
+import static
org.apache.hadoop.fs.s3a.Constants.S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_KEY;
+import static org.apache.hadoop.fs.s3a.Constants.S3_METADATA_STORE_IMPL;
+import static org.apache.hadoop.fs.s3a.S3ATestUtils.assume;
+import static
org.apache.hadoop.fs.s3a.S3ATestUtils.removeBaseAndBucketOverrides;
+import static org.apache.hadoop.fs.s3a.S3AUtils.applyLocatedFiles;
+import static org.apache.hadoop.fs.s3a.Statistic.OBJECT_LIST_REQUESTS;
+import static
org.apache.hadoop.fs.s3a.Statistic.S3GUARD_METADATASTORE_AUTHORITATIVE_DIRECTORIES_UPDATED;
+import static
org.apache.hadoop.fs.s3a.s3guard.PathMetadataDynamoDBTranslation.emptyDirectoryMarker;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+
+/**
+ * Test to verify the expected behaviour of DynamoDB and authoritative mode.
+ * The main testFS is non-auth; we also create a test FS which runs in auth
mode.
+ * Making the default FS non-auth means that test path cleanup in the
+ * superclass isn't going to get mislead by anything authoritative.
+ *
+ * For performance boosting we demand create the auth FS and its test
+ * paths on the first test setup().
+ * This also fixes the auth/nonauth paths so that a specific
+ * bit of the FS is expected to be auth in the FS.
+ *
+ * This test is designed to run in parallel mode with other tests which
+ * may or may not be auth mode.
+ *
+ * It shouldn't make any difference -tests here simply must not make
+ * any assumptions about the state of any path outside the test tree.
+ */
+@SuppressWarnings("StaticNonFinalField")
+public class ITestDynamoDBMetadataStoreAuthoritativeMode
+extends AbstractS3ATestBase {
+
+ private static final Logger LOG = LoggerFactory.getLogger(
+ ITestDynamoDBMetadataStoreAuthoritativeMode.class);
+
+ private StoreContext storeContext;
+
+ private String fsUriStr;
+
+ private DynamoDBMetadataStore metastore;
+
+ /**
+ * Authoritative FS.
+ */
+ private static S3AFileSystem authFS;
+
+ /**
+ * The unguarded file system.
+ */
+ private static S3AFileSystem unguardedFS;
+
+ private static Path basePath;
+
+ private static Path authPath;
+
+ private static Path nonauthPath;
+
+ private Path methodAuthPath;
+
+ private Path methodNonauthPath;
+
+ private AuthoritativeAuditOperation auditor;
+
+ private Path dir;
+
+ private Path dirFile;
+
+ @AfterClass
+ public static void closeFileSystems() {
+IOUtils.cleanupWithLogger(LOG, authFS, unguardedFS);
+ }
+
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.crea
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354277362
##
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/MetastoreInstrumentation.java
##
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.s3guard;
+
+public interface MetastoreInstrumentation {
Review comment:
pulled up from the implementation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354274504
##
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStore.java
##
@@ -1991,6 +2082,51 @@ public RenameTracker initiateRenameOperation(
new AncestorState(this, BulkOperationState.OperationType.Rename,
dest));
}
+ /**
+ * Mark the directories instantiated under the destination path
+ * as authoritative.
+ * @param dest destination path.
+ * @param operationState active state.
+ * @throws IOException failure.
+ * @return the number of directories marked.
+ */
+ @Override
+ public int markAsAuthoritative(final Path dest,
+ final BulkOperationState operationState) throws IOException {
+if (operationState == null) {
+ return 0;
+}
+AncestorState state = (AncestorState)operationState;
Review comment:
not actually done in casts, AFAIK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354273717
##
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInstrumentation.java
##
@@ -1172,10 +1181,25 @@ public void recordsRead(int count) {
* records have been written (including deleted).
* @param count number of records written.
*/
+@Override
public void recordsWritten(int count) {
incrementCounter(S3GUARD_METADATASTORE_RECORD_WRITES, count);
}
+@Override
+public void directoryMarkedAuthoritative() {
+ incrementCounter(S3GUARD_METADATASTORE_AUTHORITATIVE_DIRECTORIES_UPDATED,
+ 1);
+}
+
+@Override
+public void entryAdded(final long durationNanos) {
+ addValueToQuantiles(
+ S3GUARD_METADATASTORE_PUT_PATH_LATENCY,
+ durationNanos);
+ incrementCounter(S3GUARD_METADATASTORE_PUT_PATH_REQUEST, 1);
Review comment:
It's called in S3Guard.putAndReturn. This is it today
```
instrumentation.addValueToQuantiles(
S3GUARD_METADATASTORE_PUT_PATH_LATENCY,
(System.nanoTime() - startTimeNano));
instrumentation.incrementCounter(
S3GUARD_METADATASTORE_PUT_PATH_REQUEST,
1);
```
All I'm doing is moving it into the metastore instrumentation. Therefore:
this patch is consistent with today's use.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354271058
##
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##
@@ -1275,8 +1276,7 @@ public boolean rename(Path src, Path dst) throws
IOException {
dstStatus = innerGetFileStatus(dst, true, StatusProbeEnum.ALL);
// if there is no destination entry, an exception is raised.
// hence this code sequence can assume that there is something
- // at the end of the path; the only detail being what it is and
- // whether or not it can be the destination of the rename.
+ // at the end of the path; the only detail
Review comment:
restored accidentally deleted line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on a change in pull request #1707: HADOOP-16697.
Tune/audit auth mode
URL: https://github.com/apache/hadoop/pull/1707#discussion_r354271058
##
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##
@@ -1275,8 +1276,7 @@ public boolean rename(Path src, Path dst) throws
IOException {
dstStatus = innerGetFileStatus(dst, true, StatusProbeEnum.ALL);
// if there is no destination entry, an exception is raised.
// hence this code sequence can assume that there is something
- // at the end of the path; the only detail being what it is and
- // whether or not it can be the destination of the rename.
+ // at the end of the path; the only detail
Review comment:
cut the ; text
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-559224962 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 30m 14s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 8 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 7s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 18m 10s | trunk passed | | +1 :green_heart: | compile | 16m 55s | trunk passed | | +1 :green_heart: | checkstyle | 2m 36s | trunk passed | | +1 :green_heart: | mvnsite | 2m 15s | trunk passed | | +1 :green_heart: | shadedclient | 18m 41s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 12s | trunk passed | | +0 :ok: | spotbugs | 1m 10s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 11s | trunk passed | | -0 :warning: | patch | 1m 32s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 20s | hadoop-aws in the patch failed. | | -1 :x: | compile | 15m 20s | root in the patch failed. | | -1 :x: | javac | 15m 20s | root in the patch failed. | | -0 :warning: | checkstyle | 2m 37s | root: The patch generated 15 new + 85 unchanged - 0 fixed = 100 total (was 85) | | -1 :x: | mvnsite | 0m 41s | hadoop-aws in the patch failed. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 12m 51s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 13s | the patch passed | | -1 :x: | findbugs | 0m 39s | hadoop-aws in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 9m 28s | hadoop-common in the patch failed. | | -1 :x: | unit | 0m 38s | hadoop-aws in the patch failed. | | +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. | | | | 146m 9s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ha.TestZKFailoverController | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1707 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux ed9a770845eb 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 9e69628 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-compile-root.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-compile-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/diff-checkstyle-root.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/testReport/ | | Max. process+thread count | 1420 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/10/console | | versions | git=2.7.4 ma
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-556410699 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 11s | trunk passed | | +1 :green_heart: | compile | 18m 40s | trunk passed | | +1 :green_heart: | checkstyle | 2m 55s | trunk passed | | +1 :green_heart: | mvnsite | 2m 12s | trunk passed | | +1 :green_heart: | shadedclient | 20m 26s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 0s | trunk passed | | +0 :ok: | spotbugs | 1m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 13s | trunk passed | | -0 :warning: | patch | 1m 28s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 22s | the patch passed | | +1 :green_heart: | compile | 17m 48s | the patch passed | | +1 :green_heart: | javac | 17m 48s | the patch passed | | -0 :warning: | checkstyle | 2m 53s | root: The patch generated 9 new + 76 unchanged - 0 fixed = 85 total (was 76) | | +1 :green_heart: | mvnsite | 2m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 13m 40s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 0s | the patch passed | | -1 :x: | findbugs | 1m 15s | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 39s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 29s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | The patch does not generate ASF License warnings. | | | | 128m 58s | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-tools/hadoop-aws | | | Switch statement found in org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(String[], PrintStream) where one case falls through to the next case At S3GuardTool.java:PrintStream) where one case falls through to the next case At S3GuardTool.java:[lines 1246-1253] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1707 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux bebf6e37ccb5 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 3037762 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/8/artifact/out/diff-checkstyle-root.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/8/artifact/out/new-findbugs-hadoop-tools_hadoop-aws.html | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/8/testReport/ | | Max. process+thread count | 1349 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/8/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-557654978 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 1s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 20m 35s | trunk passed | | +1 :green_heart: | compile | 17m 53s | trunk passed | | +1 :green_heart: | checkstyle | 2m 49s | trunk passed | | +1 :green_heart: | mvnsite | 2m 5s | trunk passed | | +1 :green_heart: | shadedclient | 19m 56s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 58s | trunk passed | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 8s | trunk passed | | -0 :warning: | patch | 1m 28s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 20s | the patch passed | | +1 :green_heart: | compile | 17m 14s | the patch passed | | +1 :green_heart: | javac | 17m 14s | the patch passed | | -0 :warning: | checkstyle | 2m 52s | root: The patch generated 14 new + 83 unchanged - 0 fixed = 97 total (was 83) | | +1 :green_heart: | mvnsite | 2m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 14m 31s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 0s | the patch passed | | -1 :x: | findbugs | 1m 25s | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 10m 11s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 38s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. | | | | 127m 41s | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-tools/hadoop-aws | | | Switch statement found in org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(String[], PrintStream) where one case falls through to the next case At S3GuardTool.java:PrintStream) where one case falls through to the next case At S3GuardTool.java:[lines 1254-1261] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1707 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6c275f9ff29d 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 4c1a128 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/9/artifact/out/diff-checkstyle-root.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/9/artifact/out/new-findbugs-hadoop-tools_hadoop-aws.html | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/9/testReport/ | | Max. process+thread count | 1347 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1707/9/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infr
[jira] [Commented] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988721#comment-16988721 ] Adam Antal commented on HADOOP-16683: - Hi [~snemeth], The branch-3.2 and branch-3.1 patches have green jenkins build, could you please backport so we can close this jira? > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch, HADOOP-16683.branch-3.1.001.patch, > HADOOP-16683.branch-3.2.001.patch, HADOOP-16683.branch-3.2.001.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
steveloughran commented on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-562095185 # setting the readahead I have seen the Parquet readFooter code and agree, readahead matters there. For the S3A readahead (HADOOP-13208? I forget) we worked off ORC traces and optimised for the PositionedReadable reads, which offen surface as a pair of back-to-back reads; Setting a minimum read ahead range handles this codepath. If you're going near read ahead, it would be good if you implement `CanSetReadahead`, as then Applications can configure their read ahead in a filesystem independent manner. I worry a bit about this as with object stores, the effective read ahead range does seem to depend on distance from and hence latency to the store. Presumably your benchmarks were executed in Azure? That is the likely real-world deployment. I'd be curious * what experiments you've done to work out what a good range is with your store/client? * how things behave over long-distance connections? Regarding the patch: when would we ever really want to turn this off? Is a serious question. For ORC, our past studies with S3A say "readahead". For Parquet, the implication is the same as needed. What other uses it do we care about? an HBase trace would be good here. A key thing I'm trying to emphasise is that a small readahead is not expensive compared to the delays caused by having to issue two near-adjacent GET Requests. The main penalty is on a seek outside the range we have to drain the stream, which is ~(bytes-remaining/bandwidth). I would prefer that rather than a boolean value, the stream always does the readahead except when the range == 0. That way setReadahead(0) and the config options are the way to disable it. And as noted: I don't think it should be disabled on random IO. ## S3A reads always show significant speed up with readahead > 0; the only issue is what makes a good value? # Testing Looking at the test, if you add some package-private method to query the readahead state, e.g (a) the range value and (b) bytes remaining in the current buffer. Then you can verify that * read(bytes[n]) decrements the buffer by n * seek(getPos()) doesn't drain the buffer; * seek(getPos() + 1) doesn't it either. a read() should then decrement it by one * seek(0), seek(getPos()) doesn't drain the buffer; a read() should then decrement it by one * setReadahead(0) removes the buffer on the next seek+read outside the buffer. (There's no point discarding the current read, is there?) Yes I am adding extra homework. Sorry. But as others will note: I like my tests, and I like tests in the hadoop- test suites as they catch regressions without needing any external tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-561914676 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 19m 37s | trunk passed | | +1 :green_heart: | compile | 0m 35s | trunk passed | | +1 :green_heart: | checkstyle | 0m 20s | trunk passed | | +1 :green_heart: | mvnsite | 0m 33s | trunk passed | | +1 :green_heart: | shadedclient | 13m 31s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 24s | trunk passed | | +0 :ok: | spotbugs | 0m 53s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 51s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 29s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | +1 :green_heart: | javac | 0m 24s | the patch passed | | -0 :warning: | checkstyle | 0m 17s | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | +1 :green_heart: | mvnsite | 0m 27s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | -1 :x: | whitespace | 0m 0s | The patch 1 line(s) with tabs. | | +1 :green_heart: | shadedclient | 13m 41s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | | +1 :green_heart: | findbugs | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 56m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b3c0236dc821 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 8d82c44 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/artifact/out/whitespace-eol.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/artifact/out/whitespace-tabs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/testReport/ | | Max. process+thread count | 418 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/10/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-557761499 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 26m 47s | trunk passed | | +1 :green_heart: | compile | 0m 35s | trunk passed | | +1 :green_heart: | checkstyle | 0m 32s | trunk passed | | +1 :green_heart: | mvnsite | 0m 44s | trunk passed | | +1 :green_heart: | shadedclient | 17m 20s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 32s | trunk passed | | +0 :ok: | spotbugs | 1m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 7s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 19s | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedclient | 17m 36s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | | +1 :green_heart: | findbugs | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 75m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 86c14ee7f154 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 049940e | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/7/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/7/testReport/ | | Max. process+thread count | 341 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/7/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-557757453 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 18m 3s | trunk passed | | +1 :green_heart: | compile | 0m 32s | trunk passed | | +1 :green_heart: | checkstyle | 0m 25s | trunk passed | | +1 :green_heart: | mvnsite | 0m 35s | trunk passed | | +1 :green_heart: | shadedclient | 13m 49s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 26s | trunk passed | | +0 :ok: | spotbugs | 0m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 50s | trunk passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 17s | hadoop-azure in the patch failed. | | -1 :x: | compile | 0m 17s | hadoop-azure in the patch failed. | | -1 :x: | javac | 0m 17s | hadoop-azure in the patch failed. | | +1 :green_heart: | checkstyle | 0m 16s | the patch passed | | -1 :x: | mvnsite | 0m 18s | hadoop-azure in the patch failed. | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedclient | 14m 7s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | | -1 :x: | findbugs | 0m 21s | hadoop-azure in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 20s | hadoop-azure in the patch failed. | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 53m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 7507f09ecfb7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 049940e | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-compile-hadoop-tools_hadoop-azure.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-compile-hadoop-tools_hadoop-azure.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-findbugs-hadoop-tools_hadoop-azure.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/testReport/ | | Max. process+thread count | 418 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/6/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services -
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-556831352 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 18m 21s | trunk passed | | +1 :green_heart: | compile | 0m 31s | trunk passed | | +1 :green_heart: | checkstyle | 0m 24s | trunk passed | | +1 :green_heart: | mvnsite | 0m 32s | trunk passed | | +1 :green_heart: | shadedclient | 13m 25s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 26s | trunk passed | | +0 :ok: | spotbugs | 0m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 50s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 27s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | +1 :green_heart: | javac | 0m 24s | the patch passed | | -0 :warning: | checkstyle | 0m 16s | hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) | | +1 :green_heart: | mvnsite | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 13m 14s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | | +1 :green_heart: | findbugs | 0m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 54m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b07f36ebb051 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 1a0c0e4 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/3/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/3/testReport/ | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-556886591 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 18m 11s | trunk passed | | +1 :green_heart: | compile | 0m 31s | trunk passed | | +1 :green_heart: | checkstyle | 0m 24s | trunk passed | | +1 :green_heart: | mvnsite | 0m 33s | trunk passed | | +1 :green_heart: | shadedclient | 13m 19s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 27s | trunk passed | | +0 :ok: | spotbugs | 0m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 48s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | -0 :warning: | checkstyle | 0m 16s | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | +1 :green_heart: | mvnsite | 0m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 13m 22s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | | +1 :green_heart: | findbugs | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 54m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b8e49f0eae4c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 1a0c0e4 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/4/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/4/testReport/ | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/4/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
steveloughran commented on a change in pull request #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#discussion_r354250097 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -446,9 +446,7 @@ public AbfsInputStream openFileForRead(final Path path, final FileSystem.Statist // Add statistics for InputStream return new AbfsInputStream(client, statistics, - AbfsHttpConstants.FORWARD_SLASH + getRelativePath(path), contentLength, - abfsConfiguration.getReadBufferSize(), abfsConfiguration.getReadAheadQueueDepth(), - abfsConfiguration.getTolerateOobAppends(), eTag); +AbfsHttpConstants.FORWARD_SLASH + getRelativePath(path), contentLength, eTag, abfsConfiguration); Review comment: I'd prefer some structure "ReadContext" to pass in This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads.
hadoop-yetus removed a comment on issue #1708: HADOOP-16696: Always read ahead config, to use read ahead even for non sequential reads. URL: https://github.com/apache/hadoop/pull/1708#issuecomment-551998615 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1087 | trunk passed | | +1 | compile | 33 | trunk passed | | +1 | checkstyle | 26 | trunk passed | | +1 | mvnsite | 35 | trunk passed | | +1 | shadedclient | 821 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 49 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 29 | the patch passed | | +1 | compile | 25 | the patch passed | | +1 | javac | 25 | the patch passed | | -0 | checkstyle | 18 | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | +1 | mvnsite | 27 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 817 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 25 | the patch passed | | +1 | findbugs | 55 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 82 | hadoop-azure in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 3335 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1708 | | JIRA Issue | HADOOP-16696 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux ff9d08f0db60 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 42fc888 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/1/testReport/ | | Max. process+thread count | 452 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1708/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1738: HADOOP-16747. Support Python 3 in dev-support scripts.
hadoop-yetus commented on issue #1738: HADOOP-16747. Support Python 3 in dev-support scripts. URL: https://github.com/apache/hadoop/pull/1738#issuecomment-562081231 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 24m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 12s | Maven dependency ordering for branch | | +1 :green_heart: | shadedclient | 13m 48s | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for patch | | -1 :x: | pylint | 0m 7s | The patch generated 2 new + 60 unchanged - 321 fixed = 62 total (was 381) | | -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. | | +1 :green_heart: | shadedclient | 13m 14s | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 55m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1738 | | Optional Tests | dupname asflicense pylint | | uname | Linux 91221651a7ef 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 682e6fd | | pylint | https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/1/artifact/out/diff-patch-pylint.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/1/artifact/out/whitespace-tabs.txt | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: U: | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1738/1/console | | versions | git=2.7.4 maven=3.3.9 pylint=1.9.2 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16731) Unable to build Hadoop 3.2.1 from source code
[ https://issues.apache.org/jira/browse/HADOOP-16731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16731: Priority: Major (was: Blocker) > Unable to build Hadoop 3.2.1 from source code > - > > Key: HADOOP-16731 > URL: https://issues.apache.org/jira/browse/HADOOP-16731 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.2.1 > Environment: ubuntu16.04 32位 > hadoop 3.2.1 >Reporter: zhaobaoquan >Priority: Major > Fix For: 3.2.1 > > Attachments: lll.log > > > [WARNING] CMakeFiles/Makefile2:147: recipe for target > 'CMakeFiles/nativetask.dir/all' failed > [WARNING] CMakeFiles/nativetask_static.dir/build.make:465: recipe for target > 'CMakeFiles/nativetask_static.dir/main/native/src/util/Checksum.cc.o' failed > [WARNING] CMakeFiles/Makefile2:184: recipe for target > 'CMakeFiles/nativetask_static.dir/all' failed > [WARNING] Makefile:83: recipe for target 'all' failed > [WARNING] make[1]: *** [CMakeFiles/nativetask.dir/all] Error 2 > [WARNING] make[2]: *** > [CMakeFiles/nativetask_static.dir/main/native/src/util/Checksum.cc.o] Error 1 > [WARNING] make[1]: *** [CMakeFiles/nativetask_static.dir/all] Error 2 > [WARNING] make: *** [all] Error 2 > [ERROR] Failed to execute goal > org.apache.hadoop:hadoop-maven-plugins:3.2.1:cmake-compile (cmake-compile) on > project hadoop-mapreduce-client-nativetask: make failed with error code 2 -> > [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.hadoop:hadoop-maven-plugins:3.2.1:cmake-compile > (cmake-compile) on project hadoop-mapreduce-client-nativetask: make failed > with error code 2 > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:215) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) > at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke (Method.java:498) > at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced > (Launcher.java:282) > at org.codehaus.plexus.classworlds.launcher.Launcher.launch > (Launcher.java:225) > at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode > (Launcher.java:406) > at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) > Caused by: org.apache.maven.plugin.MojoExecutionException: make failed with > error code 2 > at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake > (CompileMojo.java:229) > at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute > (CompileMojo.java:98) > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:210) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (D
[jira] [Updated] (HADOOP-16696) ABFS Adding an option to Always use Read Ahead, even for non sequential reads
[ https://issues.apache.org/jira/browse/HADOOP-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16696: Summary: ABFS Adding an option to Always use Read Ahead, even for non sequential reads (was: Adding an option to Always use Read Ahead, even for non sequential reads) > ABFS Adding an option to Always use Read Ahead, even for non sequential reads > - > > Key: HADOOP-16696 > URL: https://issues.apache.org/jira/browse/HADOOP-16696 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Saurabh >Assignee: Saurabh >Priority: Major > Attachments: patch1.diff > > > Adding a config fs.azure.always.readahead, which is disabled by default, to > allow read ahead in case of non-sequential reads, such as when reading > parquet file in spark. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka opened a new pull request #1738: HADOOP-16747. Support Python 3 in dev-support scripts.
aajisaka opened a new pull request #1738: HADOOP-16747. Support Python 3 in dev-support scripts. URL: https://github.com/apache/hadoop/pull/1738 JIRA: https://issues.apache.org/jira/browse/HADOOP-16748 * Fix formatting by IntelliJ IDEA Python plugin * Updated shebang to use Python 3 * Removed code for Python 2.x * Ran the scripts with Python 3.7.5 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1530: HDFS-14869 Copy renamed files which are not excluded anymore by filter
hadoop-yetus commented on issue #1530: HDFS-14869 Copy renamed files which are not excluded anymore by filter URL: https://github.com/apache/hadoop/pull/1530#issuecomment-562045490 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 22m 22s | trunk passed | | +1 :green_heart: | compile | 0m 30s | trunk passed | | +1 :green_heart: | checkstyle | 0m 27s | trunk passed | | +1 :green_heart: | mvnsite | 0m 37s | trunk passed | | +1 :green_heart: | shadedclient | 17m 30s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 26s | trunk passed | | +0 :ok: | spotbugs | 0m 57s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 55s | trunk passed | | -0 :warning: | patch | 1m 16s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | -0 :warning: | checkstyle | 0m 19s | hadoop-tools/hadoop-distcp: The patch generated 10 new + 47 unchanged - 0 fixed = 57 total (was 47) | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 17m 55s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | | +1 :green_heart: | findbugs | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 16m 31s | hadoop-distcp in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 86m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1530/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1530 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux fe4151e5f851 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / da9aa9c | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1530/3/artifact/out/diff-checkstyle-hadoop-tools_hadoop-distcp.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1530/3/testReport/ | | Max. process+thread count | 303 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1530/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16748) Support Python 3 in dev-support scripts
Akira Ajisaka created HADOOP-16748: -- Summary: Support Python 3 in dev-support scripts Key: HADOOP-16748 URL: https://issues.apache.org/jira/browse/HADOOP-16748 Project: Hadoop Common Issue Type: Sub-task Reporter: Akira Ajisaka Assignee: Akira Ajisaka -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
