[jira] [Commented] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[
https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827564#comment-15827564
]
Hadoop QA commented on HADOOP-13119:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
14s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 10m
6s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 13m 47s{color}
| {color:red} root generated 492 new + 183 unchanged - 0 fixed = 675 total (was
183) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 6 unchanged - 3 fixed = 6 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 14s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.viewfs.TestViewFsTrash |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13119 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847998/HADOOP-13119.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 242a8dd23b7b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / e224c96 |
| Default Java | 1.8.0_111 |
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11453/artifact/patchprocess/branch-compile-root.txt
|
| findbugs | v3.0.0 |
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11453/artifact/patchprocess/diff-compile-javac-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11453/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11453/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11453/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Web UI error
[jira] [Updated] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[ https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HADOOP-13119: Attachment: HADOOP-13119.005.patch [~eyang] Thanks for reviewing. Upload v5 patch. > Web UI error accessing links which need authorization when Kerberos > --- > > Key: HADOOP-13119 > URL: https://issues.apache.org/jira/browse/HADOOP-13119 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.4 >Reporter: Jeffrey E Rodriguez >Assignee: Yuanbo Liu > Labels: security > Attachments: HADOOP-13119.001.patch, HADOOP-13119.002.patch, > HADOOP-13119.003.patch, HADOOP-13119.004.patch, HADOOP-13119.005.patch, > screenshot-1.png > > > User Hadoop on secure mode. > login as kdc user, kinit. > start firefox and enable Kerberos > access http://localhost:50070/logs/ > Get 403 authorization errors. > only hdfs user could access logs. > Would expect as a user to be able to web interface logs link. > Same results if using curl: > curl -v --negotiate -u tester: http://localhost:50070/logs/ > HTTP/1.1 403 User tester is unauthorized to access this page. > so: > 1. either don't show links if hdfs user is able to access. > 2. provide mechanism to add users to web application realm. > 3. note that we are pass authentication so the issue is authorization to > /logs/ > suspect that /logs/ path is secure in webdescriptor so suspect users by > default don't have access to secure paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13996) Fix some release build issues
[ https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827491#comment-15827491 ] Allen Wittenauer commented on HADOOP-13996: --- FWIW, I'm also seeing: * hadoop-client-minicluster throwing some serious warnings. * failure during shading of hadoop-client-runtime while building the javadocs This is with "create-release --asfrelease --docker". At this point, I'm really concerned about the stability of the tree. I've been debating adding a create-release run in the qbt build and it's looking like a better idea more and more. :( > Fix some release build issues > - > > Key: HADOOP-13996 > URL: https://issues.apache.org/jira/browse/HADOOP-13996 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: hadoop-13996.001.patch > > > Found some build issues while doing some test runs with the create-release.sh > script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827445#comment-15827445
]
Akira Ajisaka commented on HADOOP-13989:
Sorry for breaking the post commit job and thanks Andrew for reverting this.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13955) Replace deprecated HttpServer2 and SSLFactory constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827441#comment-15827441
]
John Zhuge commented on HADOOP-13955:
-
All 4 failed unit tests are unrelated and they pass locally.
> Replace deprecated HttpServer2 and SSLFactory constants
> ---
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch, HADOOP-13955.002.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13955) Replace deprecated HttpServer2 and SSLFactory constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-13955:
Summary: Replace deprecated HttpServer2 and SSLFactory constants (was:
Remove deprecated HttpServer2 and SSLFactory constants)
> Replace deprecated HttpServer2 and SSLFactory constants
> ---
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch, HADOOP-13955.002.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[ https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827411#comment-15827411 ] Eric Yang commented on HADOOP-13119: Hi [~yuanbo], Could you fix the newly introduced style check errors? Thanks > Web UI error accessing links which need authorization when Kerberos > --- > > Key: HADOOP-13119 > URL: https://issues.apache.org/jira/browse/HADOOP-13119 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.4 >Reporter: Jeffrey E Rodriguez >Assignee: Yuanbo Liu > Labels: security > Attachments: HADOOP-13119.001.patch, HADOOP-13119.002.patch, > HADOOP-13119.003.patch, HADOOP-13119.004.patch, screenshot-1.png > > > User Hadoop on secure mode. > login as kdc user, kinit. > start firefox and enable Kerberos > access http://localhost:50070/logs/ > Get 403 authorization errors. > only hdfs user could access logs. > Would expect as a user to be able to web interface logs link. > Same results if using curl: > curl -v --negotiate -u tester: http://localhost:50070/logs/ > HTTP/1.1 403 User tester is unauthorized to access this page. > so: > 1. either don't show links if hdfs user is able to access. > 2. provide mechanism to add users to web application realm. > 3. note that we are pass authentication so the issue is authorization to > /logs/ > suspect that /logs/ path is secure in webdescriptor so suspect users by > default don't have access to secure paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13955) Remove deprecated HttpServer2 and SSLFactory constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827403#comment-15827403
]
Hadoop QA commented on HADOOP-13955:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
23s{color} | {color:green} root generated 0 new + 671 unchanged - 4 fixed = 671
total (was 675) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
16s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 57s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}200m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13955 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847939/HADOOP-13955.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ec22dfa3caa5 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 47dfda7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11451/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
[jira] [Commented] (HADOOP-13996) Fix some release build issues
[ https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827306#comment-15827306 ] Allen Wittenauer commented on HADOOP-13996: --- I'm having trouble getting a full build to complete due to the gpg-agent timing out while running it in a Docker container on my Mac. Now that the build takes significant longer (YARN UI, shading, and probably some other big, time consumers have been added), we probably need to bump up the TTL too. I'll play with it some more. > Fix some release build issues > - > > Key: HADOOP-13996 > URL: https://issues.apache.org/jira/browse/HADOOP-13996 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: hadoop-13996.001.patch > > > Found some build issues while doing some test runs with the create-release.sh > script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827224#comment-15827224
]
Joe Pallas commented on HADOOP-13989:
-
So far my experiments with {{true}} have not
been successful, although the shade plugin is cited by the jar plugin
documentation for forceCreation as a use case. I'm guessing that something
about the handling of sources jars is different or broken.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab
[
https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827216#comment-15827216
]
Duo Zhang commented on HADOOP-13433:
{quote}
can you upload branch-2.7 patch also..?
{quote}
Sure. Will upload later. I think the problem is the UTs. I need to rewrite them
without lambda as we need to support JDK7 in branch-2.
> Race in UGI.reloginFromKeytab
> -
>
> Key: HADOOP-13433
> URL: https://issues.apache.org/jira/browse/HADOOP-13433
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2, 2.6.6
>
> Attachments: HADOOP-13433.patch, HADOOP-13433-v1.patch,
> HADOOP-13433-v2.patch, HADOOP-13433-v4.patch, HBASE-13433-testcase-v3.patch
>
>
> This is a problem that has troubled us for several years. For our HBase
> cluster, sometimes the RS will be stuck due to
> {noformat}
> 2016-06-20,03:44:12,936 INFO org.apache.hadoop.ipc.SecureClient: Exception
> encountered while connecting to the server :
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: The ticket
> isn't for us (35) - BAD TGS SERVER NAME)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:194)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:140)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:187)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$700(SecureClient.java:95)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:325)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1781)
> at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
> at org.apache.hadoop.hbase.security.User.call(User.java:607)
> at org.apache.hadoop.hbase.security.User.access$700(User.java:51)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:461)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:321)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1164)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1004)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:107)
> at $Proxy24.replicateLogEntries(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:962)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.runLoop(ReplicationSource.java:466)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:515)
> Caused by: GSSException: No valid credentials provided (Mechanism level: The
> ticket isn't for us (35) - BAD TGS SERVER NAME)
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:663)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:180)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:175)
> ... 23 more
> Caused by: KrbException: The ticket isn't for us (35) - BAD TGS SERVER NAME
> at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:64)
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:185)
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:294)
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:106)
> at
> sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:557)
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:594)
> ... 26 more
> Caused by: KrbException: Identifier doesn't match expected value (906)
> at sun.security.krb5.internal.KDCRep.init(KDCRep.java:133)
> at
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827195#comment-15827195
]
Hudson commented on HADOOP-13989:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11131 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11131/])
Revert "HADOOP-13989. Fix typo in hadoop-client shade configuration. (wang: rev
47dfda7c92bf6c1b460540758412a7a31b4bee54)
* (edit) hadoop-client-modules/hadoop-client-minicluster/pom.xml
* (edit) hadoop-client-modules/hadoop-client-runtime/pom.xml
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12097) Allow port range to be specified while starting webapp
[
https://issues.apache.org/jira/browse/HADOOP-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827175#comment-15827175
]
Hadoop QA commented on HADOOP-12097:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HADOOP-12097 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-12097 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12783339/HADOOP-12097.02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11452/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Allow port range to be specified while starting webapp
> ---
>
> Key: HADOOP-12097
> URL: https://issues.apache.org/jira/browse/HADOOP-12097
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: webapp
> Attachments: HADOOP-12097.01.patch, HADOOP-12097.02.patch, tests.patch
>
>
> This can be used by AM while starting its webapp.
> Currently a random port is chosen, which is never a good idea.
>
> And {{Webapps}} has facility to allow only a specific port to be configured
> which may be occupied. A port range in case of AM, hence will be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12097) Allow port range to be specified while starting webapp
[
https://issues.apache.org/jira/browse/HADOOP-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827146#comment-15827146
]
Junping Du commented on HADOOP-12097:
-
Thanks [~varun_saxena] for reporting the issue and delivering the fix. The
patch looks good in overall. Can you merge the unit test patch into the main
patch and rebase to latest trunk? I will review it from there. Thanks!
> Allow port range to be specified while starting webapp
> ---
>
> Key: HADOOP-12097
> URL: https://issues.apache.org/jira/browse/HADOOP-12097
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: webapp
> Attachments: HADOOP-12097.01.patch, HADOOP-12097.02.patch, tests.patch
>
>
> This can be used by AM while starting its webapp.
> Currently a random port is chosen, which is never a good idea.
>
> And {{Webapps}} has facility to allow only a specific port to be configured
> which may be occupied. A port range in case of AM, hence will be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827125#comment-15827125
]
Andrew Wang commented on HADOOP-13989:
--
FYI that I've reverted this from trunk for now.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-13989:
-
Fix Version/s: (was: 3.0.0-alpha2)
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13955) Remove deprecated HttpServer2 and SSLFactory constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-13955:
Summary: Remove deprecated HttpServer2 and SSLFactory constants (was: Fix
deprecation of HttpServer2 constants)
> Remove deprecated HttpServer2 and SSLFactory constants
> --
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch, HADOOP-13955.002.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13955) Fix deprecation of HttpServer2 constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-13955:
Attachment: HADOOP-13955.002.patch
Patch 002
* Deprecate SSL_ENABLED_PROTOCOLS and DEFAULT_SSL_ENABLED_PROTOCOLS
> Fix deprecation of HttpServer2 constants
>
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch, HADOOP-13955.002.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13996) Fix some release build issues
[
https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827037#comment-15827037
]
Hadoop QA commented on HADOOP-13996:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
12s{color} | {color:green} The patch generated 0 new + 106 unchanged - 1 fixed
= 106 total (was 107) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
9s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
9s{color} | {color:green} hadoop-dist in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13996 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847908/hadoop-13996.001.patch
|
| Optional Tests | asflicense shellcheck shelldocs compile javac javadoc
mvninstall mvnsite unit xml |
| uname | Linux 42f6aedbdb5b 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 4d1f3d9 |
| Default Java | 1.8.0_111 |
| shellcheck | v0.4.5 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11450/testReport/ |
| modules | C: hadoop-dist U: hadoop-dist |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11450/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix some release build issues
> -
>
> Key: HADOOP-13996
> URL: https://issues.apache.org/jira/browse/HADOOP-13996
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Blocker
> Attachments: hadoop-13996.001.patch
>
>
> Found some build issues while doing some test runs with the create-release.sh
> script.
--
This message was
[jira] [Commented] (HADOOP-13673) Update scripts to be smarter when running with privilege
[ https://issues.apache.org/jira/browse/HADOOP-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827031#comment-15827031 ] Ravi Prakash commented on HADOOP-13673: --- LGTM too! Thanks Allen and Andrew! +1 > Update scripts to be smarter when running with privilege > > > Key: HADOOP-13673 > URL: https://issues.apache.org/jira/browse/HADOOP-13673 > Project: Hadoop Common > Issue Type: New Feature > Components: scripts >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Labels: security > Attachments: HADOOP-13673.00.patch, HADOOP-13673.01.patch, > HADOOP-13673.02.patch, HADOOP-13673.03.patch, HADOOP-13673.04.patch > > > As work continues on HADOOP-13397, it's become evident that we need better > hooks to start daemons as specifically configured users. Via the > (command)_(subcommand)_USER environment variables in 3.x, we actually have a > standardized way to do that. This in turn means we can make the sbin scripts > super functional with a bit of updating: > * Consolidate start-dfs.sh and start-secure-dns.sh into one script > * Make start-\*.sh and stop-\*.sh know how to switch users when run as root > * Undeprecate start/stop-all.sh so that it could be used as root for > production purposes and as a single user for non-production users -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13976) Path globbing does not match newlines
[ https://issues.apache.org/jira/browse/HADOOP-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827002#comment-15827002 ] Hudson commented on HADOOP-13976: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11130 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11130/]) HADOOP-13976. Path globbing does not match newlines. Contributed by Eric (kihwal: rev 706d630eb9db9658083d57d1d99b6a0f11cc5657) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/GlobPattern.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGlobPattern.java > Path globbing does not match newlines > - > > Key: HADOOP-13976 > URL: https://issues.apache.org/jira/browse/HADOOP-13976 > Project: Hadoop Common > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Fix For: 3.0.0-alpha2, 2.8.1 > > Attachments: HADOOP-13976.001.patch > > > Need to add the DOTALL flag to allow for newlines to be accepted as well -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13978) Update project release notes for 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HADOOP-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827005#comment-15827005 ] Hudson commented on HADOOP-13978: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11130 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11130/]) HADOOP-13978. Update project release notes for 3.0.0-alpha2. Contributed (wang: rev 37468ebf6f424d2a4bd654c589c00894e5376575) * (edit) hadoop-project/src/site/markdown/index.md.vm > Update project release notes for 3.0.0-alpha2 > - > > Key: HADOOP-13978 > URL: https://issues.apache.org/jira/browse/HADOOP-13978 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13978.001.patch, HADOOP-13978.002.patch > > > Let's update the website release notes for 3.0.0-alpha2's changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826988#comment-15826988
]
Sean Busbey commented on HADOOP-13989:
--
Can we use the work-around suggested in MSHADE-126?
{quote}
Adding true to the maven-jar-plugin works around
this.
{quote}
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11347) RawLocalFileSystem#mkdir and create should honor umask
[
https://issues.apache.org/jira/browse/HADOOP-11347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HADOOP-11347:
---
Target Version/s: (was: )
Hadoop Flags: Incompatible change
> RawLocalFileSystem#mkdir and create should honor umask
> --
>
> Key: HADOOP-11347
> URL: https://issues.apache.org/jira/browse/HADOOP-11347
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.7.0
>Reporter: Chris Nauroth
>Assignee: Varun Saxena
> Labels: BB2015-05-RFC, incompatible
> Fix For: 3.0.0-alpha1
>
> Attachments: HADOOP-11347.001.patch, HADOOP-11347.002.patch,
> HADOOP-11347.004.patch, HADOOP-11347.03.patch, HADOOP-11347.05.patch
>
>
> The {{FileSystem}} and {{FileContext}} APIs are inconsistent in enforcement
> of umask for newly created directories. {{FileContext}} utilizes
> configuration property {{fs.permissions.umask-mode}} and runs a separate
> {{chmod}} call to guarantee bypassing the process umask. This is the
> expected behavior for Hadoop as discussed in the documentation of
> {{fs.permissions.umask-mode}}. For the equivalent {{FileSystem}} APIs, it
> does not use {{fs.permissions.umask-mode}}. Instead, the permissions end up
> getting controlled by the process umask.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13673) Update scripts to be smarter when running with privilege
[ https://issues.apache.org/jira/browse/HADOOP-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826913#comment-15826913 ] Andrew Wang commented on HADOOP-13673: -- Nice rev Allen, LGTM +1 personally. I'd like to wait for Ravi's +1 too, though I believe his review comments are also addressed or explained. > Update scripts to be smarter when running with privilege > > > Key: HADOOP-13673 > URL: https://issues.apache.org/jira/browse/HADOOP-13673 > Project: Hadoop Common > Issue Type: New Feature > Components: scripts >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Labels: security > Attachments: HADOOP-13673.00.patch, HADOOP-13673.01.patch, > HADOOP-13673.02.patch, HADOOP-13673.03.patch, HADOOP-13673.04.patch > > > As work continues on HADOOP-13397, it's become evident that we need better > hooks to start daemons as specifically configured users. Via the > (command)_(subcommand)_USER environment variables in 3.x, we actually have a > standardized way to do that. This in turn means we can make the sbin scripts > super functional with a bit of updating: > * Consolidate start-dfs.sh and start-secure-dns.sh into one script > * Make start-\*.sh and stop-\*.sh know how to switch users when run as root > * Undeprecate start/stop-all.sh so that it could be used as root for > production purposes and as a single user for non-production users -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13997) Typo in metrics docs
[
https://issues.apache.org/jira/browse/HADOOP-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned HADOOP-13997:
-
Assignee: Daniel Templeton
> Typo in metrics docs
>
>
> Key: HADOOP-13997
> URL: https://issues.apache.org/jira/browse/HADOOP-13997
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Trivial
> Labels: newbie
>
> {{myprefix.sink.file0.filename=context1.out}} should be
> {{myprefix.sink.file0.filename=context0.out}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13673) Update scripts to be smarter when running with privilege
[
https://issues.apache.org/jira/browse/HADOOP-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826906#comment-15826906
]
Andrew Wang commented on HADOOP-13673:
--
Just for completeness, here's the email exchange between myself and Allen (I
hope Allen doesn't mind me posting this):
> * hadoop_abs, does {{readlink -f}} accomplish the same thing?
Effectively yes, but unfortunately, readlink isn't POSIX. It works
differently on different operating systems, even to the point of having
radically different parameters. So we can't rely upon it. hadoop_abs, while
obviously slower, is super portable. :)
> * Was it intentional to remove hadoop_usage from start-secure-dns.sh? The
> stop script still has a usage.
I was going to replace it but I guess I got distracted. haha. I'll put
it back for now.
> * Few typos seen while reviewing: "legimately" "optinally" "definied"
> "description"
I think i got all of these.
> * I think there's an extra "resourcemanager" in this line:
>
> {code}
> + hadoop_uservar_su yarn resourcemanager proxyserver
> "${HADOOP_YARN_HOME}/bin/yarn" \
> {code}
Yup, definitely.
> * IIUC we we call {{hadoop_uservar_su}} directly in {{start-dfs.sh}} which
> requires that the user vars to be set when running as root. Noticed though
> that {{start-balancer.sh}} doesn't do this. Is this intentional or an
> omission?
Intentional. All of the single daemon scripts will switch when they
call the main hdfs/mapred/... script. For the others, --workers needs to get
called with the appropriate user so that we don't try to use root's ssh key
unless we really were meant to (e.g., secure datanode).
> * Wondering if more needs to be said in the docs about what commands support
> this. For instance, HTTPFS is off on the side, but I guess that'll be fixed
> once John finishes the conversion from Tomcat to Jetty. Are there any other
> gaps you're aware of?
Of the daemons, yeah, httpfs is a big outlier. The other ones are
rumen and sls. Now that we have dynamic commands, we should probably make them
inline as well.
> Update scripts to be smarter when running with privilege
>
>
> Key: HADOOP-13673
> URL: https://issues.apache.org/jira/browse/HADOOP-13673
> Project: Hadoop Common
> Issue Type: New Feature
> Components: scripts
>Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Labels: security
> Attachments: HADOOP-13673.00.patch, HADOOP-13673.01.patch,
> HADOOP-13673.02.patch, HADOOP-13673.03.patch, HADOOP-13673.04.patch
>
>
> As work continues on HADOOP-13397, it's become evident that we need better
> hooks to start daemons as specifically configured users. Via the
> (command)_(subcommand)_USER environment variables in 3.x, we actually have a
> standardized way to do that. This in turn means we can make the sbin scripts
> super functional with a bit of updating:
> * Consolidate start-dfs.sh and start-secure-dns.sh into one script
> * Make start-\*.sh and stop-\*.sh know how to switch users when run as root
> * Undeprecate start/stop-all.sh so that it could be used as root for
> production purposes and as a single user for non-production users
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13978) Update project release notes for 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HADOOP-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13978: - Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch Available) Committed to trunk, thanks Arun for the extra text, and to Kai and Sean for reviews! > Update project release notes for 3.0.0-alpha2 > - > > Key: HADOOP-13978 > URL: https://issues.apache.org/jira/browse/HADOOP-13978 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13978.001.patch, HADOOP-13978.002.patch > > > Let's update the website release notes for 3.0.0-alpha2's changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13978) Update project release notes for 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HADOOP-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13978: - Component/s: documentation > Update project release notes for 3.0.0-alpha2 > - > > Key: HADOOP-13978 > URL: https://issues.apache.org/jira/browse/HADOOP-13978 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13978.001.patch, HADOOP-13978.002.patch > > > Let's update the website release notes for 3.0.0-alpha2's changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[
https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826597#comment-15826597
]
Yongjun Zhang edited comment on HADOOP-13805 at 1/17/17 9:56 PM:
-
Thanks [~tucu00].
I think you listed two options, one is to make this potentially incompatible
change; the other is to create a new UGI and obsolete the old/incorrect
implementation later on.
It may not be too bad to go with option one. Say, with option two, we may hit
the issue reported here. With option one, we need to watch out how things are
broken due to the incompatible change, and fix accordingly.
If we go with option one, if client code is broken, the client code need to be
changed to do the renewal. Would you please help putting together a recommended
client code change as part of the release notes of this jira? Or any suggestion
about documenting it so user can use it as a reference.
If we go with option one, I'm +1 on Wei-Chiu's rev6 (I found that it may not be
easy to add the test you proposed as a unit test due to the run time) Would you
please also take a look at rev6?
Thanks.
was (Author: yzhangal):
Thanks [~tucu00].
I think you listed two options, one is to make this potentially incompatible
change; the other is to create a new UGI and obsolete the old/incorrect
implementation later on.
It may not be too bad to go with option one. Say, with option two, we may hit
the issue reported here. With option one, we need to watch out how things are
broken due to the incompatible change, and fix accordingly.
If we go with option one, if client code is broken, the client code need to be
changed to do the renewal. Would you please help putting together a recommended
change as part of the release notes of this jira?
If we go with option one, I'm +1 on Wei-Chiu's rev6 (I found that it may not be
easy to add the test you proposed as a unit test due to the run time) Would you
please also take a look at rev6?
Thanks.
> UGI.getCurrentUser() fails if user does not have a keytab associated
>
>
> Key: HADOOP-13805
> URL: https://issues.apache.org/jira/browse/HADOOP-13805
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2
>Reporter: Alejandro Abdelnur
>Assignee: Xiao Chen
> Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch,
> HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch,
> HADOOP-13805.05.patch
>
>
> HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the
> UGI is created from an existing Subject as in that case the keytab is not
> 'own' by UGI but by the creator of the Subject.
> In HADOOP-13558 we introduced a new private UGI constructor
> {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and
> we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}.
> The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created
> via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new
> UserGroupInformation(subject)}} which will delegate to
> {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and
> that will use externalKeyTab == *FALSE*.
> Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using
> a non-existing keytab if the TGT expired.
> This problem is experienced in {{KMSClientProvider}} when used by the HDFS
> filesystem client accessing an an encryption zone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13978) Update project release notes for 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HADOOP-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826903#comment-15826903 ] Andrew Wang commented on HADOOP-13978: -- +1 LGTM on Arun's changes. I'll commit this shortly since it's gotten eyes-on by Kai, Arun, and myself. > Update project release notes for 3.0.0-alpha2 > - > > Key: HADOOP-13978 > URL: https://issues.apache.org/jira/browse/HADOOP-13978 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: HADOOP-13978.001.patch, HADOOP-13978.002.patch > > > Let's update the website release notes for 3.0.0-alpha2's changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13997) Typo in metrics docs
Daniel Templeton created HADOOP-13997:
-
Summary: Typo in metrics docs
Key: HADOOP-13997
URL: https://issues.apache.org/jira/browse/HADOOP-13997
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Affects Versions: 2.8.0
Reporter: Daniel Templeton
Priority: Trivial
{{myprefix.sink.file0.filename=context1.out}} should be
{{myprefix.sink.file0.filename=context0.out}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13996) Fix some release build issues
[ https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826863#comment-15826863 ] Andrew Wang commented on HADOOP-13996: -- [~aw] perchance you can do a quick review? > Fix some release build issues > - > > Key: HADOOP-13996 > URL: https://issues.apache.org/jira/browse/HADOOP-13996 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: hadoop-13996.001.patch > > > Found some build issues while doing some test runs with the create-release.sh > script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13996) Fix some release build issues
[ https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13996: - Component/s: build > Fix some release build issues > - > > Key: HADOOP-13996 > URL: https://issues.apache.org/jira/browse/HADOOP-13996 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: hadoop-13996.001.patch > > > Found some build issues while doing some test runs with the create-release.sh > script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13996) Fix some release build issues
[
https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-13996:
-
Attachment: hadoop-13996.001.patch
Fixed two issues:
* Apparently you can't run {{mvn clean}} without the hadoop maven plugins, so
install those first. It was trying to download a SNAPSHOT which was failing.
* The sign plugin expects there to be a JAR generated by every project. Since
we disabled generating the dummy hadoop-dist jar, also disable the sign plugin.
> Fix some release build issues
> -
>
> Key: HADOOP-13996
> URL: https://issues.apache.org/jira/browse/HADOOP-13996
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Blocker
> Attachments: hadoop-13996.001.patch
>
>
> Found some build issues while doing some test runs with the create-release.sh
> script.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13996) Fix some release build issues
[ https://issues.apache.org/jira/browse/HADOOP-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13996: - Status: Patch Available (was: Open) > Fix some release build issues > - > > Key: HADOOP-13996 > URL: https://issues.apache.org/jira/browse/HADOOP-13996 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Attachments: hadoop-13996.001.patch > > > Found some build issues while doing some test runs with the create-release.sh > script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13996) Fix some release build issues
Andrew Wang created HADOOP-13996: Summary: Fix some release build issues Key: HADOOP-13996 URL: https://issues.apache.org/jira/browse/HADOOP-13996 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Blocker Found some build issues while doing some test runs with the create-release.sh script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826847#comment-15826847
]
Joe Pallas commented on HADOOP-13989:
-
[~busbey], it looks like there is such a known bug in the shade plugin
(MSHADE-126, resolved as Won't Fix). The workaround is to force the jar file
to be recreated every time. Doing {{clean install}} followed by {{deploy}} did
reproduce the problem in my environment, so at least I can verify an attempted
solution.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13976) Path globbing does not match newlines
[ https://issues.apache.org/jira/browse/HADOOP-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826841#comment-15826841 ] Eric Badger commented on HADOOP-13976: -- Thanks, [~kihwal]! > Path globbing does not match newlines > - > > Key: HADOOP-13976 > URL: https://issues.apache.org/jira/browse/HADOOP-13976 > Project: Hadoop Common > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Fix For: 3.0.0-alpha2, 2.8.1 > > Attachments: HADOOP-13976.001.patch > > > Need to add the DOTALL flag to allow for newlines to be accepted as well -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13994) explicitly declare the commons-lang3 dependency as 3.4
[ https://issues.apache.org/jira/browse/HADOOP-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13994: Attachment: HADOOP-13994-HADOOP-13445-001.patch patch 001, includes the ddb exclusions needed for s3guard, so the patch is against that > explicitly declare the commons-lang3 dependency as 3.4 > -- > > Key: HADOOP-13994 > URL: https://issues.apache.org/jira/browse/HADOOP-13994 > Project: Hadoop Common > Issue Type: Bug > Components: build, fs/azure, fs/s3 >Affects Versions: 3.0.0-alpha2, HADOOP-13345 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13994-HADOOP-13445-001.patch > > > Other people aren't seeing this (yet?), but unless you explicitly exclude v > 3.4 of commons-lang3 from the azure build (which HADOOP-13660 does), then the > dependency declaration of commons-lang3 v 3.3.2 is creating a resolution > conflict. That's a dependency only needed for the local dynamodb & tests. > I propose to fix this in s3guard by explicitly declaring the version used in > the tests to be that of the azure-storage one, excluding that you get for > free. It doesn't impact anything shipped in production, but puts the hadoop > build in control of what versions of commons-lang are coming in everywhere by > way of the commons-config version declared in hadoop-common -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13976) Path globbing does not match newlines
[ https://issues.apache.org/jira/browse/HADOOP-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HADOOP-13976: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.1 Status: Resolved (was: Patch Available) Thanks for fixing this, [~ebadger]. > Path globbing does not match newlines > - > > Key: HADOOP-13976 > URL: https://issues.apache.org/jira/browse/HADOOP-13976 > Project: Hadoop Common > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Fix For: 2.8.1, 3.0.0-alpha2 > > Attachments: HADOOP-13976.001.patch > > > Need to add the DOTALL flag to allow for newlines to be accepted as well -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13660) Upgrade commons-configuration version to 2.1
[ https://issues.apache.org/jira/browse/HADOOP-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826817#comment-15826817 ] Steve Loughran commented on HADOOP-13660: - thanks...what I will do is have a patch for s3guard which also fixes up the dynamo reference; this will need to go into trunk without that bit of the patch. Testing things locally now in HADOOP-13994 > Upgrade commons-configuration version to 2.1 > > > Key: HADOOP-13660 > URL: https://issues.apache.org/jira/browse/HADOOP-13660 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13660.001.patch, HADOOP-13660.002.patch, > HADOOP-13660.003.patch, HADOOP-13660.004.patch, HADOOP-13660.005.patch, > HADOOP-13660.006.patch, HADOOP-13660.007.patch, > HADOOP-13660-configuration2.001.patch > > > We're currently pulling in version 1.6 - I think we should upgrade to the > latest 1.10. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13976) Path globbing does not match newlines
[ https://issues.apache.org/jira/browse/HADOOP-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826811#comment-15826811 ] Kihwal Lee commented on HADOOP-13976: - +1 lgtm > Path globbing does not match newlines > - > > Key: HADOOP-13976 > URL: https://issues.apache.org/jira/browse/HADOOP-13976 > Project: Hadoop Common > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Attachments: HADOOP-13976.001.patch > > > Need to add the DOTALL flag to allow for newlines to be accepted as well -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13995) s3guard cli: make tests easier to run and address failure
[ https://issues.apache.org/jira/browse/HADOOP-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826810#comment-15826810 ] Aaron Fabbri commented on HADOOP-13995: --- [~mackrorysd] wants to work on this, but shout if you need any help. > s3guard cli: make tests easier to run and address failure > - > > Key: HADOOP-13995 > URL: https://issues.apache.org/jira/browse/HADOOP-13995 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Aaron Fabbri >Assignee: Sean Mackrory > > Following up on HADOOP-13650, we should: > - Make it clearer which config parameters need to be set for test to succeed, > and provide good defaults. > - Address any remaining test failures. > - Change TestS3GuardTool to an ITest -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.
[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826812#comment-15826812
]
Aaron Fabbri commented on HADOOP-13650:
---
Created HADOOP-13995.
> S3Guard: Provide command line tools to manipulate metadata store.
> -
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-13650-HADOOP-13345.000.patch,
> HADOOP-13650-HADOOP-13345.001.patch, HADOOP-13650-HADOOP-13345.002.patch,
> HADOOP-13650-HADOOP-13345.003.patch, HADOOP-13650-HADOOP-13345.004.patch,
> HADOOP-13650-HADOOP-13345.005.patch, HADOOP-13650-HADOOP-13345.006.patch,
> HADOOP-13650-HADOOP-13345.007.patch, HADOOP-13650-HADOOP-13345.008.patch,
> HADOOP-13650-HADOOP-13345.009.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13995) s3guard cli: make tests easier to run and address failure
Aaron Fabbri created HADOOP-13995: - Summary: s3guard cli: make tests easier to run and address failure Key: HADOOP-13995 URL: https://issues.apache.org/jira/browse/HADOOP-13995 Project: Hadoop Common Issue Type: Sub-task Reporter: Aaron Fabbri Assignee: Sean Mackrory Following up on HADOOP-13650, we should: - Make it clearer which config parameters need to be set for test to succeed, and provide good defaults. - Address any remaining test failures. - Change TestS3GuardTool to an ITest -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13660) Upgrade commons-configuration version to 2.1
[ https://issues.apache.org/jira/browse/HADOOP-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826802#comment-15826802 ] Sean Mackrory commented on HADOOP-13660: I don't recall but I just took a quick look. I'll bet what happened is I ran into the convergence issue in Azure (because as you said, that comes up if you remove the exclusion), and I excluded it and meant to go back and version-manage that more properly once everything else was working. All tests pass for me now, and would have back then, but that's as close as I get to being confident that all azure storage codepaths are valid without that dependency. I'd be all for properly declaring the version we depend on in the hadoop-project pom as you're suggesting... > Upgrade commons-configuration version to 2.1 > > > Key: HADOOP-13660 > URL: https://issues.apache.org/jira/browse/HADOOP-13660 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13660.001.patch, HADOOP-13660.002.patch, > HADOOP-13660.003.patch, HADOOP-13660.004.patch, HADOOP-13660.005.patch, > HADOOP-13660.006.patch, HADOOP-13660.007.patch, > HADOOP-13660-configuration2.001.patch > > > We're currently pulling in version 1.6 - I think we should upgrade to the > latest 1.10. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13994) explicitly declare the commons-lang3 dependency as 3.4
[ https://issues.apache.org/jira/browse/HADOOP-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13994: Affects Version/s: 3.0.0-alpha2 > explicitly declare the commons-lang3 dependency as 3.4 > -- > > Key: HADOOP-13994 > URL: https://issues.apache.org/jira/browse/HADOOP-13994 > Project: Hadoop Common > Issue Type: Bug > Components: build, fs/azure, fs/s3 >Affects Versions: 3.0.0-alpha2, HADOOP-13345 >Reporter: Steve Loughran >Assignee: Steve Loughran > > Other people aren't seeing this (yet?), but unless you explicitly exclude v > 3.4 of commons-lang3 from the azure build (which HADOOP-13660 does), then the > dependency declaration of commons-lang3 v 3.3.2 is creating a resolution > conflict. That's a dependency only needed for the local dynamodb & tests. > I propose to fix this in s3guard by explicitly declaring the version used in > the tests to be that of the azure-storage one, excluding that you get for > free. It doesn't impact anything shipped in production, but puts the hadoop > build in control of what versions of commons-lang are coming in everywhere by > way of the commons-config version declared in hadoop-common -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.
[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826793#comment-15826793
]
Chris Nauroth commented on HADOOP-13650:
Thanks for investigating the test failures. A separate JIRA sounds fine to me.
I'd be happy to help with code review and a test run on my machine to make
sure it works.
> S3Guard: Provide command line tools to manipulate metadata store.
> -
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-13650-HADOOP-13345.000.patch,
> HADOOP-13650-HADOOP-13345.001.patch, HADOOP-13650-HADOOP-13345.002.patch,
> HADOOP-13650-HADOOP-13345.003.patch, HADOOP-13650-HADOOP-13345.004.patch,
> HADOOP-13650-HADOOP-13345.005.patch, HADOOP-13650-HADOOP-13345.006.patch,
> HADOOP-13650-HADOOP-13345.007.patch, HADOOP-13650-HADOOP-13345.008.patch,
> HADOOP-13650-HADOOP-13345.009.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13994) explicitly declare the commons-lang3 dependency as 3.4
[ https://issues.apache.org/jira/browse/HADOOP-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13994: Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-13345) > explicitly declare the commons-lang3 dependency as 3.4 > -- > > Key: HADOOP-13994 > URL: https://issues.apache.org/jira/browse/HADOOP-13994 > Project: Hadoop Common > Issue Type: Bug > Components: build, fs/azure, fs/s3 >Affects Versions: 3.0.0-alpha2, HADOOP-13345 >Reporter: Steve Loughran >Assignee: Steve Loughran > > Other people aren't seeing this (yet?), but unless you explicitly exclude v > 3.4 of commons-lang3 from the azure build (which HADOOP-13660 does), then the > dependency declaration of commons-lang3 v 3.3.2 is creating a resolution > conflict. That's a dependency only needed for the local dynamodb & tests. > I propose to fix this in s3guard by explicitly declaring the version used in > the tests to be that of the azure-storage one, excluding that you get for > free. It doesn't impact anything shipped in production, but puts the hadoop > build in control of what versions of commons-lang are coming in everywhere by > way of the commons-config version declared in hadoop-common -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13994) explicitly declare the commons-lang3 dependency as 3.4
Steve Loughran created HADOOP-13994: --- Summary: explicitly declare the commons-lang3 dependency as 3.4 Key: HADOOP-13994 URL: https://issues.apache.org/jira/browse/HADOOP-13994 Project: Hadoop Common Issue Type: Sub-task Components: build, fs/azure, fs/s3 Affects Versions: HADOOP-13345 Reporter: Steve Loughran Assignee: Steve Loughran Other people aren't seeing this (yet?), but unless you explicitly exclude v 3.4 of commons-lang3 from the azure build (which HADOOP-13660 does), then the dependency declaration of commons-lang3 v 3.3.2 is creating a resolution conflict. That's a dependency only needed for the local dynamodb & tests. I propose to fix this in s3guard by explicitly declaring the version used in the tests to be that of the azure-storage one, excluding that you get for free. It doesn't impact anything shipped in production, but puts the hadoop build in control of what versions of commons-lang are coming in everywhere by way of the commons-config version declared in hadoop-common -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13660) Upgrade commons-configuration version to 2.1
[ https://issues.apache.org/jira/browse/HADOOP-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13660: Summary: Upgrade commons-configuration version to 2.1 (was: Upgrade commons-configuration version) > Upgrade commons-configuration version to 2.1 > > > Key: HADOOP-13660 > URL: https://issues.apache.org/jira/browse/HADOOP-13660 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Mackrory >Assignee: Sean Mackrory > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13660.001.patch, HADOOP-13660.002.patch, > HADOOP-13660.003.patch, HADOOP-13660.004.patch, HADOOP-13660.005.patch, > HADOOP-13660.006.patch, HADOOP-13660.007.patch, > HADOOP-13660-configuration2.001.patch > > > We're currently pulling in version 1.6 - I think we should upgrade to the > latest 1.10. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13956) Read ADLS credentials from Credential Provider
[
https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826778#comment-15826778
]
Hadoop QA commented on HADOOP-13956:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
36s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
50s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13956 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847886/HADOOP-13956.003.patch
|
| Optional Tests | asflicense mvnsite compile javac javadoc mvninstall
unit findbugs checkstyle |
| uname | Linux 0e4de99f0dae 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 78b487b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11449/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-tools/hadoop-azure-datalake U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11449/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Read ADLS credentials from Credential Provider
> --
>
> Key: HADOOP-13956
> URL:
[jira] [Commented] (HADOOP-13660) Upgrade commons-configuration version
[
https://issues.apache.org/jira/browse/HADOOP-13660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826764#comment-15826764
]
Steve Loughran commented on HADOOP-13660:
-
I'm staring at a build thing elsewhere related to this; convergency errors
related to commons-lang3 and that of azure and dynamodb.
# what was the reason for excluding commons-lang3 as a dependency of azure in
the azure project? I'm not faulting the decision, just trying to understand
what's happening.
# is everyone confident that all azure storage codepaths are valid without that
dependency?
# I don't see any explicit declaration on a commons-lang version. Should we
have one? Because s3guard is pulling in dynamodb local for testing, and I'm
getting some (local) convergence issues?
{code}
+-org.apache.hadoop:hadoop-aws
+-com.amazonaws:DynamoDBLocal:1.11.0
+-org.apache.commons:commons-lang3:3.3.2
{code}
This isn't surfacing in the ASF branch because of the exclusion, I'm just
wondering what to do.
FWIW, I'm thinking of declaring lang3 dependency of
{{org.apache.commons:commons-lang3:jar:3.4}} in the hadoop-project pom, and
including it in test scope for hadoop-aws+s3guard, while excludingng the
dynamodb one, it being older
> Upgrade commons-configuration version
> -
>
> Key: HADOOP-13660
> URL: https://issues.apache.org/jira/browse/HADOOP-13660
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13660.001.patch, HADOOP-13660.002.patch,
> HADOOP-13660.003.patch, HADOOP-13660.004.patch, HADOOP-13660.005.patch,
> HADOOP-13660.006.patch, HADOOP-13660.007.patch,
> HADOOP-13660-configuration2.001.patch
>
>
> We're currently pulling in version 1.6 - I think we should upgrade to the
> latest 1.10.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.
[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826762#comment-15826762
]
Sean Mackrory commented on HADOOP-13650:
I'm getting similar failures, [~cnauroth]. I had to add the endpoint to the
configuration as well. I don't get the second error you mentioned, though I do
see failures because TestS3AGuardTool creates a LocalMetastore and then asserts
that it's a DynamoDBMetastore.
> S3Guard: Provide command line tools to manipulate metadata store.
> -
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-13650-HADOOP-13345.000.patch,
> HADOOP-13650-HADOOP-13345.001.patch, HADOOP-13650-HADOOP-13345.002.patch,
> HADOOP-13650-HADOOP-13345.003.patch, HADOOP-13650-HADOOP-13345.004.patch,
> HADOOP-13650-HADOOP-13345.005.patch, HADOOP-13650-HADOOP-13345.006.patch,
> HADOOP-13650-HADOOP-13345.007.patch, HADOOP-13650-HADOOP-13345.008.patch,
> HADOOP-13650-HADOOP-13345.009.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.
[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826761#comment-15826761
]
Aaron Fabbri commented on HADOOP-13650:
---
Hi [~cnauroth]. I also hit some errors in my testing. I'm working with
[~eddyxu] and [~mackrorysd] to fix the failures and make the test easier to
run. We could change it to an ITest as well. Are you cool with doing this as
a follow-up jira? If so I'll create one now.
> S3Guard: Provide command line tools to manipulate metadata store.
> -
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-13650-HADOOP-13345.000.patch,
> HADOOP-13650-HADOOP-13345.001.patch, HADOOP-13650-HADOOP-13345.002.patch,
> HADOOP-13650-HADOOP-13345.003.patch, HADOOP-13650-HADOOP-13345.004.patch,
> HADOOP-13650-HADOOP-13345.005.patch, HADOOP-13650-HADOOP-13345.006.patch,
> HADOOP-13650-HADOOP-13345.007.patch, HADOOP-13650-HADOOP-13345.008.patch,
> HADOOP-13650-HADOOP-13345.009.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata
> store, i.e., create or delete metadata store, or {{import}}, {{sync}} the
> file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826737#comment-15826737
]
Bibin A Chundatt edited comment on HADOOP-13989 at 1/17/17 8:34 PM:
[~jpallas]
Tried reverting HADOOP-13989 locally looks like its working fine.
Locally i tried package to reproduce
{code}
mvn package -Pdist -Dtar -Pnative -Dmaven.javadoc.skip=true
-Dmaven.repo.local=/opt/apacheprojects/hadoop/repository -Pyarn-ui -DskipTests
-rf :hadoop-client-runtime --debug
{code}
{code}
hadoop.shaded.com.fasterxml.jackson.databind.Module -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
hadoop-client-runtime: Error creating shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error creating
shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:540)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.util.zip.ZipException: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232)
at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
at
org.apache.hadoop.maven.plugin.shade.resource.ServicesResourceTransformer.modifyOutputStream(ServicesResourceTransformer.java:110)
at
org.apache.maven.plugins.shade.DefaultShader.shade(DefaultShader.java:131)
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:461)
... 22 more
{code}
was (Author: bibinchundatt):
[~jpallas]
Tried reverting HADOOP-13989 in locally looks like its working fine
{code}
mvn package -Pdist -Dtar -Pnative -Dmaven.javadoc.skip=true
-Dmaven.repo.local=/opt/apacheprojects/hadoop/repository -Pyarn-ui -DskipTests
-rf :hadoop-client-runtime --debug
{code}
{code}
hadoop.shaded.com.fasterxml.jackson.databind.Module -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
hadoop-client-runtime: Error creating shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826737#comment-15826737
]
Bibin A Chundatt commented on HADOOP-13989:
---
[~jpallas]
Tried reverting HADOOP-13989 in locally looks like its working fine
{code}
mvn package -Pdist -Dtar -Pnative -Dmaven.javadoc.skip=true
-Dmaven.repo.local=/opt/apacheprojects/hadoop/repository -Pyarn-ui -DskipTests
-rf :hadoop-client-runtime --debug
{code}
{code}
hadoop.shaded.com.fasterxml.jackson.databind.Module -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
hadoop-client-runtime: Error creating shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error creating
shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:540)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.util.zip.ZipException: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232)
at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
at
org.apache.hadoop.maven.plugin.shade.resource.ServicesResourceTransformer.modifyOutputStream(ServicesResourceTransformer.java:110)
at
org.apache.maven.plugins.shade.DefaultShader.shade(DefaultShader.java:131)
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:461)
... 22 more
{code}
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826694#comment-15826694
]
Sean Busbey commented on HADOOP-13989:
--
Here's what Jenkins is doing:
{code}
${MAVEN_HOME}/bin/mvn clean install \
-DskipTests -Pnative --threads .5C \
-Dmaven.repo.local="${MAVEN_LOCAL_REPO}"
${MAVEN_HOME}/bin/mvn deploy -DdeployAtEnd=true \
-DretryFailedDeploymentCount=10 -DskipTests \
-Dmaven.repo.local="${MAVEN_LOCAL_REPO}"
{code}
The failure, as you mentioned, is in the second invocation that does deploy. I
wonder if maybe it's a bug in the shade plugin that is trying to pack more
stuff into the existing jar?
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826688#comment-15826688
]
Sean Busbey commented on HADOOP-13989:
--
looks like the post-commit job has run a few more times and consistently fails
as here.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826687#comment-15826687
]
Sean Busbey commented on HADOOP-13989:
--
I hope we don't have any SNAPSHOT dependencies, so they shouldn't have changed.
What command are you using to do the local repro? Has anyone re-run the
post-commit job yet?
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826683#comment-15826683
]
Joe Pallas commented on HADOOP-13989:
-
That's a fair point. Since I haven't been able to reproduce the problem, I'm
not able to confirm that reversion fixes it. All I have to go on is that the
last successful build was just before the change and the first failing build
only included this and YARN-6057 (changes to {{yarn-default.xml}}). So, any
changes in dependencies would have to be outside of hadoop, I think. Are there
unstable dependencies?
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826633#comment-15826633 ] Chris Nauroth commented on HADOOP-13589: The latest patch looks good to me. Unfortunately, I can't fully verify it because of some test failures that are likely unrelated. I commented on HADOOP-13650 to try to figure out what to do about those test failures. > S3Guard: Allow execution of all S3A integration tests with S3Guard enabled. > --- > > Key: HADOOP-13589 > URL: https://issues.apache.org/jira/browse/HADOOP-13589 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Chris Nauroth >Assignee: Steve Loughran > Attachments: HADOOP-13589-HADOOP-13345-001.patch, > HADOOP-13589-HADOOP-13345-002.patch, HADOOP-13589-HADOOP-13345-002.patch, > HADOOP-13589-HADOOP-13345-004.patch, HADOOP-13589-HADOOP-13345-005.patch > > > With S3Guard enabled, S3A must continue to be functionally correct. The best > way to verify this is to execute our existing S3A integration tests in a mode > with S3A enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.
[
https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826630#comment-15826630
]
Chris Nauroth commented on HADOOP-13650:
I'm getting failures in {{TestS3GuardTool}}. Is anyone else seeing this?
{code}
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.427 sec <<<
FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.TestS3GuardTool
testInitDynamoDBMetadataStore(org.apache.hadoop.fs.s3a.s3guard.TestS3GuardTool)
Time elapsed: 1.377 sec <<< FAILURE!
java.lang.AssertionError: expected:<0> but was:<-1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.fs.s3a.s3guard.TestS3GuardTool.testInitDynamoDBMetadataStore(TestS3GuardTool.java:112)
{code}
{code}
Must specify -e ENDPOINT or fs.s3a.s3guard.ddb.endpoint in conf.
{code}
I had specified {{fs.s3a.endpoint}}, but not {{fs.s3a.s3guard.ddb.endpoint}},
so I tried adding {{fs.s3a.s3guard.ddb.endpoint}}, and then I got a different
error:
{code}
2017-01-17 10:59:59,696 [main] INFO s3guard.S3GuardTool
(S3GuardTool.java:initMetadataStore(148)) - create metadata store:
dynamodb://s3guard_test_init_ddb_table scheme: dynamodb
2017-01-17 11:00:00,092 [main] INFO json.JsonContent
(JsonContent.java:parseJsonContent(74)) - Unable to parse HTTP response content
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code
60)): expected a valid value (number, String, array, object, 'true', 'false' or
'null')
at [Source: [B@2b5825fa; line: 1, column: 2]
at
com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1586)
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:521)
at
com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:450)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2631)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:854)
at
com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:748)
at
com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3847)
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3792)
at
com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2355)
at
com.amazonaws.protocol.json.JsonContent.parseJsonContent(JsonContent.java:72)
at com.amazonaws.protocol.json.JsonContent.(JsonContent.java:64)
at
com.amazonaws.protocol.json.JsonContent.createJsonContent(JsonContent.java:54)
at
com.amazonaws.http.JsonErrorResponseHandler.handle(JsonErrorResponseHandler.java:61)
at
com.amazonaws.http.JsonErrorResponseHandler.handle(JsonErrorResponseHandler.java:33)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1495)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1167)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:948)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:661)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:635)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:618)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:586)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:573)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:445)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:1722)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1698)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:1096)
at
com.amazonaws.services.dynamodbv2.document.Table.describe(Table.java:130)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:572)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:270)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.initMetadataStore(S3GuardTool.java:171)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$InitMetadata.run(S3GuardTool.java:257)
at
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826622#comment-15826622
]
Sean Busbey commented on HADOOP-13989:
--
I'd confirm that a revert fixes things locally before going that route.
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
hadoop-client-runtime: Error creating shaded jar: duplicate entry:
META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.databind.Module
-> [Help 1]
{code}
this failure happened while building the binary jar, not while creating any
source jar. It could just be coincidence that this patch showed up at the same
time as something else that changed dependencies in a breaking way.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13956) Read ADLS credentials from Credential Provider
[ https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13956: Attachment: HADOOP-13956.003.patch Patch 003 * Fix checkstyle > Read ADLS credentials from Credential Provider > -- > > Key: HADOOP-13956 > URL: https://issues.apache.org/jira/browse/HADOOP-13956 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 3.0.0-alpha2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Critical > Attachments: HADOOP-13956.001.patch, HADOOP-13956.002.patch, > HADOOP-13956.003.patch > > > Read ADLS credentials using Hadoop CredentialProvider API. See > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Pallas reopened HADOOP-13989:
-
Assignee: (was: Joe Pallas)
The build failure that was reported appears during {{mvn deploy}}, but not
during {{mvn install}}. This is puzzling, since the change is to the shade
configuration, and shade is run during the packaging phase of the lifecycle. I
haven't been able to reproduce the problem in my build environment, either.
This seemingly trivial fix may actually require someone who is much more of a
Maven wizard than I am. ([~busbey]?)
For now it looks like the only sensible thing is to revert the commit,
[~ajisakaa]. Do you agree?
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[
https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826597#comment-15826597
]
Yongjun Zhang commented on HADOOP-13805:
Thanks [~tucu00].
I think you listed two options, one is to make this potentially incompatible
change; the other is to create a new UGI and obsolete the old/incorrect
implementation later on.
It may not be too bad to go with option one. Say, with option two, we may hit
the issue reported here. With option one, we need to watch out how things are
broken due to the incompatible change, and fix accordingly.
If we go with option one, if client code is broken, the client code need to be
changed to do the renewal. Would you please help putting together a recommended
change as part of the release notes of this jira?
If we go with option one, I'm +1 on Wei-Chiu's rev6 (I found that it may not be
easy to add the test you proposed as a unit test due to the run time) Would you
please also take a look at rev6?
Thanks.
> UGI.getCurrentUser() fails if user does not have a keytab associated
>
>
> Key: HADOOP-13805
> URL: https://issues.apache.org/jira/browse/HADOOP-13805
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2
>Reporter: Alejandro Abdelnur
>Assignee: Xiao Chen
> Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch,
> HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch,
> HADOOP-13805.05.patch
>
>
> HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the
> UGI is created from an existing Subject as in that case the keytab is not
> 'own' by UGI but by the creator of the Subject.
> In HADOOP-13558 we introduced a new private UGI constructor
> {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and
> we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}.
> The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created
> via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new
> UserGroupInformation(subject)}} which will delegate to
> {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and
> that will use externalKeyTab == *FALSE*.
> Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using
> a non-existing keytab if the TGT expired.
> This problem is experienced in {{KMSClientProvider}} when used by the HDFS
> filesystem client accessing an an encryption zone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13956) Read ADLS credentials from Credential Provider
[
https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826576#comment-15826576
]
Hadoop QA commented on HADOOP-13956:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 34s{color} | {color:orange} root: The patch generated 1 new + 0 unchanged -
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
59s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
48s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13956 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847866/HADOOP-13956.002.patch
|
| Optional Tests | asflicense mvnsite compile javac javadoc mvninstall
unit findbugs checkstyle |
| uname | Linux 74ff263cfa64 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 78b487b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11448/artifact/patchprocess/diff-checkstyle-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11448/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-tools/hadoop-azure-datalake U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11448/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826471#comment-15826471
]
Joe Pallas commented on HADOOP-13989:
-
I don't get it. Pre-commit build was fine, but the build failure certainly
seems to be connected to this change.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Assignee: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13955) Fix deprecation of HttpServer2 constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826406#comment-15826406
]
John Zhuge commented on HADOOP-13955:
-
Sure [~ajisakaa].
> Fix deprecation of HttpServer2 constants
>
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13956) Read ADLS credentials from Credential Provider
[ https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13956: Attachment: HADOOP-13956.002.patch Patch 002 * Add docs to CredentialProviderAPI.md and index.md * Fix checkstyle > Read ADLS credentials from Credential Provider > -- > > Key: HADOOP-13956 > URL: https://issues.apache.org/jira/browse/HADOOP-13956 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 3.0.0-alpha2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Critical > Attachments: HADOOP-13956.001.patch, HADOOP-13956.002.patch > > > Read ADLS credentials using Hadoop CredentialProvider API. See > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13836) Securing Hadoop RPC using SSL
[ https://issues.apache.org/jira/browse/HADOOP-13836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826330#comment-15826330 ] Arun Suresh commented on HADOOP-13836: -- [~drankye], [~daryn], [~steve_l], wondering if you folks had a chance to look at the attached design doc and perf numbers. If we are ok with the design, I can start with reviewing the patch. > Securing Hadoop RPC using SSL > - > > Key: HADOOP-13836 > URL: https://issues.apache.org/jira/browse/HADOOP-13836 > Project: Hadoop Common > Issue Type: New Feature > Components: ipc >Reporter: kartheek muthyala >Assignee: kartheek muthyala > Attachments: HADOOP-13836.patch, Secure IPC OSS Proposal-1.pdf, > SecureIPC Performance Analysis-OSS.pdf > > > Today, RPC connections in Hadoop are encrypted using Simple Authentication & > Security Layer (SASL), with the Kerberos ticket based authentication or > Digest-md5 checksum based authentication protocols. This proposal is about > enhancing this cipher suite with SSL/TLS based encryption and authentication. > SSL/TLS is a proposed Internet Engineering Task Force (IETF) standard, that > provides data security and integrity across two different end points in a > network. This protocol has made its way to a number of applications such as > web browsing, email, internet faxing, messaging, VOIP etc. And supporting > this cipher suite at the core of Hadoop would give a good synergy with the > applications on top and also bolster industry adoption of Hadoop. > The Server and Client code in Hadoop IPC should support the following modes > of communication > 1.Plain > 2. SASL encryption with an underlying authentication > 3. SSL based encryption and authentication (x509 certificate) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13496) Include file lengths in Mismatch in length error for distcp
[
https://issues.apache.org/jira/browse/HADOOP-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826165#comment-15826165
]
Ted Yu commented on HADOOP-13496:
-
The above snippet was not in my patch.
I checked
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
which doesn't have "ServiceFailedException(ex.getMessage())" either.
Please clarify.
> Include file lengths in Mismatch in length error for distcp
> ---
>
> Key: HADOOP-13496
> URL: https://issues.apache.org/jira/browse/HADOOP-13496
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: distcp
> Attachments: HADOOP-13496.v1.patch, HADOOP-13496.v1.patch
>
>
> Currently RetriableFileCopyCommand doesn't show the perceived lengths in
> Mismatch in length error:
> {code}
> 2016-08-12 10:23:14,231 ERROR [LocalJobRunner Map Task Executor #0]
> util.RetriableCommand(89): Failure in Retriable command: Copying
> hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260-
> c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> to hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/10.22.9.
>171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> java.io.IOException: Mismatch in length of
> source:hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260-c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> and
> target:hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/.distcp.tmp.attempt_local344329843_0006_m_00_0
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:193)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99)
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:281)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:253)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50)
> {code}
> It would be helpful to include what's the expected length and what's the real
> length.
> Thanks to [~yzhangal] for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13988) KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
[
https://issues.apache.org/jira/browse/HADOOP-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826110#comment-15826110
]
Greg Senia commented on HADOOP-13988:
-
Also in regards to test case let me know as this class doesn't have much around
test cases around it.
> KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
>
>
> Key: HADOOP-13988
> URL: https://issues.apache.org/jira/browse/HADOOP-13988
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, kms
>Affects Versions: 2.8.0, 2.7.3
> Environment: HDP 2.5.3.0
> WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes
>Reporter: Greg Senia
> Attachments: HADOOP-13988.patch
>
>
> After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider
> issues have not been resolved. We put a test build together and applied
> HADOOP-13558 and HADOOP-13749 these two fixes did still not solve the issue
> with requests coming from WebHDFS through to Knox to a TDE zone.
> So we added some debug to our build and determined effectively what is
> happening here is a double proxy situation which does not seem to work. So we
> propose the following fix in getActualUgi Method:
> {noformat}
> }
> // Use current user by default
> UserGroupInformation actualUgi = currentUgi;
> if (currentUgi.getRealUser() != null) {
>// Use real user for proxy user
>if (LOG.isDebugEnabled()) {
> LOG.debug("using RealUser for proxyUser);
> }
>actualUgi = currentUgi.getRealUser();
>if (getDoAsUser() != null) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("doAsUser exists");
> LOG.debug("currentUGI realUser shortName: {}",
> currentUgi.getRealUser().getShortUserName());
> LOG.debug("processUGI loginUser shortName: {}",
> UserGroupInformation.getLoginUser().getShortUserName());
> }
> if (currentUgi.getRealUser().getShortUserName() !=
> UserGroupInformation.getLoginUser().getShortUserName()) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("currentUGI.realUser does not match
> UGI.processUser);
> }
> actualUgi = UserGroupInformation.getLoginUser();
> if (LOG.isDebugEnabled()) {
> LOG.debug("LoginUser for Proxy: {}",
> actualUgi.getLoginUser());
> }
> }
>}
>
> } else if (!currentUgiContainsKmsDt() &&
> !currentUgi.hasKerberosCredentials()) {
>// Use login user for user that does not have either
>// Kerberos credential or KMS delegation token for KMS operations
>if (LOG.isDebugEnabled()) {
> LOG.debug("using loginUser no KMS Delegation Token no Kerberos
> Credentials");
> }
>actualUgi = currentUgi.getLoginUser();
> }
> return actualUgi;
>}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13496) Include file lengths in Mismatch in length error for distcp
[
https://issues.apache.org/jira/browse/HADOOP-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826098#comment-15826098
]
Steve Loughran commented on HADOOP-13496:
-
oh, almost good to go. One change
{code}
} catch (Exception ex) {
- throw new ServiceFailedException(ex.getMessage());
+ throw new ServiceFailedException("RefreshAll operation failed", ex);
}
{code}
can you make it ("RefreshAll operation failed: " + ex, ex). Stops the
underlying cause being lost in the traces; handles exceptions with null
getMessage() value, e.g. NPE
> Include file lengths in Mismatch in length error for distcp
> ---
>
> Key: HADOOP-13496
> URL: https://issues.apache.org/jira/browse/HADOOP-13496
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: distcp
> Attachments: HADOOP-13496.v1.patch, HADOOP-13496.v1.patch
>
>
> Currently RetriableFileCopyCommand doesn't show the perceived lengths in
> Mismatch in length error:
> {code}
> 2016-08-12 10:23:14,231 ERROR [LocalJobRunner Map Task Executor #0]
> util.RetriableCommand(89): Failure in Retriable command: Copying
> hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260-
> c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> to hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/10.22.9.
>171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> java.io.IOException: Mismatch in length of
> source:hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260-c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182
> and
> target:hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/.distcp.tmp.attempt_local344329843_0006_m_00_0
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:193)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99)
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:281)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:253)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50)
> {code}
> It would be helpful to include what's the expected length and what's the real
> length.
> Thanks to [~yzhangal] for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[
https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825961#comment-15825961
]
Hadoop QA commented on HADOOP-13119:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 29s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 6 unchanged - 3 fixed = 8 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
39s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13119 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847779/HADOOP-13119.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 79316ff6da37 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7ee8be1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11447/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11447/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11447/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Web UI error accessing links which need authorization when Kerberos
> ---
>
> Key: HADOOP-13119
> URL: https://issues.apache.org/jira/browse/HADOOP-13119
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.8.0, 2.7.4
>Reporter:
[jira] [Commented] (HADOOP-13453) S3Guard: Instrument new functionality with Hadoop metrics.
[
https://issues.apache.org/jira/browse/HADOOP-13453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825927#comment-15825927
]
Steve Loughran commented on HADOOP-13453:
-
They're going to have to go into that file because those are the metrics
published by the S3A filesystem when deployed, returned by S3AStorageStatistics
in a call to {{S3AFileSystem.getStorageStatistics(), and printed in
{{S3AFileSystem.toString()}}. We could choose whether to add the specific
metrics to every S3a FS instance; that's something to consider. Listing the
values but returning 0 for all gauges and counters is the most consistent.
Don't worry about the class length: if you look at it in detail, there's two
nested classes + support methods explicitly for output/input streams...you
don't need to go there. The rest of the code is fairly simple
# add new values to org.apache.hadoop.fs.s3a.Statistic; prefix {{s3guard_}}
# In {{S3AInstrumentation}}, add counters to the array {{COUNTERS_TO_CREATE}};
gauges to {{GAUGES_TO_CREATE}}
# Pass in an instance of the instrumentation down to S3Guard
# have the code call incrementCounter and increment/decrementGauge as
appropriate
# I'd like a simple counter of {{s3guard_enabled}} and
{{s3guard_authoritative}}, which will be 0 when there's no s3guard running, 1
when the respective booleans are up. Why? Remote visibility
You make a good point, "where are the tests?". The answer is: the metrics can
be used to test the internal state of the S3 classes, therefore become
implicitly tested there.
Take a look at {{ITestS3ADirectoryPerformance}} for a key example of this: our
test cases use the counters of the various HTTP operations as the means to
verify that API calls work as expected. (note that s3guard, by reducing these,
has complicated the tests)
That is, you verify the counters work by asserting that they change as you make
operations to the DFS. see:
http://steveloughran.blogspot.co.uk/2016/04/distributed-testing-making-use-of.html
for more of my thinking here
bq. Sorry for the basic question, i'm really new for work on Hadoop code base.
happy to explain my reasoning. We've all started off staring at a vast amount
of code that we don't understand; there are still big bits of Hadoop that I
don't go near.
> S3Guard: Instrument new functionality with Hadoop metrics.
> --
>
> Key: HADOOP-13453
> URL: https://issues.apache.org/jira/browse/HADOOP-13453
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Chris Nauroth
>Assignee: Ai Deng
>
> Provide Hadoop metrics showing operational details of the S3Guard
> implementation.
> The metrics will be implemented in this ticket:
> ● S3GuardRechecksNthPercentileLatency (MutableQuantiles) Percentile time
> spent
> in rechecks attempting to achieve consistency. Repeated for multiple
> percentile values
> of N. This metric is an indicator of the additional latency cost of running
> S3A with
> S3Guard.
> ● S3GuardRechecksNumOps (MutableQuantiles) Number of times a consistency
> recheck was required while attempting to achieve consistency.
> ● S3GuardStoreNthPercentileLatency (MutableQuantiles) Percentile time
> spent in
> operations against the consistent store, including both write operations
> during file system
> mutations and read operations during file system consistency checks. Repeated
> for
> multiple percentile values of N. This metric is an indicator of latency to
> the consistent
> store implementation.
> ● S3GuardConsistencyStoreNumOps (MutableQuantiles) Number of operations
> against the consistent store, including both write operations during file
> system mutations
> and read operations during file system consistency checks.
> ● S3GuardConsistencyStoreFailures (MutableCounterLong) Number of failures
> during operations against the consistent store implementation.
> ● S3GuardConsistencyStoreTimeouts (MutableCounterLong) Number of timeouts
> during operations against the consistent store implementation.
> ● S3GuardInconsistencies (MutableCounterLong) C ount of times S3Guard
> failed to
> achieve consistency, even after exhausting all rechecks. A high count may
> indicate
> unexpected outofband modification of the S3 bucket contents, such as by an
> external
> tool that does not make corresponding updates to the consistent store.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[ https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HADOOP-13119: Attachment: HADOOP-13119.004.patch upload v4 patch to address failure test case. > Web UI error accessing links which need authorization when Kerberos > --- > > Key: HADOOP-13119 > URL: https://issues.apache.org/jira/browse/HADOOP-13119 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.4 >Reporter: Jeffrey E Rodriguez >Assignee: Yuanbo Liu > Labels: security > Attachments: HADOOP-13119.001.patch, HADOOP-13119.002.patch, > HADOOP-13119.003.patch, HADOOP-13119.004.patch, screenshot-1.png > > > User Hadoop on secure mode. > login as kdc user, kinit. > start firefox and enable Kerberos > access http://localhost:50070/logs/ > Get 403 authorization errors. > only hdfs user could access logs. > Would expect as a user to be able to web interface logs link. > Same results if using curl: > curl -v --negotiate -u tester: http://localhost:50070/logs/ > HTTP/1.1 403 User tester is unauthorized to access this page. > so: > 1. either don't show links if hdfs user is able to access. > 2. provide mechanism to add users to web application realm. > 3. note that we are pass authentication so the issue is authorization to > /logs/ > suspect that /logs/ path is secure in webdescriptor so suspect users by > default don't have access to secure paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13823) s3a rename: fail if dest file exists
[
https://issues.apache.org/jira/browse/HADOOP-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825850#comment-15825850
]
Steve Loughran commented on HADOOP-13823:
-
I'm throwing an exception, from innerRename((): but I'm catching it in
{{rename()}}. That {{RenameException()}} propagates the text and return code.
It's intended to be used when we add a public rename/3, in which case the
rename will forward the exception all the way up.
Are you seeing, on a branch which has this patch in, an exception propagating?
Or is your statement that "an exception is being raised" based on code review
of the patch?
> s3a rename: fail if dest file exists
>
>
> Key: HADOOP-13823
> URL: https://issues.apache.org/jira/browse/HADOOP-13823
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.7.3
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13813-branch-2-001.patch,
> HADOOP-13823-branch-2-002.patch
>
>
> HIVE-15199 shows that s3a allows rename onto an existing file, which is
> something HDFS, azure and s3n do not permit (though file:// does). This is
> breaking bits of Hive, is an inconsistency with HDFS and a regression
> compared to s3n semantics.
> I propose: rejecting the rename on a file -> file rename if the destination
> exists (easy) and changing the s3a.xml contract file to declare the behavior
> change; this is needed for
> {{AbstractContractRenameTest.testRenameFileOverExistingFile}} to handle the
> changed semantics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[
https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825848#comment-15825848
]
Hadoop QA commented on HADOOP-13119:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 40s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestAuthenticationFilter |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13119 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12847767/HADOOP-13119.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 36d0f47d21e7 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7ee8be1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11446/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11446/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11446/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11446/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Web UI error accessing links which need authorization when Kerberos
> ---
>
>
[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab
[
https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825816#comment-15825816
]
Brahma Reddy Battula commented on HADOOP-13433:
---
[~Apache9] thanks for reporting this issue and working on this..
we've got same issue from production cluster..your approach looks good to
me..can you upload branch-2.7 patch also..?
pinging [~ste...@apache.org] and [~jojochuang]
> Race in UGI.reloginFromKeytab
> -
>
> Key: HADOOP-13433
> URL: https://issues.apache.org/jira/browse/HADOOP-13433
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2, 2.6.6
>
> Attachments: HADOOP-13433.patch, HADOOP-13433-v1.patch,
> HADOOP-13433-v2.patch, HADOOP-13433-v4.patch, HBASE-13433-testcase-v3.patch
>
>
> This is a problem that has troubled us for several years. For our HBase
> cluster, sometimes the RS will be stuck due to
> {noformat}
> 2016-06-20,03:44:12,936 INFO org.apache.hadoop.ipc.SecureClient: Exception
> encountered while connecting to the server :
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: The ticket
> isn't for us (35) - BAD TGS SERVER NAME)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:194)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:140)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:187)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$700(SecureClient.java:95)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:325)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1781)
> at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
> at org.apache.hadoop.hbase.security.User.call(User.java:607)
> at org.apache.hadoop.hbase.security.User.access$700(User.java:51)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:461)
> at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:321)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1164)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1004)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:107)
> at $Proxy24.replicateLogEntries(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:962)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.runLoop(ReplicationSource.java:466)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:515)
> Caused by: GSSException: No valid credentials provided (Mechanism level: The
> ticket isn't for us (35) - BAD TGS SERVER NAME)
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:663)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:180)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:175)
> ... 23 more
> Caused by: KrbException: The ticket isn't for us (35) - BAD TGS SERVER NAME
> at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:64)
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:185)
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:294)
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:106)
> at
> sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:557)
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:594)
> ... 26 more
> Caused by: KrbException: Identifier doesn't match expected value (906)
> at
[jira] [Updated] (HADOOP-13119) Web UI error accessing links which need authorization when Kerberos
[ https://issues.apache.org/jira/browse/HADOOP-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HADOOP-13119: Attachment: HADOOP-13119.003.patch upload v3 patch for this JIRA. > Web UI error accessing links which need authorization when Kerberos > --- > > Key: HADOOP-13119 > URL: https://issues.apache.org/jira/browse/HADOOP-13119 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.4 >Reporter: Jeffrey E Rodriguez >Assignee: Yuanbo Liu > Labels: security > Attachments: HADOOP-13119.001.patch, HADOOP-13119.002.patch, > HADOOP-13119.003.patch, screenshot-1.png > > > User Hadoop on secure mode. > login as kdc user, kinit. > start firefox and enable Kerberos > access http://localhost:50070/logs/ > Get 403 authorization errors. > only hdfs user could access logs. > Would expect as a user to be able to web interface logs link. > Same results if using curl: > curl -v --negotiate -u tester: http://localhost:50070/logs/ > HTTP/1.1 403 User tester is unauthorized to access this page. > so: > 1. either don't show links if hdfs user is able to access. > 2. provide mechanism to add users to web application realm. > 3. note that we are pass authentication so the issue is authorization to > /logs/ > suspect that /logs/ path is secure in webdescriptor so suspect users by > default don't have access to secure paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13915) [JDK9] Compilation failure in hadoop-auth-examples module
[
https://issues.apache.org/jira/browse/HADOOP-13915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825737#comment-15825737
]
Akira Ajisaka commented on HADOOP-13915:
There is a workaround from
https://issues.apache.org/jira/browse/MWAR-397?focusedCommentId=15818678=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15818678
:
{code}
$ MAVEN_OPTS="--add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens
java.base/java.util=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED
--add-opens java.desktop/java.awt.font=ALL-UNNAMED" mvn install -DskipTests
{code}
> [JDK9] Compilation failure in hadoop-auth-examples module
> -
>
> Key: HADOOP-13915
> URL: https://issues.apache.org/jira/browse/HADOOP-13915
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: OpenJDK 9-ea+149
>Reporter: Akira Ajisaka
>
> {{mvn install -DskipTests}} fails.
> {noformat}
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-war-plugin:2.4:war (default-war) on project
> hadoop-auth-examples: Execution default-war of goal
> org.apache.maven.plugins:maven-war-plugin:2.4:war failed: Unable to load the
> mojo 'war' in the plugin 'org.apache.maven.plugins:maven-war-plugin:2.4' due
> to an API incompatibility:
> org.codehaus.plexus.component.repository.exception.ComponentLookupException:
> null
> [ERROR] -
> [ERROR] realm =plugin>org.apache.maven.plugins:maven-war-plugin:2.4
> [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
> [ERROR] urls[0] =
> file:/home/centos/.m2/repository/org/apache/maven/plugins/maven-war-plugin/2.4/maven-war-plugin-2.4.jar
> [ERROR] urls[1] =
> file:/home/centos/.m2/repository/org/apache/maven/reporting/maven-reporting-api/2.0.6/maven-reporting-api-2.0.6.jar
> [ERROR] urls[2] =
> file:/home/centos/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.0-alpha-7/doxia-sink-api-1.0-alpha-7.jar
> [ERROR] urls[3] =
> file:/home/centos/.m2/repository/commons-cli/commons-cli/1.0/commons-cli-1.0.jar
> [ERROR] urls[4] =
> file:/home/centos/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
> [ERROR] urls[5] =
> file:/home/centos/.m2/repository/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.jar
> [ERROR] urls[6] =
> file:/home/centos/.m2/repository/org/codehaus/plexus/plexus-io/2.0.7/plexus-io-2.0.7.jar
> [ERROR] urls[7] =
> file:/home/centos/.m2/repository/commons-io/commons-io/2.2/commons-io-2.2.jar
> [ERROR] urls[8] =
> file:/home/centos/.m2/repository/org/codehaus/plexus/plexus-archiver/2.4.1/plexus-archiver-2.4.1.jar
> [ERROR] urls[9] =
> file:/home/centos/.m2/repository/org/apache/commons/commons-compress/1.5/commons-compress-1.5.jar
> [ERROR] urls[10] =
> file:/home/centos/.m2/repository/org/tukaani/xz/1.2/xz-1.2.jar
> [ERROR] urls[11] =
> file:/home/centos/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.15/plexus-interpolation-1.15.jar
> [ERROR] urls[12] =
> file:/home/centos/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
> [ERROR] urls[13] =
> file:/home/centos/.m2/repository/com/thoughtworks/xstream/xstream/1.4.2/xstream-1.4.2.jar
> [ERROR] urls[14] =
> file:/home/centos/.m2/repository/xmlpull/xmlpull/1.1.3.1/xmlpull-1.1.3.1.jar
> [ERROR] urls[15] =
> file:/home/centos/.m2/repository/xpp3/xpp3_min/1.1.4c/xpp3_min-1.1.4c.jar
> [ERROR] urls[16] =
> file:/home/centos/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.10/plexus-utils-3.0.10.jar
> [ERROR] urls[17] =
> file:/home/centos/.m2/repository/org/apache/maven/shared/maven-filtering/1.1/maven-filtering-1.1.jar
> [ERROR] urls[18] =
> file:/home/centos/.m2/repository/org/sonatype/plexus/plexus-build-api/0.0.4/plexus-build-api-0.0.4.jar
> [ERROR] Number of foreign imports: 1
> [ERROR] import: Entry[import from realm
> ClassRealm[project>org.apache.hadoop:hadoop-main:3.0.0-alpha2-SNAPSHOT,
> parent: ClassRealm[maven.api, parent: null]]]
> [ERROR]
> [ERROR] -:
> ExceptionInInitializerError: Unable to make field private final
> java.util.Comparator java.util.TreeMap.comparator accessible: module
> java.base does not "opens java.util" to unnamed module @8ad2460
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13965) Groups should be consistent in using default group mapping class
[
https://issues.apache.org/jira/browse/HADOOP-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825719#comment-15825719
]
Akira Ajisaka commented on HADOOP-13965:
+1, thanks [~linyiqun].
> Groups should be consistent in using default group mapping class
>
>
> Key: HADOOP-13965
> URL: https://issues.apache.org/jira/browse/HADOOP-13965
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 3.0.0-alpha2
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>Priority: Minor
> Attachments: HADOOP-13965.001.patch
>
>
> {code:title=Groups.java}
> public Groups(Configuration conf, final Timer timer) {
>impl =
> ReflectionUtils.newInstance(
>
> conf.getClass(CommonConfigurationKeys.HADOOP_SECURITY_GROUP_MAPPING,
> ShellBasedUnixGroupsMapping.class,
> GroupMappingServiceProvider.class),
> conf);
> ...
> }
> {code}
> The default value of setting {{hadoop.security.group.mapping}} is different
> in code and config file. In file {{core-default.xml}}, it uses the class
> {{JniBasedUnixGroupsMappingWithFallback}} and this should be the true value.
> {code}
>
> hadoop.security.group.mapping
>
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
>
> Class for user to group mapping (get groups for a given user) for ACL.
> The default implementation,
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback,
> will determine if the Java Native Interface (JNI) is available.
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825680#comment-15825680
]
Hudson commented on HADOOP-13989:
-
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11126 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11126/])
HADOOP-13989. Fix typo in hadoop-client shade configuration. Contributed
(aajisaka: rev 7e81063c13a636c551d3281966a24ccfcfd46793)
* (edit) hadoop-client-modules/hadoop-client-minicluster/pom.xml
* (edit) hadoop-client-modules/hadoop-client-runtime/pom.xml
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Assignee: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13989:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this to trunk. Thanks [~jpallas] for the contribution!
FYI: The Fix Version/s is set by a committer when the patch is committed, so
I'll appreciate if you don't set the fix version in the next time.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Assignee: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13989:
---
Assignee: Joe Pallas
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Assignee: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13989) Fix typo in hadoop-client shade configuration
[
https://issues.apache.org/jira/browse/HADOOP-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825625#comment-15825625
]
Akira Ajisaka commented on HADOOP-13989:
+1, checking this in.
> Fix typo in hadoop-client shade configuration
> -
>
> Key: HADOOP-13989
> URL: https://issues.apache.org/jira/browse/HADOOP-13989
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha2
>Reporter: Joe Pallas
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13989.001.patch
>
>
> The pom files for hadoop-client-minicluster and hadoop-client-runtime have a
> typo in the configuration of the shade module. They say
> {{}} instead of {{}}. (This was noticed
> by IntelliJ, but not by maven.)
> Shade plugin doc is at
> [http://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createSourcesJar].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13955) Fix deprecation of HttpServer2 constants
[
https://issues.apache.org/jira/browse/HADOOP-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825619#comment-15825619
]
Akira Ajisaka commented on HADOOP-13955:
Mostly looks good to me. Can we remove {{SSL_ENABLED_PROTOCOLS}} and
{{DEFAULT_SSL_ENABLED_PROTOCOLS}} as well?
> Fix deprecation of HttpServer2 constants
>
>
> Key: HADOOP-13955
> URL: https://issues.apache.org/jira/browse/HADOOP-13955
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13955.001.patch
>
>
> A follow-up to HADOOP-13597. Fix these javac warnings about deprecated
> constants:
> {noformat}
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java:[139,20]
> [deprecation] DEFAULT_SSL_REQUIRE_CLIENT_CERT in SSLFactory has been
> deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java:[148,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java:[85,27]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> [WARNING]
> /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java:[99,40]
> [deprecation] HTTP_MAX_THREADS in HttpServer2 has been deprecated
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
