[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[ https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758379#comment-16758379 ] Hudson commented on HADOOP-16079: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15866 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15866/]) HADOOP-16079. Token.toString faulting if any token listed can't load. (stevel: rev 7f46d13dac8cf85b094f41b3dd68e02c69e5afbc) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java > Token.toString faulting if any token listed can't load. > --- > > Key: HADOOP-16079 > URL: https://issues.apache.org/jira/browse/HADOOP-16079 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.1.2, 3.2.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Fix For: 3.2.1, 3.1.3 > > Attachments: HADOOP-16079-001.patch > > > The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails > if any token in the service lists isn't known. > If any JAR lists a class in > META-INF/services/org.apache.hadoop.security.token.TokenIdentifier which > refers to a class which for any reason cannot load (e.g. it depends on a > class which is in a JAR that is not on the classpath), then the enumeration > of all tokens and their supported kinds will fail. This can surface in > Token.toString(), but it will also break the RM token renewal thread. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[ https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758367#comment-16758367 ] Steve Loughran commented on HADOOP-16079: - thanks; committed to all branches that had its predecessor patch in. Anyone who is playing games with object stores and DTs should fix this > Token.toString faulting if any token listed can't load. > --- > > Key: HADOOP-16079 > URL: https://issues.apache.org/jira/browse/HADOOP-16079 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.1.2, 3.2.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Fix For: 3.2.1, 3.1.3 > > Attachments: HADOOP-16079-001.patch > > > The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails > if any token in the service lists isn't known. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[ https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757562#comment-16757562 ] Larry McCay commented on HADOOP-16079: -- [~ste...@apache.org] - this seems an appropriate change to me. +1 > Token.toString faulting if any token listed can't load. > --- > > Key: HADOOP-16079 > URL: https://issues.apache.org/jira/browse/HADOOP-16079 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 3.1.2, 3.2.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Attachments: HADOOP-16079-001.patch > > > The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails > if any token in the service lists isn't known. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[
https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755021#comment-16755021
]
Hadoop QA commented on HADOOP-16079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
16s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16079 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956699/HADOOP-16079-001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 95093fbd210f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 5d578d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15863/testReport/ |
| Max. process+thread count | 1415 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15863/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Token.toString faulting if
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[ https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754927#comment-16754927 ] Steve Loughran commented on HADOOP-16079: - patch 001 adds the LinkageError. Does not add any more tests I'm afraid; will do more manual verification > Token.toString faulting if any token listed can't load. > --- > > Key: HADOOP-16079 > URL: https://issues.apache.org/jira/browse/HADOOP-16079 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 3.1.2, 3.2.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Attachments: HADOOP-16079-001.patch > > > The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails > if any token in the service lists isn't known. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[ https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754150#comment-16754150 ] Steve Loughran commented on HADOOP-16079: - Issue here is that HADOOP-15808 catches ServiceConfigurationError, but that turns out not to include CNFErrors, which are subsets of LinkageErrors. Two options # catch all Error # add LinkageError as a catchable I'm gong for #2 as its least risk of dropping something of a real problem. > Token.toString faulting if any token listed can't load. > --- > > Key: HADOOP-16079 > URL: https://issues.apache.org/jira/browse/HADOOP-16079 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 3.1.2, 3.2.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails > if any token in the service lists isn't known. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16079) Token.toString faulting if any token listed can't load.
[
https://issues.apache.org/jira/browse/HADOOP-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754148#comment-16754148
]
Steve Loughran commented on HADOOP-16079:
-
After creating a token of a known type, if there's a JAR on the CP registering
a different token with some transitive dependency, the toString() call on the
issued token fails
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/example/auth/TokenIdentifier
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at
org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:158)
at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:185)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.printTokensToString(DelegationTokenFetcher.java:224)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.printTokens(DelegationTokenFetcher.java:244)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:139)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:135)
at
org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:72)
Caused by: java.lang.ClassNotFoundException:
com.google.cloud.hadoop.fs.gcs.auth.AbstractGCPTokenIdentifier
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 27 more
{code}
> Token.toString faulting if any token listed can't load.
> ---
>
> Key: HADOOP-16079
> URL: https://issues.apache.org/jira/browse/HADOOP-16079
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 3.1.2, 3.2.1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> The patch in HADOOP-15808 turns out not to be enough; Token.toString() fails
> if any token in the service lists isn't known.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
