[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039779#comment-14039779 ] Hudson commented on HDFS-6222: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #590 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/590/]) HDFS-6222. Remove background token renewer from webhdfs. Contributed by Rushabh Shah and Daryn Sharp. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1604300) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/SWebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Fix For: 3.0.0, 2.5.0 Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2-v3.patch, HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039839#comment-14039839 ] Hudson commented on HDFS-6222: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1781 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1781/]) HDFS-6222. Remove background token renewer from webhdfs. Contributed by Rushabh Shah and Daryn Sharp. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1604300) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/SWebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Fix For: 3.0.0, 2.5.0 Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2-v3.patch, HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039863#comment-14039863 ] Hudson commented on HDFS-6222: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1808 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1808/]) HDFS-6222. Remove background token renewer from webhdfs. Contributed by Rushabh Shah and Daryn Sharp. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1604300) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/SWebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Fix For: 3.0.0, 2.5.0 Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2-v3.patch, HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039273#comment-14039273
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651716/HDFS-6222.branch-2-v3.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7191//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2-v2.patch,
HDFS-6222.branch-2-v3.patch, HDFS-6222.branch-2.patch,
HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch,
HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039624#comment-14039624 ] Hudson commented on HDFS-6222: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5747 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5747/]) HDFS-6222. Remove background token renewer from webhdfs. Contributed by Rushabh Shah and Daryn Sharp. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1604300) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/SWebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Fix For: 3.0.0, 2.5.0 Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2-v3.patch, HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039047#comment-14039047
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651662/HDFS-6222.trunk-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7189//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7189//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch,
HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039068#comment-14039068 ] Rushabh S Shah commented on HDFS-6222: -- The failed test runs fine on my machine. Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v3.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038014#comment-14038014
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651519/HDFS-6222.branch-2-v2.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7180//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk.patch,
HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038249#comment-14038249
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651547/HDFS-6222.trunk-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7182//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/7182//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7182//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2-v2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.branch-2.patch, HDFS-6222.trunk-v2.patch, HDFS-6222.trunk-v2.patch,
HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028002#comment-14028002
]
Chris Nauroth commented on HDFS-6222:
-
Hi, [~daryn]. Can you describe what kinds of problems you've seen for
long-running daemons? Is there a race condition where the renewer thread
might not do the renewal ahead of the client operation that needs it?
Something else?
A couple of questions/observations:
# I have a question on this logic:
{code}
} else {
token = getDelegationToken(null);
if (token != null) {
LOG.debug(Fetched new token: + token);
} else { // security is disabled
canRefreshDelegationToken = false;
}
}
{code}
In the second else block, I don't see how we can reach this code path when
security is disabled, because it's already guarded by an earlier check of
{{canRefreshDelegationToken}}, which is initialized from
{{UserGroupInformation#isSecurityEnabled}}. Also, I seem to recall you
mentioned to me offline seeing certain failures where a malformed response from
the server can cause the JSON parser to return a null object. If that happens,
then this logic would turn off future attempts to refresh. Would that
effectively leave the file system instance in an unusable state, with the only
workaround being to discard the instance and get a new one?
# It's unfortunate that we have to parse the exception strings, but I don't see
an alternative. Do you know if these exception messages are consistent across
prior versions? Is there any chance that a 2.5.0 {{WebHdfsFileSystem}}
instance would be incompatible with servers running earlier versions back to
2.2.0?
# The latency of token renewal is now synchronous with the client operation.
It's probably no big deal if your internal testing on secure clusters didn't
show a problem.
# Unless I'm misreading, it seems like much of {{TokenAspect}} has become dead
code. Is there a need for further clean-up?
# Thanks for adding those tests. They're very thorough. Do you think we need
to add assertions that we're getting the correct token kind? Is it also
worthwhile to add some tests through swebhdfs too, and assert that they get
{{SWebHdfsFileSystem#TOKEN_KIND}}?
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028080#comment-14028080 ] Daryn Sharp commented on HDFS-6222: --- Good questions. This design is the result of problems encountered by converting a mission critical production system to use webhdfs. We've been internally running in production for months with this change on 0.23, and a sandbox 2.x grid. A few of the issues: The renewer is hardcoded to assume 24h, which isn't a guarantee by any means. The filesystem can go dead for up to a day. Decreasing the token renewal on our QA clusters to 30s to stress token handling obviously didn't work either... We've also encountered class loader leaks. Filesystems would become unusable if the token expired, erroneously cancelled, or transient renewal failures such as during a NN restart. # A secure client is supposed to be able to talk to an insecure server which is why earlier logic had this same behavior. Regarding malformed responses, NPEs used to be generated, not null returns. My earlier work trapped and converted the NPEs, which in this case will trigger the retry loop. Unlike the current implementation, the fs will attempt to re-acquire a token even after one operation fails which prevents the fs from becoming unusable - # Yes, very unfortunate, but I only did it for backwards compatibility with NNs, and also cross-compatibility with DNs that don't munge the token exception. I checked earlier versions and it appears to have always been this way. # True. We've become very performance conscience, but token renewal is infrequent if ever required by non-daemons so I consider the tiny latency worth the robustness. # TokenAspect is still used by hftp or I would have happily removed it... # I think this was covered by other tests. I'll double check and add if necessary. I'm not sure how to test swebhdfs since it requires extra configuration and ssl certs to function... Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028117#comment-14028117
]
Chris Nauroth commented on HDFS-6222:
-
Thanks for the response. This all makes sense.
bq. A secure client is supposed to be able to talk to an insecure server...
Now I get it. Would you mind changing that comment to security is disabled on
server to make this clearer?
Thanks for checking earlier versions for the compatibility concern.
bq. TokenAspect is still used by hftp...
I forgot about this. I had been looking at trunk, where {{HftpFileSystem}} is
gone.
bq. I'm not sure how to test swebhdfs...
There is an example in {{TestHttpsFileSystem}}, and there are some helper
methods that take care of bootstrapping configuration, including SSL certs.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026441#comment-14026441 ] Daryn Sharp commented on HDFS-6222: --- The test failure is unrelated. Remove background token renewer from webhdfs Key: HDFS-6222 URL: https://issues.apache.org/jira/browse/HDFS-6222 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch, HDFS-6222.trunk.patch, HDFS-6222.trunk.patch The background token renewer is a source of problems for long-running daemons. Webhdfs should lazy fetch a new token when it receives an InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014058#comment-14014058
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647613/HDFS-6222.trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7014//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7014//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch,
HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6222) Remove background token renewer from webhdfs
[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010007#comment-14010007
]
Hadoop QA commented on HDFS-6222:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646923/HDFS-6222.trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6980//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6980//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6980//console
This message is automatically generated.
Remove background token renewer from webhdfs
Key: HDFS-6222
URL: https://issues.apache.org/jira/browse/HDFS-6222
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: HDFS-6222.branch-2.patch, HDFS-6222.trunk.patch
The background token renewer is a source of problems for long-running
daemons. Webhdfs should lazy fetch a new token when it receives an
InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
