[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282179#comment-15282179
]
Sangjin Lee commented on HADOOP-9317:
-
There has been no movement on this for a while. We'll need to move it out of
scope for 2.8.0 soon. Let me know if you disagree.
> User cannot specify a kerberos keytab for commands
> --
>
> Key: HADOOP-9317
> URL: https://issues.apache.org/jira/browse/HADOOP-9317
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HADOOP-9317.branch-23.patch,
> HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
> HADOOP-9317.patch, HADOOP-9317.patch
>
>
> {{UserGroupInformation}} only allows kerberos users to be logged in via the
> ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
> used, but it's only exposed programatically. This forces keytab-based users
> running hadoop commands to periodically issue a kinit from the keytab. A
> race condition exists during the kinit when the ticket cache is deleted and
> re-created. Hadoop commands will fail when the ticket cache does not
> momentarily exist.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13965823#comment-13965823
]
Mit Desai commented on HADOOP-9317:
---
Talked to Daryn offline. Removing the target version for 0.23
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch, HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13965901#comment-13965901
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584166/HADOOP-9317.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3781//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch, HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13837351#comment-13837351
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584166/HADOOP-9317.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3328//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/3328//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch, HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663430#comment-13663430
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584166/HADOOP-9317.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2554//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2554//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch, HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13583223#comment-13583223
]
Daryn Sharp commented on HADOOP-9317:
-
As background for the motivation: In some production environments we have
hundreds of job launches every few mins. The launches may perform dozens of
hadoop commands before actually submitting the job. We are seeing a huge
failure rate, necessitating unnecessary retry loops, because of this kinit
issue whether it be explicitly by the user or implicitly by hadoop's background
renewal. As the job load is increased, we are seeing more and more failures
that are breaking through the retry loop.
@Aaron:
I have not tested with IBM's java. If you have convenient access, would you be
able to test it for me? On the bright side, even if it's broken, it won't be a
problem unless the user sets the KRB5KEYTAB env to activate the new code. If
it is broken, could I file another jira to make it work for IBM's java?
@Allen:
Yes, kinit will regardless of -R, unlink the file, open/write the principal,
open/write the TGT. So your suggestion won't work because concurrent launches
issuing the kinit will still result in the race condition where one process may
be issuing the kinit while another is trying to run hadoop commands. Obtaining
a new TGT for every launch would place tremendously more pressure on the KDC,
thus why this change tries the ticket cache, falls back to the keytab, and
updates the ticket cache if it had to fallback.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582469#comment-13582469
]
Aaron T. Myers commented on HADOOP-9317:
Hey Daryn, have you tested this with IBM Java? I don't think it will quite
work, since it could result in both useDefaultCcache and useKeytab being set,
which according to [IBM's JGSS
documentation|http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.security.component.doc%2Fsecurity-component%2FjgssDocs%2Fjaas_login_user.html]
are incompatible when set in the same JAAS config.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582660#comment-13582660
]
Allen Wittenauer commented on HADOOP-9317:
--
Maybe I'm missing something, but I don't understand why just using a different
KRB5CCNAME for every invocation doesn't fix this. i.e., program flow should be:
{code}
export KRB5CCNAME=/tmp/mycoolcache.$$
kinit -k -t keytab identity
hadoop jar blah
rm /tmp/mycookcache.$$
{code}
You could even be smarter and check the creation timestamp vs. expiry.
Additionally, I'm not sure, but I don't think kinit -R removes the file. (But
I could be wrong.)
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581404#comment-13581404
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569954/HADOOP-9317.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2201//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch, HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581469#comment-13581469
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569963/HADOOP-9317.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2202//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2202//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2202//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch, HADOOP-9317.patch,
HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9317) User cannot specify a kerberos keytab for commands
[
https://issues.apache.org/jira/browse/HADOOP-9317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581502#comment-13581502
]
Hadoop QA commented on HADOOP-9317:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569971/HADOOP-9317.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2204//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2204//console
This message is automatically generated.
User cannot specify a kerberos keytab for commands
--
Key: HADOOP-9317
URL: https://issues.apache.org/jira/browse/HADOOP-9317
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-9317.branch-23.patch,
HADOOP-9317.branch-23.patch, HADOOP-9317.patch, HADOOP-9317.patch,
HADOOP-9317.patch
{{UserGroupInformation}} only allows kerberos users to be logged in via the
ticket cache when running hadoop commands. {{UGI}} allows a keytab to be
used, but it's only exposed programatically. This forces keytab-based users
running hadoop commands to periodically issue a kinit from the keytab. A
race condition exists during the kinit when the ticket cache is deleted and
re-created. Hadoop commands will fail when the ticket cache does not
momentarily exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
