[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900196#comment-15900196 ] Alejandro Abdelnur commented on HADOOP-13805: - Hi [~daryn], The exception we were hitting was: {code} Caused by: java.io.IOException: loginUserFromKeyTab must be done first at org.apache.hadoop.security.UserGroupInformation.reloginFromKeytab(UserGroupInformation.java:1055) at org.apache.hadoop.security.UserGroupInformation.checkTGTAndReloginFromKeytab(UserGroupInformation.java:1020) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:478) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:771) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$3.call(LoadBalancingKMSClientProvider.java:185) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$3.call(LoadBalancingKMSClientProvider.java:181) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:181) at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388) at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1420) {code} The {{UGI}} as created using a {{Subject}}. The UGI used by KMS client is obtained from {{UGI.getCurrentUser()}}. Regarding 'Any UGI should be able to relogin a subject regardless of who created it'. It may be the case in a conventional app, in our case, the app (StreamSets Data Collector) is a server app that is using classloaders to be able to interact with different versions of Hadoop clusters. Each classloader has its own Hadoop classes (diff versions of it). And Tthe renewal of the Kerberos credentials in the seed {{Subject}} is done from code in the bootstrap classloader. All this has worked fine for almost 2 years until HDFS encryption has been switched on and we run int the above exception. Said this, if you have a better idea how to solve this problem I'm all for it. Thanks. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895272#comment-15895272 ] Daryn Sharp commented on HADOOP-13805: -- What exactly broke in EZ because as best I can tell the whole premise of this change and the "external keytab" concept is completely broken. I think there's a deeper bug this is attempting to mask because I've seen the EZ UGI handling - it is completely screwed up and on my to-do blockers for EZ deployment. Internally we hacked out the external ugi changes because it broke us. bq. If you create a UGI from another UGI, ie via getCurrentUser(), the created UGI should not relogin from keytab, the relogin should be done by the creator UGI if it has a keytab. bq. My point is, any UGI created from a Subject (directly or via another UGI) should never attempt to relogin, it is the creator of the responsibility to do so. Wrong. Any UGI should be able to relogin a subject regardless of who created it. Why should any number of threads be dead in the water waiting for the "owner" to relogin in? It's a shared resource. bq. The bug i'm hitting now is that UGI.getCurrentUser() creates a new UGI and this tries to do relogin from keytab even if there is no keytab associated to the current UGI. This happens when HDFS client is accessing encryption zones, specifically the HDFS client interacting with the KMS client to get encryption keys. Whoa. Let's step back for a minute. The ugi knows if it's from a keytab based on whether there's a KeyTab instance present. Why does the ugi think it has a keytab but doesn't have a keytab? That makes no sense and is an indicator that the ugi is being grossly misused. [~alejandro.villa] Please provide more details or a stack trace. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894784#comment-15894784 ] Yongjun Zhang commented on HADOOP-13805: Good point [~jojochuang]. Just added it. Though it's incompatible when the fix is enabled, the behavior is the same as before without enabling the fix. Well, I still marked it as incompatible and made it clear in the release notes. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894335#comment-15894335 ] Wei-Chiu Chuang commented on HADOOP-13805: -- I didn't follow up to the end when this was resolve. Is this an incompatible change? If so this deserves a release note and we need to flag it as an incompatible change. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872212#comment-15872212 ] Xiao Chen commented on HADOOP-13805: Thanks [~yzhangal], [~jojochuang] for covering me up and working on this, and thanks [~tucu00] for reporting and reviewing! > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872209#comment-15872209 ] Hudson commented on HADOOP-13805: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11273 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11273/]) HADOOP-13805. UGI.getCurrentUser() fails if user does not have a keytab (yzhang: rev 4c26c241ad2b907dc02cecefa9846cbe2b0465ba) * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithMiniKdc.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872184#comment-15872184 ] Yongjun Zhang commented on HADOOP-13805: I committed to trunk. Thanks [~xiaochen] / [~jojochuang] for the earlier work, and [~tucu00] for the review. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872142#comment-15872142 ] Yongjun Zhang commented on HADOOP-13805: The test failures are pre-existing and reported as HADOOP-14030. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871396#comment-15871396 ] Hadoop QA commented on HADOOP-13805: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 58s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 35s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 58m 58s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.security.TestKDiag | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12853225/HADOOP-13805.010.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux bdec1aea006f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 02c5494 | | Default Java | 1.8.0_121 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/11651/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11651/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11651/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871313#comment-15871313 ] Yongjun Zhang commented on HADOOP-13805: Thanks [~tucu00]. Rebased and added an INFO message when the config is enabled, as rev10. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.010.patch, > HADOOP-13805.01.patch, HADOOP-13805.02.patch, HADOOP-13805.03.patch, > HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870923#comment-15870923 ] Alejandro Abdelnur commented on HADOOP-13805: - I've got confirmation the patch is working as expected on a live cluster. IMO, we can go for it. +1 again. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839846#comment-15839846 ] Wei-Chiu Chuang commented on HADOOP-13805: -- Hi, I am currently on PTO until Feburary 5th. While on PTO I will have limited access to Internet. For anything urgent or customer escalations, please contact cce-hdfs@, Yongjun/Xiao/John Zhuge. For any technical inquires pertains to Hadoop or HDFS, please contact Aaron T. Myer (atm@) Thanks -- A very happy Clouderan > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839844#comment-15839844 ] Alejandro Abdelnur commented on HADOOP-13805: - patch9 lgtm, +1. It would be great if you can verify it in a live cluster before committing. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.009.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838547#comment-15838547 ] Hadoop QA commented on HADOOP-13805: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s{color} | {color:green} hadoop-common-project/hadoop-common: The patch generated 0 new + 220 unchanged - 4 fixed = 220 total (was 224) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 38s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 33s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 2s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12849351/HADOOP-13805.009.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux f228227cb6f8 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / b782bf2 | | Default Java | 1.8.0_121 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11508/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11508/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch,
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837592#comment-15837592 ] Alejandro Abdelnur commented on HADOOP-13805: - patch8 looks good. A few things regarding the new {{enableRenewThreadCreationForTest}}, can we make the methods package private (the testcases using them are in the same package, and having them package private will avoid an app setting them by mistake). Can we also log a WARN message in the {{spawnAutoRenewalThreadForUserCreds()}} if in test mode? > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch, HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837286#comment-15837286 ] Hadoop QA commented on HADOOP-13805: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m 21s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 28s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch generated 1 new + 220 unchanged - 4 fixed = 221 total (was 224) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 12s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 37s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 61m 24s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12849224/HADOOP-13805.008.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux a2b6c9c5d44c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 9c0a4d3 | | Default Java | 1.8.0_121 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/11505/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11505/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11505/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837230#comment-15837230 ] Yongjun Zhang commented on HADOOP-13805: Thanks a lot for the review [~tucu00]. The failed test is actually an interesting one. It loginuser from subject, and expects the renewal thread to be created to renew the credential. Because of the fix, no matter whether we disable or enable the config, the condition for creating renewal thread is always false, thus the test failed. The reason is that this test is created after HADOOP-13558, and it depends on the behaviour of HADOOP-13558. Disabling the config will disable the HADOOP-13558 change, enabling it will fix the wrong behavior, that's why this test can't work by simply disabling or enabling the config. Discussed with [~xiaochen] who originally created the testcase, we agreed upon a solution that introduce a special field to allow the renewal thread be created for testing purpose. Uploaded rev 008 with this solution. Would you please take a look again? Thanks. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.007.patch, > HADOOP-13805.008.patch, HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch, HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826597#comment-15826597 ] Yongjun Zhang commented on HADOOP-13805: Thanks [~tucu00]. I think you listed two options, one is to make this potentially incompatible change; the other is to create a new UGI and obsolete the old/incorrect implementation later on. It may not be too bad to go with option one. Say, with option two, we may hit the issue reported here. With option one, we need to watch out how things are broken due to the incompatible change, and fix accordingly. If we go with option one, if client code is broken, the client code need to be changed to do the renewal. Would you please help putting together a recommended change as part of the release notes of this jira? If we go with option one, I'm +1 on Wei-Chiu's rev6 (I found that it may not be easy to add the test you proposed as a unit test due to the run time) Would you please also take a look at rev6? Thanks. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823796#comment-15823796 ] Alejandro Abdelnur commented on HADOOP-13805: - [~yzhangal], I understand your concern on potentially breaking existing usages if fixing the current wrong behavior. If you don't want to fix the behavior because it would account as an incompatible change, then the only option I see is a new API that will allow to create a UGI with an externalLogin; else I don't see how to create such UGI. Any suggestion? > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823079#comment-15823079 ] Hadoop QA commented on HADOOP-13805: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 14s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 30s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch generated 2 new + 94 unchanged - 2 fixed = 96 total (was 96) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 50m 16s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.security.TestUGIWithMiniKdc | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12847530/HADOOP-13805.006.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux bc2685988b5a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / ed09c14 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/11442/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/11442/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11442/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11442/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. >
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823075#comment-15823075 ] Yongjun Zhang commented on HADOOP-13805: I have been looking at rev006 quite a bit, it looks good to me, except for two things: 1. The change to constructor {code} UserGroupInformation(Subject subject) { this(subject, true); } {code} now changed the original behavior, even though it's really fixing a wrong behavior, it's an incompatible change. Other application use this API may break. Hi [~tucu00], thanks for reporting the issue and review so far. How do you think we should address that? 2. The test suggested by Alejandro at https://issues.apache.org/jira/browse/HADOOP-13805?focusedCommentId=15653489=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15653489 is better included with the patch. Thanks. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823074#comment-15823074 ] Xiao Chen commented on HADOOP-13805: Thanks for the email. I'm currently on PTO and will return on Jan. 23. I will have limited email access during this time. For Escalations, please contact cce-hdfs@, HDFS-CCE HipChat room ,or Yongjun Zhang (yzhang@).For HDFS/KMS issues, please contact int-hdfs@, HDFS HipChat room, or ATM (atm@). -- -Xiao > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.006.patch, HADOOP-13805.01.patch, > HADOOP-13805.02.patch, HADOOP-13805.03.patch, HADOOP-13805.04.patch, > HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819530#comment-15819530 ] Wei-Chiu Chuang commented on HADOOP-13805: -- [~tucu00] I am not expert in UGI and user authentication. But looking at this jira and the patch, I think you are right. UGI should not use isKeytab to determine if it should renew. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch, HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811981#comment-15811981 ] Alejandro Abdelnur commented on HADOOP-13805: - [~xiaochen], Unless I'm missing something, I don't think patch 5 will do the trick for the scenario where I've seen this issue to pop up. {{getCurrentUser()}} creates a new UGI using the {{UserGroupInformation(Subject)}} constructor, in your proposed change, the availability of the keytab is determined by inspecting the given subject. If the given Subject has a keytab but is not owned/created to the UGI (my usecase) it will create UGI with the isKeytab flag set to true and this will fail in the same way as without patch 5. Somehow, we have to make sure that if a UGI is created with an external Subject, any UGI derived from it it should not say that it has a keytab. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch, HADOOP-13805.04.patch, HADOOP-13805.05.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799225#comment-15799225 ] Hadoop QA commented on HADOOP-13805: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 8s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 53m 22s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12845597/HADOOP-13805.05.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 88ec3ca88dad 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a0a2761 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11358/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11358/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments:
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797799#comment-15797799 ] Alejandro Abdelnur commented on HADOOP-13805: - [~xiaochen], {{kinit -R}} assumes the TGT can still be renewed, if it reached it max life time it is not. So this will delay the failure until the TGT cannot be renewed anymore; at that point it will fail as you'll need to use the keytab which you don't have. Regardless, even if {{kinit -R}} would do the trick, it is not correct for UGI to take over renewal responsibilities when the TGT has not been obtained by UGI. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch, HADOOP-13805.04.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796761#comment-15796761 ] Hadoop QA commented on HADOOP-13805: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 10s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 30s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch generated 9 new + 101 unchanged - 0 fixed = 110 total (was 101) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 27s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 52m 0s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12845453/HADOOP-13805.04.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7a10fcdf0400 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8fadd69 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/11347/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11347/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11347/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741459#comment-15741459 ] Alejandro Abdelnur commented on HADOOP-13805: - [~xiaochen], sorry missed your NOV18 comment. The renewal thread should not be started if there is no keytab, there is no point to do so because it will not have the credentials (the info in the keytab) at renewal time. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737331#comment-15737331 ] Xiao Chen commented on HADOOP-13805: Thanks for the ping, I think this is major, but should target the same as HADOOP-13558. Tucu, please feel free to modify if you disagree. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736880#comment-15736880 ] Andrew Wang commented on HADOOP-13805: -- Is this a release blocker? Also HADOOP-13558 has fix versions of 2.7.4 and 2.8.0, is this targeted at those releases as well? > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >Priority: Blocker > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch, > HADOOP-13805.03.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677700#comment-15677700 ] Hadoop QA commented on HADOOP-13805: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 6s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 42s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 50m 28s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12839605/HADOOP-13805.03.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux ffb9eb3b4c18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / f6ffa11 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11101/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11101/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15675938#comment-15675938 ] Xiao Chen commented on HADOOP-13805: Hm, clearly unit test caught me. Let me look more into this... > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >Priority: Blocker > Attachments: HADOOP-13805.01.patch, HADOOP-13805.02.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15675717#comment-15675717 ] Hadoop QA commented on HADOOP-13805: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 57s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 46m 49s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.security.TestUGIWithMiniKdc | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HADOOP-13805 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12839492/HADOOP-13805.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux eab41fcf714b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 140b993 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/11096/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/11096/testReport/ | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/11096/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 >
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671471#comment-15671471 ] Alejandro Abdelnur commented on HADOOP-13805: - Patch LGTM, I was missing the point that when login from keytab the subject is created outside of the resulting UGI. I see now why you need the private constructor introduced in HADOOP-13558. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >Priority: Blocker > Attachments: HADOOP-13805.01.patch > > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15666971#comment-15666971 ] Alejandro Abdelnur commented on HADOOP-13805: - [~xiaochen], If you create a {{UGI}} from {{Subject}} externally, this can done only via the {{getUGIFromSubject(Subject)}} method, and in that case the UGI should not relogin from keytab as we already discussed. If you create a {{UGI}} from another {{UGI}}, ie via {{getCurrentUser()}}, the created {{UGI}} should not relogin from keytab, the relogin should be done by the creator {{UGI}} if it has a keytab. My point is, any {{UGI}} created from a {{Subject}} (directly or via another {{UGI}}) should never attempt to relogin, it is the creator of the responsibility to do so. The bug i'm hitting now is that {{UGI.getCurrentUser()}} creates a new UGI and this tries to do relogin from keytab even if there is no keytab associated to the current UGI. This happens when HDFS client is accessing encryption zones, specifically the HDFS client interacting with the KMS client to get encryption keys. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >Priority: Blocker > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15664854#comment-15664854 ] Xiao Chen commented on HADOOP-13805: Thanks [~tucu00] for creating this with great details / reproduction / solution. Could you help me understand more? Specifically this: currently UGI is always created with the {{UGI(Subject)}} constructor. If we set the default to true, then it seems the spawn logic won't be triggered ever. When should hadoop spawn the background thread to renew? Is the proposal to make {{false}} the default and only {{true}} when {{loginUserFromKeytab}} / {{loginUserFromKeytabAndReturnUGI}} / {{loginUserFromTicketCache}} ? Also, what about {{createRemoteUser}} and {{createProxyUser}}? > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Assignee: Xiao Chen >Priority: Blocker > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653489#comment-15653489 ] Alejandro Abdelnur commented on HADOOP-13805: - A quick way to verify the proposed solution work is using running the following snippet and wait until the TGT expires: {code} UserGroupInformation.setShouldRenewImmediatelyForTests(true); UserGroupInformation.loginUserFromSubject(subject); UserGroupInformation ugi = UserGroupInformation.getLoginUser(); for (int i = 0; i < 100; i++) { System.out.printf("***Into %dsecs\n", i * 30*1000); ugi.doAs(new PrivilegedExceptionAction() { @Override public Object run() throws Exception { URI uri = new URI("kms://http@localhost:16000/kms"); KMSClientProvider provider = (KMSClientProvider) new KMSClientProvider.Factory().createProvider(uri, hConf); System.out.println(provider.getKeys()); return null; } }); for (int j = 0; j < 30; j++) { System.out.println("! " + j); System.out.flush(); Thread.sleep(1000); } System.out.println(); } {code} > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Priority: Blocker > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. > This problem is experienced in {{KMSClientProvider}} when used by the HDFS > filesystem client accessing an an encryption zone. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13805) UGI.getCurrentUser() fails if user does not have a keytab associated
[ https://issues.apache.org/jira/browse/HADOOP-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653479#comment-15653479 ] Alejandro Abdelnur commented on HADOOP-13805: - The solution, and I think is a safe one, is that the {{UGI(Subject)}} constructor to set isExternalKeytab to *TRUE*. If an UGI is being created from a Subject, the keytab is always external and it is the Subject creator responsibility to keep the credentials valid. > UGI.getCurrentUser() fails if user does not have a keytab associated > > > Key: HADOOP-13805 > URL: https://issues.apache.org/jira/browse/HADOOP-13805 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.8.0, 2.9.0, 3.0.0-alpha2 >Reporter: Alejandro Abdelnur >Priority: Blocker > > HADOOP-13558 intention was to avoid UGI from trying to renew the TGT when the > UGI is created from an existing Subject as in that case the keytab is not > 'own' by UGI but by the creator of the Subject. > In HADOOP-13558 we introduced a new private UGI constructor > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > we use with TRUE only when doing a {{UGI.loginUserFromSubject()}}. > The problem is, when we call {{UGI.getCurrentUser()}}, and UGI was created > via a Subject (via the {{UGI.loginUserFromSubject()}} method), we call {{new > UserGroupInformation(subject)}} which will delegate to > {{UserGroupInformation(Subject subject, final boolean externalKeyTab)}} and > that will use externalKeyTab == *FALSE*. > Then the UGI returned by {{UGI.getCurrentUser()}} will attempt to login using > a non-existing keytab if the TGT expired. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org