[jira] [Commented] (HADOOP-15959) revert HADOOP-12751
[ https://issues.apache.org/jira/browse/HADOOP-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706764#comment-16706764 ] He Xiaoqiao commented on HADOOP-15959: -- hi [~ste...@apache.org], [~ajisakaa], IIUC, this is a common issue, since after revert HADOOP-12751, some auth_to_local rules if include '/' or '@' will always check fail in KerberosName#apply since it throws NoMatchingRule exception. {quote}if (result != null && nonSimplePattern.matcher(result).find()) { throw new NoMatchingRule("Non-simple name " + result + " after auth_to_local rule " + this); }{quote} another case ref HADOOP-15922. Please check +1. > revert HADOOP-12751 > --- > > Key: HADOOP-15959 > URL: https://issues.apache.org/jira/browse/HADOOP-15959 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.7.7, 2.8.5 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.2.0, 2.7.8, 3.0.4, 3.1.2, 2.8.6, 2.9.3 > > Attachments: HADOOP-15959-001.patch, HADOOP-15959-branch-2-002.patch, > HADOOP-15959-branch-2.7-003.patch > > > HADOOP-12751 doesn't quite work right. Revert. > (this patch is so jenkins can do the test runs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL
[ https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706737#comment-16706737 ] He Xiaoqiao commented on HADOOP-15922: -- [~eyang] Thanks for your feedback. Actually, if use 'fool/localhost' as impersonate user for ut, KerberosName could not check pass after revert HADOOP-12751, since KerberosName#apply check if impersonate user name include '/' or '@' when apply rule like 'RULE:[2:$1/$2]', and it throws exception because HADOOP-12751. {quote} if (result != null && nonSimplePattern.matcher(result).find()) { throw new NoMatchingRule("Non-simple name " + result + " after auth_to_local rule " + this); }{quote} Before revert HADOOP-12751, this check just LOG.info and not throw exception. IIUC, this is a common issue: if using auth_to_local and some rule include '/' or '@', it always throw exception. FYI. > DelegationTokenAuthenticationFilter get wrong doAsUser since it does not > decode URL > --- > > Key: HADOOP-15922 > URL: https://issues.apache.org/jira/browse/HADOOP-15922 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > Fix For: 3.3.0, 3.1.2, 3.2.1 > > Attachments: HADOOP-15922.001.patch, HADOOP-15922.002.patch, > HADOOP-15922.003.patch, HADOOP-15922.004.patch, HADOOP-15922.005.patch, > HADOOP-15922.006.patch > > > DelegationTokenAuthenticationFilter get wrong doAsUser when proxy user from > client is complete kerberos name (e.g., user/hostn...@realm.com, actually it > is acceptable), because DelegationTokenAuthenticationFilter does not decode > DOAS parameter in URL which is encoded by {{URLEncoder}} at client. > e.g. KMS as example: > a. KMSClientProvider creates connection to KMS Server using > DelegationTokenAuthenticatedURL#openConnection. > b. If KMSClientProvider is a doAsUser, KMSClientProvider will put {{doas}} > with url encoded user as one parameter of http request. > {code:java} > // proxyuser > if (doAs != null) { > extraParams.put(DO_AS, URLEncoder.encode(doAs, "UTF-8")); > } > {code} > c. when KMS server receives the request, it does not decode the proxy user. > As result, KMS Server will get the wrong proxy user if this proxy user is > complete Kerberos Name or it includes some special character. Some other > authentication and authorization exception will throws next to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15959) revert HADOOP-12751
[ https://issues.apache.org/jira/browse/HADOOP-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706718#comment-16706718 ] Akira Ajisaka commented on HADOOP-15959: Hi [~ste...@apache.org], this revert broke TestSecureLogins. Would you check this? {noformat} [ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.29 s <<< FAILURE! - in org.apache.hadoop.registry.secure.TestSecureLogins [ERROR] testValidKerberosName(org.apache.hadoop.registry.secure.TestSecureLogins) Time elapsed: 0.008 s <<< ERROR! org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to zookeeper/localhost at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:401) at org.apache.hadoop.registry.secure.TestSecureLogins.testValidKerberosName(TestSecureLogins.java:203) {noformat} > revert HADOOP-12751 > --- > > Key: HADOOP-15959 > URL: https://issues.apache.org/jira/browse/HADOOP-15959 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.7.7, 2.8.5 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.2.0, 2.7.8, 3.0.4, 3.1.2, 2.8.6, 2.9.3 > > Attachments: HADOOP-15959-001.patch, HADOOP-15959-branch-2-002.patch, > HADOOP-15959-branch-2.7-003.patch > > > HADOOP-12751 doesn't quite work right. Revert. > (this patch is so jenkins can do the test runs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL
[ https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706444#comment-16706444 ] Eric Yang commented on HADOOP-15922: [~hexiaoqiao] it is a better example if the user is foo/localhost because this is a real format for kerberos principal, where foo%local is not. [~daryn], without auth_to_local rule defined, the system act as pass through. Hence, revert HADOOP-12751 is a moot point unless some other code is added to safe guard against no rule. Otherwise, there is a discrepancy between your security mailing list proposal and no rule defined. Is there a Jira for the second part of your proposal? > DelegationTokenAuthenticationFilter get wrong doAsUser since it does not > decode URL > --- > > Key: HADOOP-15922 > URL: https://issues.apache.org/jira/browse/HADOOP-15922 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > Fix For: 3.3.0, 3.1.2, 3.2.1 > > Attachments: HADOOP-15922.001.patch, HADOOP-15922.002.patch, > HADOOP-15922.003.patch, HADOOP-15922.004.patch, HADOOP-15922.005.patch, > HADOOP-15922.006.patch > > > DelegationTokenAuthenticationFilter get wrong doAsUser when proxy user from > client is complete kerberos name (e.g., user/hostn...@realm.com, actually it > is acceptable), because DelegationTokenAuthenticationFilter does not decode > DOAS parameter in URL which is encoded by {{URLEncoder}} at client. > e.g. KMS as example: > a. KMSClientProvider creates connection to KMS Server using > DelegationTokenAuthenticatedURL#openConnection. > b. If KMSClientProvider is a doAsUser, KMSClientProvider will put {{doas}} > with url encoded user as one parameter of http request. > {code:java} > // proxyuser > if (doAs != null) { > extraParams.put(DO_AS, URLEncoder.encode(doAs, "UTF-8")); > } > {code} > c. when KMS server receives the request, it does not decode the proxy user. > As result, KMS Server will get the wrong proxy user if this proxy user is > complete Kerberos Name or it includes some special character. Some other > authentication and authorization exception will throws next to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL
[ https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706280#comment-16706280 ] Hadoop QA commented on HADOOP-15922: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 27s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m 18s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 57s{color} | {color:orange} hadoop-common-project: The patch generated 2 new + 97 unchanged - 0 fixed = 99 total (was 97) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 1s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 14s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 2s{color} | {color:green} hadoop-kms in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 40s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}105m 46s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f | | JIRA Issue | HADOOP-15922 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12950307/HADOOP-15922.006.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux f27c11fd6f8e 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug 17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8f3e12f | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_181 | | findbugs | v3.1.0-RC1 | | checkstyle |
[jira] [Commented] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL
[ https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706207#comment-16706207 ] He Xiaoqiao commented on HADOOP-15922: -- [~eyang] update and re-upload v006 and use special character '%' replace '/' in TestKMS#testGetDelegationTokenByProxyUser compare to v005. Since after revert of HADOOP-12751, have to config complex auth_to_local rules to pass the auth. In order to check client not double encode doAs user name only, choose another special character '%' and do not import auth_to_local rules. FYI. > DelegationTokenAuthenticationFilter get wrong doAsUser since it does not > decode URL > --- > > Key: HADOOP-15922 > URL: https://issues.apache.org/jira/browse/HADOOP-15922 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > Fix For: 3.3.0, 3.1.2, 3.2.1 > > Attachments: HADOOP-15922.001.patch, HADOOP-15922.002.patch, > HADOOP-15922.003.patch, HADOOP-15922.004.patch, HADOOP-15922.005.patch, > HADOOP-15922.006.patch > > > DelegationTokenAuthenticationFilter get wrong doAsUser when proxy user from > client is complete kerberos name (e.g., user/hostn...@realm.com, actually it > is acceptable), because DelegationTokenAuthenticationFilter does not decode > DOAS parameter in URL which is encoded by {{URLEncoder}} at client. > e.g. KMS as example: > a. KMSClientProvider creates connection to KMS Server using > DelegationTokenAuthenticatedURL#openConnection. > b. If KMSClientProvider is a doAsUser, KMSClientProvider will put {{doas}} > with url encoded user as one parameter of http request. > {code:java} > // proxyuser > if (doAs != null) { > extraParams.put(DO_AS, URLEncoder.encode(doAs, "UTF-8")); > } > {code} > c. when KMS server receives the request, it does not decode the proxy user. > As result, KMS Server will get the wrong proxy user if this proxy user is > complete Kerberos Name or it includes some special character. Some other > authentication and authorization exception will throws next to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL
[ https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Xiaoqiao updated HADOOP-15922: - Attachment: HADOOP-15922.006.patch > DelegationTokenAuthenticationFilter get wrong doAsUser since it does not > decode URL > --- > > Key: HADOOP-15922 > URL: https://issues.apache.org/jira/browse/HADOOP-15922 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > Fix For: 3.3.0, 3.1.2, 3.2.1 > > Attachments: HADOOP-15922.001.patch, HADOOP-15922.002.patch, > HADOOP-15922.003.patch, HADOOP-15922.004.patch, HADOOP-15922.005.patch, > HADOOP-15922.006.patch > > > DelegationTokenAuthenticationFilter get wrong doAsUser when proxy user from > client is complete kerberos name (e.g., user/hostn...@realm.com, actually it > is acceptable), because DelegationTokenAuthenticationFilter does not decode > DOAS parameter in URL which is encoded by {{URLEncoder}} at client. > e.g. KMS as example: > a. KMSClientProvider creates connection to KMS Server using > DelegationTokenAuthenticatedURL#openConnection. > b. If KMSClientProvider is a doAsUser, KMSClientProvider will put {{doas}} > with url encoded user as one parameter of http request. > {code:java} > // proxyuser > if (doAs != null) { > extraParams.put(DO_AS, URLEncoder.encode(doAs, "UTF-8")); > } > {code} > c. when KMS server receives the request, it does not decode the proxy user. > As result, KMS Server will get the wrong proxy user if this proxy user is > complete Kerberos Name or it includes some special character. Some other > authentication and authorization exception will throws next to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org