[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446355#comment-17446355 ] Hadoop QA commented on HADOOP-15518: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Logfile || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m 26s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 17s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 48s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 56s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 46s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 16m 7s{color} | {color:green}{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 18m 29s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are enabled, using SpotBugs. {color} | | {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 0m 56s{color} | {color:green}{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 8s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m 8s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 59s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 59s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 38s{color} | {color:orange}https://ci-hadoop.apache.org/job/PreCommit-HADOOP-Build/252/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-auth.txt{color} | {color:orange} hadoop-common-project/hadoop-auth: The patch generated 2 new + 23 unchanged - 0 fixed = 25 total (was 23) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 46s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 14m 20s{color} | {color:red}https://ci-hadoop.apache.org/job/PreCommit-HADOOP-Build/252/artifact/out/patch-shadedclient.txt{color} |
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446290#comment-17446290 ] Prabhu Joseph commented on HADOOP-15518: Have found Spark Running jobs - Executors Page is empty due to AuthenticationFilter authenticating an already authenticated request failing with below exception. This patch helped to resolve the issue. [~kminder] If you are fine, i will rebase the patch with getRemoteUser instead of getUserPrincipal. Thanks. {code} 2021-11-18 10:06:59,560 WARN server.AuthenticationFilter - Authentication exception: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34)) {code} > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch, HADOOP-15518.002.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160578#comment-17160578 ] Prabhu Joseph commented on HADOOP-15518: [~jnp] Have not faced this issue while testing Hadoop Daemons after HADOOP-16314. I think this will happen only if user has configured multiple AuthenticationFilterInitializer. But it is safer to have this fix as AuthenticationFilter is used by many other projects like Ranger, Knox, Oozie and it is difficult to find the reason for "Request is a replay attack" on some systems. As per previous comments from [~sunilg], the patch has to use *getRemoteUser* instead of *getUserPrincipal*. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch, HADOOP-15518.002.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160548#comment-17160548 ] Jitendra Nath Pandey commented on HADOOP-15518: --- [~prabhujoseph] Is this jira still relevant given bunch of changes have gone in with HADOOP-16095 and its subtasks, particularly HADOOP-16314? > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch, HADOOP-15518.002.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160215#comment-17160215 ] Hadoop QA commented on HADOOP-15518: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 7s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 31m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 16m 34s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m 47s{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m 6s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 26s{color} | {color:orange} hadoop-common-project/hadoop-auth: The patch generated 2 new + 34 unchanged - 0 fixed = 36 total (was 34) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 15m 14s{color} | {color:red} patch has errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 29s{color} | {color:green} hadoop-auth in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 45s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}109m 5s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/PreCommit-HADOOP-Build/17049/artifact/out/Dockerfile | | JIRA Issue | HADOOP-15518 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13007892/HADOOP-15518.002.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e409d92d9484 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 2ba44a73bf2 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | checkstyle |
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160153#comment-17160153 ] Xiaoyu Yao commented on HADOOP-15518: - Thanks [~jnp] for the heads up. I took a look at the patch and it looks good to me. Attach a rebased patch to the latest trunk. Given [~eyang][~sunilg] have looked at this years back, I will leave this for their feedbacks. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17158705#comment-17158705 ] Hadoop QA commented on HADOOP-15518: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color} | {color:red} HADOOP-15518 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HADOOP-15518 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12926920/HADOOP-15518-001.patch | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/17047/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17158702#comment-17158702 ] Jitendra Nath Pandey commented on HADOOP-15518: --- [~kminder], [~eyang], is this patch still valid? cc [~xyao] > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524265#comment-16524265 ] Vinod Kumar Vavilapalli commented on HADOOP-15518: -- bq. I have changed getUserPrincipal to getRemoteUser and this change seems working fine. That was my first solution too. [~kminder], would that work? > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511492#comment-16511492 ] Sunil Govindan commented on HADOOP-15518: - Thanks [~eyang] for the pointer as per offline discussion. I have changed *getUserPrincipal* to *getRemoteUser* and this change seems working fine. GSS exception could be avoided like the patch attached here. Moreover since we check for getRemoteUser, class cast exception also wont happen as in the case Eric pointed above. So i think this patch could be improved by checking for getRemoteUser. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508318#comment-16508318 ] Eric Yang commented on HADOOP-15518: [~sunilg] Multiple AuthenticationFilters configured with different service principal names is a corner case that shouldn't exist but the code is allowing this to happen. Comments in this JIRA and YARN-8108 should explain why this is unsupported use case. The casting problem is using the same HTTP principal and YARN code is activating multiple filters that based on AuthenticationFilter. Token casting issue didn't exist prior to this patch. This patch is making assumption that filters based on AuthenticationFilter would make compatible tokens, which RMAuthenticationFilter and AuthenticationFilter don't make the same type of token. Thus, the casting problem occurs. This problem can be eliminated by applying same type of AuthenticationFilter on a server port. YARN-8108 can fix YARN resource manager. There might be other places in Hadoop that might have similar problems, like KMSAuthenticationFilter and DelegationTokenAuthenticationFilter that need to be reviewed to understand the impact of this change. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507946#comment-16507946 ] Sunil Govindan commented on HADOOP-15518: - [~eyang] This latest issue (Problem accessing /proxy/application_1528498597648_0001/) is a different problem, correct? bq.If multiple AuthenticationFilters are configured, and service principal names are different Are you mentioning a case where AuthenticationFilter is added multiple time (like Spnego Filter and authentication) but will use different keytab? > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506669#comment-16506669 ] Eric Yang commented on HADOOP-15518: There seems to be a problem if multiple filters extended from AuthenticationFilter, the token casting are incompatible. Browser shows this error message: {code} HTTP ERROR 500 Problem accessing /proxy/application_1528498597648_0001/. Reason: Server Error Caused by: java.lang.ClassCastException: org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter$1$1 cannot be cast to org.apache.hadoop.security.authentication.server.AuthenticationToken at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:250) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:597) at org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:649) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:304) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:597) at org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1608) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) {code} > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority:
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506539#comment-16506539 ] genericqa commented on HADOOP-15518: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 0s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 36m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 36m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 2s{color} | {color:green} hadoop-auth in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 37s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}127m 34s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-15518 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12926920/HADOOP-15518-001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 61010ea62644 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a127244 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14748/testReport/ | | Max. process+thread count | 291 (vs. ulimit of 1) | | modules | C: hadoop-common-project/hadoop-auth U: hadoop-common-project/hadoop-auth | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14748/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Authentication filter calling
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506506#comment-16506506 ] Eric Yang commented on HADOOP-15518: One foot note about this change. If multiple AuthenticationFilters are configured, and service principal names are different. TGS granted to remote client is the principal name of the first AuthenticationFilter that gets triggered. This may look unexpected when auditing where user has been through klist. The ability to configure different HTTP principals on the same server port configuration shouldn't exist, but developer should be aware of the API imperfection to avoid getting to this hole. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506425#comment-16506425 ] Owen O'Malley commented on HADOOP-15518: This looks good, [~kminder]. +1 > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505924#comment-16505924 ] Sunil Govindan commented on HADOOP-15518: - Thanks [~kminder] I checked this change in below cases. # Accessed RM old UI and new YARN UI from kerberized browser when AuthenticationFilter is configured and Http Auth Type was kerberos. # Both UI were accessible when Http Auth Type was configured as JWTRedirectAuthenticationHandler. Before this patch, we were getting replay attack for Ui2 as multiple auth handlers were present. For ui2, I have found that initial /ui2 will get 401 from Auth handler and then later it ll accept request as proper cookie were present in new request. But jetty will do a next redirect (not from client side) to access /ui2/index.html which will not have this cookie. Hence was getting GSS exception. Since this patch checks for principal as well, I think it looks fine. cc [~vinodkv] [~eyang] > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505081#comment-16505081 ] Eric Yang commented on HADOOP-15518: [~kminder] Race condition is not a criticism of this patch. It is byproduct of having multiple instances of AuthenticationFilter that authenticate method is called more than once due to lack of checking that the current request is already authenticated. My above comment is showing the states prior to this patch. Thank you for the patch. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505025#comment-16505025 ] Kevin Minder commented on HADOOP-15518: --- [~eyang] < Can you please clarify if your comment about a race condition is a criticism of the patch? I'm having trouble rationalizing your +1 with your other comments. Are you really trying to say that the patch solves the race condition? > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504999#comment-16504999 ] Eric Yang commented on HADOOP-15518: [~lmccay] {quote}it is unclear to me where the race condition is. My assumption is that the filters are invoked linearly so a second filter shouldn't be invoked until the request state is set properly.{quote} Your assumption is correct, and the diagram might help to explain the race conditions. HIgh level of the sequence of events of a normal setup: | Time | Browser | HttpRequest | HttpResponse | | 1 | Send WWW-Authenticate 1 | | | | 2 | | AuthenticationFilter checks WWW-Authenticate 1 | | | 3 | | Call authenticate to verify WWW-Authenticate 1 ticket with KDC | | | 4 | | Set User principal and remote user via Java security callbacks | | | 5 | | | AuthenticationFilter writes WWW-Authenticate 2 | | 6 | | | Business logic | | 7 | Received WWW-Authenticate 2 | | | Events of duplicated AuthenticationFilters are: | Time | Browser | HttpRequest | HttpResponse | | 1 | WWW-Authenticate 1 | | | | 2 | | AuthenticationFilter Instance 1 Check WWW-Authenticate 1 | | | 3 | | Call authenticate to verify WWW-Authenticate 1 ticket with KDC | | | 4 | | Set User principal and remote user via Java security callbacks | | | 5 | | | AuthenticationFilter Instance 1 writes WWW-Authenticate 2 | | 6 | | AuthenticationFilter Instance 2 Check WWW-Autnenticate 1 | AuthenticationFilter Instance 2 rewrites HTTP status with 403 | Browser has never retrieved WWW-Authenticate 2 header from server because HttpResponse is still buffered on server side. The race condition is between HttpRequest at time 6 is still using existing ticket 1 without using the new ticket 2 that is issued at time 5. Second Filter is invoked at time 6 using out dated data. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504893#comment-16504893 ] Larry McCay commented on HADOOP-15518: -- [~eyang] - it is unclear to me where the race condition is. My assumption is that the filters are invoked linearly so a second filter shouldn't be invoked until the request state is set properly. [~kminder] - this seems an appropriate change to me and worth adding tests for. [~owen.omalley] - are we missing anything in this implementation and assumptions about the HTTP client only being authenticated once by AuthenticationFilter? > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Assignee: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred in the current request. This > primarily affects situations where multiple authentication mechanism has been > configured. For example when core-site.xml's has > hadoop.http.authentication.type=kerberos and yarn-site.xml has > yarn.timeline-service.http-authentication.type=kerberos the result is an > attempt to perform two Kerberos authentications for the same request. This > in turn results in Kerberos triggering a replay attack detection. The > javadocs for AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the case in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504881#comment-16504881 ] Eric Yang commented on HADOOP-15518: +1 The patch looks good to me. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred. This primarily affects situations > where multipole authentication mechanism has been configured. For example > when core-site.xml's has hadoop.http.authentication.type and yarn-site.xml > has yarn.timeline-service.http-authentication.type the result is an attempt > to perform two Kerberos authentications for the same request. This in turn > results in Kerberos triggering a replay attack detection. The javadocs for > AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the cause in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15518) Authentication filter calling handler after request already authenticated
[ https://issues.apache.org/jira/browse/HADOOP-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504877#comment-16504877 ] Eric Yang commented on HADOOP-15518: This is a race condition if multiple instances of AuthenticationFilters are chained together by accident, and the token has already been checked once by first instance of AuthenticationFilter, and the token has not been committed to HttpResponse. "authenticate" method will get called twice. The javadoc is not wrong, except how to go about validating the current request is already authenticated is not trivial when token has not been received by browser yet. It might be possible to add checks for either HttpRequest.getRemoteUser() or HttpResponse header committing state to see if Authentication had already occurred to prevent the race condition. > Authentication filter calling handler after request already authenticated > - > > Key: HADOOP-15518 > URL: https://issues.apache.org/jira/browse/HADOOP-15518 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.1 >Reporter: Kevin Minder >Priority: Major > Attachments: HADOOP-15518-001.patch > > > The hadoop-auth AuthenticationFilter will invoke its handler even if a prior > successful authentication has occurred. This primarily affects situations > where multipole authentication mechanism has been configured. For example > when core-site.xml's has hadoop.http.authentication.type and yarn-site.xml > has yarn.timeline-service.http-authentication.type the result is an attempt > to perform two Kerberos authentications for the same request. This in turn > results in Kerberos triggering a replay attack detection. The javadocs for > AuthenticationHandler > ([https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationHandler.java)] > indicate for the authenticate method that > {quote}This method is invoked by the AuthenticationFilter only if the HTTP > client request is not yet authenticated. > {quote} > This does not appear to be the cause in practice. > I've create a patch and tested on a limited number of functional use cases > (e.g. the timeline-service issue noted above). If there is general agreement > that the change is valid I'll add unit tests to the patch. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org