[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086097#comment-17086097 ] Hudson commented on HADOOP-16972: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18157 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18157/]) HADOOP-16972. Ignore AuthenticationFilterInitializer for KMSWebServer. (github: rev ac40daece17e9a6339927dbcadab76034bd7882c) * (edit) hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebServer.java * (edit) hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Blocker > Fix For: 3.3.0 > > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086064#comment-17086064 ] Eric Yang commented on HADOOP-16972: [~iwasakims] Thank you for the pointer that kms-dt is a different type from hdfs-dt. Your patch is right way to address this problem in the short term. It is not a good idea to make separate token issuer a common practice unless there are good reasons. Synchronization of session becomes a problem when token expiration unsynchronized due to API calls at different time. HttpFS is working in the absence of contacting namenode. Hence, it is kind of ok to allow HttpFS manages a separate token set for a specific use case. In theory, KMS security does not benefit from having separated token kind. This implementation is more for performance reason to reduce round trip with namenode for user credential validation. However, there are more disadvantages in doing so, like unsynchronized session, and additional logic/payload to populate different token types to the right place. Since Hadoop community has already done some of the hard work to solve the problems superficially. This patch is good stop gap solution, and I would prefer to fix KMS to use global AuthenticationFilter to avoid session problems, and reduce config logistics. These changes are beyond my participation in KMS code or scope of this issue. +1 for fixing this in 3.3.0 to prevent regression. > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085364#comment-17085364 ] Masatake Iwasaki commented on HADOOP-16972: --- [~eyang] Since KMS issues KMS delegation token, it uses its own KMSAuthenticationFilter which uses KerberosDelegationTokenAuthenticationHandler inside. The KMSAuthenticationFilter is always used since it is hardcoded in WEB-INF/web.xml (same as httpfs). We need the same workaround as webhdfs and httpfs for KMS too. > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085316#comment-17085316 ] Eric Yang commented on HADOOP-16972: HDFS uses Auth filter instead of the global AuthenticationFilter because WebHDFS issues delegation token that standard AuthenticationFilter does not have same capability. It would be better to use global authentication filter to reduce security holes. KMS server can either be protected using global authentication filter, or customize like you are suggesting. However, I do not think switching filter initialization solves the root problem, where Kerberos tgt token is reused on two different endpoints when servers are co-located on the same node. I think the unit test is passing for the wrong reason where realm information is not available and not triggering lookup. Could you verify KDC server log to make sure that authentication lookup has in fact happened? > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084052#comment-17084052 ] Masatake Iwasaki commented on HADOOP-16972: --- [~weichiu] The error is the reproducible "Request is a replay (34)". I'm trying to reproduced the error in a unit test. Using dedicated configuration files (without hadoop.http.filter.initializers) for KMS like CM worked. If you have backported HADOOP-16314 and hadoop.http.filter.initializers=AuthenticationFilterInitializer worked for KMS, the cause could be on my environment.. > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
[ https://issues.apache.org/jira/browse/HADOOP-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084038#comment-17084038 ] Wei-Chiu Chuang commented on HADOOP-16972: -- Hi [~iwasakims] when you say KMS does not work, do you mean the message "Request is a replay (34)" like in HDFS-14845? Does it reproduce all the time? Thanks. I have not yet seen this problem. But maybe it's because we have per-process core-site.xml file. We don't share config files. > Ignore AuthenticationFilterInitializer for KMSWebServer > --- > > Key: HADOOP-16972 > URL: https://issues.apache.org/jira/browse/HADOOP-16972 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 3.3.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > KMS does not work if hadoop.http.filter.initializers is set to > AuthenticationFilterInitializer since KMS uses its own authentication filter. > This is problematic when KMS is on the same node with other Hadoop services > and shares core-site.xml with them. The filter initializers configuration > should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
