When I tested the config has a specific hostname (ex. HTTP/ [email protected] - specify namenode host), namenode was ok, but datanode didn't start info server. I think datanode also use that config value ("hadoop.http.authentication.kerberos.principal").
datanode's host is dn01.hadoop.com error log is following: # error log 2013-04-02 09:46:45,651 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = false 2013-04-02 09:46:45,652 INFO org.mortbay.log: jetty-6.1.26 2013-04-02 09:46:46,651 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /hadoop/security/keytab/hdfs.keytab, for principal HTTP/ [email protected] 2013-04-02 09:46:46,658 WARN org.mortbay.log: failed authentication: javax.servlet.ServletException: javax.security.auth.login.LoginException: Unable to obtain password from user 2013-04-02 09:46:46,660 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@24c6e1ec {/,file:/hadoop/webapps/datanode} javax.servlet.ServletException: javax.security.auth.login.LoginException: Unable to obtain password from user at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:178) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.apache.hadoop.http.HttpServer.start(HttpServer.java:585) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:518) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:309) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734) at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:109) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177) the other odd thing is that I config 'hadoop.http.authentication.signature.secret.file' and 'hadoop.http.authentication.signature.secret' but warn log occur like this # warn log 2013-04-02 10:28:19,508 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: 'signature.secret' configuration not set, using a random value as secret Who know about this? Thanks. 2013/4/1 Daryn Sharp <[email protected]> > Hi, > > While it would be nice if this setting followed the convention of > allowing _HOST, the auth handler is independent of the common project which > implements that behavior. The good news is I believe the config key is > only used by the NN so you shouldn't have to reconfigure all your nodes. > Have you tested if a client fails when the config has a specific hostname? > > Daryn > > On Apr 1, 2013, at 2:50 AM, Oh Seok Keun wrote: > > Hi All > > I upgraded my hadoop cluster version to v1.1.2 last week. And I > configured hadoop security with kerberos. > When I configure some configuration for authentication hadoop http, I > failed to start NameNode web-server. > When I configure 'hadoop.http.authentication.kerberos.principal' with > proper host name (ex. HTTP/[email protected]), NameNode is > doing well. But I can't configure every node(hundred machine) with each > host name. > I guess new SPNEGO feature can't replace _HTTP with host's domain name. > Is right? > > My configuration and log are following: > > # configuration > > <property> > <name>hadoop.http.filter.initializers</name> > <value>org.apache.hadoop.security.AuthenticationFilterInitializer</value> > </property> > <property> > <name>hadoop.http.authentication.type</name> > <value>kerberos</value> > </property> > <property> > <name>hadoop.http.authentication.token.validity</name> > <value>36000</value> > </property> > <property> > <name>hadoop.http.authentication.signature.secret.file</name> > <value>/hadoop/security/conf/hadoop-http-auth-signature-secret</value> > </property> > <property> > <name>hadoop.http.authentication.cookie.domain</name> > <value>hadoop.com</value> > </property> > <property> > <name>hadoop.http.authentication.simple.anonymous.allowed</name> > <value>false</value> > </property> > <property> > <name>hadoop.http.authentication.kerberos.principal</name> > <value>HTTP/[email protected]</value> > </property> > <property> > <name>hadoop.http.authentication.kerberos.keytab</name> > <value>/hadoop/security/keytab/hdfs.keytab</value> > </property> > > > # log > > 2013-04-01 16:37:25,720 INFO org.apache.hadoop.http.HttpServer: > dfs.webhdfs.enabled = false > 2013-04-01 16:37:25,721 INFO org.apache.hadoop.http.HttpServer: Adding > Kerberos (SPNEGO) filter to getDelegationToken > 2013-04-01 16:37:25,722 INFO org.apache.hadoop.http.HttpServer: Adding > Kerberos (SPNEGO) filter to renewDelegationToken > 2013-04-01 16:37:25,723 INFO org.apache.hadoop.http.HttpServer: Adding > Kerberos (SPNEGO) filter to cancelDelegationToken > 2013-04-01 16:37:25,723 INFO org.apache.hadoop.http.HttpServer: Adding > Kerberos (SPNEGO) filter to fsck > 2013-04-01 16:37:25,724 INFO org.apache.hadoop.http.HttpServer: Adding > Kerberos (SPNEGO) filter to getimage > 2013-04-01 16:37:25,728 INFO org.apache.hadoop.http.HttpServer: Port > returned by webServer.getConnectors()[0].getLocalPort() before open() is > -1. Opening the listener on 50070 > 2013-04-01 16:37:25,730 INFO org.apache.hadoop.http.HttpServer: > listener.getLocalPort() returned 50070 > webServer.getConnectors()[0].getLocalPort() returned 50070 > 2013-04-01 16:37:25,730 INFO org.apache.hadoop.http.HttpServer: Jetty > bound to port 50070 > 2013-04-01 16:37:25,730 INFO org.mortbay.log: jetty-6.1.26 > 2013-04-01 16:37:26,091 INFO > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: > Login using keytab /hadoop/security/keytab/hdfs.keytab, for principal HTTP/_ > [email protected] > 2013-04-01 16:37:26,113 WARN org.mortbay.log: failed authentication: > javax.servlet.ServletException: javax.security.auth.login.LoginException: > Unable to obtain password from user > > 2013-04-01 16:37:26,114 WARN org.mortbay.log: Failed startup of context > org.mortbay.jetty.webapp.WebAppContext@3e7bfc04 > {/,file:/hadoop/webapps/hdfs} > javax.servlet.ServletException: javax.security.auth.login.LoginException: > Unable to obtain password from user > > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:178) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146) > at > org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) > at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) > at > org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) > at > org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) > at > org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) > at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) > at org.mortbay.jetty.Server.doStart(Server.java:224) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.apache.hadoop.http.HttpServer.start(HttpServer.java:631) > at > org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:484) > at > org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:362) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:313) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:536) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1419) > > Thanks :) > > Bill. > > >
