Hi ALL, I got root cause,Here keytab for HTTP and NN should be same...
But one more problem after successful login,I am getting NullPointerExecption. 2012-09-25 20:56:31,475 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Log Size Trigger :40000 txns 2012-09-25 20:57:31,922 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection to http://****.168:50070/getimage?getimage=1&txid=2841&storageInfo=-40:1613522265:0:CID-41727827-e93b-4764-a344-36ea765187ea<http://10.18.91.168:50070/getimage?getimage=1&txid=2841&storageInfo=-40:1613522265:0:CID-41727827-e93b-4764-a344-36ea765187ea> 2012-09-25 20:57:32,004 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Throwable Exception in doCheckpoint java.lang.RuntimeException: java.lang.NullPointerException at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1137) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2338) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:388) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.isNegotiate(KerberosAuthenticator.java:187) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:142) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:168) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:248) at org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:477) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:219) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:82) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:374) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:361) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.downloadCheckpointFiles(SecondaryNameNode.java:360) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:454) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:319) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$2.run(SecondaryNameNode.java:286) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:438) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:282) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at sun.net.www.protocol.http.NegotiateAuthentication.setHeaders(NegotiateAuthentication.java:161) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1294) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379) ... 22 more 2012-09-25 20:57:32,010 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: SHUTDOWN_MSG: ________________________________ From: Brahma Reddy Battula Sent: Tuesday, September 25, 2012 1:03 PM To: [email protected] Subject: [HADOOP-Securemode]checkpoint and fsck are failing Dear All, I started cluster in secure-mode by going through following document. http://hadoop.apache.org/docs/r1.0.3/HttpAuthentication.html Checkpoint is failing by throwing following exception..can anybody help on this.. 012-09-25 11:28:06,393 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in doCheckpoint java.io.IOException: Exception trying to open authenticated connection to http://***.168:50070/getimage?getimage=1&txid=42&storageInfo=-40:1613522265:0:CID-41727827-e93b-4764-a344-36ea765187ea<http://10.18.91.168:50070/getimage?getimage=1&txid=42&storageInfo=-40:1613522265:0:CID-41727827-e93b-4764-a344-36ea765187ea> at org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:479) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:219) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:82) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:374) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:361) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.downloadCheckpointFiles(SecondaryNameNode.java:360) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:454) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:319) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$2.run(SecondaryNameNode.java:286) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:438) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:282) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 503, message: Service Unavailable at org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:301) at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:63) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:145) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:168) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:248) at org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:477) ... 17 more
