[
https://issues.apache.org/jira/browse/HDFS-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478684#comment-13478684
]
Brahma Reddy Battula commented on HDFS-3980:
--------------------------------------------
[~atm]
Thanks a lot for reply..
{quote}
can you tell me exactly what you did to generate the keytab in question? It's
unfortunately quite easy to accidentally invalidate a keytab for a given
principal if you later export another keytab including entries for the same
principal
{quote}
I have generated keytab using following
xst -norandkey -k /etc/hadoop/hdfs.keytab hdfs/(hostname of machine)@HADOOP.COM
xst -norandkey -k /etc/hadoop/hdfs.keytab HTTP/(hostname of machine)@HADOOP.COM
and these two only I had configured..Please check following link for same..
https://issues.apache.org/jira/browse/HDFS-4043?focusedCommentId=13478670&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13478670
{quote}
I can't find the method KerberosUtil#getWebDfsPrincipalAndReset in any version
of Hadoop that I'm aware of. What exact version are you experiencing this with?
{quote}
Internally we had added this one.I removed that(getWebDfsPrincipalAndReset) one
executed even then checkpoint is failing by throwing followig exception..
{noformat}
2012-10-18 10:04:55,907 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection to
http://10.***.177:50070/getimage?getimage=1&txid=280&storageInfo=-40:85372811:0:CID-86a868d5-df3a-4a3c-b068-cc9a3bafec9b
2012-10-18 10:05:21,943 ERROR org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:hdfs/[email protected] (auth:KERBEROS)
cause:java.io.IOException: Exception trying to open authenticated connection to
http://10.***.177:50070/getimage?getimage=1&txid=280&storageInfo=-40:85372811:0:CID-86a868d5-df3a-4a3c-b068-cc9a3bafec9b
2012-10-18 10:05:21,944 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in
doCheckpoint
java.io.IOException: Exception trying to open authenticated connection to
http://10.***.177:50070/getimage?getimage=1&txid=280&storageInfo=-40:85372811:0:CID-86a868d5-df3a-4a3c-b068-cc9a3bafec9b
at
org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:510)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.doGetUrl(TransferFsImage.java:229)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:222)
at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:86)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:399)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:385)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.downloadCheckpointFiles(SecondaryNameNode.java:384)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:477)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:343)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$2.run(SecondaryNameNode.java:310)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:337)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1347)
at
org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:450)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:306)
at java.lang.Thread.run(Thread.java:662)
Caused by:
org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Server not found
in Kerberos database (7) - UNKNOWN_SERVER)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:273)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:169)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:232)
at
org.apache.hadoop.security.SecurityUtil.openSecureHttpConnection(SecurityUtil.java:508)
... 18 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Server
not found in Kerberos database (7) - UNKNOWN_SERVER)
at
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:663)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:230)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:162)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:252)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:228)
... 21 more
Caused by: KrbException: Server not found in Kerberos database (7) -
UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:64)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:185)
at
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:294)
at
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:106)
at
sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:557)
at
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:594)
... 28 more
Caused by: KrbException: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:133)
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:58)
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:53)
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:46)
... 33 more
{noformat}
Please correct me If I am wrong..
> NPE in HttpURLConnection.java while starting SecondaryNameNode.
> ---------------------------------------------------------------
>
> Key: HDFS-3980
> URL: https://issues.apache.org/jira/browse/HDFS-3980
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
> Affects Versions: 3.0.0, 2.0.1-alpha
> Reporter: Brahma Reddy Battula
> Priority: Critical
> Attachments: core-site.xml, hdfs-site.xml
>
>
> Scenario:
> ========
> I started secure cluster by going thru following..
> https://ccp.cloudera.com/display/CDHDOC/CDH3+Security+Guide..
> Here SecondaryNamenode is getting shutdown by throwing NPE..
> Please correct me If I am wrong...
> Will attach conf and logs..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
