[
https://issues.apache.org/jira/browse/HDFS-14059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684202#comment-16684202
]
Plamen Jeliazkov commented on HDFS-14059:
-----------------------------------------
Hey folks,
I seem to have uncovered 2 issues in my testing. Posting them in detail here.
If we want to make sub-tasks out of them let me know and I will do so.
(1)
I see the new transitionToObserver operation in `haadmin` but it won’t let me
do it. Did you have to `-forceManual` to get it to go into Observer? I am
guessing you guys haven’t hit this because you did not have automatic failover
enabled. Do we want to see this message or just have it go through?
Here is what I see in console:
{code}
[hdfs@*** hadoop-3.3.0-SNAPSHOT]$ bin/hdfs haadmin -transitionToObserver ha2
Automatic failover is enabled for NameNode at ****
Refusing to manually manage HA state, since it may cause
a split-brain scenario or other incorrect state.
If you are very sure you know what you are doing, please
specify the --forcemanual flag.
{code}
(2)
Seems I am encountering some issue with initializing state in the
ObserverReadProxyProvider. It seems it is expecting some configuration property
“hadoop.security.service.user.name.key” on creation of the DFSClient. My
suspicion is that this is coming from making use of HAServiceProtocol’s
getServiceState having the annotation requiring it to be set. I made sure to
grab debug logs showing you the key its looking for and the configuration
values.
HAServiceProtocol’s annotation:
{code:java}
@KerberosInfo(serverPrincipal=CommonConfigurationKeys.HADOOP_SECURITY_SERVICE_USER_NAME_KEY)
{code}
This is the error message I get:
{code:java}
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: Get token info
proto:interface org.apache.hadoop.ha.protocolPB.HAServiceProtocolPB info:null
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: tokens aren't supported
for this protocol or user doesn't have one
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: Get kerberos info
proto:interface org.apache.hadoop.ha.protocolPB.HAServiceProtocolPB
info:@org.apache.hadoop.security.KerberosInfo(clientPrincipal=,
serverPrincipal=hadoop.security.service.user.name.key)
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: getting serverKey:
hadoop.security.service.user.name.key conf value: null principal: null
java.io.IOException: Couldn't set up IO streams:
java.lang.IllegalArgumentException: Failed to specify server's Kerberos
principal name
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:876)
at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:421)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1604)
at org.apache.hadoop.ipc.Client.call(Client.java:1435)
at org.apache.hadoop.ipc.Client.call(Client.java:1388)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy8.getServiceStatus(Unknown Source)
at
org.apache.hadoop.ha.protocolPB.HAServiceProtocolClientSideTranslatorPB.getServiceStatus(HAServiceProtocolClientSideTranslatorPB.java:136)
...
{code}
> Test reads from standby on a secure cluster with Configured failover
> --------------------------------------------------------------------
>
> Key: HDFS-14059
> URL: https://issues.apache.org/jira/browse/HDFS-14059
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Priority: Major
>
> Run standard HDFS tests to verify reading from ObserverNode on a secure HA
> cluster with {{ConfiguredFailoverProxyProvider}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
