[
https://issues.apache.org/jira/browse/HDFS-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16805130#comment-16805130
]
Erik Krogen commented on HDFS-14245:
------------------------------------
Hey [~shv], I don't think that actually makes sense. Using {{HAUtil}} will work
fine at the moment of instantiation, but if the active changes later, this
proxy will be rendered completely unusable, as described in the Javadoc for
{{HAUtil.getAddressOfActive}}:
{code}
* Get the internet address of the currently-active NN. This should rarely be
* used, since callers of this method who connect directly to the NN using the
* resulting InetSocketAddress will not be able to connect to the active NN if
* a failover were to occur after this method has been called.
{code}
I also don't like this solution because it isn't very general. I am worried
that we will continue to play whack-a-mole with all of the uses of the
{{NamenodeProtocols}} which go through an HA proxy but are not
{{ClientProtocol}} (reminder that the NameNode implements 12 protocols in
total...). I think there are better ways to solve this:
* We can add a mechanism to {{NameNodeProxies}} to instantiate an entire
`NamenodeProtocols`, which implements _all_ of the NN protocols (using
{{ProxyCombiner}} from HDFS-14162), and then {{GetGroups}} (and other similar
uses) can simply request a {{NamenodeProtocols}} proxy which can then by cast
as any of the individual protocols. This will work, and should be fairly
simple, but may not have good performance as the {{ProxyCombiner}} simply tries
all of its underlying protocols via reflection until it finds one that is
valid. One way around this would be to have a way for the caller to specify a
"primary" protocol which always gets checked first. This still has the downside
that we have to change all locations which currently use one of the other
protocols via an HA proxy.
* We can change ORPP such that it will work for non-{{ClientProtocol}} proxies,
but in this case, never use the observers (always fall back to the active
proxy). This is pretty simple, and since uses of protocols besides Client are
pretty light, the performance difference shouldn’t matter. This doesn't require
any changes at the caller sites.
I am leaning towards the second: it is simple, and requires changes only within
{{ObserverReadProxyProvider}} rather than at any caller site. But I am open to
opinions.
> Class cast error in GetGroups with ObserverReadProxyProvider
> ------------------------------------------------------------
>
> Key: HDFS-14245
> URL: https://issues.apache.org/jira/browse/HDFS-14245
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: HDFS-12943
> Reporter: Shen Yinjie
> Assignee: Shen Yinjie
> Priority: Major
> Attachments: HDFS-14245.patch
>
>
> Run "hdfs groups" with ObserverReadProxyProvider, Exception throws as :
> {code:java}
> Exception in thread "main" java.io.IOException: Couldn't create proxy
> provider class
> org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
> at
> org.apache.hadoop.hdfs.NameNodeProxiesClient.createFailoverProxyProvider(NameNodeProxiesClient.java:261)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:119)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:95)
> at org.apache.hadoop.hdfs.tools.GetGroups.getUgmProtocol(GetGroups.java:87)
> at org.apache.hadoop.tools.GetGroupsBase.run(GetGroupsBase.java:71)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.GetGroups.main(GetGroups.java:96)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hdfs.NameNodeProxiesClient.createFailoverProxyProvider(NameNodeProxiesClient.java:245)
> ... 7 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hdfs.server.namenode.ha.NameNodeHAProxyFactory cannot be
> cast to org.apache.hadoop.hdfs.server.namenode.ha.ClientHAProxyFactory
> at
> org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider.<init>(ObserverReadProxyProvider.java:123)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider.<init>(ObserverReadProxyProvider.java:112)
> ... 12 more
> {code}
> similar with HDFS-14116, we did a simple fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
