Gary Helmling created HBASE-15856:
-------------------------------------
Summary: Cached Connection instances can wind up with addresses
never resolved
Key: HBASE-15856
URL: https://issues.apache.org/jira/browse/HBASE-15856
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Critical
During periods where DNS is not working properly, we can wind up caching
connections to master or regionservers where the initial hostname resolution
and the resolution is never re-attempted. This means that clients will forever
get UnknownHostException for any calls.
When constructing a BlockingRpcChannelImplementation, we instantiate the
InetSocketAddress to use for the connection. This instance is then used in the
rpc client connection, where we check isUnresolved() and throw an
UnknownHostException if that returns true. However, at this point the rpc
channel is already cached in the HConnectionImplementation map of stubs. So at
this point it will never be resolved.
Setting the config for hbase.resolve.hostnames.on.failure masks this issue,
since the stub key used is modified to contain the address. However, even in
that case, if DNS fails, an rpc channel instance with unresolved ISA will still
be cached in the stubs under the hostname only key.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
