[ 
https://issues.apache.org/jira/browse/HBASE-30376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18113814#comment-18113814
 ] 

Daniel Roudnitsky commented on HBASE-30376:
-------------------------------------------

Hi Longping, I’m not convinced HBase should maintain its own DNS cache here. 
Could stale answer behavior can be provided by the system resolver, for example 
Unbound with serve-expired, without adding another cache above the JVM and OS 
resolver ? 

> Tolerate transient DNS server failures with an opt-in client-side DNS 
> resolution cache (serve-stale fallback) for RPC connections
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-30376
>                 URL: https://issues.apache.org/jira/browse/HBASE-30376
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>    Affects Versions: 2.6.7
>            Reporter: Longping Jie
>            Assignee: Longping Jie
>            Priority: Major
>
> **Problem**
> The client resolves RegionServer/Master hostnames via `java.net.InetAddress` 
> whenever a new RPC connection is established, and twice more in the Kerberos 
> security preamble (a second forward lookup plus a PTR lookup for principal 
> `_HOST`). When the DNS server is intermittently unstable, new connections 
> fail with `UnknownHostException` until DNS recovers (retry window is only 
> tens of seconds), the blocking lookup stalls the Netty event loop and all 
> connections sharing it, and Kerberos connections fail on PTR lookups even 
> when TCP connect succeeded.
> **Fix**
> Add an opt-in cache in hbase-client with serve-stale semantics: live DNS 
> always first; on resolution failure fall back to the last successful mapping, 
> age-bounded (default 10 min) and kept fresh by a daemon background refresher 
> (degraded hosts re-probed every 2s); when no usable cache exists, throw the 
> original `UnknownHostException` unchanged. Two hook points only 
> (`RpcConnection#getRemoteInetAddress`, `RpcConnection#getServerPrincipals`), 
> so Kerberos and non-Kerberos behave the same. No server-side or wire-protocol 
> changes, no new dependencies, JDK 8/17 compatible. Gated by 
> `hbase.ipc.client.dns.fallback-cache.enabled` (default `false` = original 
> code path).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to