Botong Huang created YARN-8010:

             Summary: add config in FederationRMFailoverProxy to not bypass 
facade cache when failing over
                 Key: YARN-8010
             Project: Hadoop YARN
          Issue Type: Task
            Reporter: Botong Huang
            Assignee: Botong Huang

Today when YarnRM is failing over, the FederationRMFailoverProxy running in 
AMRMProxy will perform failover, try to get latest subcluster info from 
FederationStateStore and then retry connect to the latest YarnRM master. When 
calling getSubCluster() to FederationStateStoreFacade, it bypasses the cache 
with a flush flag. When YarnRM is failing over, every AM heartbeat thread 
creates a different thread inside FederationInterceptor, each of which keeps 
performing failover several times. This leads to a big spike of getSubCluster 
call to FederationStateStore. 

Depending on the cluster setup (e.g. putting a VIP before all YarnRMs), YarnRM 
master slave change might not result in RM addr change. In other cases, a small 
delay of getting latest subcluster information may be acceptable. This patch 
thus creates a config option, so that it is possible to ask the 
FederationRMFailoverProxy to not flush cache when calling getSubCluster(). 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to