[
https://issues.apache.org/jira/browse/HDFS-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676432#comment-17676432
]
ASF GitHub Bot commented on HDFS-16890:
---------------------------------------
simbadzina opened a new pull request, #5298:
URL: https://github.com/apache/hadoop/pull/5298
HDFS-16890: Ensures router periodically refreshes its record of a
namespace's state.
### Description of PR
When using the ObserverReadProxyProvider, clients can set
dfs.client.failover.observer.auto-msync-period... to periodically get the
Active namenode's state. When using routers without the
ObserverReadProxyProvider, this periodic update is lost.
In a busy cluster, the Router constantly gets updated with the active
namenode's state when
There is a write operation.
There is an operation (read/write) from a new clients.
However, in the scenario when there are no new clients and no write
operations, the state kept in the router can lag behind the active's. The
router does update its state with responses from the Observer, but the observer
may be lagging behind too.
We should have a periodic refresh in the router to serve a similar role as
dfs.client.failover.observer.auto-msync-period
### How was this patch tested?
New test case.
TestObserverWithRouter#testPeriodicStateRefreshUsingActiveNamenode()
### For code changes:
- [ X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
> Add period state refresh to keep router state near active namenode's
> --------------------------------------------------------------------
>
> Key: HDFS-16890
> URL: https://issues.apache.org/jira/browse/HDFS-16890
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Simbarashe Dzinamarira
> Assignee: Simbarashe Dzinamarira
> Priority: Major
>
> When using the ObserverReadProxyProvider, clients can setÂ
> *dfs.client.failover.observer.auto-msync-period...* to periodically get the
> Active namenode's state. When using routers without the
> ObserverReadProxyProvider, this periodic update is lost.
> In a busy cluster, the Router constantly gets updated with the active
> namenode's state when
> # There is a write operation.
> # There is an operation (read/write) from a new clients.
> However, in the scenario when there are no new clients and no write
> operations, the state kept in the router can lag behind the active's. The
> router does update its state with responses from the Observer, but the
> observer may be lagging behind too.
> We should have a periodic refresh in the router to serve a similar role as
> *dfs.client.failover.observer.auto-msync-period*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
