[
https://issues.apache.org/jira/browse/HDFS-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929662#comment-17929662
]
ASF GitHub Bot commented on HDFS-17735:
---------------------------------------
hfutatzhanghb commented on PR #7422:
URL: https://github.com/apache/hadoop/pull/7422#issuecomment-2677929000
> Hi, @hfutatzhanghb Regarding this fix LGTM! The current implementation of
the getSubclusterInfo method involves an asynchronous call to
getDatanodesSubcluster(). Here's the relevant code snippet:
>
> ```java
> // Fetch the mapping asynchronously
> Thread updater = new Thread(new Runnable() {
> @Override
> public void run() {
> final MembershipStore membershipStore = getMembershipStore();
> if (membershipStore == null) {
> LOG.error("Cannot access the Membership store.");
> return;
> }
>
> subclusterMapping = getSubclusterInfo(membershipStore);
> lastUpdated = monotonicNow();
> }
> });
> updater.start();
>
> // Wait until initialized
> if (subclusterMapping == null) {
> try {
> LOG.debug("Wait to get the mapping for the first time");
> updater.join();
> } catch (InterruptedException e) {
> LOG.error("Cannot wait for the updater to finish");
> }
> }
> ```
>
> When subclusterMapping is null or has not been updated for a long time, an
asynchronous thread is started to update subclusterMapping, and the handler
thread waits until subclusterMapping is updated.
>
> Instead of starting an asynchronous thread to update subclusterMapping
when it is null or stale, and having the handler thread wait for the update to
complete, we can modify the approach as follows:
>
> 1. During router startup, fetch and update subclusterMapping once.
> 2. Start a periodic thread that regularly updates subclusterMapping.
> 3. Handler threads can read from subclusterMapping without waiting for
updates, as the periodic thread ensures that subclusterMapping is kept
up-to-date.
>
> This approach ensures that the handler threads are not blocked while
waiting for subclusterMapping to be updated, improving the overall performance
and responsiveness of the system.
@KeeProMise Sir, very nice idea. I think we can implement it in the future.
> [ARR] LocalResolver#getDatanodesSubcluster adapts to async rpc
> --------------------------------------------------------------
>
> Key: HDFS-17735
> URL: https://issues.apache.org/jira/browse/HDFS-17735
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
> Labels: pull-request-available
>
> LocalResolver#getDatanodesSubcluster adapts to async rpc
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
