Hi folks, Want to bring the discussion to add a new call in the HDFS client protocol. (see the changes https://github.com/apache/hadoop/pull/3595/files).
*Context*: Currently in federated HDFS, since the routers don't know which subclusters the client is writing files to, the routers have to make a fan-out call to all the subclusters. If one of the subclusters is slow, then the routers will accumulate such renewLease calls and eventually cause router degradation. *Proposal*: We are proposing to add a new call as follows in client protocol to make renewLease call against specific subcluster. It maintains backward compatibility: 1) existing clients still make the fan-out call (renewLease(String clientName)); 2) The new clients can work with old/new versions of routers. void renewLease(String clientName, String nsId) throws IOException; Please comment. Thanks, Aihua ---------- Forwarded message --------- From: Inigo Goiri <notificati...@github.com> Date: Thu, Oct 28, 2021 at 12:09 PM Subject: Re: [apache/hadoop] HDFS-16283: RBF: improve renewLease() to call only a specific NameNod… (PR #3595) To: apache/hadoop <had...@noreply.github.com> Cc: aihuaxu <aihu...@gmail.com>, Author <aut...@noreply.github.com> *@goiri* requested changes on this pull request. ------------------------------ In hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java <https://github.com/apache/hadoop/pull/3595#discussion_r738674897>: > @@ -765,6 +765,14 @@ BatchedDirectoryListing getBatchedListing( @Idempotent void renewLease(String clientName) throws IOException; + /** + * The functionality is the same as renewLease(clientName). This is to support + * router based FileSystem to newLease against a specific target FileSystem instead + * of all the target FileSystems in each call. + */ + @Idempotent + void renewLease(String clientName, String nsId) throws IOException; Changing the client protocol requires more than just a RBF PR. We should bring this topic up in the mailing list. The good news is that this is an additional call and not modifying the existing. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/apache/hadoop/pull/3595#pullrequestreview-792299305>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGKDWK5QZVNDDH6TAN7Q2ADUJGNV5ANCNFSM5G3KXPFQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
