[
https://issues.apache.org/jira/browse/HDFS-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633539#comment-17633539
]
ASF GitHub Bot commented on HDFS-16837:
---------------------------------------
ZanderXu commented on code in PR #5123:
URL: https://github.com/apache/hadoop/pull/5123#discussion_r1021133501
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientGSIContext.java:
##########
@@ -77,11 +83,51 @@ public void
updateResponseState(RpcResponseHeaderProto.Builder header) {
@Override
public synchronized void receiveResponseState(RpcResponseHeaderProto header)
{
if (header.hasRouterFederatedState()) {
- routerFederatedState = header.getRouterFederatedState();
+ routerFederatedState =
mergeRouterFederatedState(header.getRouterFederatedState());
} else {
lastSeenStateId.accumulate(header.getStateId());
}
}
+ /**
+ * Utility function to parse routerFederatedState field in RPC headers.
+ */
+ public static Map<String, Long> getRouterFederatedStateMap(ByteString
byteString) {
+ if (byteString == null) {
+ return Collections.emptyMap();
+ }
+
+ RouterFederatedStateProto federatedState;
+ try {
+ federatedState = RouterFederatedStateProto.parseFrom(byteString);
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException(e);
+ }
+ return federatedState.getNamespaceStateIdsMap();
+ }
+
+ /**
+ * Merge the local FederatedState and RemoteFederateState to get the max
value for each namespace.
+ * @param remoteState the remote RouterFederatedState.
+ * @return one ByteString object which contains the max value of each
namespace.
+ */
+ private ByteString mergeRouterFederatedState(ByteString remoteState) {
+ Map<String, Long> localMapping = new HashMap<>(
+ getRouterFederatedStateMap(this.routerFederatedState));
+ Map<String, Long> remoteMapping = getRouterFederatedStateMap(remoteState);
+ remoteMapping.forEach((k, v) -> {
+ long localValue = localMapping.getOrDefault(k, 0L);
+ localMapping.put(k, Math.max(v, localValue));
+ });
+
+ for (String ns : remoteMapping.keySet()) {
Review Comment:
Thanks, sir. Updated
> [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max
> state id for each namespace
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-16837
> URL: https://issues.apache.org/jira/browse/HDFS-16837
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
>
> ClientGSIContext should merge local and remote RouterFederatedState to get
> the max state id for each namespace.
> And the related code as bellows:
> {code:java}
> @Override
> public synchronized void receiveResponseState(RpcResponseHeaderProto header) {
> if (header.hasRouterFederatedState()) {
> // BUG here
> routerFederatedState = header.getRouterFederatedState();
> } else {
> lastSeenStateId.accumulate(header.getStateId());
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
