[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=779765&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779765
]
ASF GitHub Bot logged work on HDFS-13522:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jun/22 03:44
Start Date: 09/Jun/22 03:44
Worklog Time Spent: 10m
Work Description: ZanderXu commented on code in PR #4311:
URL: https://github.com/apache/hadoop/pull/4311#discussion_r893033565
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -1091,6 +1091,10 @@ public void setDeferredResponse(Writable response) {
public void setDeferredError(Throwable t) {
}
+
+ public long getTimestampNanos() {
Review Comment:
It is best to replace all timestampNanos to getTimestampNanos().
Like, line583 and line3039.
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionManager.java:
##########
@@ -203,9 +210,16 @@ public ConnectionContext
getConnection(UserGroupInformation ugi,
try {
pool = this.pools.get(connectionId);
if (pool == null) {
+ if(!alignmentContexts.containsKey(nsId)) {
+ synchronized (alignmentContexts) {
+ if(!alignmentContexts.containsKey(nsId)) {
+ alignmentContexts.put(nsId, new ClientGSIContext());
+ }
+ }
+ }
Review Comment:
+1
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java:
##########
@@ -349,6 +349,18 @@ public static ClientProtocol
createProxyWithAlignmentContext(
boolean withRetries, AtomicBoolean fallbackToSimpleAuth,
AlignmentContext alignmentContext)
throws IOException {
+ if (!conf.getBoolean(HdfsClientConfigKeys.DFS_OBSERVER_READ_ENABLE,
Review Comment:
I have an idea on how to disable Observer Read.
- Proxy always with AlignmentContext, like ClientGSIContext or
RouterGSIContext
- Always update the lastSeenStateId from active
- ClientGSIContext has a boolean flag, like disableOberverRead
- If disableObserverRead is true, getLastSeenStateId() will always return a
special number, like -100
- And if we want dynamically enable ObserverRead for one ns, we just need to
set the disableObserverRead flag to false.
Our cluster with 5w+ DNs used this solution, and it is convenient to control
ObserverRead.
@simbadzina How do you think about this idea?
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterStateIdContext.java:
##########
@@ -0,0 +1,87 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdfs.server.federation.router;
+
+import java.lang.reflect.Method;
+import java.util.HashSet;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.hdfs.protocol.ClientProtocol;
+import org.apache.hadoop.hdfs.server.namenode.ha.ReadOnly;
+import org.apache.hadoop.ipc.AlignmentContext;
+import org.apache.hadoop.ipc.RetriableException;
+import org.apache.hadoop.ipc.protobuf.RpcHeaderProtos.RpcRequestHeaderProto;
+import org.apache.hadoop.ipc.protobuf.RpcHeaderProtos.RpcResponseHeaderProto;
+
+/**
+ * This is the router implementation responsible for passing
+ * client state id to next level.
+ */
[email protected]
[email protected]
+class RouterStateIdContext implements AlignmentContext {
Review Comment:
No used in this PR?
Issue Time Tracking
-------------------
Worklog Id: (was: 779765)
Time Spent: 10h 20m (was: 10h 10m)
> RBF: Support observer node from Router-Based Federation
> -------------------------------------------------------
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
> Reporter: Erik Krogen
> Assignee: Simbarashe Dzinamarira
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, RBF_ Observer support.pdf, Router+Observer RPC
> clogging.png, ShortTerm-Routers+Observer.png
>
> Time Spent: 10h 20m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
