[
https://issues.apache.org/jira/browse/HDFS-13274?focusedWorklogId=792367&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-792367
]
ASF GitHub Bot logged work on HDFS-13274:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Jul/22 22:05
Start Date: 18/Jul/22 22:05
Worklog Time Spent: 10m
Work Description: goiri commented on code in PR #4531:
URL: https://github.com/apache/hadoop/pull/4531#discussion_r923893873
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestConnectionManager.java:
##########
@@ -304,7 +343,7 @@ private void testConnectionCleanup(float ratio, int
totalConns,
addConnectionsToPool(pool, totalConns - 1, activeConns - 1);
// There are activeConn connections.
- // We can cleanup the pool
+ // We can clean up the pool
Review Comment:
"cleanup" is correct
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionPool.java:
##########
@@ -437,23 +451,31 @@ protected static <T> ConnectionContext
newConnection(Configuration conf,
}
InetSocketAddress socket = NetUtils.createSocketAddr(nnAddress);
final long version = RPC.getProtocolVersion(classes.protoPb);
- Object proxy = RPC.getProtocolProxy(classes.protoPb, version, socket, ugi,
- conf, factory, RPC.getRpcTimeout(conf), defaultPolicy,
null).getProxy();
+ Object proxy;
+ if (enableMultiSocket) {
+ FederationConnectionId connectionId = new FederationConnectionId(
+ socket, classes.protoPb, ugi, RPC.getRpcTimeout(conf),
+ defaultPolicy, conf, socketIndex);
+ proxy = RPC.getProtocolProxy(classes.protoPb, version, connectionId,
+ conf, factory).getProxy();
+ } else {
+ proxy = RPC.getProtocolProxy(classes.protoPb, version, socket, ugi,
+ conf, factory, RPC.getRpcTimeout(conf), defaultPolicy,
null).getProxy();
+ }
+
T client = newProtoClient(proto, classes, proxy);
Text dtService = SecurityUtil.buildTokenService(socket);
- ProxyAndInfo<T> clientProxy =
- new ProxyAndInfo<T>(client, dtService, socket);
- ConnectionContext connection = new ConnectionContext(clientProxy);
- return connection;
+ ProxyAndInfo<T> clientProxy = new ProxyAndInfo<T>(client, dtService,
socket);
+ return new ConnectionContext(clientProxy, maxConcurrencyPerConn);
Review Comment:
Could we make it more generic and pass the `conf` instead of ad-hoc
arguments?
Issue Time Tracking
-------------------
Worklog Id: (was: 792367)
Time Spent: 1h 40m (was: 1.5h)
> RBF: Extend RouterRpcClient to use multiple sockets
> ---------------------------------------------------
>
> Key: HDFS-13274
> URL: https://issues.apache.org/jira/browse/HDFS-13274
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Íñigo Goiri
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> HADOOP-13144 introduces the ability to create multiple connections for the
> same user and use different sockets. The RouterRpcClient should use this
> approach to get a better throughput.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
