[
https://issues.apache.org/jira/browse/HDFS-16394?focusedWorklogId=699841&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-699841
]
ASF GitHub Bot logged work on HDFS-16394:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Dec/21 08:21
Start Date: 22/Dec/21 08:21
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #3822:
URL: https://github.com/apache/hadoop/pull/3822#discussion_r773684705
##########
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
##########
@@ -1107,6 +1108,45 @@ public TestRpcService run() {
}
}
+ @Test
+ public void testNumInProcessHandlerMetrics() throws Exception {
+ UserGroupInformation ugi = UserGroupInformation.
+ createUserForTesting("user123", new String[0]);
+ // use 1 handler so the callq can be plugged
+ final Server server = setupTestServer(conf, 1);
+ try {
+ RpcMetrics rpcMetrics = server.getRpcMetrics();
+ assertEquals(rpcMetrics.getNumInProcessHandler(), 0);
+
+ ExternalCall<String> call1 = newExtCall(ugi,
+ new PrivilegedExceptionAction<String>() {
+ @Override
+ public String run() throws Exception {
+ assertTrue(rpcMetrics.getNumInProcessHandler() > 0);
Review comment:
How about exact comparison? We can use `assertEquals(1,
rpcMetrics.getNumInProcessHandler())`
##########
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
##########
@@ -1107,6 +1108,45 @@ public TestRpcService run() {
}
}
+ @Test
+ public void testNumInProcessHandlerMetrics() throws Exception {
+ UserGroupInformation ugi = UserGroupInformation.
+ createUserForTesting("user123", new String[0]);
+ // use 1 handler so the callq can be plugged
+ final Server server = setupTestServer(conf, 1);
+ try {
+ RpcMetrics rpcMetrics = server.getRpcMetrics();
+ assertEquals(rpcMetrics.getNumInProcessHandler(), 0);
+
+ ExternalCall<String> call1 = newExtCall(ugi,
+ new PrivilegedExceptionAction<String>() {
Review comment:
nit: use lambda?
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##########
@@ -3057,8 +3062,24 @@ private void internalQueueCall(Call call, boolean
blocking)
}
}
+ /**
+ * Define some states related to Handler.
+ */
+ private enum HandlerStatus {
+ /**Working for a Call.*/
+ IN_PROCESS,
+
+ /**Idle state, waiting for a new call.*/
+ IN_IDLE
+ }
Review comment:
Do we need this enum? We are updating the status to either `IN_PROCESS`
or `IN_IDLE` but not really exposing it anywhere.
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##########
@@ -498,6 +498,7 @@ protected ResponseBuffer initialValue() {
private Map<Integer, Listener> auxiliaryListenerMap;
private Responder responder = null;
private Handler[] handlers = null;
+ final private AtomicInteger numInProcessHandler = new AtomicInteger();
Review comment:
nit: `private final`
##########
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
##########
@@ -1107,6 +1108,45 @@ public TestRpcService run() {
}
}
+ @Test
+ public void testNumInProcessHandlerMetrics() throws Exception {
+ UserGroupInformation ugi = UserGroupInformation.
+ createUserForTesting("user123", new String[0]);
+ // use 1 handler so the callq can be plugged
+ final Server server = setupTestServer(conf, 1);
+ try {
+ RpcMetrics rpcMetrics = server.getRpcMetrics();
+ assertEquals(rpcMetrics.getNumInProcessHandler(), 0);
+
+ ExternalCall<String> call1 = newExtCall(ugi,
+ new PrivilegedExceptionAction<String>() {
+ @Override
+ public String run() throws Exception {
+ assertTrue(rpcMetrics.getNumInProcessHandler() > 0);
+ return UserGroupInformation.getCurrentUser().getUserName();
+ }
+ });
+ ExternalCall<Void> call2 = newExtCall(ugi,
+ new PrivilegedExceptionAction<Void>() {
Review comment:
same as above, lambda and assertEquals for exact comparison
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 699841)
Time Spent: 0.5h (was: 20m)
> RPCMetrics increases the number of handlers in processing
> ---------------------------------------------------------
>
> Key: HDFS-16394
> URL: https://issues.apache.org/jira/browse/HDFS-16394
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When using RPC, we recorded a lot of useful information, such as Queue time,
> Processing time. These are very helpful.
> But we can't know how many handlers are actually working now (only those that
> handle Call), especially when the Call Queue is very high. This is also not
> conducive to us optimizing the cluster.
> It would be very helpful if we can see the number of handlers being processed
> in RPCMetrics.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
