adoroszlai commented on a change in pull request #2900:
URL: https://github.com/apache/ozone/pull/2900#discussion_r767133363
##########
File path:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerGrpc.java
##########
@@ -195,10 +195,10 @@ public void start() throws IOException {
public void stop() {
if (isStarted) {
try {
+ server.shutdown();
Review comment:
Thanks @smengcl for the review. Initially I also suspected
`readExecutors`, but it's not really the problem.
`server.shutdown()` triggers shutdown in the server's underlying Netty
transport. The server's `terminated` flag is set via callbacks,
`server.awaitTermination()` waits for this flag to be set. It seems that this
is not happening if `eventLoopGroup` has already been shut down.
This can be reproduced easily (on current `master`) by changing
`server.awaitTermination()` to few minutes instead of few seconds. Then we can
create thread dump and see these threads being stuck at:
```
"ForkJoinPool.commonPool-worker-8" #358 daemon prio=5 os_prio=31
tid=0x00007fda0dc72800 nid=0x31d03 in Object.wait() [0x0000700026cc8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:460)
at java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:348)
at
org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl.awaitTermination(ServerImpl.java:319)
- locked <0x00000007ac151340> (a java.lang.Object)
at
org.apache.hadoop.ozone.container.common.transport.server.XceiverServerGrpc.stop(XceiverServerGrpc.java:207)
at
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.stop(OzoneContainer.java:329)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
