[
https://issues.apache.org/jira/browse/HDDS-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878596#comment-17878596
]
JiangHua Zhu commented on HDDS-11396:
-------------------------------------
The cause of this exception has been found.
When Datanode restarts, OzoneContainer will initialize many modules. Including
writeChannel, readChannel, hddsDispatcher, etc.
However, the operation on writeChannel comes first, and hddsDispatcher comes
later. Once the writeChannel is started, it can receive requests from clients.
Once the load on the machine where the Datanode is located is high,
hddsDispatcher has not had time to assign a value to clusterId. However, at
this time, some client requests arrive, and an exception will occur.
> NPE in Handler#clusterId
> ------------------------
>
> Key: HDDS-11396
> URL: https://issues.apache.org/jira/browse/HDDS-11396
> Project: Apache Ozone
> Issue Type: Bug
> Components: DN
> Affects Versions: 1.4.0
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2024-08-31-17-26-22-105.png, screenshot-1.png
>
>
> When KeyValueHandler executes handleCreateContainer, it shows that
> Handler#clusterId is null.
> Here are some logs:
> {code:java}
> 2024-08-31 13:29:30,924
> [1134f6e4-49f6-4831-ad13-6bdb8ea23409-ContainerOp-113adaf2-c5da-479c-b677-5ec11ac5d97a-2]
> WARN org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler: Operation:
> CreateContainer , Trace ID: , Message: java.lang.NullPointerException:
> clusterId cannot be null , Result: CONTAINER_INTERNAL_ERROR ,
> StorageContainerException Occurred.
> org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
> java.lang.NullPointerException: clusterId cannot be null
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.handle(KeyValueHandler.java:225)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.createContainer(HddsDispatcher.java:469)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatchRequest(HddsDispatcher.java:275)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.lambda$dispatch$0(HddsDispatcher.java:179)
> at
> org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:89)
> at
> org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatch(HddsDispatcher.java:178)
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.dispatchCommand(ContainerStateMachine.java:485)
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.lambda$applyTransaction$9(ContainerStateMachine.java:900)
> at org.apache.ratis.util.TaskQueue.lambda$submit$0(TaskQueue.java:121)
> at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:38)
> at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:78)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: java.lang.NullPointerException: clusterId cannot be null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921)
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueContainer.create(KeyValueContainer.java:148)
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.handleCreateContainer(KeyValueHandler.java:367)
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.dispatchRequest(KeyValueHandler.java:239)
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.handle(KeyValueHandler.java:222)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
