Lokesh Jain created HDDS-5248:
---------------------------------
Summary: SCM HA: Continuous PipelineNotFoundException seen in SCM
log
Key: HDDS-5248
URL: https://issues.apache.org/jira/browse/HDDS-5248
Project: Apache Ozone
Issue Type: Bug
Components: SCM, SCM HA
Reporter: Nilotpal Nandi
Assignee: Lokesh Jain
After running decommissioning tests and aborting few decommissioning command,
seeing PipelineNotFoundException INFO logs continuously in scm log.
Although these are INFO logs, raising the issue as the exceptions are thrown in
loop.
{code:java}
2021-05-16 20:06:53,421 INFO org.apache.hadoop.ipc.Server: IPC Server handler
53 on 9860, call Call#102 Retry#23
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocol.submitRequest
from 172.27.176.196:41982
org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException:
PipelineID=b1431673-adf0-402f-968f-e9241ea37609 not found
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.getPipeline(PipelineStateMap.java:121)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateManagerV2Impl.getPipeline(PipelineStateManagerV2Impl.java:125)
at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.hadoop.hdds.scm.ha.SCMHAInvocationHandler.invokeLocal(SCMHAInvocationHandler.java:83)
at
org.apache.hadoop.hdds.scm.ha.SCMHAInvocationHandler.invoke(SCMHAInvocationHandler.java:68)
at com.sun.proxy.$Proxy14.getPipeline(Unknown Source)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineManagerV2Impl.getPipeline(PipelineManagerV2Impl.java:198)
at
org.apache.hadoop.hdds.scm.server.SCMClientProtocolServer.getPipeline(SCMClientProtocolServer.java:529)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.getPipeline(StorageContainerLocationProtocolServerSideTranslatorPB.java:497)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.processRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:302)
at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.submitRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:153)
at
org.apache.hadoop.hdds.protocol.proto.StorageContainerLocationProtocolProtos$StorageContainerLocationProtocolService$2.callBlockingMethod(StorageContainerLocationProtocolProtos.java:46400)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:986)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:914)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2887)
2021-05-16 20:06:55,426 INFO org.apache.hadoop.ipc.Server: IPC Server handler
84 on 9860, call Call#102 Retry#24
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocol.submitRequest
from 172.27.176.196:41982
org.apache.hadoop.hdds.scm.pipeline.PipelineNotFoundException:
PipelineID=b1431673-adf0-402f-968f-e9241ea37609 not found
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.getPipeline(PipelineStateMap.java:121)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineStateManagerV2Impl.getPipeline(PipelineStateManagerV2Impl.java:125)
at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.hadoop.hdds.scm.ha.SCMHAInvocationHandler.invokeLocal(SCMHAInvocationHandler.java:83)
at
org.apache.hadoop.hdds.scm.ha.SCMHAInvocationHandler.invoke(SCMHAInvocationHandler.java:68)
at com.sun.proxy.$Proxy14.getPipeline(Unknown Source)
at
org.apache.hadoop.hdds.scm.pipeline.PipelineManagerV2Impl.getPipeline(PipelineManagerV2Impl.java:198)
at
org.apache.hadoop.hdds.scm.server.SCMClientProtocolServer.getPipeline(SCMClientProtocolServer.java:529)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.getPipeline(StorageContainerLocationProtocolServerSideTranslatorPB.java:497)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.processRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:302)
at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
at
org.apache.hadoop.hdds.scm.protocol.StorageContainerLocationProtocolServerSideTranslatorPB.submitRequest(StorageContainerLocationProtocolServerSideTranslatorPB.java:153)
at
org.apache.hadoop.hdds.protocol.proto.StorageContainerLocationProtocolProtos$StorageContainerLocationProtocolService$2.callBlockingMethod(StorageContainerLocationProtocolProtos.java:46400)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:986)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:914)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2887)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
