Jyotirmoy Sinha created HDDS-10595:
--------------------------------------
Summary: [snapshot-LR] OM node is not shown in 'om roles' as
follower node
Key: HDDS-10595
URL: https://issues.apache.org/jira/browse/HDDS-10595
Project: Apache Ozone
Issue Type: Bug
Components: OM, Snapshot
Reporter: Jyotirmoy Sinha
OM node is not shown in 'om roles' as follower node
OM roles - CLI -
{code:java}
[root@node~]# ozone admin om roles -id=ozone1709211738
om233 : FOLLOWER (node1.domain.com)
om232 : LEADER (node2.domain.com) {code}
OM log output -
{code:java}
2024-03-19 00:00:01,203 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: Failed appendEntries as snapshot (30450723)
installation is in progress
2024-03-19 00:00:01,203 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: inconsistency entries.
Reply:om232<-om363#14732148:FAIL-t54,INCONSISTENCY,nextIndex=0,followerCommit=-1,matchIndex=-1
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: receive installSnapshot:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: reply installSnapshot:
om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastRequest:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastReply: null
2024-03-19 00:00:01,203 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: Failed appendEntries as snapshot (30450723)
installation is in progress
2024-03-19 00:00:01,203 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: inconsistency entries.
Reply:om232<-om363#14732149:FAIL-t54,INCONSISTENCY,nextIndex=0,followerCommit=-1,matchIndex=-1
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: receive installSnapshot:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: reply installSnapshot:
om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastRequest:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,203 INFO
[grpc-default-executor-6]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastReply: null
2024-03-19 00:00:01,204 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: Failed appendEntries as snapshot (30450723)
installation is in progress
2024-03-19 00:00:01,204 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: inconsistency entries.
Reply:om232<-om363#14732150:FAIL-t54,INCONSISTENCY,nextIndex=0,followerCommit=-1,matchIndex=-1
2024-03-19 00:00:01,204 INFO
[grpc-default-executor-7]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: receive installSnapshot:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,204 INFO
[grpc-default-executor-7]-org.apache.ratis.server.impl.SnapshotInstallationHandler:
om363@group-A826AF593A36: reply installSnapshot:
om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:00:01,204 INFO
[grpc-default-executor-7]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastRequest:
om232->om234#0-t54,notify:(t:52, i:30450723)
2024-03-19 00:00:01,204 INFO
[grpc-default-executor-7]-org.apache.ratis.grpc.server.GrpcServerProtocolService:
om363: Completed INSTALL_SNAPSHOT, lastReply: null
2024-03-19 00:00:01,204 INFO
[om363-server-thread17]-org.apache.ratis.server.RaftServer$Division:
om363@group-A826AF593A36: Failed appendEntries as snapshot (30450723)
installation is in progress {code}
OM Leader log output pertaining to above OM Follower -
{code:java}
2024-03-19 00:03:11,531 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,531 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,532 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,532 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,533 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,534 INFO
[grpc-default-executor-26]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,534 INFO
[grpc-default-executor-26]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS
2024-03-19 00:03:11,535 INFO
[grpc-default-executor-75]-org.apache.ratis.grpc.server.GrpcLogAppender:
om232@group-A826AF593A36->om234-InstallSnapshotResponseHandler: received a
reply om232<-om363#0:FAIL-t54,IN_PROGRESS {code}
>From logs the snapshot installation is failing in the the follower OM, this
>log is flooding in the follower OM and similar error is present in leader
>indicating the snapshot-installation is failing/in-progress.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
