Jyotirmoy Sinha created HDDS-7955:
-------------------------------------
Summary: [snapshot] SnapshotDiff of 2 snapshots of a 2nd
volume-bucket throws java.util.ConcurrentModificationException
Key: HDDS-7955
URL: https://issues.apache.org/jira/browse/HDDS-7955
Project: Apache Ozone
Issue Type: Bug
Reporter: Jyotirmoy Sinha
Steps :
# Create a volume-bucket and create 500 snapshots of it with 500 revisions.
# Create a 2nd volume-bucket and create 4 snapshots of it.
# Perform a snapshotDiff of any 2 snapshots of 2nd volume-bucket
Snapshot info -
{code:java}
[root@jspriv01-1 ~]# ozone fs -ls ofs://ozone1/vol2/buck1/.snapshot/snap1
Found 1 items
-rw-rw-rw- 3 om om 20844 2023-02-10 07:24
ofs://ozone1/vol2/buck1/.snapshot/snap1/key1
[root@jspriv01-1 ~]# ozone fs -ls ofs://ozone1/vol2/buck1/.snapshot/snap2
Found 2 items
-rw-rw-rw- 3 om om 20844 2023-02-10 07:24
ofs://ozone1/vol2/buck1/.snapshot/snap2/key1
-rw-rw-rw- 3 om om 9175 2023-02-10 07:25
ofs://ozone1/vol2/buck1/.snapshot/snap2/key2
[root@jspriv01-1 ~]# ozone fs -ls ofs://ozone1/vol2/buck1/.snapshot/snap3
Found 1 items
-rw-rw-rw- 3 om om 532 2023-02-10 07:26
ofs://ozone1/vol2/buck1/.snapshot/snap3/key1
[root@jspriv01-1 ~]# ozone fs -ls ofs://ozone1/vol2/buck1/.snapshot/snap4
Found 2 items
-rw-rw-rw- 3 om om 532 2023-02-10 07:26
ofs://ozone1/vol2/buck1/.snapshot/snap4/key1
-rw-rw-rw- 3 om om 532 2023-02-10 07:27
ofs://ozone1/vol2/buck1/.snapshot/snap4/key2 {code}
SnapshotDiff output -
{code:java}
[root@jspriv01-1 ~]# ozone sh snapshot snapshotDiff vol2/buck1 snap1 snap2
23/02/10 11:04:12 INFO retry.RetryInvocationHandler:
com.google.protobuf.ServiceException:
org.apache.hadoop.ipc.RemoteException(java.util.ConcurrentModificationException):
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)
at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.filterRelevantSstFiles(RocksDBCheckpointDiffer.java:826)
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.getSSTDiffList(RocksDBCheckpointDiffer.java:818)
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.getSSTDiffListWithFullPath(RocksDBCheckpointDiffer.java:763)
at
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.getDeltaFiles(SnapshotDiffManager.java:231)
at
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.getSnapshotDiffReport(SnapshotDiffManager.java:134)
at
org.apache.hadoop.ozone.om.OmSnapshotManager.getSnapshotDiffReport(OmSnapshotManager.java:250)
at
org.apache.hadoop.ozone.om.OzoneManager.snapshotDiff(OzoneManager.java:4381)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.snapshotDiff(OzoneManagerRequestHandler.java:1216)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:298)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:223)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:177)
at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
, while invoking $Proxy17.submitRequest over
nodeId=om2,nodeAddress=jspriv01-7.jspriv01.root.hwx.site:9862. Trying to
failover immediately. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
