[
https://issues.apache.org/jira/browse/HDDS-9207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758304#comment-17758304
]
George Jahad commented on HDDS-9207:
------------------------------------
This error can cause the following exception:
{{java.lang.IllegalStateException: java.io.IOException: Failed init RocksDB, db
path :
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9,
exception :org.rocksdb.RocksDBException file is too short (1311480 bytes) to
be an
sstable/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/266072.sst
in file
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/MANIFEST-266083}}
{{ at
org.apache.hadoop.ozone.om.snapshot.SnapshotCache.lambda$1(SnapshotCache.java:177)}}
{{ at
java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)}}
{{ at
org.apache.hadoop.ozone.om.snapshot.SnapshotCache.get(SnapshotCache.java:165)}}
{{ at
org.apache.hadoop.ozone.om.OmSnapshotManager.checkForSnapshot(OmSnapshotManager.java:688)}}
{{ at
org.apache.hadoop.ozone.om.response.key.OMKeyPurgeResponse.addToDBBatch(OMKeyPurgeResponse.java:85)}}
{{ at
org.apache.hadoop.ozone.om.response.OMClientResponse.checkAndUpdateDB(OMClientResponse.java:73)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.lambda$5(OzoneManagerDoubleBuffer.java:409)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatchWithTrace(OzoneManagerDoubleBuffer.java:237)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatch(OzoneManagerDoubleBuffer.java:408)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushBatch(OzoneManagerDoubleBuffer.java:335)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushCurrentBuffer(OzoneManagerDoubleBuffer.java:314)}}
{{ at
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushTransactions(OzoneManagerDoubleBuffer.java:279)}}
{{ at java.lang.Thread.run(Thread.java:750)}}
{{Caused by: java.io.IOException: Failed init RocksDB, db path :
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9,
exception :org.rocksdb.RocksDBException file is too short (1311480 bytes) to
be an
sstable/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/266072.sst
in file
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/MANIFEST-266083}}
{{ at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:174)}}
{{ at
org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:220)}}
{{ at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:591)}}
{{ at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:399)}}
{{ at
org.apache.hadoop.ozone.om.OmSnapshotManager$1.load(OmSnapshotManager.java:356)}}
{{ at
org.apache.hadoop.ozone.om.OmSnapshotManager$1.load(OmSnapshotManager.java:1)}}
{{ at
org.apache.hadoop.ozone.om.snapshot.SnapshotCache.lambda$1(SnapshotCache.java:169)}}
{{ ... 12 more}}
{{Caused by: java.io.IOException: class
org.apache.hadoop.hdds.utils.db.RocksDatabase: Failed to open
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9;
status : Corruption; message : file is too short (1311480 bytes) to be an
sstable/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/266072.sst
in file
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/MANIFEST-266083}}
{{ at
org.apache.hadoop.hdds.utils.HddsServerUtil.toIOException(HddsServerUtil.java:667)}}
{{ at
org.apache.hadoop.hdds.utils.db.RocksDatabase.toIOException(RocksDatabase.java:90)}}
{{ at
org.apache.hadoop.hdds.utils.db.RocksDatabase.open(RocksDatabase.java:167)}}
{{ at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:112)}}
{{ ... 18 more}}
{{Caused by: org.rocksdb.RocksDBException: file is too short (1311480 bytes) to
be an
sstable/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/266072.sst
in file
/hadoop/ozone/data/disk1/om/db.snapshots/checkpointState/om.db-60fe8379-01d3-4a59-a8c4-bcaf9f600ee9/MANIFEST-266083}}
{{ at org.rocksdb.RocksDB.open(Native Method)}}
{{ at org.rocksdb.RocksDB.open(RocksDB.java:307)}}
{{ at
org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB.open(ManagedRocksDB.java:75)}}
{{ at
org.apache.hadoop.hdds.utils.db.RocksDatabase.open(RocksDatabase.java:153)}}
{{ ... 19 more}}
> Snapshot Bootstrap creates incorrect hard links.
> ------------------------------------------------
>
> Key: HDDS-9207
> URL: https://issues.apache.org/jira/browse/HDDS-9207
> Project: Apache Ozone
> Issue Type: Bug
> Components: Snapshot
> Reporter: George Jahad
> Assignee: George Jahad
> Priority: Critical
>
> The Snapshot bootstrap tarball creation code was incorrectly assuming that
> two sst files with the same name, (but different directories,) were hard
> linked to each other.
> This can be false in the following way:
> Suppose the active fs and snap1 consist of sst files: s1.sst-s10.sst. Then
> some delete records get added to snap1. Those get put into s11.sst.
> Meanwhile more keyTable entries are added to the active fs; they could get
> put into a different file named also s11.sst.
> To remedy this, the tarball creator must check the INodes of each file to
> confirm they are equal before linking them together.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
