[
https://issues.apache.org/jira/browse/HDDS-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17808005#comment-17808005
]
Hemant Kumar commented on HDDS-10149:
-------------------------------------
I ran it 10x10 times and there was no failure
[https://github.com/hemantk-12/ozone/actions/runs/7563557065]
Similar issue was report in HDDS-9711. When we looked into HDDS-9711, it
suspected that it could be because we limit number of files opened for a
snapshotted RocksDB to 100 and RocksDB might be closing file while it is read
by another thread. But in this scenario, it is on active FS's RocksDB and there
is no limit on number of open files for active FS's RocksDB
> Bad file descriptor in
> TestOmSnapshotFsoWithNativeLib.testSnapshotCompactionDag
> -------------------------------------------------------------------------------
>
> Key: HDDS-10149
> URL: https://issues.apache.org/jira/browse/HDDS-10149
> Project: Apache Ozone
> Issue Type: Bug
> Components: Snapshot
> Reporter: Attila Doroszlai
> Assignee: Swaminathan Balachandran
> Priority: Critical
> Attachments:
> org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithNativeLib-output.txt,
> org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithNativeLib.txt
>
>
> {{TestOmSnapshotFsoWithNativeLib}} failed in a CI run where it was really
> using the native library:
> {code}
> org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithNativeLib.testSnapshotCompactionDag
> -- Time elapsed: 2.844 s <<< ERROR!
> java.io.IOException:
> RocksDatabase[/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-d6a9ee09-9b79-4e8b-9241-4df752521eb9/ozone-meta/om.db]:
> Failed to compactRange; status : IOError; message : While pread offset 0 len
> 176:
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-d6a9ee09-9b79-4e8b-9241-4df752521eb9/ozone-meta/om.db/000508.sst:
> Bad file descriptor
> at
> org.apache.hadoop.hdds.utils.HddsServerUtil.toIOException(HddsServerUtil.java:667)
> at
> org.apache.hadoop.hdds.utils.db.RocksDatabase.toIOException(RocksDatabase.java:98)
> at
> org.apache.hadoop.hdds.utils.db.RocksDatabase.compactRange(RocksDatabase.java:595)
> at
> org.apache.hadoop.hdds.utils.db.RocksDatabase.compactDB(RocksDatabase.java:551)
> at org.apache.hadoop.hdds.utils.db.RDBStore.compactDB(RDBStore.java:216)
> at
> org.apache.hadoop.ozone.om.snapshot.TestOmSnapshot.testSnapshotCompactionDag(TestOmSnapshot.java:2415)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> Caused by: org.rocksdb.RocksDBException: While pread offset 0 len 176:
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-d6a9ee09-9b79-4e8b-9241-4df752521eb9/ozone-meta/om.db/000508.sst:
> Bad file descriptor
> at org.rocksdb.RocksDB.compactRange(Native Method)
> at org.rocksdb.RocksDB.compactRange(RocksDB.java:3416)
> at
> org.apache.hadoop.hdds.utils.db.RocksDatabase.compactRange(RocksDatabase.java:592)
> ... 6 more
> {code}
> CC [~smeng], [~hemantk]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
