[
https://issues.apache.org/jira/browse/HDDS-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-8557:
-----------------------------------
Description:
{{RocksDBCheckpointDiffer}} has single instance per DB, which may cause failure
if MiniOzoneCluster reuses the same directory between runs. Apparently this
can happen if cluster is started in {{@BeforeAll}} method and test is repeated
due to {{@Flaky}} annotation. The result is that if the first run fails,
repeated attempts always run into the error.
{code:title=https://github.com/apache/ozone/actions/runs/4901282858/jobs/8752465727#step:6:3795}
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient Time elapsed: 0.649 s
<<< ERROR!
java.lang.RuntimeException: Failed to append compaction log to
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.appendToCurrentCompactionLog(RocksDBCheckpointDiffer.java:381)
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.setCurrentCompactionLog(RocksDBCheckpointDiffer.java:314)
at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:155)
at
org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:212)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:558)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:526)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:516)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:325)
at
org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:768)
at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:648)
at
org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:733)
at
org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.createOM(MiniOzoneClusterImpl.java:811)
at
org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.build(MiniOzoneClusterImpl.java:603)
at
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClientAbstract.startCluster(TestOzoneRpcClientAbstract.java:211)
at
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient.init(TestOzoneRpcClient.java:51)
...
Caused by: java.nio.file.NoSuchFileException:
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
{code}
was:
{{RocksDBCheckpointDiffer}} has single instance per DB, which may cause failure
if MiniOzoneCluster reuses the same directory between runs. Apparently this
can happen if cluster is started in {{@BeforeAll}} method and test is repeated
due to {{@Flaky}} annotation. The result is that if the first run fails,
repeated attempts always run into the error.
{code:title=https://github.com/apache/ozone/actions/runs/4901282858/jobs/8752465727#step:6:3795}
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient Time elapsed: 0.649 s
<<< ERROR!
java.lang.RuntimeException: Failed to append compaction log to
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.appendToCurrentCompactionLog(RocksDBCheckpointDiffer.java:381)
at
org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.setCurrentCompactionLog(RocksDBCheckpointDiffer.java:314)
at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:155)
at
org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:212)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:558)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:526)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:516)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:325)
at
org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:768)
at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:648)
at
org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:733)
at
org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.createOM(MiniOzoneClusterImpl.java:811)
at
org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.build(MiniOzoneClusterImpl.java:603)
at
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClientAbstract.startCluster(TestOzoneRpcClientAbstract.java:211)
at
org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient.init(TestOzoneRpcClient.java:51)
{code}
> Cannot reuse MiniOzoneCluster dir due to RocksDBCheckpointDiffer instance
> cache
> -------------------------------------------------------------------------------
>
> Key: HDDS-8557
> URL: https://issues.apache.org/jira/browse/HDDS-8557
> Project: Apache Ozone
> Issue Type: Bug
> Components: test
> Affects Versions: 1.4.0
> Reporter: Attila Doroszlai
> Priority: Major
>
> {{RocksDBCheckpointDiffer}} has single instance per DB, which may cause
> failure if MiniOzoneCluster reuses the same directory between runs.
> Apparently this can happen if cluster is started in {{@BeforeAll}} method and
> test is repeated due to {{@Flaky}} annotation. The result is that if the
> first run fails, repeated attempts always run into the error.
> {code:title=https://github.com/apache/ozone/actions/runs/4901282858/jobs/8752465727#step:6:3795}
> org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient Time elapsed: 0.649 s
> <<< ERROR!
> java.lang.RuntimeException: Failed to append compaction log to
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.appendToCurrentCompactionLog(RocksDBCheckpointDiffer.java:381)
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.setCurrentCompactionLog(RocksDBCheckpointDiffer.java:314)
> at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:155)
> at
> org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:212)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:558)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:526)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:516)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:325)
> at
> org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:768)
> at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:648)
> at
> org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:733)
> at
> org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.createOM(MiniOzoneClusterImpl.java:811)
> at
> org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.build(MiniOzoneClusterImpl.java:603)
> at
> org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClientAbstract.startCluster(TestOzoneRpcClientAbstract.java:211)
> at
> org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient.init(TestOzoneRpcClient.java:51)
> ...
> Caused by: java.nio.file.NoSuchFileException:
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
