[
https://issues.apache.org/jira/browse/IGNITE-22690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-22690:
-------------------------------------
Description:
{code:java}
11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
<false>
11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
<false>
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:183)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:966)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:928)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:914)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.testVacuumPersistentStateAfterCleanupDelayAndVolatileStateVacuum(ItTxResourcesVacuumTest.java:631)
at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
{code}
[TC|https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_IntegrationTests_ModuleTable/8295684?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandBuildProblemsSection=true&expandBuildTestsSection=true&expandBuildChangesSection=true&showLog=8295684_14816_86.14435.14816&logFilter=debug&logView=flowAware]
Not reproduced locally 300/300 green runs, Success Rate: 99.0% on TC.
>From the log we can see that cleanup was has completed "Test: cleanup
>completed." and according to the the way how it's checked in the test, that
>means that cleanup completion timestamp was populated at least within one node
{code:java}
private void waitForCleanupCompletion(Set<String> commitPartitionNodeNames,
UUID txId) throws InterruptedException {
...
res = res || txStateMeta != null &&
txStateMeta.cleanupCompletionTimestamp() != null;
...
} {code}
We also can see that vacuumization has started, but only volatile storage was
vacuumized
{code:java}
[2024-07-08T08:02:08,659][INFO ][Test worker][VolatileTxStateMetaStorage]
Vacuum finished [vacuumObservationTimestamp=1720425728659, txnResourceTtl=0,
vacuumizedTxnsCount=1, vacuumizedPersistentTxnStatesCount=0,
markedAsInitiallyDetectedTxnsCount=0, alreadyMarkedTxnsCount=0,
skippedForFurtherProcessingUnfinishedTxnsCount=0]. {code}
Test failed because persistent storage wasn't cleaned up:
{code:java}
[2024-07-08T08:02:08,661][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_0, state=null].
[2024-07-08T08:02:08,661][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_0, state=TxMeta [txState=COMMITTED,
commitTimestamp=HybridTimestamp [physical=2024-07-08 08:02:08:623 +0000,
logical=1, composite=112749820551036929]]].
[2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_1, state=null].
[2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_1, state=TxMeta [txState=COMMITTED,
commitTimestamp=HybridTimestamp [physical=2024-07-08 08:02:08:623 +0000,
logical=1, composite=112749820551036929]]].
[2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_2, state=null].
[2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
node=itrvt_tvpsacdavsv_2, state=null]. {code}
was:
{code:java}
11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
<false>
11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
<false>
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:183)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:966)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:928)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:914)
at
app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.testVacuumPersistentStateAfterCleanupDelayAndVolatileStateVacuum(ItTxResourcesVacuumTest.java:631)
at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
{code}
[TC|https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_IntegrationTests_ModuleTable/8295684?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandBuildProblemsSection=true&expandBuildTestsSection=true&expandBuildChangesSection=true&showLog=8295684_14816_86.14435.14816&logFilter=debug&logView=flowAware]
> ItTxResourcesVacuumTest.testVacuumPersistentStateAfterCleanupDelayAndVolatileStateVacuum
> is flaky
> -------------------------------------------------------------------------------------------------
>
> Key: IGNITE-22690
> URL: https://issues.apache.org/jira/browse/IGNITE-22690
> Project: Ignite
> Issue Type: Bug
> Reporter: Alexander Lapin
> Assignee: Denis Chudov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> {code:java}
> 11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
> <false>
> 11:02:10 org.opentest4j.AssertionFailedError: expected: <true> but was:
> <false>
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:183)
> at
> app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:966)
> at
> app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:928)
> at
> app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.assertTxStateVacuumized(ItTxResourcesVacuumTest.java:914)
> at
> app//org.apache.ignite.internal.table.ItTxResourcesVacuumTest.testVacuumPersistentStateAfterCleanupDelayAndVolatileStateVacuum(ItTxResourcesVacuumTest.java:631)
> at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
> at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
> at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
> {code}
> [TC|https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_IntegrationTests_ModuleTable/8295684?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandBuildProblemsSection=true&expandBuildTestsSection=true&expandBuildChangesSection=true&showLog=8295684_14816_86.14435.14816&logFilter=debug&logView=flowAware]
> Not reproduced locally 300/300 green runs, Success Rate: 99.0% on TC.
> From the log we can see that cleanup was has completed "Test: cleanup
> completed." and according to the the way how it's checked in the test, that
> means that cleanup completion timestamp was populated at least within one node
>
> {code:java}
> private void waitForCleanupCompletion(Set<String> commitPartitionNodeNames,
> UUID txId) throws InterruptedException {
> ...
> res = res || txStateMeta != null &&
> txStateMeta.cleanupCompletionTimestamp() != null;
> ...
> } {code}
> We also can see that vacuumization has started, but only volatile storage was
> vacuumized
> {code:java}
> [2024-07-08T08:02:08,659][INFO ][Test worker][VolatileTxStateMetaStorage]
> Vacuum finished [vacuumObservationTimestamp=1720425728659, txnResourceTtl=0,
> vacuumizedTxnsCount=1, vacuumizedPersistentTxnStatesCount=0,
> markedAsInitiallyDetectedTxnsCount=0, alreadyMarkedTxnsCount=0,
> skippedForFurtherProcessingUnfinishedTxnsCount=0]. {code}
> Test failed because persistent storage wasn't cleaned up:
> {code:java}
> [2024-07-08T08:02:08,661][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_0, state=null].
> [2024-07-08T08:02:08,661][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_0, state=TxMeta [txState=COMMITTED,
> commitTimestamp=HybridTimestamp [physical=2024-07-08 08:02:08:623 +0000,
> logical=1, composite=112749820551036929]]].
> [2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_1, state=null].
> [2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_1, state=TxMeta [txState=COMMITTED,
> commitTimestamp=HybridTimestamp [physical=2024-07-08 08:02:08:623 +0000,
> logical=1, composite=112749820551036929]]].
> [2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: volatile state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_2, state=null].
> [2024-07-08T08:02:08,662][INFO ][Test worker][ItTxResourcesVacuumTest]
> Test: persistent state [tx=0190915d-4686-0000-0b11-5e8c00000001,
> node=itrvt_tvpsacdavsv_2, state=null]. {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
