[
https://issues.apache.org/jira/browse/CASSANDRA-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777150#comment-17777150
]
Jacek Lewandowski commented on CASSANDRA-18747:
-----------------------------------------------
Few problems with testing...
On 4.1, one junit test failed but I don't believe that is related:
https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/977/workflows/f6f81939-5b1e-4e17-b756-bfb8791dc23d/jobs/38838/tests
On 5.0, it turned out that {{ShortPaxosSimulationTest}} is failing. I've
retried both locally and on CI with {{cassandra-5.0}} and it fails there as
well:
https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/1013/workflows/132ec030-5eae-4622-ad92-ffea76cbb5a4/jobs/39718
Additionally, {{LeaveAndBootstrapTest}} turned out to be flaky. I did some
investigation and discovered that this particular test is very sensitive to the
execution speed - when the test is executed a bit slower, it fails consistently
in the same way (both {{cassandra-5.0}} and this feature branch). The key
temporal option in this case is either Gossiper delay/interval or quarantine
delay. The failed runs were slower and Gossiper check state task was executed
wiping some endpoint from the state map due to elapsed quarantine time. Thas is
clearly visible in the logs. I'm hoping to fix this issue by increasing the
quarantine time so that it is less likely to elapse during the test.
On trunk, all tests passed.
> Test failure: Fix assertion error AssertionError: Unknown keyspace
> system_auth\n\tat
> org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:324)\n\tat
> org.apache.cassandra.db.Keyspace.lambda$open$0(Keyspace.java:162)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18747
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18747
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema, Test/dtest/python
> Reporter: Ekaterina Dimitrova
> Assignee: Jacek Lewandowski
> Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> I've been seeing this assertion error in different tests lately.
> Full error message:
> {code:java}
> failed on teardown with "Unexpected error found in node logs (see stdout for
> full details). Errors: [[node2] 'ERROR [PendingRangeCalculator:1] 2023-08-11
> 16:35:14,445 JVMStabilityInspector.java:70 - Exception in thread
> Thread[PendingRangeCalculator:1,5,PendingRangeCalculator]\njava.lang.AssertionError:
> Unknown keyspace system_auth\n\tat
> org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:324)\n\tat
> org.apache.cassandra.db.Keyspace.lambda$open$0(Keyspace.java:162)\n\tat
> org.apache.cassandra.utils.concurrent.LoadingMap.blockingLoadIfAbsent(LoadingMap.java:105)\n\tat
>
> org.apache.cassandra.schema.Schema.maybeAddKeyspaceInstance(Schema.java:251)\n\tat
> org.apache.cassandra.db.Keyspace.open(Keyspace.java:162)\n\tat
> org.apache.cassandra.db.Keyspace.open(Keyspace.java:151)\n\tat
> org.apache.cassandra.service.PendingRangeCalculatorService.lambda$new$1(PendingRangeCalculatorService.java:58)\n\tat
>
> org.apache.cassandra.concurrent.SingleThreadExecutorPlus$AtLeastOnce.run(SingleThreadExecutorPlus.java:60)\n\tat
>
> org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)\n\tat
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']" Unexpected error found in
> node logs (see stdout for full details). Errors: [[node2] 'ERROR
> [PendingRangeCalculator:1] 2023-08-11 16:35:14,445
> JVMStabilityInspector.java:70 - Exception in thread
> Thread[PendingRangeCalculator:1,5,PendingRangeCalculator]\njava.lang.AssertionError:
> Unknown keyspace system_auth\n\tat
> org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:324)\n\tat
> org.apache.cassandra.db.Keyspace.lambda$open$0(Keyspace.java:162)\n\tat
> org.apache.cassandra.utils.concurrent.LoadingMap.blockingLoadIfAbsent(LoadingMap.java:105)\n\tat
>
> org.apache.cassandra.schema.Schema.maybeAddKeyspaceInstance(Schema.java:251)\n\tat
> org.apache.cassandra.db.Keyspace.open(Keyspace.java:162)\n\tat
> org.apache.cassandra.db.Keyspace.open(Keyspace.java:151)\n\tat
> org.apache.cassandra.service.PendingRangeCalculatorService.lambda$new$1(PendingRangeCalculatorService.java:58)\n\tat
>
> org.apache.cassandra.concurrent.SingleThreadExecutorPlus$AtLeastOnce.run(SingleThreadExecutorPlus.java:60)\n\tat
>
> org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)\n\tat
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']{code}
> Example failures:
> test_failed_snitch_update_property_file_snitch -
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2475/workflows/2086619e-0f21-464b-a866-84aca516b5e5/jobs/36716/tests]
> test_gcgs_validation -
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1666/testReport/junit/dtest.materialized_views_test/TestMaterializedViews/test_gcgs_validation/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
