[
https://issues.apache.org/jira/browse/CASSANDRA-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553609#comment-17553609
]
Andres de la Peña commented on CASSANDRA-17307:
-----------------------------------------------
[Repeated
runs|https://app.circleci.com/pipelines/github/adelapena/cassandra/1680/workflows/f8c846a3-7768-4326-b7cb-e25da4dab2d7/jobs/17618]
of this test hit both the timeout mentioned on the description and the failure
reported on CASSANDRA-17641, CASSANDRA-17642, CASSANDRA-17651 and
CASSANDRA-17652.
The test combines queries expected to succeed with writes expected to timeout.
One one hand, the queries that should succeed require a timeout config large
enough to avoid accidentally timeouts queries due to a slow env. One the other
hand, the queries that should timeout benefit from a not-so-long timeout config
so the entire test doesn't hit a JUnit timeout.
I guess that what we can do here is increasing the query timeout config, so the
queries expected to succeed don't timeout on the coordinator, and also split
the test into multiple classes so there are less queries per test. That ways
the queries that should timeout on the coordinator wouldn't produce a Junit
timeout.
The problem is that the {{ShutdownException}} reported on the other tickets is
far more common than the timeout, so it's difficult to solve the timeout
problem without fixing the {{ShutdownException}} problem before.
> Test Failure:
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityV30Test.testAvailability
> ----------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-17307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17307
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
> Reporter: Josh McKenzie
> Assignee: Andres de la Peña
> Priority: Normal
> Fix For: 4.1-beta, 4.x
>
>
> No known failures. Flakiness 0%, Stability 100%
> Error Message
> Unexpected error while reading in case write-read consistency QUORUM-QUORUM
> with not upgraded coordinator and 1 nodes down
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: Unexpected error while reading in case
> write-read consistency QUORUM-QUORUM with not upgraded coordinator and 1
> nodes down
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase$Tester.test(MixedModeAvailabilityTestBase.java:142)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.lambda$testAvailability$2(MixedModeAvailabilityTestBase.java:91)
> at
> org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:231)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.testAvailability(MixedModeAvailabilityTestBase.java:93)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.testAvailability(MixedModeAvailabilityTestBase.java:62)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase.testAvailability(MixedModeAvailabilityTestBase.java:56)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityV30Test.testAvailability(MixedModeAvailabilityV30Test.java:33)
> Caused by: java.lang.RuntimeException:
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
> received only 1 responses.
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:218)
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$5(IsolatedExecutor.java:109)
> at
> org.apache.cassandra.distributed.impl.Coordinator.executeWithResult(Coordinator.java:69)
> at
> org.apache.cassandra.distributed.api.ICoordinator.execute(ICoordinator.java:32)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase$Tester.lambda$test$1(MixedModeAvailabilityTestBase.java:135)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase$Tester.maybeFail(MixedModeAvailabilityTestBase.java:155)
> at
> org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityTestBase$Tester.test(MixedModeAvailabilityTestBase.java:134)
> Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
> timed out - received only 1 responses.
> at
> org.apache.cassandra.service.ReadCallback.awaitResults(ReadCallback.java:136)
> at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:142)
> at
> org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:145)
> at
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.awaitResultsAndRetryOnDigestMismatch(StorageProxy.java:1833)
> at
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1782)
> at
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1720)
> at
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1629)
> at
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1166)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:302)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:263)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:115)
> at
> org.apache.cassandra.distributed.impl.Coordinator.executeInternal(Coordinator.java:107)
> at
> org.apache.cassandra.distributed.impl.Coordinator.lambda$executeWithResult$0(Coordinator.java:69)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
