[
https://issues.apache.org/jira/browse/CASSANDRA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347264#comment-17347264
]
Ekaterina Dimitrova commented on CASSANDRA-16657:
-------------------------------------------------
Please ignore my previous comment, after adding more logs and revising again
what I have I realized I made a mistake and the last two INSERTs insert keys
corresponding to tokens which belong respectively to node 2 and node 3 and now
everything clicks! So the issue is really that in rare cases node 3 is not
ready after being started for the last INSERT and then we see the mentioned
failure.
I strengthened the test to ensure the replica is up before the respective
INSERT.
1000 successful runs done [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/837/workflows/b04066b9-ceab-4b31-8d9c-9a180d97dfda
] and [Java 11 |
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/837/workflows/b941b1b2-1655-40fd-ab80-2a9c6fa2e46a]
> Flaky TestPaxos
> ---------------
>
> Key: CASSANDRA-16657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16657
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Berenguer Blasi
> Assignee: Ekaterina Dimitrova
> Priority: Normal
> Fix For: 4.0-rc
>
>
> During testing for some other ticket I found in a test run
> [this|https://ci-cassandra.apache.org/job/Cassandra-devbranch/736/testReport/junit/dtest-novnode.paxos_test/TestPaxos/test_cluster_availability/]
> paxos failure
> {noformat}
> Error Message
> cassandra.Unavailable: Error from server: code=1000 [Unavailable exception]
> message="Cannot achieve consistency level SERIAL" info={'consistency':
> 'SERIAL', 'required_replicas': 1, 'alive_replicas': 0}
> Stacktrace
> self = <paxos_test.TestPaxos object at 0x7fafbd3f4850>
> @pytest.mark.no_vnodes
> def test_cluster_availability(self):
> # Warning, a change in partitioner or a change in CCM token allocation
> # may require the partition keys of these inserts to be changed.
> # This must not use vnodes as it relies on assumed token values.
>
> session = self.prepare(nodes=3)
> session.execute("CREATE TABLE test (k int PRIMARY KEY, v int)")
> session.execute("INSERT INTO test (k, v) VALUES (0, 0) IF NOT EXISTS")
>
> self.cluster.nodelist()[2].stop()
> session.execute("INSERT INTO test (k, v) VALUES (1, 1) IF NOT EXISTS")
>
> self.cluster.nodelist()[1].stop()
> session.execute("INSERT INTO test (k, v) VALUES (3, 2) IF NOT EXISTS")
>
> self.cluster.nodelist()[1].start()
> session.execute("INSERT INTO test (k, v) VALUES (5, 5) IF NOT EXISTS")
>
> self.cluster.nodelist()[2].start()
> > session.execute("INSERT INTO test (k, v) VALUES (6, 6) IF NOT EXISTS")
> paxos_test.py:83:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../venv/src/cassandra-driver/cassandra/cluster.py:2618: in execute
> return self.execute_async(query, parameters, trace, custom_payload,
> timeout, execution_profile, paging_state, host, execute_as).result()
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self = <ResponseFuture: query='<SimpleStatement query="INSERT INTO test (k,
> v) VALUES (6, 6) IF NOT EXISTS", consistency=Not ...el SERIAL"
> info={'consistency': 'SERIAL', 'required_replicas': 1, 'alive_replicas': 0}
> coordinator_host=127.0.0.2:9042>
> def result(self):
> """
> Return the final result or raise an Exception if errors were
> encountered. If the final result or error has not been set
> yet, this method will block until it is set, or the timeout
> set for the request expires.
>
> Timeout is specified in the Session request execution functions.
> If the timeout is exceeded, an :exc:`cassandra.OperationTimedOut`
> will be raised.
> This is a client-side timeout. For more information
> about server-side coordinator timeouts, see
> :class:`.policies.RetryPolicy`.
>
> Example usage::
>
> >>> future = session.execute_async("SELECT * FROM mycf")
> >>> # do other stuff...
>
> >>> try:
> ... rows = future.result()
> ... for row in rows:
> ... ... # process results
> ... except Exception:
> ... log.exception("Operation failed:")
>
> """
> self._event.wait()
> if self._final_result is not _NOT_SET:
> return ResultSet(self, self._final_result)
> else:
> > raise self._final_exception
> E cassandra.Unavailable: Error from server: code=1000 [Unavailable
> exception] message="Cannot achieve consistency level SERIAL"
> info={'consistency': 'SERIAL', 'required_replicas': 1, 'alive_replicas': 0}
> ../venv/src/cassandra-driver/cassandra/cluster.py:4894: Unavailable
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
