[
https://issues.apache.org/jira/browse/CASSANDRA-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090329#comment-17090329
]
Dmitry Kropachev commented on CASSANDRA-15034:
----------------------------------------------
[~djoshi], It does exactly the same under ccm. in this particular case it does
not matter since all we need is to get metadata read and get an error on CQL
operation from driver level or below.
I'am not trying to hide legitimate exception/error.
With the fix you will still see only "legitamate error", i.e.:
All host(s) tried for query failed (tried: /172.17.0.3:9042
(com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas
available for query at consistency QUORUM (2 required but only 1 alive)))
And what you will not see is tons of these guys:
java.util.NoSuchElementExceptionjava.util.NoSuchElementExceptionjava.util.NoSuchElementException
java.util.NoSuchElementExceptionjava.util.NoSuchElementExceptionjava.util.NoSuchElementException
java.util.NoSuchElementExceptionjava.util.NoSuchElementExceptionjava.util.NoSuchElementException
java.io.IOException: Operation x10 on key(s) [347f2f5b27831048d6f5]: Error
executing: (NoSuchElementException)
at org.apache.cassandra.stress.Operation.error(Operation.java:127)
at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)
at
org.apache.cassandra.stress.operations.userdefined.SchemaQuery.run(SchemaQuery.java:108)
at org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:466)
These messages are purely internal thing, there is no point to show them to the
people, it is just signaling that there is no records in the partition and it
needs to be reiterated, and it is reiterated but with clumsy error message that
does not tell you anything.
This fix is making these reiterations silent.
And the reason why you would need to hide these meaningless messages is that
when you run c-s in curcumstances of unstable network you can endup with
hundreds of megabytes of output 9/10 of which is totally useless.
Also there are cases when IO error happens once, and operation is silently
retrying, but on the next retry you can hit empty partition case and becasue of
this issue c-s will return -1 at the end, as result of the bug.
This fix fixes this problem to, just making reiteration on empty partition
silently.
> cassandra-stress fails to retry user profile insert and query operations
> -------------------------------------------------------------------------
>
> Key: CASSANDRA-15034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15034
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/stress
> Reporter: Shlomi Livne
> Assignee: Dinesh Joshi
> Priority: Normal
> Attachments: 0001-Fix-retry-of-userdefined-operations.patch,
> stress.yaml
>
>
> cassandra-stress that is run with a user profile against a cluster will fail
> to try operations when a node is killed.
> To reproduce:
> # Create a 3 ndoe cluster with ccm
> {code:java}
> ccm create cas-3 --vnodes -n 1 --version=3.11.3``{code}
> # start the cluster
> {code:java}
> ccm start{code}
> # run
> {code:java}
> tools/bin/cassandra-stress user profile=stress.yaml n=100000 ops(insert=1)
> no-warmup cl=QUORUM -node 127.0.0.2 -rate threads=10{code}
> or run
> {code:java}
> tools/bin/cassandra-stress user profile=stress.yaml n=100000 ops(simple=1)
> no-warmup cl=QUORUM -node 127.0.0.2 -rate threads=10{code}
> # while the stress is progressing kill a node
> {code:java}
> ccm node1 stop --not-gently{code}
> # wait for cassandra-stress to end
> for the insert case check
> {code:java}
> ccm node2 cqlsh -e " select count(*) FROM stresscql.tb ;"{code}
> we are missing rows
> for the query case (simple) the following errors will be reported
> {code:java}
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.io.IOException: Operation x10 on key(s) [49a26f35297303469236]: Error
> executing: (NoSuchElementException)
> at
> org.apache.cassandra.stress.Operation.error(Operation.java:127)java.util.NoSuchElementException
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:105)com.datastax.driver.core.exceptions.TransportException:
> [/127.0.0.1:9042] Connection has been closed
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException{code}
> profile: stress.yaml[^stress.yaml]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
