[jira] [Commented] (CASSANDRA-9875) Rebuild from targeted replica
[ https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218755#comment-17218755 ] Cameron Zemek commented on CASSANDRA-9875: -- I have attached backport of this patch for 2.1 in order to support rebuild tool we are releasing. > Rebuild from targeted replica > - > > Key: CASSANDRA-9875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9875 > Project: Cassandra > Issue Type: Improvement >Reporter: Sankalp Kohli >Assignee: Geoffrey Yu >Priority: Low > Labels: lhf > Fix For: 3.10 > > Attachments: 9875-dtest-master-v2.txt, 9875-dtest-master.txt, > 9875-trunk-v2.txt, 9875-trunk.txt, rebuild21.patch > > > Nodetool rebuild command will rebuild all the token ranges handled by the > endpoint. Sometimes we want to rebuild only a certain token range. We should > add this ability to rebuild command. We should also add the ability to stream > from a given replica. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-9875) Rebuild from targeted replica
[ https://issues.apache.org/jira/browse/CASSANDRA-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cameron Zemek updated CASSANDRA-9875: - Attachment: rebuild21.patch > Rebuild from targeted replica > - > > Key: CASSANDRA-9875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9875 > Project: Cassandra > Issue Type: Improvement >Reporter: Sankalp Kohli >Assignee: Geoffrey Yu >Priority: Low > Labels: lhf > Fix For: 3.10 > > Attachments: 9875-dtest-master-v2.txt, 9875-dtest-master.txt, > 9875-trunk-v2.txt, 9875-trunk.txt, rebuild21.patch > > > Nodetool rebuild command will rebuild all the token ranges handled by the > endpoint. Sometimes we want to rebuild only a certain token range. We should > add this ability to rebuild command. We should also add the ability to stream > from a given replica. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218733#comment-17218733 ] Berenguer Blasi edited comment on CASSANDRA-16121 at 10/22/20, 5:27 AM: Correct, those are 2 diff sets of tests. At the time of this PR `test_describe_cluster_output` was failing (CASSANDRA-16098) but wasn't getting reported in jenkins, this is what trigerred me raising this ticket. So this SHA fails this test and you can run it locally to doublecheck. It is not fixed/rebased in this PR bc I needed a failing test to see that circleci was reporting failures correctly :-) Should we rebase and run a new circleci/jenkins and then we have confirmation it A. reports failures B. now passes green? Edit: That was a silly question lol. Rebased as it was an old PR, pushed and new CI passing green in the PR. Now we know both it's green and does indeed report failures. was (Author: bereng): Correct, those are 2 diff sets of tests. At the time of this PR `test_describe_cluster_output` was failing (CASSANDRA-16098) but wasn't getting reported in jenkins, this is what trigerred me raising this ticket. So this SHA fails this test and you can run it locally to doublecheck. It is not fixed/rebased in this PR bc I needed a failing test to see that circleci was reporting failures correctly :-) Should we rebase and run a new circleci/jenkins and then we have confirmation it A. reports failures B. now passes green? > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218733#comment-17218733 ] Berenguer Blasi commented on CASSANDRA-16121: - Correct, those are 2 diff sets of tests. At the time of this PR `test_describe_cluster_output` was failing (CASSANDRA-16098) but wasn't getting reported in jenkins, this is what trigerred me raising this ticket. So this SHA fails this test and you can run it locally to doublecheck. It is not fixed/rebased in this PR bc I needed a failing test to see that circleci was reporting failures correctly :-) Should we rebase and run a new circleci/jenkins and then we have confirmation it A. reports failures B. now passes green? > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218671#comment-17218671
]
Yifan Cai commented on CASSANDRA-16103:
---
After scanning the server logs for this error, there are 2 groups of errors
grouping by the difference of the expected and the actual size.
The differences are either 8 or -1.
It is caused by the extra 1 at {{serializedHeaderSizePost40()}}
[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/Message.java#L720].
It leads to the calculated serialized size mismatch with the actual one, in
the case of some special combinations of message creation and expiration time.
* When creationTime - expirationTime == (1L << 7) - 1 ms (or (1L << 14) - 1
ms), we get the difference of -1.
* When creationTime - expirationTime == -1 ms, we get the difference of 8. The
reason we are seeing {{creationTime > expirationTime}} is that there is no
constraint to assure expirationTime is greater than creationTime in the message
constructor.
The response message shares the same expiration time with the request message,
but it assigns the current time to the creationTime. There is a small chance
that we got response messages with {{creationTime > expirationTime}}.
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218655#comment-17218655 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 10/22/20, 12:16 AM: - Jenkins run is lost. I just [rebased | https://github.com/ekaterinadimitrova2/cassandra/tree/C-15897-3.0], reverted the local temporary check for the SSTables from CASSANDRA-16063, and ran CI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/424/workflows/fb25652e-1297-4a24-a5b1-b31afb732333]. There are four tests that will need attention. I will look at them tomorrow. - test_prefer_local_reconnect_on_listen_address - snitch_test.TestGossipingPropertyFileSnitch - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/dtest.snitch_test/] - testDropSSTables - org.apache.cassandra.db.lifecycle.TrackerTest - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/org.apache.cassandra.db.lifecycle/] - testDropCompactWithClusteringAndValueColumnWithDeletesAndWrites - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest - testDropCompactWithClusteringAndValueColumn - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest was (Author: e.dimitrova): Jenkins run is lost. I just rebased, reverted the local temporary check for the SSTables from CASSANDRA-16063, and ran CI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/424/workflows/fb25652e-1297-4a24-a5b1-b31afb732333]. There are four tests that will need attention. I will look at them tomorrow. - test_prefer_local_reconnect_on_listen_address - snitch_test.TestGossipingPropertyFileSnitch - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/dtest.snitch_test/] - testDropSSTables - org.apache.cassandra.db.lifecycle.TrackerTest - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/org.apache.cassandra.db.lifecycle/] - testDropCompactWithClusteringAndValueColumnWithDeletesAndWrites - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest - testDropCompactWithClusteringAndValueColumn - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 3.0.x, 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218657#comment-17218657 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - [~marcuse], [~ifesdjeen] and [~aleksey], just to confirm you are ok with the approach proposed by Sylvain? Wanted to be sure before I invest time here. Also, do you accept my suggestion to pull update to upgradesstable (or scrub) to a followup ticket? > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 3.0.x, 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218657#comment-17218657 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 10/21/20, 11:57 PM: - [~marcuse], [~ifesdjeen] and [~aleksey], just to confirm you are ok with the approach proposed by Sylvain? Wanted to be sure before I invest time here. Also, do you accept the suggestion to pull update to upgradesstable (or scrub) to a followup ticket? was (Author: e.dimitrova): [~marcuse], [~ifesdjeen] and [~aleksey], just to confirm you are ok with the approach proposed by Sylvain? Wanted to be sure before I invest time here. Also, do you accept my suggestion to pull update to upgradesstable (or scrub) to a followup ticket? > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 3.0.x, 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218655#comment-17218655 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 10/21/20, 11:53 PM: - Jenkins run is lost. I just rebased, reverted the local temporary check for the SSTables from CASSANDRA-16063, and ran CI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/424/workflows/fb25652e-1297-4a24-a5b1-b31afb732333]. There are four tests that will need attention. I will look at them tomorrow. - test_prefer_local_reconnect_on_listen_address - snitch_test.TestGossipingPropertyFileSnitch - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/dtest.snitch_test/] - testDropSSTables - org.apache.cassandra.db.lifecycle.TrackerTest - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/org.apache.cassandra.db.lifecycle/] - testDropCompactWithClusteringAndValueColumnWithDeletesAndWrites - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest - testDropCompactWithClusteringAndValueColumn - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest was (Author: e.dimitrova): Jenkins run is lost. I just rebased, reverted the local temporary check for the SSTables from CASSANDRA-16063, and ran CI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/424/workflows/fb25652e-1297-4a24-a5b1-b31afb732333]. There are three tests that will need attention. I will look at them tomorrow. - test_prefer_local_reconnect_on_listen_address - snitch_test.TestGossipingPropertyFileSnitch - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/dtest.snitch_test/] - testDropSSTables - org.apache.cassandra.db.lifecycle.TrackerTest - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/org.apache.cassandra.db.lifecycle/] - testDropCompactWithClusteringAndValueColumnWithDeletesAndWrites - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest - testDropCompactWithClusteringAndValueColumn - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 3.0.x, 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218655#comment-17218655 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - Jenkins run is lost. I just rebased, reverted the local temporary check for the SSTables from CASSANDRA-16063, and ran CI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/424/workflows/fb25652e-1297-4a24-a5b1-b31afb732333]. There are three tests that will need attention. I will look at them tomorrow. - test_prefer_local_reconnect_on_listen_address - snitch_test.TestGossipingPropertyFileSnitch - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/dtest.snitch_test/] - testDropSSTables - org.apache.cassandra.db.lifecycle.TrackerTest - stable in [Jenkins | https://jenkins-cm4.apache.org/job/Cassandra-3.0/25/testReport/org.apache.cassandra.db.lifecycle/] - testDropCompactWithClusteringAndValueColumnWithDeletesAndWrites - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest - testDropCompactWithClusteringAndValueColumn - org.apache.cassandra.distributed.upgrade.CompactStorage2to3UpgradeTest > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 3.0.x, 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-13325:
-
Test and Documentation Plan: Unit tests
Status: Patch Available (was: Open)
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.x
>
> Attachments: trunk.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218641#comment-17218641
]
Paulo Motta commented on CASSANDRA-16205:
-
bq. For CASSANDRA-16079 isn't the code (and the imbalance) parity what we want?
That is we want tests to be running on the same setups that
allocate_tokens_for_local_replication_factor generates, since that is our
default (soon to be).
It would be ideal to have all dtests using the new token allocator but given
that slowed down dtests I find it suboptimal to create and maintain a new tool
that is only used for this purpose. Auto_bootstrap=true is also a default, but
we set it to false to speed up dtests so we have precedent here.
I don't see why we need to run all dtests (for instance auth, snitch,
notifications, cql, ttl etc) using the token allocation algorithm given these
tests are agnostic to the ring layout? As long as unit tests guarantee
correctness of the token allocator, and we have a few dtests ensuring its
correctness during bootstraps, we're safe to use random allocation for all
dtests?
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218639#comment-17218639
]
Dinesh Joshi commented on CASSANDRA-16144:
--
[~jmeredithco] I think I'm +1. If there are any minor nits, please fix them on
commit. [~dcapwell] do you want to get this once [~jmeredithco]'s finished
addressing your comments?
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
>
[jira] [Updated] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-16144:
-
Status: Ready to Commit (was: Review In Progress)
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
>
[jira] [Updated] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-16152: - Status: Ready to Commit (was: Review In Progress) > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218637#comment-17218637 ] Dinesh Joshi commented on CASSANDRA-16152: -- I'm +1 with these changes too. [~dcapwell] could you commit this? > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16218) Simplify the almost duplicated code for calculating serialization size and serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-16218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218634#comment-17218634
]
Yifan Cai commented on CASSANDRA-16218:
---
The benchmark result shows the new approach takes *14% more time* on
calculating the serialized size. Attached the benchmark. So.. it is probably
not worthy. We may instead just fix CASSANDRA-16103 and pay more attention when
reviewing changes in serialize() and serializedSize().
{code:java}
c-16218
[java] BenchmarkMode Cnt Score Error
Units
[java] SerializationSizeBench.getSerializeSize avgt6 2.290 ± 0.489
ns/op
trunk
[java] BenchmarkMode Cnt Score Error
Units
[java] SerializationSizeBench.getSerializeSize avgt6 2.003 ± 0.163
ns/op{code}
> Simplify the almost duplicated code for calculating serialization size and
> serialization
>
>
> Key: CASSANDRA-16218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16218
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client, Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: SerializationSizeBench.java
>
>
> The current pattern of counting the serialization size and the actual
> serialization in the codebase is error-prone and hard to maintain. Those 2
> code paths have almost the same code repeated.
>
> The pattern can be found in {{org.apache.cassandra.net.Message#Serializer}}
> and numerous locations that use {{org.apache.cassandra.transport.CBCodec}}.
>
> I am proposing a new approach that lets both methods share the same code
> path. The code basically looks like the below (in the case of
> {{org.apache.cassandra.net.Message#Serializer}}).
> {code:java}
> // A fake DataOutputPlus that simply increment the size when invoking write*
> methods
> public class SizeCountingDataOutput implements DataOutputPlus
> {
> private int size = 0;
> public int getSize()
> {
>return size;
> }
> public void write(int b)
> {
>size += 1;
> }
> public void write(byte[] b)
> {
>size += b.length;
> }
> ...
> }
> {code}
> Therefore, in the size calculation, we can supply the fake data output and
> call serialize to get the size.
> {code:java}
> private int serializedSize(Message message, int version)
> {
> SizeCountingDataOutput out = new SizeCountingDataOutput();
> try
> {
>serialize(message, out, version);
> }
> catch (IOException exception)
> {
>throw new AssertionError("It should not happen!", exception);
> }
> return out.getSize();
> // The original implementation
> // return version >= VERSION_40 ? serializedSizePost40(message, version) :
> serializedSizePre40(message, version);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16218) Simplify the almost duplicated code for calculating serialization size and serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-16218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16218:
--
Attachment: SerializationSizeBench.java
> Simplify the almost duplicated code for calculating serialization size and
> serialization
>
>
> Key: CASSANDRA-16218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16218
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client, Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: SerializationSizeBench.java
>
>
> The current pattern of counting the serialization size and the actual
> serialization in the codebase is error-prone and hard to maintain. Those 2
> code paths have almost the same code repeated.
>
> The pattern can be found in {{org.apache.cassandra.net.Message#Serializer}}
> and numerous locations that use {{org.apache.cassandra.transport.CBCodec}}.
>
> I am proposing a new approach that lets both methods share the same code
> path. The code basically looks like the below (in the case of
> {{org.apache.cassandra.net.Message#Serializer}}).
> {code:java}
> // A fake DataOutputPlus that simply increment the size when invoking write*
> methods
> public class SizeCountingDataOutput implements DataOutputPlus
> {
> private int size = 0;
> public int getSize()
> {
>return size;
> }
> public void write(int b)
> {
>size += 1;
> }
> public void write(byte[] b)
> {
>size += b.length;
> }
> ...
> }
> {code}
> Therefore, in the size calculation, we can supply the fake data output and
> call serialize to get the size.
> {code:java}
> private int serializedSize(Message message, int version)
> {
> SizeCountingDataOutput out = new SizeCountingDataOutput();
> try
> {
>serialize(message, out, version);
> }
> catch (IOException exception)
> {
>throw new AssertionError("It should not happen!", exception);
> }
> return out.getSize();
> // The original implementation
> // return version >= VERSION_40 ? serializedSizePost40(message, version) :
> serializedSizePre40(message, version);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218611#comment-17218611
]
Adam Holmberg commented on CASSANDRA-16183:
---
in-jvm dtests present a bit of a roadblock for this set of metrics, in the fact
that they [disable MBean
registration|https://github.com/apache/cassandra-in-jvm-dtest-api/blob/trunk/src/main/java/org/apache/cassandra/distributed/api/ICluster.java#L99].
This means that in order to test these metrics I would need to make the
[metrics in
StorageProxy|https://github.com/apache/cassandra/blob/095540d54a07d2c35bd9260e065fcf346ad36164/src/java/org/apache/cassandra/service/StorageProxy.java#L185-L192]
public/VisibleForTesting and reach in using
{IInvocableInstance::callsOnInstance}. This seems to work, but it's made more
painful because the metrics classes are not serializable. We would need to
marshal into a new structure to get the results back (so far have not found a
way to get any custom class across). I haven't found a great way to do this
that isn't coupled and brittle.
It's starting to feel like Python dtest might be the way to go with this.
Please let me know if anyone has further thoughts.
> Add tests to cover ClientRequest metrics
> -
>
> Key: CASSANDRA-16183
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16183
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Benjamin Lerer
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We do not have test that covers the ClientRequest metrics.
> * ClientRequestMetrics
> * CASClientRequestMetrics
> * CASClientWriteRequestMetrics
> * ClientWriteRequestMetrics
> * ViewWriteMetrics
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218604#comment-17218604 ] David Capwell commented on CASSANDRA-16121: --- Talking to [~e.dimitrova] in slack she showed me how to find all tests that ran and I was also looking at Jenkins wrong; here is the test results in Jenkins for the test that failed: https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/lastCompletedBuild/cython=no,jdk=jdk_1.8_latest,label=cassandra/testReport/cqlshlib.python3.jdk8.no_cython.test.test_cqlsh_output/TestCqlshOutput/test_describe_cluster_output/history/ In Jenkins this test isn't flaky, so wondering why it failed here. > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218601#comment-17218601 ] David Capwell commented on CASSANDRA-16121: --- Left comments in PR, but won't to know why this acts differently than Jenkins, if you could look into that it would be great (we don't archive the test xml so I can't view it). > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218600#comment-17218600 ] David Capwell commented on CASSANDRA-16121: --- [~Bereng] I see Jenkins runs these tests and is green, yet the test results linked are failing with different test names Jenkins: https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/lastCompletedBuild/cython=no,jdk=jdk_1.8_latest,label=cassandra/testReport/ Jenkins build def: https://github.com/apache/cassandra-builds/blob/6d556fe8873296c0a48f747bd9855e462193252d/jenkins-dsl/cassandra_job_dsl_seed.groovy#L344 Circle CI: https://app.circleci.com/pipelines/github/bereng/cassandra/129/workflows/10497110-d938-4500-8ef3-eb3d0e815b6e/jobs/1053/tests > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16121: -- Reviewers: David Capwell, Ekaterina Dimitrova (was: Ekaterina Dimitrova) > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well
[ https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218589#comment-17218589 ] David Capwell commented on CASSANDRA-16121: --- I can start reviewing. [~e.dimitrova] did you +1 or still pending? I read your comment as pending? > Circleci should run cqlshlib tests as well > -- > > Key: CASSANDRA-16121 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16121 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Currently circleci is not running cqlshlib tests. This resulted in some bugs > not being caught before committing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0
[
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218585#comment-17218585
]
David Capwell commented on CASSANDRA-16083:
---
bq. now we don't plan to do any removals, at most deprecation in beta, right?
We should no longer remove without approval in dev@, and those should only be
things added in 4.0.
IMO we should develop the pattern of X.0 we depreciate, (X + 1).0 we remove,
X.Y deprecate and (X + 2).0 remove; this should give plenty of time for users
to migrate.
For this ticket, the remaining metrics are
{code}
Objects:
"org.apache.cassandra.metrics:type=ThreadPools.*",
"org.apache.cassandra.internal:.*",
"org.apache.cassandra.metrics:type=DroppedMessage.*",
"org.apache.cassandra.metrics:type=ClientRequest,scope=CASRead,name=ConditionNotMet",
Attributes:
MaxNativeProtocolVersion
{code}
other than internal (personal bias is anything named internal is free to break
at any point in time), all look useful; I do see DroppedMessage used in tooling.
So to me the question is, do we add them back?
> Missing JMX objects and attributes upgrading from 3.0 to 4.0
>
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics
> missing in 4.0 but present in 3.0. The work here is to make sure we had
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime
> org.apache.cassandra.db:type=HintedHandoffManager
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime
>
[jira] [Updated] (CASSANDRA-16221) Jenkins doesn't run high resource python dtests with novonode so missing some tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16221: -- Test and Documentation Plan: not sure how to Status: Patch Available (was: Open) > Jenkins doesn't run high resource python dtests with novonode so missing some > tests > --- > > Key: CASSANDRA-16221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16221 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > In CASSANDRA-16220 it was found that a test is failing on trunk, and its > because of the address to address with port changes; looking closer into it > it was found that Jenkins ignores the novnode case which causes this test to > get skipped. > We should enable novnode as well to match the rest of the pipeline, but also > to handle some of the missing tests not currently run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16221) Jenkins doesn't run high resource python dtests with novonode so missing some tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16221: -- Change Category: Quality Assurance Complexity: Low Hanging Fruit Fix Version/s: NA Priority: Low (was: Normal) Status: Open (was: Triage Needed) > Jenkins doesn't run high resource python dtests with novonode so missing some > tests > --- > > Key: CASSANDRA-16221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16221 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > In CASSANDRA-16220 it was found that a test is failing on trunk, and its > because of the address to address with port changes; looking closer into it > it was found that Jenkins ignores the novnode case which causes this test to > get skipped. > We should enable novnode as well to match the rest of the pipeline, but also > to handle some of the missing tests not currently run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16221) Jenkins doesn't run high resource python dtests with novonode so missing some tests
David Capwell created CASSANDRA-16221: - Summary: Jenkins doesn't run high resource python dtests with novonode so missing some tests Key: CASSANDRA-16221 URL: https://issues.apache.org/jira/browse/CASSANDRA-16221 Project: Cassandra Issue Type: Improvement Components: Build Reporter: David Capwell Assignee: David Capwell In CASSANDRA-16220 it was found that a test is failing on trunk, and its because of the address to address with port changes; looking closer into it it was found that Jenkins ignores the novnode case which causes this test to get skipped. We should enable novnode as well to match the rest of the pipeline, but also to handle some of the missing tests not currently run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16218) Simplify the almost duplicated code for calculating serialization size and serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-16218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218577#comment-17218577
]
David Capwell commented on CASSANDRA-16218:
---
Idea sounds good to me, would simplify serializer logic. I took a quick scan
and the only negative I can find is extra function calls in some paths, but the
common case had roughly equal.
> Simplify the almost duplicated code for calculating serialization size and
> serialization
>
>
> Key: CASSANDRA-16218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16218
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client, Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The current pattern of counting the serialization size and the actual
> serialization in the codebase is error-prone and hard to maintain. Those 2
> code paths have almost the same code repeated.
>
> The pattern can be found in {{org.apache.cassandra.net.Message#Serializer}}
> and numerous locations that use {{org.apache.cassandra.transport.CBCodec}}.
>
> I am proposing a new approach that lets both methods share the same code
> path. The code basically looks like the below (in the case of
> {{org.apache.cassandra.net.Message#Serializer}}).
> {code:java}
> // A fake DataOutputPlus that simply increment the size when invoking write*
> methods
> public class SizeCountingDataOutput implements DataOutputPlus
> {
> private int size = 0;
> public int getSize()
> {
>return size;
> }
> public void write(int b)
> {
>size += 1;
> }
> public void write(byte[] b)
> {
>size += b.length;
> }
> ...
> }
> {code}
> Therefore, in the size calculation, we can supply the fake data output and
> call serialize to get the size.
> {code:java}
> private int serializedSize(Message message, int version)
> {
> SizeCountingDataOutput out = new SizeCountingDataOutput();
> try
> {
>serialize(message, out, version);
> }
> catch (IOException exception)
> {
>throw new AssertionError("It should not happen!", exception);
> }
> return out.getSize();
> // The original implementation
> // return version >= VERSION_40 ? serializedSizePost40(message, version) :
> serializedSizePre40(message, version);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218570#comment-17218570
]
David Capwell commented on CASSANDRA-16103:
---
Linking CASSANDRA-16218 as it found a similar issue, but caused by us
calculating message header with ts + 1, which causes vint to add an extra byte
to the size; this looks different than this issue as it was an off by one issue
only there, and this can be off by several bytes.
> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception;
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb HINT_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16219) when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
[ https://issues.apache.org/jira/browse/CASSANDRA-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16219: -- Source Control Link: https://github.com/apache/cassandra-dtest/commit/53cb98dc67bf9887898032a3423d667ce57a7b77 Resolution: Fixed Status: Resolved (was: Ready to Commit) > when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra > configurations, tests fail as they don't expect this flag in stderr > - > > Key: CASSANDRA-16219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16219 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > Time Spent: 20m > Remaining Estimate: 0h > > We have many flags that are set as system properties, and rather than > changing the tests to support the different versions, we try to run the tests > with different flag settings; this causes some tests to fail as stderr will > contain something like the following > JAVA_TOOL_OPTIONS: -Dcassandra.consistent.simultaneousmoves.allow=true > We should update our tests to ignore this stderr message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16219) when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
[ https://issues.apache.org/jira/browse/CASSANDRA-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16219: -- Status: Ready to Commit (was: Review In Progress) > when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra > configurations, tests fail as they don't expect this flag in stderr > - > > Key: CASSANDRA-16219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16219 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > Time Spent: 20m > Remaining Estimate: 0h > > We have many flags that are set as system properties, and rather than > changing the tests to support the different versions, we try to run the tests > with different flag settings; this causes some tests to fail as stderr will > contain something like the following > JAVA_TOOL_OPTIONS: -Dcassandra.consistent.simultaneousmoves.allow=true > We should update our tests to ignore this stderr message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: When running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 53cb98d When running python dtest and using JAVA_TOOL_OPTIONS to
inject cassandra configurations, tests fail as they don't expect this flag in
stderr
53cb98d is described below
commit 53cb98dc67bf9887898032a3423d667ce57a7b77
Author: David Capwell
AuthorDate: Wed Oct 21 12:46:26 2020 -0700
When running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra
configurations, tests fail as they don't expect this flag in stderr
patch by David Capwell; reviewed by Jordan West for CASSANDRA-16219
---
hintedhandoff_test.py | 3 ++-
nodetool_test.py | 22 +++---
offline_tools_test.py | 7 ++-
snitch_test.py| 3 ++-
4 files changed, 21 insertions(+), 14 deletions(-)
diff --git a/hintedhandoff_test.py b/hintedhandoff_test.py
index dc5d64c..0d2d6e8 100644
--- a/hintedhandoff_test.py
+++ b/hintedhandoff_test.py
@@ -7,6 +7,7 @@ from cassandra import ConsistencyLevel
from dtest import Tester, create_ks
from tools.data import create_c1c2_table, insert_c1c2, query_c1c2
+from tools.assertions import assert_stderr_clean
since = pytest.mark.since
logger = logging.getLogger(__name__)
@@ -43,7 +44,7 @@ class TestHintedHandoffConfig(Tester):
Launch a nodetool command and check there is no error, return the
result
"""
out, err, _ = node.nodetool(cmd)
-assert '' == err
+assert_stderr_clean(err)
return out
def _do_hinted_handoff(self, node1, node2, enabled, keyspace='ks'):
diff --git a/nodetool_test.py b/nodetool_test.py
index 09e1dff..800181c 100644
--- a/nodetool_test.py
+++ b/nodetool_test.py
@@ -8,7 +8,7 @@ from cassandra.query import SimpleStatement
from ccmlib.node import ToolError
from dtest import Tester, create_ks
-from tools.assertions import assert_all, assert_invalid, assert_none
+from tools.assertions import assert_all, assert_invalid, assert_none,
assert_stderr_clean
from tools.jmxutils import JolokiaAgent, make_mbean
since = pytest.mark.since
@@ -35,7 +35,7 @@ class TestNodetool(Tester):
node.decommission()
assert not "Expected nodetool error"
except ToolError as e:
-assert '' == e.stderr
+assert_stderr_clean(e.stderr)
assert 'Unsupported operation' in e.stdout
def test_correct_dc_rack_in_nodetool_info(self):
@@ -58,7 +58,7 @@ class TestNodetool(Tester):
for i, node in enumerate(cluster.nodelist()):
out, err, _ = node.nodetool('info')
-assert 0 == len(err), err
+assert_stderr_clean(err)
out_str = out
if isinstance(out, (bytes, bytearray)):
out_str = out.decode("utf-8")
@@ -91,19 +91,19 @@ class TestNodetool(Tester):
# read all of the timeouts, make sure we get a sane response
for timeout_type in types:
out, err, _ = node.nodetool('gettimeout {}'.format(timeout_type))
-assert 0 == len(err), err
+assert_stderr_clean(err)
logger.debug(out)
assert re.search(r'.* \d+ ms', out)
# set all of the timeouts to 123
for timeout_type in types:
_, err, _ = node.nodetool('settimeout {} 123'.format(timeout_type))
-assert 0 == len(err), err
+assert_stderr_clean(err)
# verify that they're all reported as 123
for timeout_type in types:
out, err, _ = node.nodetool('gettimeout {}'.format(timeout_type))
-assert 0 == len(err), err
+assert_stderr_clean(err)
logger.debug(out)
assert re.search(r'.* 123 ms', out)
@@ -197,7 +197,7 @@ class TestNodetool(Tester):
# Do a first try without any keypace, we shouldn't have the notice
out, err, _ = node.nodetool('status')
-assert 0 == len(err), err
+assert_stderr_clean(err)
assert not re.search(notice_message, out)
session = self.patient_cql_connection(node)
@@ -205,21 +205,21 @@ class TestNodetool(Tester):
# With 1 keyspace, we should still not get the notice
out, err, _ = node.nodetool('status')
-assert 0 == len(err), err
+assert_stderr_clean(err)
assert not re.search(notice_message, out)
session.execute("CREATE KEYSPACE ks2 WITH replication = {
'class':'SimpleStrategy', 'replication_factor':1}")
# With 2 keyspaces with the same settings, we should not get the notice
out, err, _ = node.nodetool('status')
-assert 0 == len(err), err
+assert_stderr_clean(err)
assert not re.search(notice_message, out)
session.execute("CREATE KEYSPACE ks3 WITH replication = {
[jira] [Commented] (CASSANDRA-16219) when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
[ https://issues.apache.org/jira/browse/CASSANDRA-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218551#comment-17218551 ] Jordan West commented on CASSANDRA-16219: - Thanks. Merge away! > when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra > configurations, tests fail as they don't expect this flag in stderr > - > > Key: CASSANDRA-16219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16219 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > We have many flags that are set as system properties, and rather than > changing the tests to support the different versions, we try to run the tests > with different flag settings; this causes some tests to fail as stderr will > contain something like the following > JAVA_TOOL_OPTIONS: -Dcassandra.consistent.simultaneousmoves.allow=true > We should update our tests to ignore this stderr message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16091) rpc server gets wrongly initialized with rpc_enabled:false
[
https://issues.apache.org/jira/browse/CASSANDRA-16091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218547#comment-17218547
]
David Capwell commented on CASSANDRA-16091:
---
[~saprykin] yes, this issue is fixed in 16127.
> rpc server gets wrongly initialized with rpc_enabled:false
> --
>
> Key: CASSANDRA-16091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16091
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Tom van der Woerdt
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> After upgrading to Cassandra 3.11.8, Cassandra no longer starts. An exception
> is thrown:
> {code:java}
> java.lang.RuntimeException: Client SSL is not supported for non-blocking
> sockets (hsha). Please remove client ssl from the configuration.
> at
> org.apache.cassandra.thrift.THsHaDisruptorServer$Factory.buildTServer(THsHaDisruptorServer.java:74)
> at
> org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:55)
> at
> org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.(ThriftServer.java:128)
> at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:55)
> at
> org.apache.cassandra.service.CassandraDaemon.startNativeTransport(CassandraDaemon.java:713)
> at
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:538)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:643)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:768)
> {code}
> No configuration changed between 3.11.7 and 3.11.8. rpc_enabled is false in
> both versions.
> I created this Jira issue because clearly something changed between 3.11.7
> and 3.11.8. Maybe intentional, maybe not. Changing `rpc_server_type` (which
> is not clearly documented to be about Thrift only) from `hsha` to `sync` does
> resolve the issue, as expected, but this does seem like a regression, hence
> the Jira issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16209) Log Warning Rather than Verbose Trace when Preview Repair Validation Conflicts with Incremental Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218533#comment-17218533 ] David Capwell commented on CASSANDRA-16209: --- You mean https://app.circleci.com/pipelines/github/dcapwell/cassandra/705/workflows/e53c72aa-dc4c-4dcc-811d-87aedd75f89a/jobs/3931 consistency_test.py::TestConsistency::test_12872? I remember this test class failing but didn't check if there was a jira; took the fact it passed in the other version to tell me it was flaky. > Log Warning Rather than Verbose Trace when Preview Repair Validation > Conflicts with Incremental Repair > -- > > Key: CASSANDRA-16209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16209 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a preview repair on repaired data identifies which SSTables to validate, > it might come across an SSTable that's still pending for an in-progress > incremental repair session. It makes sense that we immediately fail the > preview repair in that case, but the resulting error and verbose stack trace > in the logs is a bit too severe a reaction. We should downgrade this to a > simple warning message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218474#comment-17218474
]
David Capwell commented on CASSANDRA-16114:
---
left comments in PR
> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
>
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Build:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
> at
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
> at
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16114:
--
Reviewers: Benjamin Lerer, David Capwell (was: Benjamin Lerer)
> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
>
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Build:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
> at
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
> at
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16209) Log Warning Rather than Verbose Trace when Preview Repair Validation Conflicts with Incremental Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218470#comment-17218470 ] Caleb Rackliffe commented on CASSANDRA-16209: - [~dcapwell] It looks like we've seen that test fail before [here|https://issues.apache.org/jira/browse/CASSANDRA-15984?focusedCommentId=17165894=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17165894]? > Log Warning Rather than Verbose Trace when Preview Repair Validation > Conflicts with Incremental Repair > -- > > Key: CASSANDRA-16209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16209 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a preview repair on repaired data identifies which SSTables to validate, > it might come across an SSTable that's still pending for an in-progress > incremental repair session. It makes sense that we immediately fail the > preview repair in that case, but the resulting error and verbose stack trace > in the logs is a bit too severe a reaction. We should downgrade this to a > simple warning message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16219) when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
[ https://issues.apache.org/jira/browse/CASSANDRA-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218466#comment-17218466 ] David Capwell commented on CASSANDRA-16219: --- [~jwest] posted 3.0 and trunk builds, and both are normal. > when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra > configurations, tests fail as they don't expect this flag in stderr > - > > Key: CASSANDRA-16219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16219 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > We have many flags that are set as system properties, and rather than > changing the tests to support the different versions, we try to run the tests > with different flag settings; this causes some tests to fail as stderr will > contain something like the following > JAVA_TOOL_OPTIONS: -Dcassandra.consistent.simultaneousmoves.allow=true > We should update our tests to ignore this stderr message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218462#comment-17218462 ] Ekaterina Dimitrova commented on CASSANDRA-16061: - Hey [~ifesdjeen], [~bdeggleston], I see you added this test some time ago and it was initially marked as flakey. The commit description points to CASSANDRA-14404 but I believe the test was added as part of different ticket which I wasn't able to identify to read the reasoning behind the initial flaky mark. Any thoughts and advice on this and the failures we see? Thanks in advance! > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218456#comment-17218456
]
David Capwell commented on CASSANDRA-16220:
---
In slack https://the-asf.slack.com/archives/CK23JSY2K/p1603301859245500 I
pointed out that Jenkins doesn't run this test, as Jenkins only runs the tests
with vnodes, and this test is marked no_vnode; we should fix Jenkins to run
these test to see these issues (though 100% fine in a different ticket).
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out
[jira] [Updated] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16220:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-beta
Severity: Normal
Status: Open (was: Triage Needed)
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))
> if process and not
[jira] [Created] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
David Capwell created CASSANDRA-16220:
-
Summary: python dtest
pending_range_test.py::TestPendingRangeMovements::test_pending_range
(@pytest.mark.resource_intensive) fails on trunk
Key: CASSANDRA-16220
URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: David Capwell
The test has the following section
{code}
if cluster.version() >= '2.2':
node2.watch_log_for('127.0.0.1 state moving', timeout=10,
filename='debug.log’)
{code}
The issue is that in trunk we have the port attached to the log, so the log is
now
{code}
DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
/127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
/127.0.0.1:7000 state moving, new token -634023222112864484
{code}
Since the log now contains the port, this test always times out on trunk when
it hits this
{code}
self =
@pytest.mark.resource_intensive
def test_pending_range(self):
"""
@jira_ticket CASSANDRA-10887
"""
cluster = self.cluster
# If we are on 2.1, we need to set the log level to debug or higher, as
debug.log does not exist.
if cluster.version() < '2.2':
cluster.set_log_level('DEBUG')
# Create 5 node cluster
cluster.populate(5).start()
node1, node2 = cluster.nodelist()[0:2]
# Set up RF=3 keyspace
session = self.patient_cql_connection(node1)
create_ks(session, 'ks', 3)
session.execute("CREATE TABLE users (login text PRIMARY KEY, email
text, name text, login_count int)")
# We use the partition key 'jdoe3' because it belongs to node1.
# The key MUST belong to node1 to repro the bug.
session.execute("INSERT INTO users (login, email, name, login_count)
VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
lwt_query = SimpleStatement("UPDATE users SET email = 'jane...@abc.com'
WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
# Show we can execute LWT no problem
for i in range(1000):
session.execute(lwt_query)
token = '-634023222112864484'
mark = node1.mark_log()
# Move a node
node1.nodetool('move {}'.format(token))
# Watch the log so we know when the node is moving
node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
from_mark=mark)
node1.watch_log_for('Sleeping 3 ms before start streaming/fetching
ranges', timeout=10, from_mark=mark)
if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log')
pending_range_test.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process = None
verbose = False, filename = 'debug.log'
def watch_log_for(self, exprs, from_mark=None, timeout=600, process=None,
verbose=False, filename='system.log'):
"""
Watch the log until one or more (regular) expressions are found or
timeouts (a
TimeoutError is then raised). On successful completion, a list of
pair (line matched,
match object) is returned.
"""
start = time.time()
tofind = [exprs] if isinstance(exprs, string_types) else exprs
tofind = [re.compile(e) for e in tofind]
matchings = []
reads = ""
if len(tofind) == 0:
return None
log_file = os.path.join(self.log_directory(), filename)
output_read = False
while not os.path.exists(log_file):
time.sleep(.5)
if start + timeout < time.time():
raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
created.".format(log_file))
if process and not output_read:
process.poll()
if process.returncode is not None:
self.print_process_output(self.name, process, verbose)
output_read = True
if process.returncode != 0:
raise RuntimeError() # Shouldn't reuse RuntimeError
but I'm lazy
with open(log_file) as f:
if from_mark:
f.seek(from_mark)
while True:
# First, if we have a process to check, then check it.
# Skip on Windows - stdout/stderr is cassandra.bat
if not common.is_win() and not output_read:
if process:
process.poll()
if
[jira] [Commented] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218429#comment-17218429 ] Josh McKenzie commented on CASSANDRA-15588: --- We settled on the following recommended and supported paths on the dev list: # 2.1 -> 3.11 -> 4.0 # 3.0 -> 4.0 # 3.11 -> 4.0 One group was going to take on the 3.0 -> 4.0 path and the other the 2.1 -> 3.11 -> 4.0 We need to better articulate what test coverage we consider sufficient for confidence in these upgrade paths. > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16219) when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra configurations, tests fail as they don't expect this flag in stderr
[ https://issues.apache.org/jira/browse/CASSANDRA-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218422#comment-17218422 ] David Capwell commented on CASSANDRA-16219: --- sure, will create one now, will trigger on 3.0 and trunk. > when running python dtest and using JAVA_TOOL_OPTIONS to inject cassandra > configurations, tests fail as they don't expect this flag in stderr > - > > Key: CASSANDRA-16219 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16219 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > We have many flags that are set as system properties, and rather than > changing the tests to support the different versions, we try to run the tests > with different flag settings; this causes some tests to fail as stderr will > contain something like the following > JAVA_TOOL_OPTIONS: -Dcassandra.consistent.simultaneousmoves.allow=true > We should update our tests to ignore this stderr message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15935: -- Status: Ready to Commit (was: Changes Suggested) +1 with latest changes > Improve machinery for testing consistency in presence of range movements > > > Key: CASSANDRA-15935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently, we can test range movements only by adding and bootstrapping a new > node. This is both inefficient and insufficient for large-scale tests. We > need a possibility to dynamically change ring ownership over the lifetime of > cluster, with a flexibility to changing gossip status of the node from > perspective of other participants, adding and removing nodes from other > nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance
[ https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218407#comment-17218407 ] Ekaterina Dimitrova commented on CASSANDRA-15586: - Thanks [~yukim]! Please link the tickets to this one for visibility, when they are opened. Thanks in advance! > 4.0 quality testing: Cluster Setup and Maintenance > -- > > Key: CASSANDRA-15586 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15586 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Assignee: Ekaterina Dimitrova >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-beta > > > We want 4.0 to be easy for users to setup out of the box and just work. This > means having low friction when users download the Cassandra package and start > running it. For example, users should be able to easily configure and start > new 4.0 clusters and have tokens distributed evenly. Another example is > packaging, it should be easy to install Cassandra on all supported platforms > (e.g. packaging) and have Cassandra use standard platform integrations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218404#comment-17218404 ] Adam Holmberg commented on CASSANDRA-16183: --- Thanks for the input. I have a vote against #1 and an aversion to #2, so I will attempt #3. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218388#comment-17218388 ] Benjamin Lerer edited comment on CASSANDRA-16183 at 10/21/20, 4:44 PM: --- I have a different view than [~brandon.williams] :( . It makes sense to me to test those metrics using distributed tests. So, I would not go for the approach 1). Regarding python dtests vs in-JVM dtests, I do not have a strong opinion. Learning how in-JVM tests works will always be useful anyway, so I let it up to you. was (Author: blerer): I have a different view that [~brandon.williams] :( . It makes sense to me to test those metrics using distributed tests. So, I would not go for the approach 1). Regarding python dtests vs in-JVM dtests, I do not have a strong opinion. Learning how in-JVM tests works will always be useful anyway, so I let it up to you. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218388#comment-17218388 ] Benjamin Lerer commented on CASSANDRA-16183: I have a different view that [~brandon.williams] :( . It makes sense to me to test those metrics using distributed tests. So, I would not go for the approach 1). Regarding python dtests vs in-JVM dtests, I do not have a strong opinion. Learning how in-JVM tests works will always be useful anyway, so I let it up to you. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16159) Reduce the Severity of Errors Reported in FailureDetector#isAlive()
[
https://issues.apache.org/jira/browse/CASSANDRA-16159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Arora reassigned CASSANDRA-16159:
-
Assignee: Shubham Arora (was: Uchenna)
> Reduce the Severity of Errors Reported in FailureDetector#isAlive()
> ---
>
> Key: CASSANDRA-16159
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16159
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Caleb Rackliffe
>Assignee: Shubham Arora
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Noticed the following error in the failure detector during a host replacement:
> {noformat}
> java.lang.IllegalArgumentException: Unknown endpoint: 10.38.178.98:7000
> at
> org.apache.cassandra.gms.FailureDetector.isAlive(FailureDetector.java:281)
> at
> org.apache.cassandra.service.StorageService.handleStateBootreplacing(StorageService.java:2502)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2182)
> at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:3145)
> at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1242)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1368)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:77)
> at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:93)
> at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:44)
> at
> org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:884)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {noformat}
> This particular error looks benign, given that even if it occurs, the node
> continues to handle the {{BOOT_REPLACE}} state. There are two things we might
> be able to do to improve {{FailureDetector#isAlive()}} though:
> 1.) We don’t short circuit in the case that the endpoint in question is in
> quarantine after being removed. It may be useful to check for this so we can
> avoid logging an ERROR when the endpoint is clearly doomed/dead. (Quarantine
> works great when the gossip message is _from_ a quarantined endpoint, but in
> this case, that would be the new/replacing and not the old/replaced one.)
> 2.) We can reduce the severity of the logging from ERROR to WARN and provide
> better context around how to determine whether or not there’s actually a
> problem. (ex. “If this occurs while trying to determine liveness for a node
> that is currently being replaced, it can be safely ignored.”)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218376#comment-17218376 ] Brandon Williams commented on CASSANDRA-16183: -- Given that we're just talking about metrics here, I think the first approach is sufficient. Firing up multiple machines for this seems very heavy handed, so the second is my least favorite. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16180) 4.0 quality testing: Coordination
[ https://issues.apache.org/jira/browse/CASSANDRA-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16180: -- Change Category: Quality Assurance Status: Open (was: Triage Needed) > 4.0 quality testing: Coordination > - > > Key: CASSANDRA-16180 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16180 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.0 > > > This is a subtask of CASSANDRA-15579 focusing on coordination. > I think that the main reference dtest for this is > [consistency_test.py|https://github.com/apache/cassandra-dtest/blob/master/consistency_test.py]. > We should identify which other tests cover this and identify what should be > extended, similarly to what has been done with CASSANDRA-15977. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[ https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15865: Status: In Progress (was: Patch Available) > Flaky dtest > hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow > --- > > Key: CASSANDRA-15865 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15865 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Sam Tunnicliffe >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Attachments: logsGoodNBad.zip > > Time Spent: 20m > Remaining Estimate: 0h > > I've seen this fail a couple of times under JDK11, when it doesn't appear to > be related to the changes under test. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg reassigned CASSANDRA-16061: - Assignee: (was: Adam Holmberg) > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218319#comment-17218319 ] Adam Holmberg edited comment on CASSANDRA-16183 at 10/21/20, 2:42 PM: -- I started looking at this yesterday, and thought I would drop a note here to see if there are preferences on how to proceed. I have considered three approaches: 1.) Unit tests could be added touching all of these metrics. It would be straightforward to validate CL mapping, counters, latency sanity (i.e. non-zero and correct relative to each other. What we would not achieve is validating Unavailables and Timeouts. We would just test that they are there, and zero. If that is not sufficient we could augment this approach by tacking on validation in dtests where those conditions are already being created. 2.) If the above is not rigorous enough, we could instead create new tests in Python dtests. I think all the building blocks are there. What I am less sure about is our current aversion to adding tests to this suite. 3.) If we need dtests and don't want to expand Python suite, it's probably possible to do in-JVM dtests. This one is probably the heaviest lift for me just because I have no exposure to that machinery yet. I'm happy to figure it out, but the upfront learning will be more. [~blerer]I'm interested in your thoughts, or anyone else who cares to weigh in. was (Author: aholmber): I started looking at this yesterday, and thought I would drop a note here to see if there are preferences on how to proceed. I have considered three approaches: 1.) Unit tests could be added touching all of these metrics. It would be straightforward to validate CL mapping, counters, latency sanity (i.e. non-zero and correct relative to each other. What we would not achieve is validating Unavailables and Timeouts. We would just test that they are there, and zero. If that is not sufficient we could augment this approach by tacking on validation in dtests where those conditions are already being created. 2.) If the above is not rigorous enough, we could create new tests in Python dtests. I think all the building blocks are there. What I am less sure about is our current aversion to adding tests to this suite. 3.) If we need dtests and don't want to expand Python suite, it's probably possible to do in-JVM dtests. This one is probably the heaviest lift for me just because I have no exposure to that machinery yet. I'm happy to figure it out, but the upfront learning will be more. [~blerer]I'm interested in your thoughts, or anyone else who cares to weigh in. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218319#comment-17218319 ] Adam Holmberg commented on CASSANDRA-16183: --- I started looking at this yesterday, and thought I would drop a note here to see if there are preferences on how to proceed. I have considered three approaches: 1.) Unit tests could be added touching all of these metrics. It would be straightforward to validate CL mapping, counters, latency sanity (i.e. non-zero and correct relative to each other. What we would not achieve is validating Unavailables and Timeouts. We would just test that they are there, and zero. If that is not sufficient we could augment this approach by tacking on validation in dtests where those conditions are already being created. 2.) If the above is not rigorous enough, we could create new tests in Python dtests. I think all the building blocks are there. What I am less sure about is our current aversion to adding tests to this suite. 3.) If we need dtests and don't want to expand Python suite, it's probably possible to do in-JVM dtests. This one is probably the heaviest lift for me just because I have no exposure to that machinery yet. I'm happy to figure it out, but the upfront learning will be more. [~blerer]I'm interested in your thoughts, or anyone else who cares to weigh in. > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15789) Rows can get duplicated in mixed major-version clusters and after full upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15789:
---
Fix Version/s: (was: 4.0-alpha)
4.0-beta1
4.0
> Rows can get duplicated in mixed major-version clusters and after full upgrade
> --
>
> Key: CASSANDRA-15789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15789
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Aleksey Yeschenko
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1
>
>
> In a mixed 2.X/3.X major version cluster a sequence of row deletes,
> collection overwrites, paging, and read repair can cause 3.X nodes to split
> individual rows into several rows with identical clustering. This happens due
> to 2.X paging and RT semantics, and a 3.X {{LegacyLayout}} deficiency.
> To reproduce, set up a 2-node mixed major version cluster with the following
> table:
> {code}
> CREATE TABLE distributed_test_keyspace.tlb (
> pk int,
> ck int,
> v map,
> PRIMARY KEY (pk, ck)
> );
> {code}
> 1. Using either node as the coordinator, delete the row with ck=2 using
> timestamp 1
> {code}
> DELETE FROM tbl USING TIMESTAMP 1 WHERE pk = 1 AND ck = 2;
> {code}
> 2. Using either node as the coordinator, insert the following 3 rows:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 1, {'e':'f'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 3;
> INSERT INTO tbl (pk, ck, v) VALUES (1, 3, {'i':'j'}) USING TIMESTAMP 3;
> {code}
> 3. Flush the table on both nodes
> 4. Using the 2.2 node as the coordinator, force read repar by querying the
> table with page size = 2:
>
> {code}
> SELECT * FROM tbl;
> {code}
> 5. Overwrite the row with ck=2 using timestamp 5:
> {code}
> INSERT INTO tbl (pk, ck, v) VALUES (1, 2, {'g':'h'}) USING TIMESTAMP 5;}}
> {code}
> 6. Query the 3.0 node and observe the split row:
> {code}
> cqlsh> select * from distributed_test_keyspace.tlb ;
> pk | ck | v
> ++
> 1 | 1 | {'e': 'f'}
> 1 | 2 | {'g': 'h'}
> 1 | 2 | {'k': 'l'}
> 1 | 3 | {'i': 'j'}
> {code}
> This happens because the read to query the second page ends up generating the
> following mutation for the 3.0 node:
> {code}
> ColumnFamily(tbl -{deletedAt=-9223372036854775808, localDeletion=2147483647,
> ranges=[2:v:_-2:v:!, deletedAt=2, localDeletion=1588588821]
> [2:v:!-2:!, deletedAt=1, localDeletion=1588588821]
> [3:v:_-3:v:!, deletedAt=2, localDeletion=1588588821]}-
> [2:v:63:false:1@3,])
> {code}
> Which on 3.0 side gets incorrectly deserialized as
> {code}
> Mutation(keyspace='distributed_test_keyspace', key='0001', modifications=[
> [distributed_test_keyspace.tbl] key=1
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [v]]
> Row[info=[ts=-9223372036854775808] ]: ck=2 | del(v)=deletedAt=2,
> localDeletion=1588588821, [v[c]=d ts=3]
> Row[info=[ts=-9223372036854775808] del=deletedAt=1,
> localDeletion=1588588821 ]: ck=2 |
> Row[info=[ts=-9223372036854775808] ]: ck=3 | del(v)=deletedAt=2,
> localDeletion=1588588821
> ])
> {code}
> {{LegacyLayout}} correctly interprets a range tombstone whose start and
> finish {{collectionName}} values don't match as a wrapping fragment of a
> legacy row deletion that's being interrupted by a collection deletion
> (correctly) - see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1874-L1889].
> Quoting the comment inline:
> {code}
> // Because of the way RangeTombstoneList work, we can have a tombstone where
> only one of
> // the bound has a collectionName. That happens if we have a big tombstone A
> (spanning one
> // or multiple rows) and a collection tombstone B. In that case,
> RangeTombstoneList will
> // split this into 3 RTs: the first one from the beginning of A to the
> beginning of B,
> // then B, then a third one from the end of B to the end of A. To make this
> simpler, if
> // we detect that case we transform the 1st and 3rd tombstone so they don't
> end in the middle
> // of a row (which is still correct).
> {code}
> {{LegacyLayout#addRowTombstone()}} method then chokes when it encounters such
> a tombstone in the middle of an existing row - having seen {{v[c]=d}} first,
> and mistakenly starts a new row, while in the middle of an existing one: (see
> [code|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/LegacyLayout.java#L1500-L1501]).
[cassandra-dtest] branch master updated: Revert "Revert "Add test_truncate_failure""
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new ded6a96 Revert "Revert "Add test_truncate_failure""
ded6a96 is described below
commit ded6a967601f554770c533984a42ae841a20943d
Author: Brandon Williams
AuthorDate: Wed Oct 21 09:12:37 2020 -0500
Revert "Revert "Add test_truncate_failure""
This restores test_truncate_failure.
This reverts commit 016a0eb38db25ab36e1adabbc0bfe9575212b2ec.
---
byteman/truncate_fail.btm | 8
cql_test.py | 33 +
2 files changed, 41 insertions(+)
diff --git a/byteman/truncate_fail.btm b/byteman/truncate_fail.btm
new file mode 100644
index 000..fa9caba
--- /dev/null
+++ b/byteman/truncate_fail.btm
@@ -0,0 +1,8 @@
+RULE Throw during truncate operation
+CLASS org.apache.cassandra.db.ColumnFamilyStore
+METHOD truncateBlocking()
+AT ENTRY
+IF TRUE
+DO
+ throw new RuntimeException("Dummy failure");
+ENDRULE
\ No newline at end of file
diff --git a/cql_test.py b/cql_test.py
index eced21d..dde7b7d 100644
--- a/cql_test.py
+++ b/cql_test.py
@@ -1,4 +1,5 @@
import itertools
+import re
import struct
import time
import pytest
@@ -764,6 +765,38 @@ class TestMiscellaneousCQL(CQLTester):
[2, None, 2, None],
[3, None, 3, None]])
+@since("4.0")
+def test_truncate_failure(self):
+"""
+@jira_ticket CASSANDRA-16208
+Tests that if a TRUNCATE query fails on some replica, the coordinator
will immediately return an error to the
+client instead of waiting to time out because it couldn't get the
necessary number of success acks.
+"""
+cluster = self.cluster
+cluster.populate(3, install_byteman=True).start()
+node1, _, node3 = cluster.nodelist()
+node3.byteman_submit(['./byteman/truncate_fail.btm'])
+
+session = self.patient_exclusive_cql_connection(node1)
+create_ks(session, 'ks', 3)
+
+logger.debug("Creating data table")
+session.execute("CREATE TABLE data (id int PRIMARY KEY, data text)")
+session.execute("UPDATE data SET data = 'Awesome' WHERE id = 1")
+
+self.fixture_dtest_setup.ignore_log_patterns = ['Dummy failure']
+logger.debug("Truncating data table (error expected)")
+
+thrown = False
+exception = None
+try:
+session.execute("TRUNCATE data")
+except Exception as e:
+exception = e
+thrown = True
+
+assert thrown, "No exception has been thrown"
+assert re.search("Truncate failed on replica /127.0.0.3",
str(exception)) is not None
@since('3.2')
class AbortedQueryTester(CQLTester):
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218304#comment-17218304 ] Ekaterina Dimitrova edited comment on CASSANDRA-16217 at 10/21/20, 2:03 PM: Hi [~ifesdjeen], Thanks for pinging. ??Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these.?? I am +1 on adding the minimal support and not having breaking changes. was (Author: e.dimitrova): Hi @Alex, Thanks for pinging. ??Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these.?? I am +1 on adding the minimal support and not having breaking changes. > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > Preliminary patch: [https://github.com/apache/cassandra/pull/785] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218304#comment-17218304 ] Ekaterina Dimitrova commented on CASSANDRA-16217: - Hi @Alex, Thanks for pinging. ??Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these.?? I am +1 on adding the minimal support and not having breaking changes. > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > Preliminary patch: [https://github.com/apache/cassandra/pull/785] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem
[
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-15584:
--
Description:
Reference [doc from
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
for context.
*Shepherd: Benjamin Lerer*
Many users of Apache Cassandra employ open source tooling to automate Cassandra
configuration, runtime management, and repair scheduling. Prior to release, we
need to confirm that popular third-party tools function properly.
Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH
ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]|
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT
STARTED* | [~stefan.miklosovic]|
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra
operator|https://github.com/instaclustr/cassandra-operator]|
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Esop |
https://github.com/instaclustr/instaclustr-esop]|{color:#00875A}*DONE*{color} |
[~stefan.miklosovic]|
| [Instaclustr Icarus |
https://github.com/instaclustr/instaclustr-icarus]|{color:#00875A}*DONE*{color}
| [~stefan.miklosovic]|
| [Cassandra SSTable generator |
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
[~stefan.miklosovic]|
| [Cassandra TTL Remover | https://github.com/instaclustr/TTLRemover] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Cassandra Everywhere Strategy |
https://github.com/instaclustr/cassandra-everywhere-strategy] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Cassandra LDAP Authenticator |
https://github.com/instaclustr/cassandra-ldap] | {color:#00875A}*DONE*{color} |
[~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} |
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *IN PROGRESS*|
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck
Dehay|
|
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
{color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]|
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]|
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment
API|https://github.com/datastax/management-api-for-apache-cassandra]|
{color:#00875A}*DONE*{color}| [~tjake]|
Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any issue
and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if testing 4.0 is
part of your CI process.
* *Contact*: The person acting as the contact point for that tool.
was:
Reference [doc from
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
for context.
*Shepherd: Benjamin Lerer*
Many users of Apache Cassandra employ open source tooling to automate Cassandra
configuration, runtime management, and repair scheduling. Prior to release, we
need to confirm that popular third-party tools function properly.
Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH
ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]|
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT
STARTED* | [~stefan.miklosovic]|
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra
operator|https://github.com/instaclustr/cassandra-operator]|
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Esop |
https://github.com/instaclustr/instaclustr-esop]|{color:#00875A}*DONE*{color} |
[~stefan.miklosovic]|
| [Instaclustr Icarus |
https://github.com/instaclustr/instaclustr-icarus]|{color:#00875A}*DONE*{color}
| [~stefan.miklosovic]|
| [Cassandra SSTable generator |
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
[~stefan.miklosovic]|
| [Cassandra TTL Remover | https://github.com/instaclustr/TTLRemover] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Cassandra Everywhere Strategy |
https://github.com/instaclustr/cassandra-everywhere-strategy] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} |
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *IN PROGRESS*|
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]|
[jira] [Commented] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[
https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218301#comment-17218301
]
Thomas Steinmaurer commented on CASSANDRA-16201:
[~marcuse], first impression from our comparison infrastructure regarding the
3.0, 3.11 and 4.0 patches.
When having a look on 2 high-level metrics:
* JVM suspension, marked as "1" in the dashboard below
* Cassandra dropped messages, marked as "2" in the dashboard below
!screenshot-3.png|width=100%!
* Cassandra 3.0: No positive impact on suspension
* Cassandra 3.11: Huge positive impact on suspension
* Cassandra 4.0: Huge positive impact on suspension + no dropped messages with
the patch
I will keep that running over the night and provide another set of JFR files
for 3.0, 3.11 and 4.0 with the patch.
Thanks for your efforts!
> Reduce amount of allocations during batch statement execution
> -
>
> Key: CASSANDRA-16201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16201
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Thomas Steinmaurer
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile,
> we see 4.0b2 going OOM from time to time. According to a heap dump, we have
> multiple NTR threads in a 3-digit MB range.
> This is likely related to object array pre-allocations at the size of
> {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always
> only 1 {{BTreeRow}} in the {{BTree}}.
> !screenshot-1.png|width=100%!
> So it seems we have many, many 20K elemnts pre-allocated object arrays
> resulting in a shallow heap of 80K each, although there is only one element
> in the array.
> This sort of pre-allocation is causing a lot of memory pressure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[
https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Steinmaurer updated CASSANDRA-16201:
---
Attachment: screenshot-3.png
> Reduce amount of allocations during batch statement execution
> -
>
> Key: CASSANDRA-16201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16201
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Thomas Steinmaurer
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile,
> we see 4.0b2 going OOM from time to time. According to a heap dump, we have
> multiple NTR threads in a 3-digit MB range.
> This is likely related to object array pre-allocations at the size of
> {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always
> only 1 {{BTreeRow}} in the {{BTree}}.
> !screenshot-1.png|width=100%!
> So it seems we have many, many 20K elemnts pre-allocated object arrays
> resulting in a shallow heap of 80K each, although there is only one element
> in the array.
> This sort of pre-allocation is causing a lot of memory pressure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16151:
--
Since Version: 0.7 beta 3
Source Control Link:
https://github.com/apache/cassandra/commit/72941b9ec14e64af9e64365027d542b4fff41d81
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218270#comment-17218270
]
Andres de la Peña commented on CASSANDRA-16151:
---
Great, thanks, nice patches.
Committed to {{cassandra-2.2}} as
[72941b9ec14e64af9e64365027d542b4fff41d81|https://github.com/apache/cassandra/commit/72941b9ec14e64af9e64365027d542b4fff41d81]
and merged up to
[{{3.0}}|https://github.com/apache/cassandra/commit/d44dbd91c1752a71f4819be92fd75a52be9f0118],
[{{3.11}}|https://github.com/apache/cassandra/commit/a2f59be94388754e667f545b02798810056f9b1b]
and
[{{trunk}}|https://github.com/apache/cassandra/commit/095540d54a07d2c35bd9260e065fcf346ad36164].
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16151:
--
Fix Version/s: (was: 4.0-beta)
4.0-beta3
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (2edc5bb -> 095540d)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 2edc5bb Log Warning Rather than Verbose Trace when Preview Repair Validation Conflicts with Incremental Repair new 72941b9 Package tools/bin scripts as executable new d44dbd9 Merge branch 'cassandra-2.2' into cassandra-3.0 new a2f59be Merge branch 'cassandra-3.0' into cassandra-3.11 new 095540d Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Package tools/bin scripts as executable
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new 72941b9 Package tools/bin scripts as executable 72941b9 is described below commit 72941b9ec14e64af9e64365027d542b4fff41d81 Author: Angelo Polo AuthorDate: Wed Oct 21 13:09:51 2020 +0100 Package tools/bin scripts as executable patch by Angelo Polo; reviewed by Andres de la Peña for CASSANDRA-16151 --- CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 1274689..8a78993 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.19 + * Package tools/bin scripts as executable (CASSANDRA-16151) * Fix ExceptionInInitializerError when data_file_directories is not set (CASSANDRA-16008) * Fixed a NullPointerException when calling nodetool enablethrift (CASSANDRA-16127) diff --git a/build.xml b/build.xml index 2037f65..4f3818c 100644 --- a/build.xml +++ b/build.xml @@ -1139,14 +1139,14 @@ + - - - + + @@ -1164,6 +1164,7 @@ + @@ -1171,16 +1172,21 @@ - + + + + + + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (bca91ca -> a2f59be)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from bca91ca Synchronize Keyspace instance store/clear new 72941b9 Package tools/bin scripts as executable new d44dbd9 Merge branch 'cassandra-2.2' into cassandra-3.0 new a2f59be Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 095540d54a07d2c35bd9260e065fcf346ad36164 Merge: 2edc5bb a2f59be Author: Andrés de la Peña AuthorDate: Wed Oct 21 13:24:18 2020 +0100 Merge branch 'cassandra-3.11' into trunk CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --cc CHANGES.txt index 04af373,af44f96..97762c1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -34,36 -9,16 +34,37 @@@ Merged from 3.0 * Handle unexpected columns due to schema races (CASSANDRA-15899) * Add flag to ignore unreplicated keyspaces during repair (CASSANDRA-15160) Merged from 2.2: + * Package tools/bin scripts as executable (CASSANDRA-16151) * Fixed a NullPointerException when calling nodetool enablethrift (CASSANDRA-16127) + * Automatically drop compact storage on tables for which it is safe (CASSANDRA-16048) -3.11.8 +4.0-beta2 + * Add addition incremental repair visibility to nodetool repair_admin (CASSANDRA-14939) + * Always access system properties and environment variables via the new CassandraRelevantProperties and CassandraRelevantEnv classes (CASSANDRA-15876) + * Remove deprecated HintedHandOffManager (CASSANDRA-15939) + * Prevent repair from overrunning compaction (CASSANDRA-15817) + * fix cqlsh COPY functions in Python 3.8 on Mac (CASSANDRA-16053) + * Strip comment blocks from cqlsh input before processing statements (CASSANDRA-15802) + * Fix unicode chars error input (CASSANDRA-15990) + * Improved testability for CacheMetrics and ChunkCacheMetrics (CASSANDRA-15788) + * Handle errors in StreamSession#prepare (CASSANDRA-15852) + * FQL replay should have options to ignore DDL statements (CASSANDRA-16039) + * Remove COMPACT STORAGE internals (CASSANDRA-13994) + * Make TimestampSerializer accept fractional seconds of varying precision (CASSANDRA-15976) + * Improve cassandra-stress logging when using a profile file that doesn't exist (CASSANDRA-14425) + * Improve logging for socket connection/disconnection (CASSANDRA-15980) + * Throw FSWriteError upon write failures in order to apply DiskFailurePolicy (CASSANDRA-15928) + * Forbid altering UDTs used in partition keys (CASSANDRA-15933) + * Fix version parsing logic when upgrading from 3.0 (CASSANDRA-15973) + * Optimize NoSpamLogger use in hot paths (CASSANDRA-15766) + * Verify sstable components on startup (CASSANDRA-15945) + * Resolve JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node (CASSANDRA-15937) +Merged from 3.11: * Correctly interpret SASI's `max_compaction_flush_memory_in_mb` setting in megabytes not bytes (CASSANDRA-16071) * Fix short read protection for GROUP BY queries (CASSANDRA-15459) + * stop_paranoid disk failure policy is ignored on CorruptSSTableException after node is up (CASSANDRA-15191) * Frozen RawTuple is not annotated with frozen in the toString method (CASSANDRA-15857) Merged from 3.0: - * Use IF NOT EXISTS for index and UDT create statements in snapshot schema files (CASSANDRA-13935) * Fix gossip shutdown order (CASSANDRA-15816) * Remove broken 'defrag-on-read' optimization (CASSANDRA-15432) * Check for endpoint collision with hibernating nodes (CASSANDRA-14599) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (5bb76ba -> d44dbd9)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5bb76ba Merge branch 'cassandra-2.2' into cassandra-3.0 new 72941b9 Package tools/bin scripts as executable new d44dbd9 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d44dbd91c1752a71f4819be92fd75a52be9f0118 Merge: 5bb76ba 72941b9 Author: Andrés de la Peña AuthorDate: Wed Oct 21 13:19:13 2020 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 # Conflicts: # CHANGES.txt CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --cc CHANGES.txt index 0f9694e,8a78993..af7245a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,9 +1,24 @@@ -2.2.19 +3.0.23: + * Check SSTables for latest version before dropping compact storage (CASSANDRA-16063) + * Handle unexpected columns due to schema races (CASSANDRA-15899) + * Avoid failing compactions with very large partitions (CASSANDRA-15164) + * Use IF NOT EXISTS for index and UDT create statements in snapshot schema files (CASSANDRA-13935) + * Add flag to ignore unreplicated keyspaces during repair (CASSANDRA-15160) +Merged from 2.2: + * Package tools/bin scripts as executable (CASSANDRA-16151) - * Fix ExceptionInInitializerError when data_file_directories is not set (CASSANDRA-16008) * Fixed a NullPointerException when calling nodetool enablethrift (CASSANDRA-16127) -2.2.18 +3.0.22: + * Fix gossip shutdown order (CASSANDRA-15816) + * Remove broken 'defrag-on-read' optimization (CASSANDRA-15432) + * Check for endpoint collision with hibernating nodes (CASSANDRA-14599) + * Operational improvements and hardening for replica filtering protection (CASSANDRA-15907) + * stop_paranoid disk failure policy is ignored on CorruptSSTableException after node is up (CASSANDRA-15191) + * 3.x fails to start if commit log has range tombstones from a column which is also deleted (CASSANDRA-15970) + * Forbid altering UDTs used in partition keys (CASSANDRA-15933) + * Fix empty/null json string representation (CASSANDRA-15896) + * Handle difference in timestamp precision between java8 and java11 in LogFIle.java (CASSANDRA-16050) +Merged from 2.2: * Fix CQL parsing of collections when the column type is reversed (CASSANDRA-15814) Merged from 2.1: * Only allow strings to be passed to JMX authentication (CASSANDRA-16077) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit a2f59be94388754e667f545b02798810056f9b1b Merge: bca91ca d44dbd9 Author: Andrés de la Peña AuthorDate: Wed Oct 21 13:19:50 2020 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt | 1 + build.xml | 14 ++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --cc CHANGES.txt index 6c41cee,af7245a..af44f96 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,23 -1,14 +1,24 @@@ -3.0.23: +3.11.9 + * Synchronize Keyspace instance store/clear (CASSANDRA-16210) + * Fix ColumnFilter to avoid querying cells of unselected complex columns (CASSANDRA-15977) + * Fix memory leak in CompressedChunkReader (CASSANDRA-15880) + * Don't attempt value skipping with mixed version cluster (CASSANDRA-15833) + * Avoid failing compactions with very large partitions (CASSANDRA-15164) + * Make sure LCS handles duplicate sstable added/removed notifications correctly (CASSANDRA-14103) +Merged from 3.0: * Check SSTables for latest version before dropping compact storage (CASSANDRA-16063) * Handle unexpected columns due to schema races (CASSANDRA-15899) - * Avoid failing compactions with very large partitions (CASSANDRA-15164) - * Use IF NOT EXISTS for index and UDT create statements in snapshot schema files (CASSANDRA-13935) * Add flag to ignore unreplicated keyspaces during repair (CASSANDRA-15160) Merged from 2.2: + * Package tools/bin scripts as executable (CASSANDRA-16151) * Fixed a NullPointerException when calling nodetool enablethrift (CASSANDRA-16127) -3.0.22: +3.11.8 + * Correctly interpret SASI's `max_compaction_flush_memory_in_mb` setting in megabytes not bytes (CASSANDRA-16071) + * Fix short read protection for GROUP BY queries (CASSANDRA-15459) + * Frozen RawTuple is not annotated with frozen in the toString method (CASSANDRA-15857) +Merged from 3.0: + * Use IF NOT EXISTS for index and UDT create statements in snapshot schema files (CASSANDRA-13935) * Fix gossip shutdown order (CASSANDRA-15816) * Remove broken 'defrag-on-read' optimization (CASSANDRA-15432) * Check for endpoint collision with hibernating nodes (CASSANDRA-14599) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15690) Single partition queries can mistakenly omit partition deletions and resurrect data
[ https://issues.apache.org/jira/browse/CASSANDRA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15690: --- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.0.21 3.11.7 > Single partition queries can mistakenly omit partition deletions and > resurrect data > --- > > Key: CASSANDRA-15690 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15690 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Coordination >Reporter: Aleksey Yeschenko >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1 > > > We have logic that allows us to exclude sstables with partition deletions > that are older than the minimum collected timestamp in a local request. > However, it’s possible that another node could have rows that aren’t known to > the local node that are in turn older than the excluded partition deletion. > In such a scenario, those will be mistakenly resurrected, which is a > correctness issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16151:
--
Fix Version/s: 3.0.23
2.2.19
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 2.2.19, 3.0.23, 3.11.9
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16151:
--
Status: Ready to Commit (was: Review In Progress)
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 3.11.9
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[
https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-16201:
Test and Documentation Plan: cci runs
Status: Patch Available (was: Open)
4.0:
[patch|https://github.com/krummas/cassandra/commits/marcuse/16201-4.0-new],
[cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-4.0-new]
3.11:
[patch|https://github.com/krummas/cassandra/commits/marcuse/16201-3.11-new],
[cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-3.11-new]
3.0:
[patch|https://github.com/krummas/cassandra/commits/marcuse/16201-3.0-new],
[cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-3.0-new]
this focuses only on getMutations as that is where the OOM happened - there is
much more to do, but I think these fixes are fairly safe and make performance
slightly more acceptable
> Reduce amount of allocations during batch statement execution
> -
>
> Key: CASSANDRA-16201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16201
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Thomas Steinmaurer
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile,
> we see 4.0b2 going OOM from time to time. According to a heap dump, we have
> multiple NTR threads in a 3-digit MB range.
> This is likely related to object array pre-allocations at the size of
> {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always
> only 1 {{BTreeRow}} in the {{BTree}}.
> !screenshot-1.png|width=100%!
> So it seems we have many, many 20K elemnts pre-allocated object arrays
> resulting in a shallow heap of 80K each, although there is only one element
> in the array.
> This sort of pre-allocation is causing a lot of memory pressure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16114:
---
Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Status: Review In Progress (was: Patch Available)
> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
>
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Build:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
> at
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
> at
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16114:
---
Reviewers: Benjamin Lerer
> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
>
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Build:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
> at
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
> at
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218214#comment-17218214
]
Angelo Polo commented on CASSANDRA-16151:
-
Patches for 2.2 and 3.0 attached!
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 3.11.9
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Angelo Polo updated CASSANDRA-16151:
Attachment: 3.0-Package-tools-bin-scripts-as-executable.patch
2.2-Package-tools-bin-scripts-as-executable.patch
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 3.11.9
>
> Attachments: 2.2-Package-tools-bin-scripts-as-executable.patch,
> 3.0-Package-tools-bin-scripts-as-executable.patch,
> 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[
https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218202#comment-17218202
]
Berenguer Blasi edited comment on CASSANDRA-15865 at 10/21/20, 9:51 AM:
Some progress. The only failure's logs look like
{noformat}
grep -ri "is now" node1_debug.log
INFO [GossipStage:1] 2020-10-19 15:34:07,204 Gossiper.java:1235 - Node
/127.0.0.2:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:34:07,264 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:19,054 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:34:31,968 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,652 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,653 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:11,715 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:11,719 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:24,727 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:24,762 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
grep -ri "is now" node2_debug.log
INFO [GossipStage:1] 2020-10-19 15:34:07,930 Gossiper.java:1235 - Node
/127.0.0.1:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:34:07,950 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:31,988 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:45,688 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:04,633 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,652 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:24,791 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:25,316 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
{noformat}
whereas all the other million non-failing runs diplay
{noformat}
grep -ri "is now" node1_debug.log
INFO [GossipStage:1] 2020-10-19 15:35:40,341 Gossiper.java:1235 - Node
/127.0.0.2:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:35:40,391 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:52,169 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:05,123 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:44,959 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:57,732 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
grep -ri "is now" node2_debug.log
INFO [GossipStage:1] 2020-10-19 15:35:41,061 Gossiper.java:1235 - Node
/127.0.0.1:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:35:41,094 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:05,150 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:18,807 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:37,710 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,728 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:57,756 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:58,660 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
{noformat}
We can see node2 behaves exactly the same whereas node1 displays an extra
'DOWN'. Matching timestamps this is what's going on in the logs for a bad run
{noformat}
DEBUG [GossipStage:1] 2020-10-19 15:35:11,626 MigrationManager.java:89 - Not
pulling schema from /127.0.0.2:7000, because schema versions match
(7e1669d0-936a-331b-8aca-b6a47709bc60)
INFO [GossipStage:1] 2020-10-19 15:35:11,715 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
DEBUG [GossipStage:1] 2020-10-19 15:35:11,716 FailureDetector.java:354 -
Forcing conviction of /127.0.0.2:7000
DEBUG [GossipStage:1] 2020-10-19 15:35:11,719
[jira] [Updated] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[ https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15865: Attachment: logsGoodNBad.zip > Flaky dtest > hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow > --- > > Key: CASSANDRA-15865 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15865 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Sam Tunnicliffe >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Attachments: logsGoodNBad.zip > > Time Spent: 20m > Remaining Estimate: 0h > > I've seen this fail a couple of times under JDK11, when it doesn't appear to > be related to the changes under test. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[
https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218202#comment-17218202
]
Berenguer Blasi commented on CASSANDRA-15865:
-
Some progress. The only failure's logs look like
{noformat}
grep -ri "is now" node1_debug.log
INFO [GossipStage:1] 2020-10-19 15:34:07,204 Gossiper.java:1235 - Node
/127.0.0.2:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:34:07,264 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:19,054 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:34:31,968 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,652 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,653 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:11,715 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:11,719 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:24,727 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:24,762 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
grep -ri "is now" node2_debug.log
INFO [GossipStage:1] 2020-10-19 15:34:07,930 Gossiper.java:1235 - Node
/127.0.0.1:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:34:07,950 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:31,988 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:34:45,688 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:35:04,633 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:04,652 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:24,791 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:25,316 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
{noformat}
whereas all the other million non-failing runs diplay
{noformat}
grep -ri "is now" node1_debug.log
INFO [GossipStage:1] 2020-10-19 15:35:40,341 Gossiper.java:1235 - Node
/127.0.0.2:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:35:40,391 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:35:52,169 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:05,123 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,729 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:44,959 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:57,732 Gossiper.java:1193 - InetAddress
/127.0.0.2:7000 is now UP
grep -ri "is now" node2_debug.log
INFO [GossipStage:1] 2020-10-19 15:35:41,061 Gossiper.java:1235 - Node
/127.0.0.1:7000 is now part of the cluster
INFO [GossipStage:1] 2020-10-19 15:35:41,094 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:05,150 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:18,807 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
INFO [GossipStage:1] 2020-10-19 15:36:37,710 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:37,728 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:57,756 Gossiper.java:1193 - InetAddress
/127.0.0.1:7000 is now UP
INFO [GossipStage:1] 2020-10-19 15:36:58,660 Gossiper.java:1211 - InetAddress
/127.0.0.1:7000 is now DOWN
{noformat}
We can see node2 behaves exactly the same whereas node1 displays an extra
'DOWN'. Matching timestamps this is what's going on in the logs for a bad run
{noformat}
DEBUG [GossipStage:1] 2020-10-19 15:35:11,626 MigrationManager.java:89 - Not
pulling schema from /127.0.0.2:7000, because schema versions match
(7e1669d0-936a-331b-8aca-b6a47709bc60)
INFO [GossipStage:1] 2020-10-19 15:35:11,715 Gossiper.java:1211 - InetAddress
/127.0.0.2:7000 is now DOWN
DEBUG [GossipStage:1] 2020-10-19 15:35:11,716 FailureDetector.java:354 -
Forcing conviction of /127.0.0.2:7000
DEBUG [GossipStage:1] 2020-10-19 15:35:11,719 Gossiper.java:1192 - removing
expire time for endpoint :
[jira] [Comment Edited] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218182#comment-17218182
]
Michael Semb Wever edited comment on CASSANDRA-16205 at 10/21/20, 8:18 AM:
---
bq. If the objective here is to pre-generate tokens to speed up dtests on
CASSANDRA-16079, couldn't we generate them using the latter approach on
ccm/dtest ?
For CASSANDRA-16079 isn't the code (and the imbalance) parity what we want?
That is we want tests to be running on the same setups that
{{allocate_tokens_for_local_replication_factor}} generates, since that is our
default (soon to be).
I agree that the script could be extended to generate "balanced", but the user
will still want the RF algorithm approach for those adding nodes to an existing
cluster situations (for the same reason we have the algorithm inside C*). If
you agree, can this then be a follow on ticket?
was (Author: michaelsembwever):
bq. If the objective here is to pre-generate tokens to speed up dtests on
CASSANDRA-16079, couldn't we generate them using the latter approach on
ccm/dtest ?
For CASSANDRA-16079 isn't the code (and the imbalance) parity what we want?
That is we want tests to be running on the same setups that
allocate_tokens_for_local_replication_factor generates, since that is our
default (soon to be).
I agree that the script could be extended to generate "balanced", but the user
will still want the RF algorithm approach for those adding nodes to an existing
cluster situations (for the same reason we have the algorithm inside C*). If
you agree, can this then be a follow on ticket?
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218182#comment-17218182
]
Michael Semb Wever commented on CASSANDRA-16205:
bq. If the objective here is to pre-generate tokens to speed up dtests on
CASSANDRA-16079, couldn't we generate them using the latter approach on
ccm/dtest ?
For CASSANDRA-16079 isn't the code (and the imbalance) parity what we want?
That is we want tests to be running on the same setups that
allocate_tokens_for_local_replication_factor generates, since that is our
default (soon to be).
I agree that the script could be extended to generate "balanced", but the user
will still want the RF algorithm approach for those adding nodes to an existing
cluster situations (for the same reason we have the algorithm inside C*). If
you agree, can this then be a follow on ticket?
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16185) Add tests to cover CommitLog metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yasar Arafath Baigh reassigned CASSANDRA-16185:
---
Assignee: Yasar Arafath Baigh (was: Uchenna)
> Add tests to cover CommitLog metrics
>
>
> Key: CASSANDRA-16185
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16185
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Assignee: Yasar Arafath Baigh
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The only metrics that seems to be covered by unit test for the CommitLog
> metrics is {{oversizedMutations}}. We should add testing the other ones.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15944) ASF CI unit tests on JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218132#comment-17218132 ] Michael Semb Wever commented on CASSANDRA-15944: Fixed with [6d556fe8873296c0a48f747bd9855e462193252d|https://github.com/apache/cassandra-builds/commit/6d556fe8873296c0a48f747bd9855e462193252d]. > ASF CI unit tests on JDK11 > -- > > Key: CASSANDRA-15944 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15944 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta3 > > > ASF CI tests today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > screenshot for naming specifics, attached in CASSANDRA-15809 > > This ticket is to add JDK11 test targets on Cassandra-trunk and > Cassandra-devbranch, for parity to CircleCI's workflows. > > The JDK is specified in the groovy DSL: > > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > This is a continuation from CASSANDRA-15809 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15944) ASF CI unit tests on JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15944: --- Status: Resolved (was: Open) > ASF CI unit tests on JDK11 > -- > > Key: CASSANDRA-15944 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15944 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta3 > > > ASF CI tests today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > screenshot for naming specifics, attached in CASSANDRA-15809 > > This ticket is to add JDK11 test targets on Cassandra-trunk and > Cassandra-devbranch, for parity to CircleCI's workflows. > > The JDK is specified in the groovy DSL: > > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > This is a continuation from CASSANDRA-15809 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: ninja-fix: jvm-dtest-upgrade are not (yet) JDK11 compatible.
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new 6d556fe ninja-fix: jvm-dtest-upgrade are not (yet) JDK11 compatible.
6d556fe is described below
commit 6d556fe8873296c0a48f747bd9855e462193252d
Author: Mick Semb Wever
AuthorDate: Wed Oct 21 08:14:34 2020 +0200
ninja-fix: jvm-dtest-upgrade are not (yet) JDK11 compatible.
ASF CI unit tests on JDK11 (CASSANDRA-15944)
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 607d612..ddd1284 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -470,7 +470,8 @@ cassandraBranches.each {
disabled(false)
using('Cassandra-template-test')
axes {
-if (branchName == 'trunk') {
+// jvm-dtest-upgrade would require mixed JDK compilations
to support JDK11+
+if (branchName == 'trunk' && targetName !=
'jvm-dtest-upgrade') {
jdk(jdkLabel,'jdk_11_latest')
} else {
jdk(jdkLabel)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
