[jira] [Commented] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217260#comment-17217260 ] Berenguer Blasi commented on CASSANDRA-16061: - Increasing the timeout didn't fix it. I managed to repro locally and indeed tested that with no luck. Try setting a VM and reduce RAM and CPU until you hit it. If I come back to this guy I will post exact repro steps. > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16184) ClientRequestSize metric tests should be improved
[ https://issues.apache.org/jira/browse/CASSANDRA-16184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217259#comment-17217259 ] Berenguer Blasi commented on CASSANDRA-16184: - +1 LGTM. > ClientRequestSize metric tests should be improved > - > > Key: CASSANDRA-16184 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16184 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > The current test for {{ClientRequestSize}} only test that the metrics are > updated on request. It does not test that the returned values are the > expected ones. Some test comparisons are done with the wrong values. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16193) Add tests for Messaging/Internode metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217209#comment-17217209 ] maxwellguo commented on CASSANDRA-16193: [~blerer] hi I want to know if I can do some help? > Add tests for Messaging/Internode metrics > - > > Key: CASSANDRA-16193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16193 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > We do not seems to have tests for the Messaging/Internode metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217164#comment-17217164 ] David Capwell commented on CASSANDRA-16152: --- Review (all minor/nit) * https://github.com/apache/cassandra/compare/trunk...jonmeredith:C16152-trunk#diff-19e4d42c9846cd1362ee2f70c66444953892b84c38d5c9c415d42e24de9c88b0R433 Could simplify to {code} public I getFirstRunningInstance() { return stream().filter(i -> !i.isShutdown()).findFirst().orElseThrow(() -> new IllegalStateException("All instances are shutdown")); } {code} * https://github.com/apache/cassandra/compare/trunk...jonmeredith:C16152-trunk#diff-19e4d42c9846cd1362ee2f70c66444953892b84c38d5c9c415d42e24de9c88b0R535 Instance has a coordinator method, so you can call that rather than refetch the instance from the cluster * https://github.com/apache/cassandra/compare/trunk...jonmeredith:C16152-trunk#diff-321b52220c5bd0aaadf275a845143eb208c889c2696ba0d48a5fc880551131d8R508 comment isn't needed, we don't need to extend the dtest api as we know its always that config type as this is the implementation. Assuming the tests pass, the patch LGTM; +1. > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217153#comment-17217153 ] David Capwell commented on CASSANDRA-15935: --- I did my final review, so everything looks mostly fine, so once those comments are dealt with I am good. > Improve machinery for testing consistency in presence of range movements > > > Key: CASSANDRA-15935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently, we can test range movements only by adding and bootstrapping a new > node. This is both inefficient and insufficient for large-scale tests. We > need a possibility to dynamically change ring ownership over the lifetime of > cluster, with a flexibility to changing gossip status of the node from > perspective of other participants, adding and removing nodes from other > nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217152#comment-17217152 ] David Capwell commented on CASSANDRA-15935: --- Left feedback in PR; main comments were * upgrade tests are broken * (not a blocker) methods that use [calls|runs]OnInstance rather than their nodetool versions * StorageService#waitForSchema looks to be used wrong in src/java and causes waits far longer than expected * refactor joinTokenRing rather than fork (not a blocker, but makes the util harder to use as it behaves differently and less likely to be maintained) * gossip helper removeFromRing assert fails with all input * should we provide ApplicationState.NET_VERSION rather than updating messaging service version on all gossip mock events > Improve machinery for testing consistency in presence of range movements > > > Key: CASSANDRA-15935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently, we can test range movements only by adding and bootstrapping a new > node. This is both inefficient and insufficient for large-scale tests. We > need a possibility to dynamically change ring ownership over the lifetime of > cluster, with a flexibility to changing gossip status of the node from > perspective of other participants, adding and removing nodes from other > nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[ https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217144#comment-17217144 ] David Capwell commented on CASSANDRA-16103: --- [~aholmber] bq. For example the schema, workload, and topology? Was the cluster facing outages or adverse conditions? Looking at the clusters impacted, we have 3 categories: diff test (clone cluster in 3.0 and 4.0, and diff 100% of rows), ad-hoc (random people deploying random stuff), test cluster (user tests). Out of these 3 categories the ad-hoc clusters hit this the most, followed by diff test (read only); the test cluster is low traffic and only seen this a small number of times in the past week. Diff is heavy read, that is the only workload I can talk about (because I don't know what people do on the ad-hoc clusters). Now, for the first test that saw this issue, I think it was the following: * tip-stress https://github.com/thelastpickle/tlp-stress/blob/master/src/main/kotlin/com/thelastpickle/tlpstress/profiles/KeyValue.kt * 6 node cluster, single DC * rate 800 qps * concurrency 100 * consistency level LOCAL_ONE I do not remember the read/write percentage, but seeing how this happens for all response types... I don't think it matters I sadly don't remember well if this is correct or not (mostly the profile used is likely to change, the other variables are my defaults), I didn't document this in CASSANDRA-16097 =( > Invalid serialized size for responses > - > > Key: CASSANDRA-16103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16103 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Was running a benchmark at LOCAL_ONE and eventually saw the below exception; > this is related to CASSANDRA-16097 as it was found during the same test. > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb HINT_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb MUTATION_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[ https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217134#comment-17217134 ] David Capwell commented on CASSANDRA-16103: --- Update, looked at all the logs and found the following list {code} verb MUTATION_RSP READ_REPAIR_RSP HINT_RSP READ_RSP FAILURE_RSP PAXOS_COMMIT_RSP {code} > Invalid serialized size for responses > - > > Key: CASSANDRA-16103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16103 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Was running a benchmark at LOCAL_ONE and eventually saw the below exception; > this is related to CASSANDRA-16097 as it was found during the same test. > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb HINT_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb MUTATION_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16183) Add tests to cover ClientRequest metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg reassigned CASSANDRA-16183: - Assignee: Adam Holmberg > Add tests to cover ClientRequest metrics > - > > Key: CASSANDRA-16183 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16183 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > We do not have test that covers the ClientRequest metrics. > * ClientRequestMetrics > * CASClientRequestMetrics > * CASClientWriteRequestMetrics > * ClientWriteRequestMetrics > * ViewWriteMetrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16168) Rename master branch to trunk in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16168: --- Fix Version/s: 4.0-beta3 Source Control Link: https://github.com/apache/cassandra-diff/commits/trunk Resolution: Fixed Status: Resolved (was: Ready to Commit) > Rename master branch to trunk in cassandra-diff > --- > > Key: CASSANDRA-16168 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16168 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta3 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16168) Rename master branch to trunk in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16168: --- Reviewers: Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Michael Semb Wever, Michael Semb Wever Status: Review In Progress (was: Patch Available) > Rename master branch to trunk in cassandra-diff > --- > > Key: CASSANDRA-16168 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16168 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16168) Rename master branch to trunk in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16168: --- Test and Documentation Plan: ci-cassandra.a.o Status: Patch Available (was: In Progress) > Rename master branch to trunk in cassandra-diff > --- > > Key: CASSANDRA-16168 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16168 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[ https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217097#comment-17217097 ] Adam Holmberg commented on CASSANDRA-16103: --- [~dcapwell] is there anything more you can tell us about the test that produced this? For example the schema, workload, and topology? Was the cluster facing outages or adverse conditions? > Invalid serialized size for responses > - > > Key: CASSANDRA-16103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16103 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Was running a benchmark at LOCAL_ONE and eventually saw the below exception; > this is related to CASSANDRA-16097 as it was found during the same test. > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb HINT_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb MUTATION_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16168) Rename master branch to trunk in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16168: --- Status: Ready to Commit (was: Review In Progress) > Rename master branch to trunk in cassandra-diff > --- > > Key: CASSANDRA-16168 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16168 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16061: -- Status: Patch Available (was: In Progress) > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217083#comment-17217083 ] Adam Holmberg commented on CASSANDRA-16061: --- I let this run for quite some time in the background and didn't reproduce for myself. There's not much to go on with the error Berenguer was describing. All failures, in including several on [ci-cassandra|https://ci-cassandra.apache.org/job/Cassandra-trunk/77/testReport/dtest-novnode.transient_replication_ring_test/TestTransientReplicationRing/] right now, are simple client timeouts. For now I think we should just increase the timeout on that request. I'm open to suggestion on whether we should keep this open for a time, or reopen if it continues. [patch|https://github.com/apache/cassandra-dtest/compare/master...aholmberg:CASSANDRA-16061?expand=1] [ci|https://app.circleci.com/pipelines/github/aholmberg/cassandra?branch=CASSANDRA-16061] > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16151) Package tools/bin scripts as executable
[ https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217071#comment-17217071 ] Andres de la Peña commented on CASSANDRA-16151: --- Looks good to me, I think we should also patch 2.2 and 3.0 since they also seem affected. > Package tools/bin scripts as executable > --- > > Key: CASSANDRA-16151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16151 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Labels: patch > Fix For: 4.0-beta, 3.11.9 > > Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch, > trunk-Package-tools-bin-scripts-as-executable.patch > > > The tools/bin scripts aren't packaged as executable in the source > distributions, though in the repository the scripts have the right bits. > This causes, on 3.11.8 for example, the tests in > org.apache.cassandra.cql3.EmptyValuesTest to fail: > {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13, > Permission denied}} > {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}} > {{[junit-timeout] at > org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}} > {{[junit-timeout] at > org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}} > See attached patch of build.xml for the trunk and cassandra-3.11 branches. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[ https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-16151: -- Reviewers: Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Andres de la Peña, Andres de la Peña (was: Andres de la Peña) Status: Review In Progress (was: Patch Available) > Package tools/bin scripts as executable > --- > > Key: CASSANDRA-16151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16151 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Labels: patch > Fix For: 4.0-beta, 3.11.9 > > Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch, > trunk-Package-tools-bin-scripts-as-executable.patch > > > The tools/bin scripts aren't packaged as executable in the source > distributions, though in the repository the scripts have the right bits. > This causes, on 3.11.8 for example, the tests in > org.apache.cassandra.cql3.EmptyValuesTest to fail: > {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13, > Permission denied}} > {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}} > {{[junit-timeout] at > org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}} > {{[junit-timeout] at > org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}} > See attached patch of build.xml for the trunk and cassandra-3.11 branches. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master created (now 5d53454)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git. at 5d53454 Rename master branch to trunk in cassandra-builds (CASSANDRA-16163) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217055#comment-17217055 ] Melissa Logan commented on CASSANDRA-16115: --- As the team dug into using Antora for the main website (in addition to docs), it became clear that it would be far simpler to use a static site generator (SSG) to vastly reduce complexity. Captured thoughts about the recommendation below and would appreciate feedback. Flagging in particular for [~mck], [~polandll], [~Anthony Grasso]. Recommendation Since one of the benefits of a static site generator (SSG) is that content folks can manage content with markdown instead of HTML, we looked at *markdown -> HTML* parsers and found some popular Github repos. The attached files use one called “Marked”. In order to run the attached files, you need a local server running on your computer (since the index.html file needs to fetch the test.md file). For folks who don’t have a local server or aren’t familiar with running one, you can also view the link below. The link itself conveys very simply what the attached files do: namely, all content would be added to the “test.md” file, while all the HTML/JS would be done in the .html file for layout, interactions, etc. This allows for a separation of concerns so developers can modify layout/interaction in the .html files, while content folks can do their thing in the .md file, just like how Antora/Jekyll work (but without the large overhead/complexity those SSR platforms bring). One illustration of how it could work is that the homepage would have a corresponding .md file for each section. The homepage-hero.md file, for example, would have all content that appears at the top of the page. The “What is Apache Cassandra” section with the nine icons would be another markdown file where content could be modified. The idea of having separate files is for to better separate what markdown needs to be parsed for what section. Done this way, we eliminate the setup, overhead, complexity, and build processes that SSGs require. We instead would simply have .html files the developers work with, and .md files the content folks work with. If needed, we can stand up a simple remote server (like [cassandra.traverstodd.com|http://cassandra.traverstodd.com/]) where we’d push these two attached files, so that it can be seen in action. [Demo link here|https://marked.js.org/demo/?text=Marked%20-%20Markdown%20Parser%0A%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%3D%0A%0A%5BMarked%5D%20lets%20you%20convert%20%5BMarkdown%5D%20into%20HTML.%20%20Markdown%20is%20a%20simple%20text%20format%20whose%20goal%20is%20to%20be%20very%20easy%20to%20read%20and%20write%2C%20even%20when%20not%20converted%20to%20HTML.%20%20This%20demo%20page%20will%20let%20you%20type%20anything%20you%20like%20and%20see%20how%20it%20gets%20converted.%20%20Live.%20%20No%20more%20waiting%20around.%0A%0AHow%20To%20Use%20The%20Demo%0A---%0A%0A1.%20Type%20in%20stuff%20on%20the%20left.%0A2.%20See%20the%20live%20updates%20on%20the%20right.%0A%0AThat%27s%20it.%20%20Pretty%20simple.%20%20There%27s%20also%20a%20drop-down%20option%20in%20the%20upper%20right%20to%20switch%20between%20various%20views%3A%0A%0A-%20**Preview%3A**%20%20A%20live%20display%20of%20the%20generated%20HTML%20as%20it%20would%20render%20in%20a%20browser.%0A-%20**HTML%20Source%3A**%20%20The%20generated%20HTML%20before%20your%20browser%20makes%20it%20pretty.%0A-%20**Lexer%20Data%3A**%20%20What%20%5Bmarked%5D%20uses%20internally%2C%20in%20case%20you%20like%20gory%20stuff%20like%20this.%0A-%20**Quick%20Reference%3A**%20%20A%20brief%20run-down%20of%20how%20to%20format%20things%20using%20markdown.%0A%0AWhy%20Markdown%3F%0A-%0A%0AIt%27s%20easy.%20%20It%27s%20not%20overly%20bloated%2C%20unlike%20HTML.%20%20Also%2C%20as%20the%20creator%20of%20%5Bmarkdown%5D%20says%2C%0A%0A%3E%20The%20overriding%20design%20goal%20for%20Markdown%27s%0A%3E%20formatting%20syntax%20is%20to%20make%20it%20as%20readable%0A%3E%20as%20possible.%20The%20idea%20is%20that%20a%0A%3E%20Markdown-formatted%20document%20should%20be%0A%3E%20publishable%20as-is%2C%20as%20plain%20text%2C%20without%0A%3E%20looking%20like%20it%27s%20been%20marked%20up%20with%20tags%0A%3E%20or%20formatting%20instructions.%0A%0AReady%20to%20start%20writing%3F%20%20Either%20start%20changing%20stuff%20on%20the%20left%20or%0A%5Bclear%20everything%5D(%2Fdemo%2F%3Ftext%3D)%20with%20a%20simple%20click.%0A%0A%5BMarked%5D%3A%20https%3A%2F%2Fgithub.com%2Fmarkedjs%2Fmarked%2F%0A%5BMarkdown%5D%3A%20http%3A%2F%2Fdaringfireball.net%2Fprojects%2Fmarkdown%2F%0A==master]. > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL:
[jira] [Updated] (CASSANDRA-16164) Rename master branch to trunk in cassandra-builds
[ https://issues.apache.org/jira/browse/CASSANDRA-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16164: --- Fix Version/s: 4.0-beta3 Source Control Link: https://github.com/apache/cassandra-builds/commit/5d5345478f6a89ef5131bde086c68d98c1146d8b Resolution: Fixed Status: Resolved (was: Ready to Commit) > Rename master branch to trunk in cassandra-builds > - > > Key: CASSANDRA-16164 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16164 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta3 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16164) Rename master branch to trunk in cassandra-builds
[ https://issues.apache.org/jira/browse/CASSANDRA-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16164: --- Status: Ready to Commit (was: Review In Progress) > Rename master branch to trunk in cassandra-builds > - > > Key: CASSANDRA-16164 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16164 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16164) Rename master branch to trunk in cassandra-builds
[ https://issues.apache.org/jira/browse/CASSANDRA-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16164: --- Test and Documentation Plan: ci-cassandra.a.o Status: Patch Available (was: In Progress) > Rename master branch to trunk in cassandra-builds > - > > Key: CASSANDRA-16164 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16164 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16164) Rename master branch to trunk in cassandra-builds
[ https://issues.apache.org/jira/browse/CASSANDRA-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16164: --- Reviewers: Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Michael Semb Wever, Michael Semb Wever Status: Review In Progress (was: Patch Available) > Rename master branch to trunk in cassandra-builds > - > > Key: CASSANDRA-16164 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16164 > Project: Cassandra > Issue Type: Sub-task > Components: Build >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: Rename master branch to trunk in cassandra-builds (CASSANDRA-16163)
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new 5d53454 Rename master branch to trunk in cassandra-builds (CASSANDRA-16163) 5d53454 is described below commit 5d5345478f6a89ef5131bde086c68d98c1146d8b Author: Mick Semb Wever AuthorDate: Mon Oct 19 21:36:23 2020 +0200 Rename master branch to trunk in cassandra-builds (CASSANDRA-16163) --- jenkins-dsl/cassandra_job_dsl_seed.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy b/jenkins-dsl/cassandra_job_dsl_seed.groovy index b706ff8..f779d24 100644 --- a/jenkins-dsl/cassandra_job_dsl_seed.groovy +++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy @@ -34,7 +34,7 @@ def buildsRepo = "https://gitbox.apache.org/repos/asf/cassandra-builds.git; if(binding.hasVariable("CASSANDRA_BUILDS_GIT_URL")) { buildsRepo = "${CASSANDRA_BUILDS_GIT_URL}" } -def buildsBranch = "master" +def buildsBranch = "trunk" if(binding.hasVariable("CASSANDRA_BUILDS_BRANCH")) { buildsBranch = "${CASSANDRA_BUILDS_BRANCH}" } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16057: -- Status: Resolved (was: Ready to Commit) Backport complete https://github.com/apache/cassandra/commit/f15c6b8c06c9588bd96615fabfc36b11857cd4bb > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217007#comment-17217007 ] David Capwell edited comment on CASSANDRA-16057 at 10/19/20, 7:35 PM: -- CI Results: Yellow, expected failures only Branch: cassandra-2.2 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-2.2-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/115/ Branch: cassandra-3.0 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.0-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/116/ Branch: cassandra-3.11 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.11-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/117/ was (Author: dcapwell): Starting commit CI Results (pending): Branch: cassandra-2.2 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-2.2-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/115/ Branch: cassandra-3.0 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.0-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/116/ Branch: cassandra-3.11 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.11-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/117/ > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new f15c6b8 Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool f15c6b8 is described below commit f15c6b8c06c9588bd96615fabfc36b11857cd4bb Author: Yifan Cai AuthorDate: Mon Oct 19 11:09:11 2020 -0700 Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool patch by Yifan Cai; reviewed by Alex Petrov, David Capwell for CASSANDRA-16057 --- src/java/org/apache/cassandra/tools/NodeProbe.java | 15 src/java/org/apache/cassandra/tools/NodeTool.java | 52 - .../tools/{nodetool/Version.java => Output.java} | 22 +++--- .../cassandra/tools/nodetool/BootstrapResume.java | 4 +- .../apache/cassandra/tools/nodetool/Cleanup.java | 7 +- .../cassandra/tools/nodetool/ClearSnapshot.java| 4 +- .../tools/nodetool/CompactionHistory.java | 12 +-- .../cassandra/tools/nodetool/CompactionStats.java | 11 +-- .../cassandra/tools/nodetool/DescribeCluster.java | 16 ++-- .../cassandra/tools/nodetool/DescribeRing.java | 8 +- .../tools/nodetool/FailureDetectorInfo.java| 4 +- .../apache/cassandra/tools/nodetool/GcStats.java | 6 +- .../tools/nodetool/GetCompactionThreshold.java | 8 +- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../cassandra/tools/nodetool/GetEndpoints.java | 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 2 +- .../cassandra/tools/nodetool/GetLoggingLevels.java | 6 +- .../cassandra/tools/nodetool/GetSSTables.java | 4 +- .../tools/nodetool/GetStreamThroughput.java| 4 +- .../tools/nodetool/GetTraceProbability.java| 2 +- .../cassandra/tools/nodetool/GossipInfo.java | 4 +- .../org/apache/cassandra/tools/nodetool/Info.java | 41 +- .../cassandra/tools/nodetool/ListSnapshots.java| 14 ++-- .../apache/cassandra/tools/nodetool/NetStats.java | 44 ++- .../cassandra/tools/nodetool/ProxyHistograms.java | 15 ++-- .../cassandra/tools/nodetool/RangeKeySample.java | 6 +- .../apache/cassandra/tools/nodetool/Refresh.java | 2 +- .../cassandra/tools/nodetool/RemoveNode.java | 6 +- .../apache/cassandra/tools/nodetool/Repair.java| 2 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 29 +++ .../org/apache/cassandra/tools/nodetool/Scrub.java | 2 +- .../apache/cassandra/tools/nodetool/Snapshot.java | 12 +-- .../apache/cassandra/tools/nodetool/Status.java| 34 .../cassandra/tools/nodetool/StatusBackup.java | 4 +- .../cassandra/tools/nodetool/StatusBinary.java | 4 +- .../cassandra/tools/nodetool/StatusGossip.java | 4 +- .../cassandra/tools/nodetool/StatusHandoff.java| 4 +- .../cassandra/tools/nodetool/StatusThrift.java | 4 +- .../cassandra/tools/nodetool/StopDaemon.java | 4 +- .../cassandra/tools/nodetool/TableHistograms.java | 20 +++-- .../cassandra/tools/nodetool/TableStats.java | 90 +++--- .../cassandra/tools/nodetool/TopPartitions.java| 18 +++-- .../apache/cassandra/tools/nodetool/TpStats.java | 12 +-- .../cassandra/tools/nodetool/UpgradeSSTable.java | 2 +- .../apache/cassandra/tools/nodetool/Verify.java| 4 +- .../apache/cassandra/tools/nodetool/Version.java | 4 +- .../cassandra/distributed/impl/Instance.java | 62 +-- .../test/ClientNetworkStopStartTest.java | 28 ++- .../cassandra/distributed/test/NodeToolTest.java | 16 +++- 49 files changed, 398 insertions(+), 288 deletions(-) diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java b/src/java/org/apache/cassandra/tools/NodeProbe.java index 9798763..2ebef5d 100644 --- a/src/java/org/apache/cassandra/tools/NodeProbe.java +++ b/src/java/org/apache/cassandra/tools/NodeProbe.java @@ -115,6 +115,7 @@ public class NodeProbe implements AutoCloseable protected CacheServiceMBean cacheService; protected StorageProxyMBean spProxy; protected HintedHandOffManagerMBean hhProxy; +protected Output output; private boolean failed; /** @@ -133,6 +134,7 @@ public class NodeProbe implements AutoCloseable this.port = port; this.username = username; this.password = password; +this.output = Output.CONSOLE; connect(); } @@ -147,6 +149,7 @@ public class NodeProbe implements AutoCloseable { this.host = host; this.port = port; +this.output = Output.CONSOLE; connect(); } @@ -160,6 +163,7 @@ public class NodeProbe implements AutoCloseable { this.host = host; this.port = defaultPort; +this.output = Output.CONSOLE; connect(); } @@ -168,6 +172,7 @@
[cassandra] branch cassandra-3.0 updated (7b3a15d -> 5bb76ba)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7b3a15d Merge branch 'cassandra-2.2' into cassandra-3.0 new f15c6b8 Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool new 5bb76ba Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 72 - src/java/org/apache/cassandra/tools/NodeProbe.java | 15 src/java/org/apache/cassandra/tools/NodeTool.java | 52 - .../tools/{GetVersion.java => Output.java} | 17 +++- .../cassandra/tools/nodetool/BootstrapResume.java | 4 +- .../apache/cassandra/tools/nodetool/Cleanup.java | 4 +- .../cassandra/tools/nodetool/ClearSnapshot.java| 4 +- .../tools/nodetool/CompactionHistory.java | 12 +-- .../cassandra/tools/nodetool/CompactionStats.java | 8 +- .../cassandra/tools/nodetool/DescribeCluster.java | 16 ++-- .../cassandra/tools/nodetool/DescribeRing.java | 8 +- .../tools/nodetool/FailureDetectorInfo.java| 4 +- .../apache/cassandra/tools/nodetool/GcStats.java | 6 +- .../tools/nodetool/GetCompactionThreshold.java | 8 +- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../cassandra/tools/nodetool/GetEndpoints.java | 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 2 +- .../cassandra/tools/nodetool/GetLoggingLevels.java | 6 +- .../cassandra/tools/nodetool/GetSSTables.java | 4 +- .../tools/nodetool/GetStreamThroughput.java| 4 +- .../tools/nodetool/GetTraceProbability.java| 2 +- .../cassandra/tools/nodetool/GossipInfo.java | 4 +- .../org/apache/cassandra/tools/nodetool/Info.java | 41 +- .../cassandra/tools/nodetool/ListSnapshots.java| 12 +-- .../apache/cassandra/tools/nodetool/NetStats.java | 44 ++- .../cassandra/tools/nodetool/ProxyHistograms.java | 15 ++-- .../cassandra/tools/nodetool/RangeKeySample.java | 6 +- .../apache/cassandra/tools/nodetool/Refresh.java | 2 +- .../cassandra/tools/nodetool/RemoveNode.java | 6 +- .../apache/cassandra/tools/nodetool/Repair.java| 2 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 29 +++ .../org/apache/cassandra/tools/nodetool/Scrub.java | 5 +- .../apache/cassandra/tools/nodetool/Snapshot.java | 12 +-- .../apache/cassandra/tools/nodetool/Status.java| 34 .../cassandra/tools/nodetool/StatusBackup.java | 4 +- .../cassandra/tools/nodetool/StatusBinary.java | 4 +- .../cassandra/tools/nodetool/StatusGossip.java | 4 +- .../cassandra/tools/nodetool/StatusHandoff.java| 9 ++- .../cassandra/tools/nodetool/StatusThrift.java | 4 +- .../cassandra/tools/nodetool/TableHistograms.java | 20 +++-- .../cassandra/tools/nodetool/TableStats.java | 90 +++--- .../cassandra/tools/nodetool/TopPartitions.java| 18 +++-- .../apache/cassandra/tools/nodetool/TpStats.java | 12 +-- .../cassandra/tools/nodetool/UpgradeSSTable.java | 4 +- .../apache/cassandra/tools/nodetool/Verify.java| 4 +- .../apache/cassandra/tools/nodetool/Version.java | 4 +- .../cassandra/distributed/impl/Instance.java | 62 +-- .../test/ClientNetworkStopStartTest.java | 28 ++- .../cassandra/distributed/test/NodeToolTest.java | 16 +++- 49 files changed, 435 insertions(+), 316 deletions(-) copy src/java/org/apache/cassandra/tools/{GetVersion.java => Output.java} (73%) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5bb76ba97aa4decc8a0af676873e810ea036968e Merge: 7b3a15d f15c6b8 Author: David Capwell AuthorDate: Mon Oct 19 12:29:33 2020 -0700 Merge branch 'cassandra-2.2' into cassandra-3.0 .circleci/config.yml | 72 - src/java/org/apache/cassandra/tools/NodeProbe.java | 15 src/java/org/apache/cassandra/tools/NodeTool.java | 52 - .../tools/{nodetool/Version.java => Output.java} | 22 +++--- .../cassandra/tools/nodetool/BootstrapResume.java | 4 +- .../apache/cassandra/tools/nodetool/Cleanup.java | 4 +- .../cassandra/tools/nodetool/ClearSnapshot.java| 4 +- .../tools/nodetool/CompactionHistory.java | 12 +-- .../cassandra/tools/nodetool/CompactionStats.java | 8 +- .../cassandra/tools/nodetool/DescribeCluster.java | 16 ++-- .../cassandra/tools/nodetool/DescribeRing.java | 8 +- .../tools/nodetool/FailureDetectorInfo.java| 4 +- .../apache/cassandra/tools/nodetool/GcStats.java | 6 +- .../tools/nodetool/GetCompactionThreshold.java | 8 +- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../cassandra/tools/nodetool/GetEndpoints.java | 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 2 +- .../cassandra/tools/nodetool/GetLoggingLevels.java | 6 +- .../cassandra/tools/nodetool/GetSSTables.java | 4 +- .../tools/nodetool/GetStreamThroughput.java| 4 +- .../tools/nodetool/GetTraceProbability.java| 2 +- .../cassandra/tools/nodetool/GossipInfo.java | 4 +- .../org/apache/cassandra/tools/nodetool/Info.java | 41 +- .../cassandra/tools/nodetool/ListSnapshots.java| 12 +-- .../apache/cassandra/tools/nodetool/NetStats.java | 44 ++- .../cassandra/tools/nodetool/ProxyHistograms.java | 15 ++-- .../cassandra/tools/nodetool/RangeKeySample.java | 6 +- .../apache/cassandra/tools/nodetool/Refresh.java | 2 +- .../cassandra/tools/nodetool/RemoveNode.java | 6 +- .../apache/cassandra/tools/nodetool/Repair.java| 2 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 29 +++ .../org/apache/cassandra/tools/nodetool/Scrub.java | 5 +- .../apache/cassandra/tools/nodetool/Snapshot.java | 12 +-- .../apache/cassandra/tools/nodetool/Status.java| 34 .../cassandra/tools/nodetool/StatusBackup.java | 4 +- .../cassandra/tools/nodetool/StatusBinary.java | 4 +- .../cassandra/tools/nodetool/StatusGossip.java | 4 +- .../cassandra/tools/nodetool/StatusHandoff.java| 9 ++- .../cassandra/tools/nodetool/StatusThrift.java | 4 +- .../cassandra/tools/nodetool/TableHistograms.java | 20 +++-- .../cassandra/tools/nodetool/TableStats.java | 90 +++--- .../cassandra/tools/nodetool/TopPartitions.java| 18 +++-- .../apache/cassandra/tools/nodetool/TpStats.java | 12 +-- .../cassandra/tools/nodetool/UpgradeSSTable.java | 4 +- .../apache/cassandra/tools/nodetool/Verify.java| 4 +- .../apache/cassandra/tools/nodetool/Version.java | 4 +- .../cassandra/distributed/impl/Instance.java | 62 +-- .../test/ClientNetworkStopStartTest.java | 28 ++- .../cassandra/distributed/test/NodeToolTest.java | 16 +++- 49 files changed, 434 insertions(+), 322 deletions(-) diff --cc .circleci/config.yml index 1fb2fcb,d5efe4f..2e40513 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@@ -1,97 -1,5 +1,97 @@@ version: 2 jobs: + j8_jvm_upgrade_dtests: +docker: +- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306 - resource_class: medium ++resource_class: xlarge +working_directory: ~/ +shell: /bin/bash -eo pipefail -l +parallelism: 1 +steps: +- attach_workspace: +at: /home/cassandra +- run: +name: Determine distributed Tests to Run +command: | + # reminder: this code (along with all the steps) is independently executed on every circle container + # so the goal here is to get the circleci script to return the tests *this* container will run + # which we do via the `circleci` cli tool. + + rm -fr ~/cassandra-dtest/upgrade_tests + echo "***java tests***" + + # get all of our unit test filenames + set -eo pipefail && circleci tests glob "$HOME/cassandra/test/distributed/**/*.java" > /tmp/all_java_unit_tests.txt + + # split up the unit tests into groups based on the number of containers we have + set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt + set -eo pipefail && cat
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 95d60a06f298cf3187c1f6cd7aa5cdb344f4f08c Merge: 92f7c8d 7c407c4 Author: David Capwell AuthorDate: Mon Oct 19 12:32:29 2020 -0700 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (c3636bf -> 7c407c4)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from c3636bf Fix ColumnFilter to avoid querying cells of unselected complex columns new f15c6b8 Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool new 5bb76ba Merge branch 'cassandra-2.2' into cassandra-3.0 new 7c407c4 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 82 +++--- src/java/org/apache/cassandra/tools/NodeProbe.java | 15 src/java/org/apache/cassandra/tools/NodeTool.java | 52 -- .../tools/{GetVersion.java => Output.java} | 17 +++-- .../cassandra/tools/nodetool/BootstrapResume.java | 4 +- .../apache/cassandra/tools/nodetool/Cleanup.java | 2 +- .../cassandra/tools/nodetool/ClearSnapshot.java| 4 +- .../tools/nodetool/CompactionHistory.java | 2 +- .../cassandra/tools/nodetool/CompactionStats.java | 22 +++--- .../cassandra/tools/nodetool/DescribeCluster.java | 18 ++--- .../cassandra/tools/nodetool/DescribeRing.java | 8 ++- .../tools/nodetool/FailureDetectorInfo.java| 4 +- .../cassandra/tools/nodetool/GarbageCollect.java | 2 +- .../apache/cassandra/tools/nodetool/GcStats.java | 6 +- .../tools/nodetool/GetCompactionThreshold.java | 8 +-- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../tools/nodetool/GetConcurrentCompactors.java| 2 +- .../cassandra/tools/nodetool/GetEndpoints.java | 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 2 +- .../cassandra/tools/nodetool/GetLoggingLevels.java | 6 +- .../cassandra/tools/nodetool/GetSSTables.java | 4 +- .../tools/nodetool/GetStreamThroughput.java| 4 +- .../cassandra/tools/nodetool/GetTimeout.java | 2 +- .../tools/nodetool/GetTraceProbability.java| 2 +- .../cassandra/tools/nodetool/GossipInfo.java | 4 +- .../org/apache/cassandra/tools/nodetool/Info.java | 45 ++-- .../cassandra/tools/nodetool/ListSnapshots.java| 13 ++-- .../apache/cassandra/tools/nodetool/NetStats.java | 44 ++-- .../cassandra/tools/nodetool/ProxyHistograms.java | 27 +++ .../cassandra/tools/nodetool/RangeKeySample.java | 6 +- .../apache/cassandra/tools/nodetool/Refresh.java | 2 +- .../cassandra/tools/nodetool/RemoveNode.java | 6 +- .../apache/cassandra/tools/nodetool/Repair.java| 2 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 27 +++ .../org/apache/cassandra/tools/nodetool/Scrub.java | 5 +- .../apache/cassandra/tools/nodetool/Snapshot.java | 12 ++-- .../apache/cassandra/tools/nodetool/Status.java| 32 + .../cassandra/tools/nodetool/StatusBackup.java | 4 +- .../cassandra/tools/nodetool/StatusBinary.java | 4 +- .../cassandra/tools/nodetool/StatusGossip.java | 4 +- .../cassandra/tools/nodetool/StatusHandoff.java| 9 ++- .../cassandra/tools/nodetool/StatusThrift.java | 4 +- .../cassandra/tools/nodetool/TableHistograms.java | 19 ++--- .../cassandra/tools/nodetool/TableStats.java | 2 +- .../cassandra/tools/nodetool/TopPartitions.java| 18 ++--- .../apache/cassandra/tools/nodetool/TpStats.java | 4 +- .../cassandra/tools/nodetool/UpgradeSSTable.java | 4 +- .../apache/cassandra/tools/nodetool/Verify.java| 4 +- .../apache/cassandra/tools/nodetool/Version.java | 4 +- .../cassandra/tools/nodetool/ViewBuildStatus.java | 10 +-- .../cassandra/distributed/impl/Instance.java | 62 ++-- .../test/ClientNetworkStopStartTest.java | 28 ++-- .../cassandra/distributed/test/NodeToolTest.java | 16 - .../apache/cassandra/stress/CompactionStress.java | 2 +- 54 files changed, 405 insertions(+), 294 deletions(-) copy src/java/org/apache/cassandra/tools/{GetVersion.java => Output.java} (73%) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7c407c4380cd2adfed5f6b1df10dbc18e9c522b1 Merge: c3636bf 5bb76ba Author: David Capwell AuthorDate: Mon Oct 19 12:30:25 2020 -0700 Merge branch 'cassandra-3.0' into cassandra-3.11 .circleci/config.yml | 82 +++--- src/java/org/apache/cassandra/tools/NodeProbe.java | 15 src/java/org/apache/cassandra/tools/NodeTool.java | 52 -- .../tools/{nodetool/Version.java => Output.java} | 22 +++--- .../cassandra/tools/nodetool/BootstrapResume.java | 4 +- .../apache/cassandra/tools/nodetool/Cleanup.java | 2 +- .../cassandra/tools/nodetool/ClearSnapshot.java| 4 +- .../tools/nodetool/CompactionHistory.java | 2 +- .../cassandra/tools/nodetool/CompactionStats.java | 22 +++--- .../cassandra/tools/nodetool/DescribeCluster.java | 18 ++--- .../cassandra/tools/nodetool/DescribeRing.java | 8 ++- .../tools/nodetool/FailureDetectorInfo.java| 4 +- .../cassandra/tools/nodetool/GarbageCollect.java | 2 +- .../apache/cassandra/tools/nodetool/GcStats.java | 6 +- .../tools/nodetool/GetCompactionThreshold.java | 8 +-- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../tools/nodetool/GetConcurrentCompactors.java| 2 +- .../cassandra/tools/nodetool/GetEndpoints.java | 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 2 +- .../cassandra/tools/nodetool/GetLoggingLevels.java | 6 +- .../cassandra/tools/nodetool/GetSSTables.java | 4 +- .../tools/nodetool/GetStreamThroughput.java| 4 +- .../cassandra/tools/nodetool/GetTimeout.java | 2 +- .../tools/nodetool/GetTraceProbability.java| 2 +- .../cassandra/tools/nodetool/GossipInfo.java | 4 +- .../org/apache/cassandra/tools/nodetool/Info.java | 45 ++-- .../cassandra/tools/nodetool/ListSnapshots.java| 13 ++-- .../apache/cassandra/tools/nodetool/NetStats.java | 44 ++-- .../cassandra/tools/nodetool/ProxyHistograms.java | 27 +++ .../cassandra/tools/nodetool/RangeKeySample.java | 6 +- .../apache/cassandra/tools/nodetool/Refresh.java | 2 +- .../cassandra/tools/nodetool/RemoveNode.java | 6 +- .../apache/cassandra/tools/nodetool/Repair.java| 2 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 27 +++ .../org/apache/cassandra/tools/nodetool/Scrub.java | 5 +- .../apache/cassandra/tools/nodetool/Snapshot.java | 12 ++-- .../apache/cassandra/tools/nodetool/Status.java| 32 + .../cassandra/tools/nodetool/StatusBackup.java | 4 +- .../cassandra/tools/nodetool/StatusBinary.java | 4 +- .../cassandra/tools/nodetool/StatusGossip.java | 4 +- .../cassandra/tools/nodetool/StatusHandoff.java| 9 ++- .../cassandra/tools/nodetool/StatusThrift.java | 4 +- .../cassandra/tools/nodetool/TableHistograms.java | 19 ++--- .../cassandra/tools/nodetool/TableStats.java | 2 +- .../cassandra/tools/nodetool/TopPartitions.java| 18 ++--- .../apache/cassandra/tools/nodetool/TpStats.java | 4 +- .../cassandra/tools/nodetool/UpgradeSSTable.java | 4 +- .../apache/cassandra/tools/nodetool/Verify.java| 4 +- .../apache/cassandra/tools/nodetool/Version.java | 4 +- .../cassandra/tools/nodetool/ViewBuildStatus.java | 10 +-- .../cassandra/distributed/impl/Instance.java | 62 ++-- .../test/ClientNetworkStopStartTest.java | 28 ++-- .../cassandra/distributed/test/NodeToolTest.java | 16 - .../apache/cassandra/stress/CompactionStress.java | 2 +- 54 files changed, 404 insertions(+), 300 deletions(-) diff --cc .circleci/config.yml index ebc13d3,2e40513..37dcf7a --- a/.circleci/config.yml +++ b/.circleci/config.yml @@@ -3,10 -3,10 +3,10 @@@ jobs j8_jvm_upgrade_dtests: docker: - image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306 - resource_class: medium + resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l --parallelism: 1 ++parallelism: 2 steps: - attach_workspace: at: /home/cassandra @@@ -326,56 -326,10 +326,56 @@@ - CASSANDRA_SKIP_SYNC: true - DTEST_REPO: git://github.com/apache/cassandra-dtest.git - DTEST_BRANCH: master - - CCM_MAX_HEAP_SIZE: 1024M - - CCM_HEAP_NEWSIZE: 256M + - CCM_MAX_HEAP_SIZE: 2048M + - CCM_HEAP_NEWSIZE: 512M - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64 + utests_stress: +docker: +- image: spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306 - resource_class: medium ++resource_class: xlarge +working_directory: ~/ +shell: /bin/bash -eo pipefail -l +parallelism: 1 +steps: +- attach_workspace: +at:
[cassandra] branch trunk updated (92f7c8d -> 95d60a0)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 92f7c8d Ignore COMPACT STORAGE flag for tables for which its safe to do so new f15c6b8 Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool new 5bb76ba Merge branch 'cassandra-2.2' into cassandra-3.0 new 7c407c4 Merge branch 'cassandra-3.0' into cassandra-3.11 new 95d60a0 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[ https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-16048: Source Control Link: https://github.com/apache/cassandra/commit/92f7c8db1444bf5d757cd50dba2211a446f3b22c Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/92f7c8db1444bf5d757cd50dba2211a446f3b22c > Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and > Value Columns > -- > > Key: CASSANDRA-16048 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16048 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Some compact storage tables, specifically those where the user has defined > both at least one clustering and the value column, can be safely handled in > 4.0 because besides the DENSE flag they are not materially different post 3.0 > and there is no visible change to the user facing schema after dropping > compact storage. We can detect this case and allow these tables to silently > drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE > tables that don’t meet the criteria. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Ignore COMPACT STORAGE flag for tables for which its safe to do so
This is an automated email from the ASF dual-hosted git repository. jwest pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 92f7c8d Ignore COMPACT STORAGE flag for tables for which its safe to do so 92f7c8d is described below commit 92f7c8db1444bf5d757cd50dba2211a446f3b22c Author: Jordan West AuthorDate: Mon Aug 17 14:20:32 2020 -0700 Ignore COMPACT STORAGE flag for tables for which its safe to do so patch by Jordan West; Reviewed by Marcus Eriksson and Caleb Rackliffe for CASSANDRA-16048 --- CHANGES.txt| 1 + .../apache/cassandra/schema/SchemaKeyspace.java| 60 ++- .../cassandra/distributed/impl/Instance.java | 10 ++ .../upgrade/CompactStorage3to4UpgradeTest.java | 190 + 4 files changed, 255 insertions(+), 6 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 70bbd4f..04af373 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -35,6 +35,7 @@ Merged from 3.0: * Add flag to ignore unreplicated keyspaces during repair (CASSANDRA-15160) Merged from 2.2: * Fixed a NullPointerException when calling nodetool enablethrift (CASSANDRA-16127) + * Automatically drop compact storage on tables for which it is safe (CASSANDRA-16048) 4.0-beta2 * Add addition incremental repair visibility to nodetool repair_admin (CASSANDRA-14939) diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java index 0333ee6..f6dbe03 100644 --- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java +++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java @@ -866,15 +866,34 @@ public final class SchemaKeyspace StringBuilder messages = new StringBuilder(); for (UntypedResultSet.Row row : query(query)) { -if (SchemaConstants.isLocalSystemKeyspace(row.getString("keyspace_name"))) +String keyspaceName = row.getString("keyspace_name"); +if (SchemaConstants.isLocalSystemKeyspace(keyspaceName)) continue; -Set flags = row.getFrozenSet("flags", UTF8Type.instance); -if (TableMetadata.Flag.isLegacyCompactTable(TableMetadata.Flag.fromStringSet(flags))) +Set flags = TableMetadata.Flag.fromStringSet(row.getFrozenSet("flags", UTF8Type.instance)); +if (TableMetadata.Flag.isLegacyCompactTable(flags)) { -messages.append(String.format("ALTER TABLE %s.%s DROP COMPACT STORAGE;\n", - maybeQuote(row.getString("keyspace_name")), - maybeQuote(row.getString("table_name"; +String tableName = row.getString("table_name"); +if (isSafeToDropCompactStorage(keyspaceName, tableName)) +{ +flags.remove(TableMetadata.Flag.DENSE); +flags.add(TableMetadata.Flag.COMPOUND); +String update = String.format("UPDATE %s.%s SET flags={%s} WHERE keyspace_name='%s' AND table_name='%s'", + SchemaConstants.SCHEMA_KEYSPACE_NAME, TABLES, + TableMetadata.Flag.toStringSet(flags).stream() + .map(f -> "'" + f + "'") + .collect(Collectors.joining(", ")), + keyspaceName, tableName); + +logger.info("Safely dropping COMPACT STORAGE on {}.{}", keyspaceName, tableName); +executeInternal(update); +} +else +{ +messages.append(String.format("ALTER TABLE %s.%s DROP COMPACT STORAGE;\n", + maybeQuote(row.getString("keyspace_name")), + maybeQuote(tableName))); +} } } @@ -889,6 +908,35 @@ public final class SchemaKeyspace } } +private static boolean isSafeToDropCompactStorage(String keyspaceName, String tableName) +{ +if (!Boolean.parseBoolean(System.getProperty("cassandra.auto_drop_compact_storage", "false"))) +return false; + +String columnQuery = String.format("SELECT kind, type FROM %s.%s WHERE keyspace_name='%s' and table_name='%s'", + SchemaConstants.SCHEMA_KEYSPACE_NAME, COLUMNS, keyspaceName, tableName); + +String simpleType = "empty"; +int simpleCount = 0; +for (UntypedResultSet.Row row : query(columnQuery)) +{ +String kind
[jira] [Updated] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[ https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16048: Status: Ready to Commit (was: Review In Progress) +1 > Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and > Value Columns > -- > > Key: CASSANDRA-16048 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16048 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Some compact storage tables, specifically those where the user has defined > both at least one clustering and the value column, can be safely handled in > 4.0 because besides the DENSE flag they are not materially different post 3.0 > and there is no visible change to the user facing schema after dropping > compact storage. We can detect this case and allow these tables to silently > drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE > tables that don’t meet the criteria. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[ https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216958#comment-17216958 ] Jordan West edited comment on CASSANDRA-16048 at 10/19/20, 6:53 PM: Thanks [~marcuse]. Addressed all comments and added the flag. My concern was what it meant to opt-in then opt-out but I think its minimal in reality. I wanted to make it a static startup flag but that was a pain for testing so its checked on each call. This should be fine since its called at startup only. was (Author: jrwest): Thanks [~marcuse]. Addressed all comments and added the flag. My concern was what it meant to opt-in then opt-out but I think its minimal in reality. I made it a start-up flag since changing it at runtime wouldn't have meaning. > Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and > Value Columns > -- > > Key: CASSANDRA-16048 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16048 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Some compact storage tables, specifically those where the user has defined > both at least one clustering and the value column, can be safely handled in > 4.0 because besides the DENSE flag they are not materially different post 3.0 > and there is no visible change to the user facing schema after dropping > compact storage. We can detect this case and allow these tables to silently > drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE > tables that don’t meet the criteria. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15234: --- Reviewers: Benjamin Lerer, David Capwell (was: Benjamin Lerer, David Capwell, Michael Semb Wever) > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.0 > > Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217007#comment-17217007 ] David Capwell commented on CASSANDRA-16057: --- Starting commit CI Results (pending): Branch: cassandra-2.2 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-2.2-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/115/ Branch: cassandra-3.0 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.0-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/116/ Branch: cassandra-3.11 Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16057-cassandra-3.11-676928E7-73DD-4B98-8D97-2B57E11D1034 Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/117/ > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217004#comment-17217004 ] Benjamin Lerer commented on CASSANDRA-16192: [~ua038697] My apologies. I got confused reading my emails. I though Fabio was still the assignee but apparently he had unassigned himself already. You did not need to ask. > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: In Jenkins use the new JDK labels (INFRA-20858, CASSANDRA-15943)
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new abc40e8 In Jenkins use the new JDK labels (INFRA-20858,CASSANDRA-15943) abc40e8 is described below commit abc40e88198c46434e2e9caa141b6d5f712ecf1a Author: Mick Semb Wever AuthorDate: Mon Oct 19 20:17:04 2020 +0200 In Jenkins use the new JDK labels (INFRA-20858,CASSANDRA-15943) ref: https://lists.apache.org/thread.html/rb4c2834b9874b9f4a74c528de9055958483d2bc6e62c3464bc5c053f%40%3Cbuilds.apache.org%3E --- jenkins-dsl/cassandra_job_dsl_seed.groovy | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy b/jenkins-dsl/cassandra_job_dsl_seed.groovy index 73129cc..b706ff8 100644 --- a/jenkins-dsl/cassandra_job_dsl_seed.groovy +++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy @@ -10,7 +10,7 @@ def jobDescription = 'http://cassandra.apache.org/img/cassandra_logo.png; />Apache Cassandra DSL-generated job - DSL git repo: https://gitbox.apache.org/repos/asf?p=cassandra-builds.git;>cassandra-builds' -def jdkLabel = 'JDK 1.8 (latest)' +def jdkLabel = 'jdk_1.8_latest' if(binding.hasVariable("CASSANDRA_JDK_LABEL")) { jdkLabel = "${CASSANDRA_JDK_LABEL}" } @@ -439,9 +439,9 @@ cassandraBranches.each { using('Cassandra-template-artifacts') axes { if (branchName == 'trunk') { -jdk('JDK 1.8 (latest)','JDK 11 (latest)') +jdk('jdk_1.8_latest','jdk_11_latest') } else { -jdk('JDK 1.8 (latest)') +jdk('jdk_1.8_latest') } label('label', slaveLabel) } @@ -592,7 +592,7 @@ matrixJob('Cassandra-devbranch-artifacts') { description(jobDescription) concurrentBuild() axes { -jdk('JDK 1.8 (latest)','JDK 11 (latest)') +jdk('jdk_1.8_latest','jdk_11_latest') label('label', slaveLabel) } compressBuildLog() - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16103) Invalid serialized size for responses
[ https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216972#comment-17216972 ] David Capwell commented on CASSANDRA-16103: --- Seen with the following as well {code} Invalid serialized size; expected 13, actual size at least 21, for verb READ_REPAIR_RSP {code} > Invalid serialized size for responses > - > > Key: CASSANDRA-16103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16103 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Was running a benchmark at LOCAL_ONE and eventually saw the below exception; > this is related to CASSANDRA-16097 as it was found during the same test. > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb HINT_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} > {code} > org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized > size; expected 14, actual size at least 13, for verb MUTATION_RSP > at > org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816) > at > org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687) > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216971#comment-17216971 ] Uchenna commented on CASSANDRA-16192: - [~blerer] Just got approval. Will ask before reassigning next time, my trigger finger was a little active today. > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uchenna reassigned CASSANDRA-16192: --- Assignee: Uchenna (was: Fábio Takeo Ueno) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Uchenna >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uchenna reassigned CASSANDRA-16192: --- Assignee: Mohamed Zafraan (was: Uchenna) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[ https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216958#comment-17216958 ] Jordan West commented on CASSANDRA-16048: - Thanks [~marcuse]. Addressed all comments and added the flag. My concern was what it meant to opt-in then opt-out but I think its minimal in reality. I made it a start-up flag since changing it at runtime wouldn't have meaning. > Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and > Value Columns > -- > > Key: CASSANDRA-16048 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16048 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Some compact storage tables, specifically those where the user has defined > both at least one clustering and the value column, can be safely handled in > 4.0 because besides the DENSE flag they are not materially different post 3.0 > and there is no visible change to the user facing schema after dropping > compact storage. We can detect this case and allow these tables to silently > drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE > tables that don’t meet the criteria. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16210) Synchronize Keyspace instance store/clear
[ https://issues.apache.org/jira/browse/CASSANDRA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214734#comment-17214734 ] Ekaterina Dimitrova edited comment on CASSANDRA-16210 at 10/19/20, 5:17 PM: The issue is that it was possible to open a new Keyspace instance in the middle of Schema.dropKeyspace(). To see the problem the drop has to progress to the following [state|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/Schema.java#L657]: 1) Keyspace instance doesn't exist - it has been already removed. 2) KeyspaceMetadata still exists Keyspace.open in this state creates a new Keyspace instance (with ColumnFamilyStore instances) and stores it in Schema.keyspaceInstances. This is an object leak. [3.11 |https://github.com/ekaterinadimitrova2/cassandra/pull/59] CI run: [Java8 | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/104/#showFailuresLink] EDIT: Not applicable to 4.0 -[4.0 | https://github.com/ekaterinadimitrova2/cassandra/pull/62] CI run: [Java 8 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9d0905bd-6ca6-480a-862b-35d5842ed5ef] | [Java 11 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9476a603-a494-4da3-bf69-9498d40ae29a]- was (Author: e.dimitrova): The issue is that it was possible to open a new Keyspace instance in the middle of Schema.dropKeyspace(). To see the problem the drop has to progress to the following [state|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/Schema.java#L657]: 1) Keyspace instance doesn't exist - it has been already removed. 2) KeyspaceMetadata still exists Keyspace.open in this state creates a new Keyspace instance (with ColumnFamilyStore instances) and stores it in Schema.keyspaceInstances. This is an object leak. [3.11 |https://github.com/ekaterinadimitrova2/cassandra/pull/59] CI run: [Java8 | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/104/#showFailuresLink] [4.0 | https://github.com/ekaterinadimitrova2/cassandra/pull/62] CI run: [Java 8 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9d0905bd-6ca6-480a-862b-35d5842ed5ef] | [Java 11 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9476a603-a494-4da3-bf69-9498d40ae29a] > Synchronize Keyspace instance store/clear > - > > Key: CASSANDRA-16210 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16210 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x > > > DTest failure: > dtest-large.repair_tests.repair_test.TestRepairDataSystemTable.test_repair_table > (vnodes) - one random failure was reported which pointed to a race condition > to be spotted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16210) Synchronize Keyspace instance store/clear
[ https://issues.apache.org/jira/browse/CASSANDRA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214734#comment-17214734 ] Ekaterina Dimitrova edited comment on CASSANDRA-16210 at 10/19/20, 5:17 PM: The issue is that it was possible to open a new Keyspace instance in the middle of Schema.dropKeyspace(). To see the problem the drop has to progress to the following [state|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/Schema.java#L657]: 1) Keyspace instance doesn't exist - it has been already removed. 2) KeyspaceMetadata still exists Keyspace.open in this state creates a new Keyspace instance (with ColumnFamilyStore instances) and stores it in Schema.keyspaceInstances. This is an object leak. [3.11 |https://github.com/ekaterinadimitrova2/cassandra/pull/59] CI run: [Java8 | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/104/#showFailuresLink] EDIT: Not applicable to 4.0 -[4.0 | https://github.com/ekaterinadimitrova2/cassandra/pull/62]- -CI run: [Java 8 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9d0905bd-6ca6-480a-862b-35d5842ed5ef] | [Java 11 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9476a603-a494-4da3-bf69-9498d40ae29a]- was (Author: e.dimitrova): The issue is that it was possible to open a new Keyspace instance in the middle of Schema.dropKeyspace(). To see the problem the drop has to progress to the following [state|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/config/Schema.java#L657]: 1) Keyspace instance doesn't exist - it has been already removed. 2) KeyspaceMetadata still exists Keyspace.open in this state creates a new Keyspace instance (with ColumnFamilyStore instances) and stores it in Schema.keyspaceInstances. This is an object leak. [3.11 |https://github.com/ekaterinadimitrova2/cassandra/pull/59] CI run: [Java8 | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/104/#showFailuresLink] EDIT: Not applicable to 4.0 -[4.0 | https://github.com/ekaterinadimitrova2/cassandra/pull/62] CI run: [Java 8 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9d0905bd-6ca6-480a-862b-35d5842ed5ef] | [Java 11 | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/414/workflows/9476a603-a494-4da3-bf69-9498d40ae29a]- > Synchronize Keyspace instance store/clear > - > > Key: CASSANDRA-16210 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16210 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x > > > DTest failure: > dtest-large.repair_tests.repair_test.TestRepairDataSystemTable.test_repair_table > (vnodes) - one random failure was reported which pointed to a race condition > to be spotted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16210) Synchronize Keyspace instance store/clear
[ https://issues.apache.org/jira/browse/CASSANDRA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216939#comment-17216939 ] Ekaterina Dimitrova commented on CASSANDRA-16210: - Agreed, as per my initial comment - schema race conditions solved for 4.0 with CASSANDRA-9425 Thanks [~mck]!! > Synchronize Keyspace instance store/clear > - > > Key: CASSANDRA-16210 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16210 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x > > > DTest failure: > dtest-large.repair_tests.repair_test.TestRepairDataSystemTable.test_repair_table > (vnodes) - one random failure was reported which pointed to a race condition > to be spotted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216935#comment-17216935 ] Benjamin Lerer commented on CASSANDRA-16192: [~ua038697] Before reasigning this ticket did you check with [~fabioueno] if he was fine with it? > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Fábio Takeo Ueno >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-16192: -- Assignee: Fábio Takeo Ueno (was: Mohamed Zafraan) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Fábio Takeo Ueno >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uchenna reassigned CASSANDRA-16192: --- Assignee: Mohamed Zafraan > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16200: - Source Control Link: https://github.com/apache/cassandra/commit/f6281619aef740d6e57ffc3b5e4183bf83651e9a (was: https://github.com/apache/cassandra/commit/c54da9e243f03daf92514db06d9ecfcb802e127e) Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed. > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16200: - Status: Ready to Commit (was: Review In Progress) > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16200: - Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams) Brandon Williams, Brandon Williams (was: Brandon Williams) Status: Review In Progress (was: Patch Available) > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Nodetool ring testing
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new f628161 Nodetool ring testing f628161 is described below commit f6281619aef740d6e57ffc3b5e4183bf83651e9a Author: Bereng AuthorDate: Mon Oct 19 07:27:59 2020 +0200 Nodetool ring testing Patch by Berenguer Blasi, reviewed by brandonwilliams for CASSANDRA-16200 --- build.xml | 16 +- .../org/apache/cassandra/tools/nodetool/Ring.java | 4 +- .../distributed/test/NodeToolRingTest.java | 172 + .../org/apache/cassandra/tools/ToolRunner.java | 89 +-- 4 files changed, 265 insertions(+), 16 deletions(-) diff --git a/build.xml b/build.xml index 4f7d217..4418534 100644 --- a/build.xml +++ b/build.xml @@ -18,7 +18,9 @@ ~ under the License. --> + xmlns:artifact="antlib:org.apache.maven.artifact.ant" + xmlns:if="ant:if" + xmlns:unless="ant:unless"> @@ -1610,7 +1612,8 @@ --> - + + @@ -1624,7 +1627,8 @@ --> - + + @@ -1635,7 +1639,8 @@ --> - + + @@ -2127,7 +2132,8 @@ --> - + + diff --git a/src/java/org/apache/cassandra/tools/nodetool/Ring.java b/src/java/org/apache/cassandra/tools/nodetool/Ring.java index 8b23717..134c85d 100644 --- a/src/java/org/apache/cassandra/tools/nodetool/Ring.java +++ b/src/java/org/apache/cassandra/tools/nodetool/Ring.java @@ -244,7 +244,9 @@ public class Ring extends NodeToolCmd for (HostStatWithPort stat : hoststats) { -tokens.addAll(endpointsToTokens.get(stat.endpoint.toString())); +// Remove extra '/' from address +String addressNPort = stat.endpointWithPort.toString().replaceAll("^/", ""); +tokens.addAll(endpointsToTokens.get(addressNPort)); lastToken = tokens.get(tokens.size() - 1); } diff --git a/test/distributed/org/apache/cassandra/distributed/test/NodeToolRingTest.java b/test/distributed/org/apache/cassandra/distributed/test/NodeToolRingTest.java new file mode 100644 index 000..008dd6b --- /dev/null +++ b/test/distributed/org/apache/cassandra/distributed/test/NodeToolRingTest.java @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.cassandra.distributed.test; + +import java.io.IOException; +import java.util.Arrays; + +import org.junit.AfterClass; +import org.junit.Before; +import org.junit.Test; + +import org.apache.cassandra.distributed.api.ICluster; +import org.apache.cassandra.tools.ToolRunner; +import org.apache.cassandra.tools.ToolRunner.ToolResult; +import org.assertj.core.api.Assertions; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +public class NodeToolRingTest extends TestBaseImpl +{ +private static ICluster cluster; + +@Before +public void setupEnv() throws IOException +{ +if (cluster == null) +cluster = init(builder().withNodes(1).start()); +} + +@AfterClass +public static void teardownEnv() throws Exception +{ +cluster.close(); +} + +@Test +public void testWrongArgFailsAndPrintsHelp() +{ +ToolResult tool = ToolRunner.invokeNodetoolJvmDtest(cluster.get(1), "--wrongarg", "ring"); +Assertions.assertThat(tool.getStdout()).containsIgnoringCase("nodetool help"); +assertEquals(1, tool.getExitCode()); +assertTrue(tool.getCleanedStderr().isEmpty()); +} + +@Test +public void testMaybeChangeDocs() +{ +// If you added, modified options or help, please update docs if necessary + +ToolResult tool = ToolRunner.invokeNodetoolJvmDtest(cluster.get(1), "help", "ring"); + +String help = "NAME\n" + "nodetool ring - Print information about the token ring\n" +
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216816#comment-17216816 ] Aleksey Yeschenko commented on CASSANDRA-15158: --- Left a small comment on the 3.0 branch. Also, the following nits for {{MigrationCoordinator}}: 1. A bunch of unused imports 2. {{shouldApplySchemaFrom()}} has an unused argument 3. {{requestQueue}} could be an {{ArrayDequeue}} instead of a {{LinkedList}} - should set a good example for anyone randomly reading this code, even if it's not critical to do the right thing in this context > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Blake Eggleston >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15158: -- Status: Ready to Commit (was: Review In Progress) > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Blake Eggleston >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16210) Synchronize Keyspace instance store/clear
[ https://issues.apache.org/jira/browse/CASSANDRA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16210: --- Fix Version/s: (was: 4.0-beta3) > Synchronize Keyspace instance store/clear > - > > Key: CASSANDRA-16210 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16210 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x > > > DTest failure: > dtest-large.repair_tests.repair_test.TestRepairDataSystemTable.test_repair_table > (vnodes) - one random failure was reported which pointed to a race condition > to be spotted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216805#comment-17216805 ] Alex Petrov commented on CASSANDRA-15935: - [~dcapwell] [~aleksey] thank you both for the review & feedback. Hopefully, by now everything should be incorporated. [~dcapwell] could you take another look? I went through your comments once again just now to make sure I haven't missed anything. > Improve machinery for testing consistency in presence of range movements > > > Key: CASSANDRA-15935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently, we can test range movements only by adding and bootstrapping a new > node. This is both inefficient and insufficient for large-scale tests. We > need a possibility to dynamically change ring ownership over the lifetime of > cluster, with a flexibility to changing gossip status of the node from > perspective of other participants, adding and removing nodes from other > nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15977) 4.0 quality testing: Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-15977: -- Fix Version/s: (was: 4.0-beta) 4.0-beta3 3.11.9 Source Control Link: https://github.com/apache/cassandra/commit/c3636bf5996fe1dc4541d89d979f6ca18a54fd83 Resolution: Fixed Status: Resolved (was: Ready to Commit) > 4.0 quality testing: Read Repair > > > Key: CASSANDRA-15977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15977 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/unit >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 3.11.9, 4.0-beta3 > > Time Spent: 13h 20m > Remaining Estimate: 0h > > This is a subtask of CASSANDRA-15579 focusing on read repair. > [This > document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing] > lists and describes the existing functional tests for read repair, so we can > have a broad view of what is currently covered. We can comment on this > document and add ideas for new cases/tests, so it can gradually evolve to a > more or less detailed test plan. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15977) 4.0 quality testing: Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216779#comment-17216779 ] Andres de la Peña commented on CASSANDRA-15977: --- Fix in {{ColumnFilter}} committed to {{cassandra-3.11}} as [c3636bf5996fe1dc4541d89d979f6ca18a54fd83|https://github.com/apache/cassandra/commit/c3636bf5996fe1dc4541d89d979f6ca18a54fd83] and merged up with the new tests to trunk as [83b41b6b69b6f020401ecb94137d0cf6fa2b67e4|https://github.com/apache/cassandra/commit/83b41b6b69b6f020401ecb94137d0cf6fa2b67e4]. Dtest changes committed to {{master}} as [cbb35e8db8e4641ea8f6b8363ac6ed3a93529362|https://github.com/apache/cassandra-dtest/commit/cbb35e8db8e4641ea8f6b8363ac6ed3a93529362]. Thanks for the reviews. > 4.0 quality testing: Read Repair > > > Key: CASSANDRA-15977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15977 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/unit >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 13h 20m > Remaining Estimate: 0h > > This is a subtask of CASSANDRA-15579 focusing on read repair. > [This > document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing] > lists and describes the existing functional tests for read repair, so we can > have a broad view of what is currently covered. We can comment on this > document and add ideas for new cases/tests, so it can gradually evolve to a > more or less detailed test plan. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Skip read repair tests ported to in-JMV
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/master by this push: new cbb35e8 Skip read repair tests ported to in-JMV cbb35e8 is described below commit cbb35e8db8e4641ea8f6b8363ac6ed3a93529362 Author: Andrés de la Peña AuthorDate: Mon Oct 19 15:42:08 2020 +0100 Skip read repair tests ported to in-JMV patch by Andrés de la Peña; reviewed by Caleb Rackliffe and Benjamin Lerer for CASSANDRA-15977 --- conftest.py | 41 + consistency_test.py | 2 ++ read_repair_test.py | 8 ++-- 3 files changed, 49 insertions(+), 2 deletions(-) diff --git a/conftest.py b/conftest.py index a17e876..51d2a58 100644 --- a/conftest.py +++ b/conftest.py @@ -484,6 +484,47 @@ def fixture_ported_to_in_jvm(request, fixture_dtest_setup): if skip_msg: pytest.skip(skip_msg) +def _skip_ported_msg(current_running_version, ported_from_version): +if loose_version_compare(current_running_version, ported_from_version) >= 0: +return "ported to in-JVM from %s >= %s" % (ported_from_version, current_running_version) + + +@pytest.fixture(autouse=True) +def fixture_ported_to_in_jvm(request, fixture_dtest_setup): +marker = request.node.get_closest_marker('ported_to_in_jvm') +if marker and not request.config.getoption("--use-vnodes"): + +if not marker.args: +pytest.skip("ported to in-jvm") + +from_str = marker.args[0] +ported_from_version = LooseVersion(from_str) + +# For upgrade tests don't run the test if any of the involved versions +# are excluded by the annotation +if hasattr(request.cls, "UPGRADE_PATH"): +upgrade_path = request.cls.UPGRADE_PATH +ccm_repo_cache_dir, _ = ccmlib.repository.setup(upgrade_path.starting_meta.version) +starting_version = get_version_from_build(ccm_repo_cache_dir) +skip_msg = _skip_ported_msg(starting_version, ported_from_version) +if skip_msg: +pytest.skip(skip_msg) +ccm_repo_cache_dir, _ = ccmlib.repository.setup(upgrade_path.upgrade_meta.version) +ending_version = get_version_from_build(ccm_repo_cache_dir) +skip_msg = _skip_ported_msg(ending_version, ported_from_version) +if skip_msg: +pytest.skip(skip_msg) +else: +# For regular tests the value in the current cluster actually means something so we should +# use that to check. +# Use cassandra_version_from_build as it's guaranteed to be a LooseVersion +# whereas cassandra_version may be a string if set in the cli options +current_running_version = fixture_dtest_setup.dtest_config.cassandra_version_from_build +skip_msg = _skip_ported_msg(current_running_version, ported_from_version) +if skip_msg: +pytest.skip(skip_msg) + + @pytest.fixture(autouse=True) def fixture_skip_version(request, fixture_dtest_setup): marker = request.node.get_closest_marker('skip_version') diff --git a/consistency_test.py b/consistency_test.py index e422b81..2c9f868 100644 --- a/consistency_test.py +++ b/consistency_test.py @@ -17,6 +17,7 @@ from tools.data import (create_c1c2_table, insert_c1c2, insert_columns, query_c1c2, rows_to_list) since = pytest.mark.since +ported_to_in_jvm = pytest.mark.ported_to_in_jvm logger = logging.getLogger(__name__) ExpectedConsistency = namedtuple('ExpectedConsistency', ('num_write_nodes', 'num_read_nodes', 'is_strong')) @@ -1499,6 +1500,7 @@ class TestConsistency(Tester): node3.stop(wait_other_notice=True) assert_none(session, "SELECT * FROM t WHERE id = 0 LIMIT 1", cl=ConsistencyLevel.QUORUM) +@ported_to_in_jvm('4.0') def test_readrepair(self): cluster = self.cluster cluster.set_configuration_options(values={'hinted_handoff_enabled': False}) diff --git a/read_repair_test.py b/read_repair_test.py index 58c324f..f3cb6c2 100644 --- a/read_repair_test.py +++ b/read_repair_test.py @@ -19,6 +19,7 @@ from tools.jmxutils import JolokiaAgent, make_mbean from tools.misc import retry_till_success since = pytest.mark.since +ported_to_in_jvm = pytest.mark.ported_to_in_jvm logger = logging.getLogger(__name__) def byteman_validate(node, script, verbose=False, opts=None): @@ -100,6 +101,7 @@ class TestReadRepair(Tester): cluster.start() @since('3.0') +@ported_to_in_jvm('4.0') def test_alter_rf_and_run_read_repair(self, fixture_set_cluster_settings): """ @jira_ticket CASSANDRA-10655 @@ -272,6 +274,7 @@ class TestReadRepair(Tester): raise NotRepairedException() @since('2.0') +
[cassandra] branch cassandra-3.11 updated: Fix ColumnFilter to avoid querying cells of unselected complex columns
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.11 by this push: new c3636bf Fix ColumnFilter to avoid querying cells of unselected complex columns c3636bf is described below commit c3636bf5996fe1dc4541d89d979f6ca18a54fd83 Author: Andrés de la Peña AuthorDate: Fri Sep 25 16:30:28 2020 +0100 Fix ColumnFilter to avoid querying cells of unselected complex columns patch by Andrés de la Peña; reviewed by Caleb Rackliffe and Benjamin Lerer for CASSANDRA-15977 --- CHANGES.txt| 1 + .../apache/cassandra/db/filter/ColumnFilter.java | 7 +- .../cassandra/db/filter/ColumnFilterTest.java | 115 - 3 files changed, 120 insertions(+), 3 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 98c23c5..df8e6ba 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.11.9 + * Fix ColumnFilter to avoid querying cells of unselected complex columns (CASSANDRA-15977) * Fix memory leak in CompressedChunkReader (CASSANDRA-15880) * Don't attempt value skipping with mixed version cluster (CASSANDRA-15833) * Avoid failing compactions with very large partitions (CASSANDRA-15164) diff --git a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java index 3c79539..e18717c 100644 --- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java +++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java @@ -180,7 +180,12 @@ public class ColumnFilter public boolean fetchedCellIsQueried(ColumnDefinition column, CellPath path) { assert path != null; -if (!isFetchAll || subSelections == null) + +// first verify that the column to which the cell belongs is queried +if (!fetchedColumnIsQueried(column)) +return false; + +if (subSelections == null) return true; SortedSet s = subSelections.get(column.name); diff --git a/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java b/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java index 41fd0a7..5bbd2f5 100644 --- a/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java +++ b/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java @@ -23,7 +23,10 @@ import org.junit.Test; import junit.framework.Assert; import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.ColumnDefinition; +import org.apache.cassandra.db.PartitionColumns; import org.apache.cassandra.db.marshal.Int32Type; +import org.apache.cassandra.db.marshal.SetType; +import org.apache.cassandra.db.rows.CellPath; import org.apache.cassandra.dht.Murmur3Partitioner; import org.apache.cassandra.io.util.DataInputBuffer; import org.apache.cassandra.io.util.DataInputPlus; @@ -31,9 +34,11 @@ import org.apache.cassandra.io.util.DataOutputBuffer; import org.apache.cassandra.net.MessagingService; import org.apache.cassandra.utils.ByteBufferUtil; +import static org.junit.Assert.assertEquals; + public class ColumnFilterTest { -final static ColumnFilter.Serializer serializer = new ColumnFilter.Serializer(); +private static final ColumnFilter.Serializer serializer = new ColumnFilter.Serializer(); @Test public void columnFilterSerialisationRoundTrip() throws Exception @@ -56,7 +61,7 @@ public class ColumnFilterTest testRoundTrip(ColumnFilter.selection(metadata.partitionColumns().without(v1)), metadata, MessagingService.VERSION_3014); } -static void testRoundTrip(ColumnFilter columnFilter, CFMetaData metadata, int version) throws Exception +private static void testRoundTrip(ColumnFilter columnFilter, CFMetaData metadata, int version) throws Exception { DataOutputBuffer output = new DataOutputBuffer(); serializer.serialize(columnFilter, output, version); @@ -64,4 +69,110 @@ public class ColumnFilterTest DataInputPlus input = new DataInputBuffer(output.buffer(), false); Assert.assertEquals(serializer.deserialize(input, version, metadata), columnFilter); } + +/** + * Tests whether a filter fetches and/or queries columns and cells. + */ +@Test +public void testFetchedQueried() +{ +CFMetaData metadata = CFMetaData.Builder.create("ks", "table") + .withPartitioner(Murmur3Partitioner.instance) + .addPartitionKey("k", Int32Type.instance) + .addRegularColumn("simple", Int32Type.instance) + .addRegularColumn("complex", SetType.getInstance(Int32Type.instance, true)) + .build(); + +
[cassandra] branch trunk updated (fde6600 -> 83b41b6)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from fde6600 Revert "Nodetool ring testing" new c3636bf Fix ColumnFilter to avoid querying cells of unselected complex columns new 83b41b6 Merge branch 'cassandra-3.11' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt|1 + build.xml |2 +- .../apache/cassandra/db/filter/ColumnFilter.java |7 +- .../cassandra/metrics/ReadRepairMetrics.java |1 - test/data/jmxdump/cassandra-4.0-jmx.yaml | 12 - .../test/ReadRepairEmptyRangeTombstonesTest.java | 290 + .../distributed/test/ReadRepairQueryTypesTest.java | 1198 .../cassandra/distributed/test/ReadRepairTest.java | 299 +++-- .../distributed/test/ReadRepairTester.java | 165 +++ .../distributed/test/SimpleReadWriteTest.java | 133 +-- .../cassandra/distributed/test/TestBaseImpl.java | 105 ++ .../upgrade/MixedModeReadRepairTest.java | 190 +++- test/unit/org/apache/cassandra/cql3/CQLTester.java |2 +- .../cassandra/db/filter/ColumnFilterTest.java | 112 +- 14 files changed, 2300 insertions(+), 217 deletions(-) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ReadRepairEmptyRangeTombstonesTest.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ReadRepairQueryTypesTest.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ReadRepairTester.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16210) Synchronize Keyspace instance store/clear
[ https://issues.apache.org/jira/browse/CASSANDRA-16210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216764#comment-17216764 ] Michael Semb Wever commented on CASSANDRA-16210: I think you were right originally about this patch not applying to 4.0. The call hierarchy to the private {{Schema.dropKeyspace(..)}} method is only three only methods, all of which are {{synchronized}}. In 3.11: at runtime it's only accessed by method in {{SchemaKeyspace.mergeSchema(..)}} which is also {{synchronized}}. But the method itself is public and not {{synchronized}} so the tests can get into the race condition. Ideally i would think it's better to have synchronized at the test layer, but that's maybe asking a bit much for such a small fix, so I'm +1 on the 3.11 patch. > Synchronize Keyspace instance store/clear > - > > Key: CASSANDRA-16210 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16210 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0-beta3 > > > DTest failure: > dtest-large.repair_tests.repair_test.TestRepairDataSystemTable.test_repair_table > (vnodes) - one random failure was reported which pointed to a race condition > to be spotted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216741#comment-17216741 ] Aleksey Yeschenko commented on CASSANDRA-15935: --- +1d with some nits. > Improve machinery for testing consistency in presence of range movements > > > Key: CASSANDRA-15935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently, we can test range movements only by adding and bootstrapping a new > node. This is both inefficient and insufficient for large-scale tests. We > need a possibility to dynamically change ring ownership over the lifetime of > cluster, with a flexibility to changing gossip status of the node from > perspective of other participants, adding and removing nodes from other > nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0
[ https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216709#comment-17216709 ] Ekaterina Dimitrova commented on CASSANDRA-16083: - I've seen the same story as [~brandon.williams]. Just one question on my mind, now we don't plan to do any removals, at most deprecation in beta, right? > Missing JMX objects and attributes upgrading from 3.0 to 4.0 > > > Key: CASSANDRA-16083 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16083 > Project: Cassandra > Issue Type: Task > Components: Observability/Metrics >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Using the tools added in CASSANDRA-16082, below are the list of metrics > missing in 4.0 but present in 3.0. The work here is to make sure we had > proper deprecation for each metric, and if not to add it back. > {code} > $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml > --ignore-missing-on-left > Objects not in right: > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency > org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram > org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime > org.apache.cassandra.db:type=HintedHandoffManager > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed > org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime > org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MigrationStage,name=CompletedTasks > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize > org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=ViewReadTime > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=BloomFilterFalsePositives > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=CompressionMetadataOffHeapMemoryUsed > org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=TotalBlockedTasks > org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=LiveScannedHistogram >
[jira] [Commented] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth
[ https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216646#comment-17216646 ] Marcus Eriksson commented on CASSANDRA-15369: - btw, are we targetting 3.0+ here? I'd probably prefer this just going to 4.0 - otherwise I'd like to see patches for 3.0/3.11 as well > Fake row deletions and range tombstones, causing digest mismatch and sstable > growth > --- > > Key: CASSANDRA-15369 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15369 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Coordination, Local/Memtable, Local/SSTable >Reporter: Benedict Elliott Smith >Assignee: Zhao Yang >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > As assessed in CASSANDRA-15363, we generate fake row deletions and fake > tombstone markers under various circumstances: > * If we perform a clustering key query (or select a compact column): > * Serving from a {{Memtable}}, we will generate fake row deletions > * Serving from an sstable, we will generate fake row tombstone markers > * If we perform a slice query, we will generate only fake row tombstone > markers for any range tombstone that begins or ends outside of the limit of > the requested slice > * If we perform a multi-slice or IN query, this will occur for each > slice/clustering > Unfortunately, these different behaviours can lead to very different data > stored in sstables until a full repair is run. When we read-repair, we only > send these fake deletions or range tombstones. A fake row deletion, > clustering RT and slice RT, each produces a different digest. So for each > single point lookup we can produce a digest mismatch twice, and until a full > repair is run we can encounter an unlimited number of digest mismatches > across different overlapping queries. > Relatedly, this seems a more problematic variant of our atomicity failures > caused by our monotonic reads, since RTs can have an atomic effect across (up > to) the entire partition, whereas the propagation may happen on an > arbitrarily small portion. If the RT exists on only one node, this could > plausibly lead to fairly problematic scenario if that node fails before the > range can be repaired. > At the very least, this behaviour can lead to an almost unlimited amount of > extraneous data being stored until the range is repaired and compaction > happens to overwrite the sub-range RTs and row deletions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth
[ https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216645#comment-17216645 ] Marcus Eriksson commented on CASSANDRA-15369: - Just one more question; Before, in [searchIterator()|https://github.com/apache/cassandra/blob/ccab496d2d37c86341d364dea6c27513fda27331/src/java/org/apache/cassandra/db/partitions/AbstractBTreePartition.java#L139] we returned EMPTY_STATIC_ROW, [now|https://github.com/apache/cassandra/pull/473/files#diff-6e27ca8fc225036969f774910f2142568fcf85ab588a100c5d3484ac412048f3R121] we return null, why is that? Don't think it makes a difference but would probably be good with a comment why we return null. > Fake row deletions and range tombstones, causing digest mismatch and sstable > growth > --- > > Key: CASSANDRA-15369 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15369 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Coordination, Local/Memtable, Local/SSTable >Reporter: Benedict Elliott Smith >Assignee: Zhao Yang >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > As assessed in CASSANDRA-15363, we generate fake row deletions and fake > tombstone markers under various circumstances: > * If we perform a clustering key query (or select a compact column): > * Serving from a {{Memtable}}, we will generate fake row deletions > * Serving from an sstable, we will generate fake row tombstone markers > * If we perform a slice query, we will generate only fake row tombstone > markers for any range tombstone that begins or ends outside of the limit of > the requested slice > * If we perform a multi-slice or IN query, this will occur for each > slice/clustering > Unfortunately, these different behaviours can lead to very different data > stored in sstables until a full repair is run. When we read-repair, we only > send these fake deletions or range tombstones. A fake row deletion, > clustering RT and slice RT, each produces a different digest. So for each > single point lookup we can produce a digest mismatch twice, and until a full > repair is run we can encounter an unlimited number of digest mismatches > across different overlapping queries. > Relatedly, this seems a more problematic variant of our atomicity failures > caused by our monotonic reads, since RTs can have an atomic effect across (up > to) the entire partition, whereas the propagation may happen on an > arbitrarily small portion. If the RT exists on only one node, this could > plausibly lead to fairly problematic scenario if that node fails before the > range can be repaired. > At the very least, this behaviour can lead to an almost unlimited amount of > extraneous data being stored until the range is repaired and compaction > happens to overwrite the sub-range RTs and row deletions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16209) Log Warning Rather than Verbose Trace when Preview Repair Validation Conflicts with Incremental Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16209: Status: Ready to Commit (was: Review In Progress) lgtm, +1 > Log Warning Rather than Verbose Trace when Preview Repair Validation > Conflicts with Incremental Repair > -- > > Key: CASSANDRA-16209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16209 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a preview repair on repaired data identifies which SSTables to validate, > it might come across an SSTable that's still pending for an in-progress > incremental repair session. It makes sense that we immediately fail the > preview repair in that case, but the resulting error and verbose stack trace > in the logs is a bit too severe a reaction. We should downgrade this to a > simple warning message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16209) Log Warning Rather than Verbose Trace when Preview Repair Validation Conflicts with Incremental Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16209: Reviewers: Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Status: Review In Progress (was: Patch Available) > Log Warning Rather than Verbose Trace when Preview Repair Validation > Conflicts with Incremental Repair > -- > > Key: CASSANDRA-16209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16209 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a preview repair on repaired data identifies which SSTables to validate, > it might come across an SSTable that's still pending for an in-progress > incremental repair session. It makes sense that we immediately fail the > preview repair in that case, but the resulting error and verbose stack trace > in the logs is a bit too severe a reaction. We should downgrade this to a > simple warning message. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16114: Test and Documentation Plan: See PR for CI results Status: Patch Available (was: In Progress) > Fix tests CQLTester.assertLastSchemaChange causes ClassCastException > > > Key: CASSANDRA-16114 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16114 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > Build: > https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681 > {code} > java.lang.ClassCastException: > org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to > org.apache.cassandra.transport.messages.ResultMessage$SchemaChange > at > org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916) > at > org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16184) ClientRequestSize metric tests should be improved
[ https://issues.apache.org/jira/browse/CASSANDRA-16184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16184: Reviewers: Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi) Berenguer Blasi, Berenguer Blasi Status: Review In Progress (was: Patch Available) > ClientRequestSize metric tests should be improved > - > > Key: CASSANDRA-16184 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16184 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > The current test for {{ClientRequestSize}} only test that the metrics are > updated on request. It does not test that the returned values are the > expected ones. Some test comparisons are done with the wrong values. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi reassigned CASSANDRA-16114: --- Assignee: Berenguer Blasi (was: Cedric Nabaa) > Fix tests CQLTester.assertLastSchemaChange causes ClassCastException > > > Key: CASSANDRA-16114 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16114 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Build: > https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681 > {code} > java.lang.ClassCastException: > org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to > org.apache.cassandra.transport.messages.ResultMessage$SchemaChange > at > org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916) > at > org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[ https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216533#comment-17216533 ] Berenguer Blasi commented on CASSANDRA-16114: - Hi, I see no activity from [~cedric.nabaa] for the last month nor a reply so I hijacked this one. I hope that is ok, apologies otherwise. > Fix tests CQLTester.assertLastSchemaChange causes ClassCastException > > > Key: CASSANDRA-16114 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16114 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Cedric Nabaa >Priority: Normal > Fix For: 4.0-beta > > > Build: > https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681 > {code} > java.lang.ClassCastException: > org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to > org.apache.cassandra.transport.messages.ResultMessage$SchemaChange > at > org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916) > at > org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[ https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216490#comment-17216490 ] Marcus Eriksson commented on CASSANDRA-16048: - Added a few nits on the commit, and, as Caleb and Sylvain mentioned, having some logging around this would be good. bq. I skipped adding a flag since we can't detect and "undo" updating the tables that were updated. Could you expand on this? Not sure I follow - sounds like having the flag would be better? > Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and > Value Columns > -- > > Key: CASSANDRA-16048 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16048 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Some compact storage tables, specifically those where the user has defined > both at least one clustering and the value column, can be safely handled in > 4.0 because besides the DENSE flag they are not materially different post 3.0 > and there is no visible change to the user facing schema after dropping > compact storage. We can detect this case and allow these tables to silently > drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE > tables that don’t meet the criteria. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216486#comment-17216486 ] Berenguer Blasi edited comment on CASSANDRA-16200 at 10/19/20, 6:44 AM: It started failing after rebasing indeed. Pushed changes + new CI runs in the PR. Reviewer notes: Code is exactly the same. The only change is that now stdout/err _sometimes_ are captured within the jvm dtest result sometime not. [This|https://github.com/apache/cassandra/pull/768/files#diff-5cec6e6ccb0e8b90f1cd424e02b5cfce4d3e9a2a387127842ce6f1b0366788d7R276-R282] is the change where both the captured and jvmDtestResult stdout/err are concatenated. was (Author: bereng): It started failing after rebasing indeed. Pushed changes + new CI runs in the PR. > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16200: Status: Patch Available (was: In Progress) > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16200) Nodetool ring unit testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216486#comment-17216486 ] Berenguer Blasi commented on CASSANDRA-16200: - It started failing after rebasing indeed. Pushed changes + new CI runs in the PR. > Nodetool ring unit testing > -- > > Key: CASSANDRA-16200 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16200 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > Time Spent: 50m > Remaining Estimate: 0h > > Add nodetool ring testing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org