[jira] [Updated] (CASSANDRA-16476) Update the Compact Storage related docs
[ https://issues.apache.org/jira/browse/CASSANDRA-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16476: Reviewers: Alex Petrov, Berenguer Blasi (was: Alex Petrov) > Update the Compact Storage related docs > --- > > Key: CASSANDRA-16476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16476 > Project: Cassandra > Issue Type: Task > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 4.0-rc > > > Compact Storage partial support was returned recently. > We need to elaborate in the docs on what does this mean to the users. > * Compact Storage support was returned, except for the supercolumn tables. > * This was done because during the big 4.0 beta testing changes of behavior > after DROP COMPACT STORAGE usage were revealed. > /CC [~ifesdjeen] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16476) Update the Compact Storage related docs
[ https://issues.apache.org/jira/browse/CASSANDRA-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299379#comment-17299379 ] Berenguer Blasi edited comment on CASSANDRA-16476 at 3/11/21, 7:54 AM: --- I think I spotted a few typos and suggestions. Discalimer: I am no English native speaker so take them lightly :-) Edit: I wonder if you want to also modify sthg [here|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile] as well. was (Author: bereng): I think I spotted a few typos and suggestions. Discalimer: I am no English native speaker so take them lightly :-) > Update the Compact Storage related docs > --- > > Key: CASSANDRA-16476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16476 > Project: Cassandra > Issue Type: Task > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 4.0-rc > > > Compact Storage partial support was returned recently. > We need to elaborate in the docs on what does this mean to the users. > * Compact Storage support was returned, except for the supercolumn tables. > * This was done because during the big 4.0 beta testing changes of behavior > after DROP COMPACT STORAGE usage were revealed. > /CC [~ifesdjeen] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16476) Update the Compact Storage related docs
[ https://issues.apache.org/jira/browse/CASSANDRA-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299379#comment-17299379 ] Berenguer Blasi commented on CASSANDRA-16476: - I think I spotted a few typos and suggestions. Discalimer: I am no English native speaker so take them lightly :-) > Update the Compact Storage related docs > --- > > Key: CASSANDRA-16476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16476 > Project: Cassandra > Issue Type: Task > Components: Legacy/Local Write-Read Paths >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 4.0-rc > > > Compact Storage partial support was returned recently. > We need to elaborate in the docs on what does this mean to the users. > * Compact Storage support was returned, except for the supercolumn tables. > * This was done because during the big 4.0 beta testing changes of behavior > after DROP COMPACT STORAGE usage were revealed. > /CC [~ifesdjeen] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299303#comment-17299303 ] Tom Whitmore commented on CASSANDRA-16499: -- Hi [~benedict], [~clohfink], I was able to do some testing on Linux, and got some positive results. I was able to find *significant* (+30.9%) throughput improvements in the single-thread case, with improvements varying from small to marginal over 10, 50 and 200 thread cases. Tests were with an m5d.2xlarge EC2 instance with 300 GB local SSD, running Cassandra 4 beta 4 on Amazon Linux and Java 8. Results from Cassandra Stress: ||Threads||Baseline Op/s||Baseline Latency mean||Baseline Latency p99.9||Patched Op/s||Patched Latency mean||Patched Latency p99.9||Op/s Difference|| |1|4979|0.2|0.3|6520|0.1|0.2|+30.9%| |10|33220|0.3|1.35|33882|0.3|1.4|+2.0%| |50|49686|1|24.3|54473|0.9|16.5|+9.6%| |200|65646|3|94.3|67303|2.9|91.8|+2.5%| Benefits drop off at larger numbers of cassandra-stress threads, however latencies still seem to be comparable or slightly better. See spreadsheet & detailed test results: * [^MaybeStartSpinning Unpark fix; Linux benchmarks -- 07.xlsx] * [^AMI Linux test -- 09.txt] My intuition regarding unparking threads on enqueuing a task, was that it was something that logically might well apply across platforms – not just Windows. We now have the following test results: * +30.9% single-thread throughput on Linux * improved latency on Linux under single-thread/ low load conditions (-50% mean, -33% p99.9; though measurements aren't very accurate) * small to marginal improvements on Linux for more threads (10, 50, 200) * a basic "smell test" of load-test latencies (p99, p99.9) etc on Linux up to 200 cassandra-stress threads suggest it's plausibly stable * 18x faster single-thread throughput on Windows * 5x faster 10-thread throughput on Windows * improvements for Windows seem so big as to potentially make a previously unviable platform, now viable as to performance. Given this evidence, could it now make sense to consider this as a serious enhancement candidate to incorporate into SEPExecutor for all platforms? > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: AMI Linux test -- 09.txt, Cassandra Write trace 5; > warmed up -- 02.txt, MaybeStartSpinning Unpark fix on 4beta4; Cassandra > Stress results -- 01.txt, MaybeStartSpinning Unpark fix; Cassandra Stress > results -- 02.txt, MaybeStartSpinning Unpark fix; Linux benchmarks -- > 07.xlsx, SEPWorker trace 2 delay examples -- 01.txt, SEPWorker trace 2 > delays.txt, SEPWorker trace 3 delays; with proposed fix.txt, Single-thread > Latencies report -- 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- > 01.txt, Stress Write sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, > proposed fix patches.zip, tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern
[jira] [Updated] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Whitmore updated CASSANDRA-16499: - Attachment: AMI Linux test -- 09.txt > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: AMI Linux test -- 09.txt, Cassandra Write trace 5; > warmed up -- 02.txt, MaybeStartSpinning Unpark fix on 4beta4; Cassandra > Stress results -- 01.txt, MaybeStartSpinning Unpark fix; Cassandra Stress > results -- 02.txt, MaybeStartSpinning Unpark fix; Linux benchmarks -- > 07.xlsx, SEPWorker trace 2 delay examples -- 01.txt, SEPWorker trace 2 > delays.txt, SEPWorker trace 3 delays; with proposed fix.txt, Single-thread > Latencies report -- 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- > 01.txt, Stress Write sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, > proposed fix patches.zip, tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern of threads waiting in > AbstractWriteResponseHandler while nothing else is busy > * checked out Cassandra 3.11.10 source, built it, debugged & stepped > through key areas to try and understand behavior. > * instrumented key areas with custom tracing code & timestamps to 0.01 > millisecond. > ** _see patch attached._ > * benchmarked Cassandra 4 beta 4 & verified delays also present > * shown & traced delays with my healthcare scenario benchmark > * shown & traced delays with the +Cassandra stress-test+ tool. > The configuration was: > * single-node Cassandra running locally, on a recent Dell laptop with NVmE > SSD. > * for the healthcare scenario: > ** Java client app running 1 or 10 threads; > ** trialled LOCAL_ONE and ANY consistency levels; > ** trialled unbatched, BatchType.UNLOGGED and BatchType.LOGGED. > * for 'cassandra-stress': > ** cassandra-stress.bat write n=1 -rate threads=1 > Without deeply understanding the code, I have considered a couple of possible > areas/ ideas as to improvement. I worked on the 3.11.10 codebase. I'd be > interested to understand whether or not these might be sound or not; note > that neither achieves as much improvement as might theoretically be hoped for. > My investigations based on the key observation of large delays between > StorageProxy.performLocally() being invoked and the LocalMutationRunnable > actually executing, for single-threaded workloads. > What I looked at: > * Without fully understanding SEPExecutor.takeWorkPermit() – it answers true > to execute immediately, false if scheduled – for this workload scheduling > seemed slow. > ** takeWorkPermit() answers false if no work permits are available. > ** I noticed takeWorkPermit() also answers false if no task permits are > available, +even if no task permit need be taken.+ > ** by changing this condition I was able to gain +45% performance. > * Without deeply understanding SEP Executor/ Worker or sleep algorithms, I >
[jira] [Updated] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Whitmore updated CASSANDRA-16499: - Attachment: MaybeStartSpinning Unpark fix; Linux benchmarks -- 07.xlsx > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: AMI Linux test -- 09.txt, Cassandra Write trace 5; > warmed up -- 02.txt, MaybeStartSpinning Unpark fix on 4beta4; Cassandra > Stress results -- 01.txt, MaybeStartSpinning Unpark fix; Cassandra Stress > results -- 02.txt, MaybeStartSpinning Unpark fix; Linux benchmarks -- > 07.xlsx, SEPWorker trace 2 delay examples -- 01.txt, SEPWorker trace 2 > delays.txt, SEPWorker trace 3 delays; with proposed fix.txt, Single-thread > Latencies report -- 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- > 01.txt, Stress Write sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, > proposed fix patches.zip, tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern of threads waiting in > AbstractWriteResponseHandler while nothing else is busy > * checked out Cassandra 3.11.10 source, built it, debugged & stepped > through key areas to try and understand behavior. > * instrumented key areas with custom tracing code & timestamps to 0.01 > millisecond. > ** _see patch attached._ > * benchmarked Cassandra 4 beta 4 & verified delays also present > * shown & traced delays with my healthcare scenario benchmark > * shown & traced delays with the +Cassandra stress-test+ tool. > The configuration was: > * single-node Cassandra running locally, on a recent Dell laptop with NVmE > SSD. > * for the healthcare scenario: > ** Java client app running 1 or 10 threads; > ** trialled LOCAL_ONE and ANY consistency levels; > ** trialled unbatched, BatchType.UNLOGGED and BatchType.LOGGED. > * for 'cassandra-stress': > ** cassandra-stress.bat write n=1 -rate threads=1 > Without deeply understanding the code, I have considered a couple of possible > areas/ ideas as to improvement. I worked on the 3.11.10 codebase. I'd be > interested to understand whether or not these might be sound or not; note > that neither achieves as much improvement as might theoretically be hoped for. > My investigations based on the key observation of large delays between > StorageProxy.performLocally() being invoked and the LocalMutationRunnable > actually executing, for single-threaded workloads. > What I looked at: > * Without fully understanding SEPExecutor.takeWorkPermit() – it answers true > to execute immediately, false if scheduled – for this workload scheduling > seemed slow. > ** takeWorkPermit() answers false if no work permits are available. > ** I noticed takeWorkPermit() also answers false if no task permits are > available, +even if no task permit need be taken.+ > ** by changing this condition I was able to gain +45% performance. > * Without deeply understanding SEP Executor/
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299259#comment-17299259 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 3/11/21, 3:08 AM: --- DTests committed [here|https://github.com/apache/cassandra-dtest/commit/700f712c0ee87ea90c960e7fe92343bff7b2ba23]. 3.0 committed [here|https://github.com/apache/cassandra/commit/efde6a76f9807eaa0af80b5e85aa350ac8709a81]. 3.11 committed [here|https://github.com/apache/cassandra/commit/b6540009365b1986950510ecc28ee5740c64412e]. 4.0 committed [here|https://github.com/apache/cassandra/commit/c7432e98a381c16cb63692e727c35e2909d7200a]. In 3.0 and 3.11 the docker image was not updated in config.yml after the latest changes, only LOWRES was updated so I used the opportunity to correct that. Thank you [~slebresne] for the patch. Thank you for the reviews and help [~blerer] and [~brandon.williams] was (Author: e.dimitrova): DTests committed [here|https://github.com/apache/cassandra-dtest/commit/700f712c0ee87ea90c960e7fe92343bff7b2ba23]. 3.0 committed [here|https://github.com/apache/cassandra/commit/efde6a76f9807eaa0af80b5e85aa350ac8709a81]. 3.11 committed [here|https://github.com/apache/cassandra/commit/b6540009365b1986950510ecc28ee5740c64412e]. 4.0 committed [here|https://github.com/apache/cassandra/commit/c7432e98a381c16cb63692e727c35e2909d7200a]. In 3.0 and 3.11 the docker image was not updated in config.yml after the latest changes only in LOWRES so I used the opportunity to correct that. Thank you [~slebresne] for the patch. Thank you for the reviews and help [~blerer] and [~brandon.williams] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta5, 4.0, 3.0.x, 3.11.x > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Since Version: (was: 3.0.0) > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Since Version: 3.0.0 Source Control Link: https://issues.apache.org/jira/browse/CASSANDRA-15897?focusedCommentId=17299259=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17299259 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Fix Version/s: (was: 4.0-beta) 4.0 4.0-beta5 > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta5, 4.0, 3.0.x, 3.11.x > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15897: Test and Documentation Plan: https://issues.apache.org/jira/browse/CASSANDRA-15897?focusedCommentId=17299259=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17299259 (was: https://issues.apache.org/jira/browse/CASSANDRA-15897?focusedCommentId=17248009=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17248009) > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299259#comment-17299259 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - DTests committed [here|https://github.com/apache/cassandra-dtest/commit/700f712c0ee87ea90c960e7fe92343bff7b2ba23]. 3.0 committed [here|https://github.com/apache/cassandra/commit/efde6a76f9807eaa0af80b5e85aa350ac8709a81]. 3.11 committed [here|https://github.com/apache/cassandra/commit/b6540009365b1986950510ecc28ee5740c64412e]. 4.0 committed [here|https://github.com/apache/cassandra/commit/c7432e98a381c16cb63692e727c35e2909d7200a]. In 3.0 and 3.11 the docker image was not updated in config.yml after the latest changes only in LOWRES so I used the opportunity to correct that. Thank you [~slebresne] for the patch. Thank you for the reviews and help [~blerer] and [~brandon.williams] > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (dc82d48 -> b654000)
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from dc82d48 Merge branch 'cassandra-3.0' into cassandra-3.11
add efde6a7 Refuse DROP COMPACT STORAGE if some 2.x sstables are in use
patch by Sylvain Lebresne; review changes and tests by Ekaterina Dimitrova;
reviewed by Benjamin Lerer and Brandon Williams for CASSANDRA-15897
add b654000 Merge branch 'cassandra-3.0' into cassandra-3.11
No new revisions were added by this update.
Summary of changes:
.circleci/config.yml | 22 +-
CHANGES.txt| 1 +
build.xml | 11 +
.../cql3/statements/AlterTableStatement.java | 133 --
.../org/apache/cassandra/db/ColumnFamilyStore.java | 4 +
.../db/compaction/CompactionStrategyManager.java | 3 +-
.../org/apache/cassandra/db/lifecycle/Tracker.java | 43 ++--
.../org/apache/cassandra/gms/ApplicationState.java | 23 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 18 ++
.../org/apache/cassandra/gms/VersionedValue.java | 11 +
.../apache/cassandra/io/sstable/Descriptor.java| 1 +
.../io/sstable/format/VersionAndType.java | 93 +++
...n.java => InitialSSTableAddedNotification.java} | 11 +-
.../cassandra/service/SSTablesGlobalTracker.java | 284 +
.../SSTablesVersionsInUseChangeNotification.java | 50
.../apache/cassandra/service/StorageService.java | 19 ++
.../upgrade/CompactStorage2to3UpgradeTest.java | 73 +++---
.../upgrade/DropCompactStorageTest.java| 72 ++
.../apache/cassandra/db/lifecycle/TrackerTest.java | 31 ++-
.../io/sstable/format/VersionAndTypeTest.java | 51
.../service/SSTablesGlobalTrackerTest.java | 158
21 files changed, 1007 insertions(+), 105 deletions(-)
create mode 100644
src/java/org/apache/cassandra/io/sstable/format/VersionAndType.java
copy
src/java/org/apache/cassandra/notifications/{SSTableAddedNotification.java =>
InitialSSTableAddedNotification.java} (69%)
create mode 100644
src/java/org/apache/cassandra/service/SSTablesGlobalTracker.java
create mode 100644
src/java/org/apache/cassandra/service/SSTablesVersionsInUseChangeNotification.java
create mode 100644
test/distributed/org/apache/cassandra/distributed/upgrade/DropCompactStorageTest.java
create mode 100644
test/unit/org/apache/cassandra/io/sstable/format/VersionAndTypeTest.java
create mode 100644
test/unit/org/apache/cassandra/service/SSTablesGlobalTrackerTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit c7432e98a381c16cb63692e727c35e2909d7200a Merge: 0e990d7 b654000 Author: Ekaterina Dimitrova AuthorDate: Wed Mar 10 20:54:07 2021 -0500 Merge branch 'cassandra-3.11' into trunk CHANGES.txt| 1 + build.xml | 2 +- .../statements/schema/AlterTableStatement.java | 93 ++- .../org/apache/cassandra/db/ColumnFamilyStore.java | 4 + .../db/compaction/CompactionStrategyManager.java | 5 +- .../org/apache/cassandra/db/lifecycle/Tracker.java | 61 +++-- .../org/apache/cassandra/gms/ApplicationState.java | 17 ++ .../cassandra/gms/GossipDigestAckVerbHandler.java | 2 +- src/java/org/apache/cassandra/gms/Gossiper.java| 42 ++- .../org/apache/cassandra/gms/VersionedValue.java | 12 +- .../apache/cassandra/io/sstable/Descriptor.java| 7 + .../cassandra/io/sstable/format/Version.java | 1 - .../io/sstable/format/VersionAndType.java | 93 +++ .../InitialSSTableAddedNotification.java} | 27 +- src/java/org/apache/cassandra/schema/Schema.java | 3 +- .../cassandra/schema/SchemaTransformation.java | 4 +- .../cassandra/service/SSTablesGlobalTracker.java | 284 + .../SSTablesVersionsInUseChangeNotification.java | 50 .../apache/cassandra/service/StartupChecks.java| 5 +- .../apache/cassandra/service/StorageService.java | 19 ++ .../upgrade/CompactStorage3to4UpgradeTest.java | 4 + .../validation/operations/CompactTableTest.java| 1 - .../apache/cassandra/db/lifecycle/TrackerTest.java | 31 ++- .../io/sstable/format/VersionAndTypeTest.java | 51 .../service/SSTablesGlobalTrackerTest.java | 158 25 files changed, 899 insertions(+), 78 deletions(-) diff --cc CHANGES.txt index ffec3cb,1e74bf6..592c9c3 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,50 -1,10 +1,51 @@@ -3.11.11 +4.0-beta5 + * Send FAILED_SESSION_MSG on shutdown and on in-progress repairs during startup (CASSANDRA-16425) + * Reinstate removed ApplicationState padding (CASSANDRA-16484) + * Expose data dirs to ColumnFamilyStoreMBean (CASSANDRA-16335) + * Add possibility to copy SSTables in SSTableImporter instead of moving them (CASSANDRA-16407) + * Fix DESCRIBE statement for CUSTOM indices with options (CASSANDRA-16482) + * Fix cassandra-stress JMX connection (CASSANDRA-16473) + * Avoid processing redundant application states on endpoint changes (CASSANDRA-16381) + * Prevent parent repair sessions leak (CASSANDRA-16446) + * Fix timestamp issue in SinglePartitionSliceCommandTest testPartitionD…eletionRowDeletionTie (CASSANDRA-16443) + * Promote protocol V5 out of beta (CASSANDRA-14973) + * Fix incorrect encoding for strings can be UTF8 (CASSANDRA-16429) + * Fix node unable to join when RF > N in multi-DC with added warning (CASSANDRA-16296) + * Add an option to nodetool tablestats to check sstable location correctness (CASSANDRA-16344) + * Unable to ALTER KEYSPACE while decommissioned/assassinated nodes are in gossip (CASSANDRA-16422) + * Metrics backward compatibility restored after CASSANDRA-15066 (CASSANDRA-16083) + * Reduce new reserved keywords introduced since 3.0 (CASSANDRA-16439) + * Improve system tables handling in case of disk failures (CASSANDRA-14793) + * Add access and datacenters to unreserved keywords (CASSANDRA-16398) + * Fix nodetool ring, status output when DNS resolution or port printing are in use (CASSANDRA-16283) + * Upgrade Jacoco to 0.8.6 (for Java 11 support) (CASSANDRA-16365) + * Move excessive repair debug loggings to trace level (CASSANDRA-16406) + * Restore validation of each message's protocol version (CASSANDRA-16374) + * Upgrade netty and chronicle-queue dependencies to get Auditing and native library loading working on arm64 architectures (CASSANDRA-16384,CASSANDRA-16392) + * Release StreamingTombstoneHistogramBuilder spool when switching writers (CASSANDRA-14834) + * Correct memtable on-heap size calculations to match actual use (CASSANDRA-16318) + * Fix client notifications in CQL protocol v5 (CASSANDRA-16353) + * Too defensive check when picking sstables for preview repair (CASSANDRA-16284) + * Ensure pre-negotiation native protocol responses have correct stream id (CASSANDRA-16376) + * Fix check for -Xlog in cassandra-env.sh (CASSANDRA-16279) + * SSLFactory should initialize SSLContext before setting protocols (CASSANDRA-16362) + * Restore sasi dependencies jflex, snowball-stemmer, and concurrent-trees, in the cassandra-all pom (CASSANDRA-16303) + * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) +Merged from 3.11: * Upgrade jackson-databind to 2.9.10.8 (CASSANDRA-16462) + * Fix digest computation for queries with fetched but non queried
[cassandra] branch trunk updated (0e990d7 -> c7432e9)
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 0e990d7 Merge branch 'cassandra-3.11' into trunk
add efde6a7 Refuse DROP COMPACT STORAGE if some 2.x sstables are in use
patch by Sylvain Lebresne; review changes and tests by Ekaterina Dimitrova;
reviewed by Benjamin Lerer and Brandon Williams for CASSANDRA-15897
add b654000 Merge branch 'cassandra-3.0' into cassandra-3.11
new c7432e9 Merge branch 'cassandra-3.11' into trunk
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.txt| 1 +
build.xml | 2 +-
.../statements/schema/AlterTableStatement.java | 93 ++-
.../org/apache/cassandra/db/ColumnFamilyStore.java | 4 +
.../db/compaction/CompactionStrategyManager.java | 5 +-
.../org/apache/cassandra/db/lifecycle/Tracker.java | 61 +++--
.../org/apache/cassandra/gms/ApplicationState.java | 17 ++
.../cassandra/gms/GossipDigestAckVerbHandler.java | 2 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 42 ++-
.../org/apache/cassandra/gms/VersionedValue.java | 12 +-
.../apache/cassandra/io/sstable/Descriptor.java| 7 +
.../cassandra/io/sstable/format/Version.java | 1 -
.../io/sstable/format/VersionAndType.java | 93 +++
...n.java => InitialSSTableAddedNotification.java} | 13 +-
src/java/org/apache/cassandra/schema/Schema.java | 3 +-
.../cassandra/schema/SchemaTransformation.java | 4 +-
.../cassandra/service/SSTablesGlobalTracker.java | 284 +
.../SSTablesVersionsInUseChangeNotification.java | 50
.../apache/cassandra/service/StartupChecks.java| 5 +-
.../apache/cassandra/service/StorageService.java | 19 ++
.../upgrade/CompactStorage3to4UpgradeTest.java | 4 +
.../validation/operations/CompactTableTest.java| 1 -
.../apache/cassandra/db/lifecycle/TrackerTest.java | 31 ++-
.../io/sstable/format/VersionAndTypeTest.java | 51
.../service/SSTablesGlobalTrackerTest.java | 158
25 files changed, 891 insertions(+), 72 deletions(-)
create mode 100644
src/java/org/apache/cassandra/io/sstable/format/VersionAndType.java
copy
src/java/org/apache/cassandra/notifications/{SSTableDeletingNotification.java
=> InitialSSTableAddedNotification.java} (65%)
create mode 100644
src/java/org/apache/cassandra/service/SSTablesGlobalTracker.java
create mode 100644
src/java/org/apache/cassandra/service/SSTablesVersionsInUseChangeNotification.java
create mode 100644
test/unit/org/apache/cassandra/io/sstable/format/VersionAndTypeTest.java
create mode 100644
test/unit/org/apache/cassandra/service/SSTablesGlobalTrackerTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (9c7b69e -> efde6a7)
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 9c7b69e Fix ColumnFilter::toString not returning a valid CQL fragment
add efde6a7 Refuse DROP COMPACT STORAGE if some 2.x sstables are in use
patch by Sylvain Lebresne; review changes and tests by Ekaterina Dimitrova;
reviewed by Benjamin Lerer and Brandon Williams for CASSANDRA-15897
No new revisions were added by this update.
Summary of changes:
.circleci/config.yml | 20 +-
CHANGES.txt| 1 +
build.xml | 12 +-
.../cql3/statements/AlterTableStatement.java | 95 ++-
.../org/apache/cassandra/db/ColumnFamilyStore.java | 5 +
.../org/apache/cassandra/db/lifecycle/Tracker.java | 43 ++--
.../org/apache/cassandra/gms/ApplicationState.java | 23 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 18 ++
.../org/apache/cassandra/gms/VersionedValue.java | 11 +
.../apache/cassandra/io/sstable/Descriptor.java| 1 +
.../io/sstable/format/VersionAndType.java | 93 +++
...n.java => InitialSSTableAddedNotification.java} | 11 +-
.../notifications/SSTableAddedNotification.java| 1 +
.../cassandra/service/SSTablesGlobalTracker.java | 284 +
.../SSTablesVersionsInUseChangeNotification.java | 50
.../apache/cassandra/service/StorageService.java | 19 ++
.../upgrade/CompactStorage2to3UpgradeTest.java | 25 +-
.../upgrade/DropCompactStorageTest.java| 69 +
.../apache/cassandra/db/lifecycle/TrackerTest.java | 31 ++-
.../io/sstable/format/VersionAndTypeTest.java | 51
.../service/SSTablesGlobalTrackerTest.java | 162
21 files changed, 964 insertions(+), 61 deletions(-)
create mode 100644
src/java/org/apache/cassandra/io/sstable/format/VersionAndType.java
copy
src/java/org/apache/cassandra/notifications/{SSTableAddedNotification.java =>
InitialSSTableAddedNotification.java} (69%)
create mode 100644
src/java/org/apache/cassandra/service/SSTablesGlobalTracker.java
create mode 100644
src/java/org/apache/cassandra/service/SSTablesVersionsInUseChangeNotification.java
create mode 100644
test/distributed/org/apache/cassandra/distributed/upgrade/DropCompactStorageTest.java
create mode 100644
test/unit/org/apache/cassandra/io/sstable/format/VersionAndTypeTest.java
create mode 100644
test/unit/org/apache/cassandra/service/SSTablesGlobalTrackerTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Tests fixed and new ones added for CASSANDRA-15897
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 700f712 Tests fixed and new ones added for CASSANDRA-15897
700f712 is described below
commit 700f712c0ee87ea90c960e7fe92343bff7b2ba23
Author: Ekaterina Dimitrova
AuthorDate: Fri Dec 11 10:06:19 2020 -0500
Tests fixed and new ones added for CASSANDRA-15897
patch by Ekaterina Dimitrova; reviewed by Benjamin Lerer and Brandon
Williams
---
snitch_test.py | 16 +--
upgrade_tests/drop_compact_storage_upgrade_test.py | 131 ++---
2 files changed, 124 insertions(+), 23 deletions(-)
diff --git a/snitch_test.py b/snitch_test.py
index 5b640c8..a931303 100644
--- a/snitch_test.py
+++ b/snitch_test.py
@@ -127,26 +127,26 @@ class TestGossipingPropertyFileSnitch(Tester):
logger.debug(out)
assert "/{}".format(NODE1_BROADCAST_ADDRESS) in out
-assert "INTERNAL_IP:{}:{}".format('9' if running40 else '6',
NODE1_LISTEN_ADDRESS) in out
+assert "INTERNAL_IP:{}:{}".format('10' if running40 else '7',
NODE1_LISTEN_ADDRESS) in out
assert "/{}".format(NODE2_BROADCAST_ADDRESS) in out
if running40:
-assert
"INTERNAL_ADDRESS_AND_PORT:7:{}".format(NODE1_40_LISTEN_ADDRESS) in out
-assert
"INTERNAL_ADDRESS_AND_PORT:7:{}".format(NODE2_40_LISTEN_ADDRESS) in out
+assert
"INTERNAL_ADDRESS_AND_PORT:8:{}".format(NODE1_40_LISTEN_ADDRESS) in out
+assert
"INTERNAL_ADDRESS_AND_PORT:8:{}".format(NODE2_40_LISTEN_ADDRESS) in out
else:
-assert "INTERNAL_IP:{}:{}".format('6', NODE2_LISTEN_ADDRESS) in out
+assert "INTERNAL_IP:{}:{}".format('7', NODE2_LISTEN_ADDRESS) in out
out, err, _ = node2.nodetool('gossipinfo')
assert_stderr_clean(err)
logger.debug(out)
assert "/{}".format(NODE1_BROADCAST_ADDRESS) in out
-assert "INTERNAL_IP:{}:{}".format('9' if running40 else '6',
NODE2_LISTEN_ADDRESS) in out
+assert "INTERNAL_IP:{}:{}".format('10' if running40 else '7',
NODE2_LISTEN_ADDRESS) in out
assert "/{}".format(NODE2_BROADCAST_ADDRESS) in out
if running40:
-assert
"INTERNAL_ADDRESS_AND_PORT:7:{}".format(NODE1_40_LISTEN_ADDRESS) in out
-assert
"INTERNAL_ADDRESS_AND_PORT:7:{}".format(NODE2_40_LISTEN_ADDRESS) in out
+assert
"INTERNAL_ADDRESS_AND_PORT:8:{}".format(NODE1_40_LISTEN_ADDRESS) in out
+assert
"INTERNAL_ADDRESS_AND_PORT:8:{}".format(NODE2_40_LISTEN_ADDRESS) in out
else:
-assert "INTERNAL_IP:{}:{}".format('6', NODE1_LISTEN_ADDRESS) in out
+assert "INTERNAL_IP:{}:{}".format('7', NODE1_LISTEN_ADDRESS) in out
class TestDynamicEndpointSnitch(Tester):
@pytest.mark.resource_intensive
diff --git a/upgrade_tests/drop_compact_storage_upgrade_test.py
b/upgrade_tests/drop_compact_storage_upgrade_test.py
index 695e82f..338eb26 100644
--- a/upgrade_tests/drop_compact_storage_upgrade_test.py
+++ b/upgrade_tests/drop_compact_storage_upgrade_test.py
@@ -1,3 +1,5 @@
+import time
+
import pytest
import logging
@@ -8,20 +10,18 @@ from dtest import Tester
since = pytest.mark.since
logger = logging.getLogger(__name__)
+VERSION_30 = 'github:apache/cassandra-3.0'
+VERSION_311 = 'github:apache/cassandra-3.11'
+VERSION_TRUNK = 'github:apache/trunk'
@pytest.mark.upgrade_test
-@since('3.0', max_version='3.11')
class TestDropCompactStorage(Tester):
-def test_drop_compact_storage(self):
-"""
-Test to verify that dropping compact storage is not possible prior
running `nodetool upgradesstables`. The
-current solution and test take care only about the local sstables.
Global check of the sstables will be added as
-part of CASSANDRA-15897.
-@jira_ticket CASSANDRA-16063
-"""
+def prepare(self):
cluster = self.cluster
-cluster.populate(2)
-node1, node2 = cluster.nodelist()
+cluster.populate(3)
+node1, node2, node3 = cluster.nodelist()
+
+# Forcing cluster version on purpose
cluster.set_install_dir(version="2.1.14")
cluster.start(wait_for_binary_proto=True)
@@ -35,6 +35,33 @@ class TestDropCompactStorage(Tester):
session.execute(
"INSERT INTO drop_compact_storage_test.test (a, b, c) VALUES
('{}', '{}', '{}');".format(i, i + 1,
i + 2))
+return cluster
+
+def drop_compact_storage(self, session, assert_msg):
+try:
+session.execute("ALTER TABLE drop_compact_storage_test.test DROP
COMPACT STORAGE")
+self.fail("No exception has been thrown")
+
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299135#comment-17299135 ] Brandon Williams commented on CASSANDRA-15897: -- +1, though I'll note the addition of getApplicationState to the Gossiper has been handled by a subsequent call from getEndpointState historically, but I'm not against the addition of the method here since there is no need to distinguish between not knowing the endpoint and it not having a given state. In fact, I hope we use it more going forward. > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299133#comment-17299133 ] Benedict Elliott Smith commented on CASSANDRA-16499: In the meantime, if it helps you, your changes look perfectly safe. So if it helps your use case you should certainly consider running a patched version. If nobody else has time beforehand, I'll take a look when I can find time to setup a suitable test environment and can consider the problem properly. > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: Cassandra Write trace 5; warmed up -- 02.txt, > MaybeStartSpinning Unpark fix on 4beta4; Cassandra Stress results -- 01.txt, > MaybeStartSpinning Unpark fix; Cassandra Stress results -- 02.txt, SEPWorker > trace 2 delay examples -- 01.txt, SEPWorker trace 2 delays.txt, SEPWorker > trace 3 delays; with proposed fix.txt, Single-thread Latencies report -- > 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- 01.txt, Stress Write > sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, proposed fix patches.zip, > tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern of threads waiting in > AbstractWriteResponseHandler while nothing else is busy > * checked out Cassandra 3.11.10 source, built it, debugged & stepped > through key areas to try and understand behavior. > * instrumented key areas with custom tracing code & timestamps to 0.01 > millisecond. > ** _see patch attached._ > * benchmarked Cassandra 4 beta 4 & verified delays also present > * shown & traced delays with my healthcare scenario benchmark > * shown & traced delays with the +Cassandra stress-test+ tool. > The configuration was: > * single-node Cassandra running locally, on a recent Dell laptop with NVmE > SSD. > * for the healthcare scenario: > ** Java client app running 1 or 10 threads; > ** trialled LOCAL_ONE and ANY consistency levels; > ** trialled unbatched, BatchType.UNLOGGED and BatchType.LOGGED. > * for 'cassandra-stress': > ** cassandra-stress.bat write n=1 -rate threads=1 > Without deeply understanding the code, I have considered a couple of possible > areas/ ideas as to improvement. I worked on the 3.11.10 codebase. I'd be > interested to understand whether or not these might be sound or not; note > that neither achieves as much improvement as might theoretically be hoped for. > My investigations based on the key observation of large delays between > StorageProxy.performLocally() being invoked and the LocalMutationRunnable > actually executing, for single-threaded workloads. > What I looked at: > * Without fully understanding SEPExecutor.takeWorkPermit() – it answers true > to execute immediately, false if scheduled – for this workload scheduling > seemed slow. > ** takeWorkPermit() answers false if no work permits are available. > ** I noticed takeWorkPermit() also
[jira] [Updated] (CASSANDRA-16495) Scheduled (Delayed) Schema Pull Tasks May Run After MIGRATION Stage Shutdown During Decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-16495:
Reviewers: Caleb Rackliffe
> Scheduled (Delayed) Schema Pull Tasks May Run After MIGRATION Stage Shutdown
> During Decommission
>
>
> Key: CASSANDRA-16495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16495
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Caleb Rackliffe
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> A new test added in CASSANDRA-16181 stumbled across this, although it doesn’t
> happen consistently. When [failure
> occurs|https://app.circleci.com/pipelines/github/maedhroz/cassandra/235/workflows/eb8133ce-9373-4136-b404-ceca167353f6/jobs/1355/tests],
> it appears to be because a delayed schema pull happens after decommission
> shuts down the MIGRATION stage’s thread pool.
> {noformat}
> ERROR [node1_isolatedExecutor:1] node1 2021-02-15 19:35:36,284
> CassandraDaemon.java:579 - Exception in thread
> Thread[node1_NonPeriodicTasks:1,5,node1]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:72)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
>
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:176)
>
> at
> java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
> at org.apache.cassandra.concurrent.Stage.submit(Stage.java:129)
> at
> org.apache.cassandra.schema.MigrationCoordinator.lambda$scheduleSchemaPull$2(MigrationCoordinator.java:362)
>
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
> A fix might be as simple as shutting down ScheduledExecutors.nonPeriodicTasks
> in StorageService#decommission(). See the original discussion
> [here|https://issues.apache.org/jira/browse/CASSANDRA-16181?focusedCommentId=17293329=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17293329].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16504) GSoC Blog Post
[ https://issues.apache.org/jira/browse/CASSANDRA-16504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-16504: Fix Version/s: 4.0 4.0-beta5 Source Control Link: https://github.com/apache/cassandra-website/commit/18f3266a5941ba49ed276bf3c8e7543c94d5a785 Resolution: Fixed Status: Resolved (was: Ready to Commit) > GSoC Blog Post > -- > > Key: CASSANDRA-16504 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16504 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta5, 4.0 > > > Ticket to track review and publishing of GSoC blog post. > The blog post is available [google > document|https://docs.google.com/document/d/1YLiyZ_quiDpCUyT3tyuWmVKiuj7h3HZdeBvqLzuMGes/edit]. > This draft was announced on #cassandra-dev on slack and already reviewed by > some community members. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16504) GSoC Blog Post
[ https://issues.apache.org/jira/browse/CASSANDRA-16504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-16504: Status: Review In Progress (was: Patch Available) > GSoC Blog Post > -- > > Key: CASSANDRA-16504 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16504 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Normal > Labels: pull-request-available > > Ticket to track review and publishing of GSoC blog post. > The blog post is available [google > document|https://docs.google.com/document/d/1YLiyZ_quiDpCUyT3tyuWmVKiuj7h3HZdeBvqLzuMGes/edit]. > This draft was announced on #cassandra-dev on slack and already reviewed by > some community members. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16504) GSoC Blog Post
[ https://issues.apache.org/jira/browse/CASSANDRA-16504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-16504: Status: Ready to Commit (was: Review In Progress) > GSoC Blog Post > -- > > Key: CASSANDRA-16504 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16504 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Normal > Labels: pull-request-available > > Ticket to track review and publishing of GSoC blog post. > The blog post is available [google > document|https://docs.google.com/document/d/1YLiyZ_quiDpCUyT3tyuWmVKiuj7h3HZdeBvqLzuMGes/edit]. > This draft was announced on #cassandra-dev on slack and already reviewed by > some community members. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16504) GSoC Blog Post
[
https://issues.apache.org/jira/browse/CASSANDRA-16504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299109#comment-17299109
]
Paulo Motta commented on CASSANDRA-16504:
-
Merged to \{{apache/cassandra-website}} on
{{18f3266a5941ba49ed276bf3c8e7543c94d5a785}}. Thanks!
> GSoC Blog Post
> --
>
> Key: CASSANDRA-16504
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16504
> Project: Cassandra
> Issue Type: Task
> Components: Documentation/Blog
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Normal
> Labels: pull-request-available
>
> Ticket to track review and publishing of GSoC blog post.
> The blog post is available [google
> document|https://docs.google.com/document/d/1YLiyZ_quiDpCUyT3tyuWmVKiuj7h3HZdeBvqLzuMGes/edit].
> This draft was announced on #cassandra-dev on slack and already reviewed by
> some community members.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16495) Scheduled (Delayed) Schema Pull Tasks May Run After MIGRATION Stage Shutdown During Decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova reassigned CASSANDRA-16495:
---
Assignee: Ekaterina Dimitrova
> Scheduled (Delayed) Schema Pull Tasks May Run After MIGRATION Stage Shutdown
> During Decommission
>
>
> Key: CASSANDRA-16495
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16495
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: Caleb Rackliffe
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> A new test added in CASSANDRA-16181 stumbled across this, although it doesn’t
> happen consistently. When [failure
> occurs|https://app.circleci.com/pipelines/github/maedhroz/cassandra/235/workflows/eb8133ce-9373-4136-b404-ceca167353f6/jobs/1355/tests],
> it appears to be because a delayed schema pull happens after decommission
> shuts down the MIGRATION stage’s thread pool.
> {noformat}
> ERROR [node1_isolatedExecutor:1] node1 2021-02-15 19:35:36,284
> CassandraDaemon.java:579 - Exception in thread
> Thread[node1_NonPeriodicTasks:1,5,node1]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:72)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
>
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:176)
>
> at
> java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
> at org.apache.cassandra.concurrent.Stage.submit(Stage.java:129)
> at
> org.apache.cassandra.schema.MigrationCoordinator.lambda$scheduleSchemaPull$2(MigrationCoordinator.java:362)
>
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
> A fix might be as simple as shutting down ScheduledExecutors.nonPeriodicTasks
> in StorageService#decommission(). See the original discussion
> [here|https://issues.apache.org/jira/browse/CASSANDRA-16181?focusedCommentId=17293329=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17293329].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15148) Add in-jvm dtest for Gossip to trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299069#comment-17299069
]
Ekaterina Dimitrova edited comment on CASSANDRA-15148 at 3/10/21, 6:53 PM:
---
Hey [~samt],
CASSANDRA-15239 is opened for this test on 3.0; the test is flaky. Then I
figured out that it Is missing in trunk and that led me to this ticket.
I see all tickets mentioned by you being already closed, should I add the test
now in trunk? Any concerns I need to know about?
was (Author: e.dimitrova):
Hey [~samt],
CASSANDRA-15239 is opened for this test on 3.0; the test is flaky. Then I
figured out that it Is missing in trunk and that led me to this ticket.
I see all tickets mentioned by you being already closed, should I add the test
now? Any concerns I need to know about?
> Add in-jvm dtest for Gossip to trunk
>
>
> Key: CASSANDRA-15148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15148
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> When merging CASSANDRA-15120 to trunk, the in-jvm dtest support for gossip
> and networking were ommitted as much of the work to support that is already
> done and due to land in CASSANDRA-15066. Once that patch is committed, we
> should add the {{GossipTest}} from CASSANDRA-15120 to trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15239) [flaky in-mem dtest] nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299070#comment-17299070
]
Ekaterina Dimitrova commented on CASSANDRA-15239:
-
Un-assigning myself for now as I am still struggling to reproduce the issue in
any environment. I might come back to this again but if someone has something
in the meanwhile, feel free to add it
> [flaky in-mem dtest] nodeDownDuringMove -
> org.apache.cassandra.distributed.test.GossipTest
> --
>
> Key: CASSANDRA-15239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15239
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Jay Zhuang
>Priority: Normal
> Fix For: 3.0.x
>
>
> The in-mem dtest fail from time to time:
> {noformat}
> nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> {noformat}
> [https://circleci.com/gh/Instagram/cassandra/98]
> More details:
> {noformat}
> Testcase:
> nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest): Caused
> an ERROR
> java.lang.IllegalStateException: Unable to contact any seeds!
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:166)
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$4(IsolatedExecutor.java:69)
> at
> org.apache.cassandra.distributed.impl.Instance.startup(Instance.java:322)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:148)
> at
> org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:96)
> Caused by: java.lang.IllegalStateException: Unable to contact any seeds!
> at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1261)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:921)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$6(Instance.java:301)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> Test org.apache.cassandra.distributed.test.GossipTest FAILED
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15148) Add in-jvm dtest for Gossip to trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299069#comment-17299069
]
Ekaterina Dimitrova commented on CASSANDRA-15148:
-
Hey [~samt],
CASSANDRA-15239 is opened for this test on 3.0; the test is flaky. Then I
figured out that it Is missing in trunk and that led me to this ticket.
I see all tickets mentioned by you being already closed, should I add the test
now? Any concerns I need to know about?
> Add in-jvm dtest for Gossip to trunk
>
>
> Key: CASSANDRA-15148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15148
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> When merging CASSANDRA-15120 to trunk, the in-jvm dtest support for gossip
> and networking were ommitted as much of the work to support that is already
> done and due to land in CASSANDRA-15066. Once that patch is committed, we
> should add the {{GossipTest}} from CASSANDRA-15120 to trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15239) [flaky in-mem dtest] nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova reassigned CASSANDRA-15239:
---
Assignee: (was: Ekaterina Dimitrova)
> [flaky in-mem dtest] nodeDownDuringMove -
> org.apache.cassandra.distributed.test.GossipTest
> --
>
> Key: CASSANDRA-15239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15239
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Jay Zhuang
>Priority: Normal
> Fix For: 3.0.x
>
>
> The in-mem dtest fail from time to time:
> {noformat}
> nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> {noformat}
> [https://circleci.com/gh/Instagram/cassandra/98]
> More details:
> {noformat}
> Testcase:
> nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest): Caused
> an ERROR
> java.lang.IllegalStateException: Unable to contact any seeds!
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:166)
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$4(IsolatedExecutor.java:69)
> at
> org.apache.cassandra.distributed.impl.Instance.startup(Instance.java:322)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:148)
> at
> org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:96)
> Caused by: java.lang.IllegalStateException: Unable to contact any seeds!
> at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1261)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:921)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$6(Instance.java:301)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> Test org.apache.cassandra.distributed.test.GossipTest FAILED
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15239) [flaky in-mem dtest] nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299066#comment-17299066
]
Ekaterina Dimitrova commented on CASSANDRA-15239:
-
I found CASSANDRA-15148 opened already for the missing test in trunk. It was
waiting on few tickets to be closed because the patch for the scenario tested
in trunk was different. I suggest we keep this for 3.0 and investigate it
separately.
> [flaky in-mem dtest] nodeDownDuringMove -
> org.apache.cassandra.distributed.test.GossipTest
> --
>
> Key: CASSANDRA-15239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15239
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Jay Zhuang
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x
>
>
> The in-mem dtest fail from time to time:
> {noformat}
> nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> {noformat}
> [https://circleci.com/gh/Instagram/cassandra/98]
> More details:
> {noformat}
> Testcase:
> nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest): Caused
> an ERROR
> java.lang.IllegalStateException: Unable to contact any seeds!
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:166)
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$4(IsolatedExecutor.java:69)
> at
> org.apache.cassandra.distributed.impl.Instance.startup(Instance.java:322)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:148)
> at
> org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:96)
> Caused by: java.lang.IllegalStateException: Unable to contact any seeds!
> at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1261)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:921)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$6(Instance.java:301)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> Test org.apache.cassandra.distributed.test.GossipTest FAILED
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15239) [flaky in-mem dtest] nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15239:
Fix Version/s: (was: 4.0-rc)
> [flaky in-mem dtest] nodeDownDuringMove -
> org.apache.cassandra.distributed.test.GossipTest
> --
>
> Key: CASSANDRA-15239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15239
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Jay Zhuang
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.0.x
>
>
> The in-mem dtest fail from time to time:
> {noformat}
> nodeDownDuringMove - org.apache.cassandra.distributed.test.GossipTest
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> {noformat}
> [https://circleci.com/gh/Instagram/cassandra/98]
> More details:
> {noformat}
> Testcase:
> nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest): Caused
> an ERROR
> java.lang.IllegalStateException: Unable to contact any seeds!
> java.lang.RuntimeException: java.lang.IllegalStateException: Unable to
> contact any seeds!
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:166)
> at
> org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$4(IsolatedExecutor.java:69)
> at
> org.apache.cassandra.distributed.impl.Instance.startup(Instance.java:322)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.startup(AbstractCluster.java:148)
> at
> org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:96)
> Caused by: java.lang.IllegalStateException: Unable to contact any seeds!
> at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1261)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:921)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$6(Instance.java:301)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
> at java.lang.Thread.run(Thread.java:748)
> Test org.apache.cassandra.distributed.test.GossipTest FAILED
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15148) Add in-jvm dtest for Gossip to trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15148:
Fix Version/s: 4.0-rc
4.0
> Add in-jvm dtest for Gossip to trunk
>
>
> Key: CASSANDRA-15148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15148
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-rc
>
>
> When merging CASSANDRA-15120 to trunk, the in-jvm dtest support for gossip
> and networking were ommitted as much of the work to support that is already
> done and due to land in CASSANDRA-15066. Once that patch is committed, we
> should add the {{GossipTest}} from CASSANDRA-15120 to trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15148) Add in-jvm dtest for Gossip to trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15148:
Change Category: Semantic
Complexity: Low Hanging Fruit
Assignee: Ekaterina Dimitrova
Priority: Low (was: Normal)
Status: Open (was: Triage Needed)
> Add in-jvm dtest for Gossip to trunk
>
>
> Key: CASSANDRA-15148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15148
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java
>Reporter: Sam Tunnicliffe
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> When merging CASSANDRA-15120 to trunk, the in-jvm dtest support for gossip
> and networking were ommitted as much of the work to support that is already
> done and due to land in CASSANDRA-15066. Once that patch is committed, we
> should add the {{GossipTest}} from CASSANDRA-15120 to trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] 02/02: generate docs for 7daffb6d
This is an automated email from the ASF dual-hosted git repository. paulo pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git commit f8f3ae33fb3ea1d481e64c24adc651b1ecceb2df Author: Paulo Motta AuthorDate: Wed Mar 10 11:21:36 2021 -0300 generate docs for 7daffb6d --- .../blog/2021/03/10/join_cassandra_gsoc_2021.html | 181 + content/blog/index.html| 16 +- content/blog/page2/index.html | 18 +- content/blog/page3/index.html | 9 + .../configuration/cassandra_config_file.html | 4 +- content/doc/4.0-beta5/cql/definitions.html | 6 +- .../doc/4.0-beta5/getting_started/installing.html | 2 +- content/doc/4.0-beta5/new/streaming.html | 10 +- content/doc/4.0-beta5/searchindex.js | 2 +- content/doc/4.0-beta5/tools/cqlsh.html | 4 +- content/doc/4.0-beta5/tools/nodetool/import.html | 9 +- .../configuration/cassandra_config_file.html | 4 +- content/doc/latest/cql/definitions.html| 6 +- content/doc/latest/getting_started/installing.html | 2 +- content/doc/latest/new/streaming.html | 10 +- content/doc/latest/searchindex.js | 2 +- content/doc/latest/tools/cqlsh.html| 4 +- content/doc/latest/tools/nodetool/import.html | 9 +- content/feed.xml | 164 --- .../configuration/cassandra_config_file.rst.txt| 4 +- src/doc/4.0-beta5/_sources/cql/definitions.rst.txt | 6 +- .../_sources/getting_started/installing.rst.txt| 2 +- src/doc/4.0-beta5/_sources/new/streaming.rst.txt | 13 +- src/doc/4.0-beta5/_sources/tools/cqlsh.rst.txt | 4 +- .../configuration/cassandra_config_file.html | 4 +- src/doc/4.0-beta5/cql/definitions.html | 6 +- src/doc/4.0-beta5/getting_started/installing.html | 2 +- src/doc/4.0-beta5/new/streaming.html | 10 +- src/doc/4.0-beta5/searchindex.js | 2 +- src/doc/4.0-beta5/tools/cqlsh.html | 4 +- src/doc/4.0-beta5/tools/nodetool/import.html | 9 +- 31 files changed, 321 insertions(+), 207 deletions(-) diff --git a/content/blog/2021/03/10/join_cassandra_gsoc_2021.html b/content/blog/2021/03/10/join_cassandra_gsoc_2021.html new file mode 100644 index 000..2aa7c59 --- /dev/null +++ b/content/blog/2021/03/10/join_cassandra_gsoc_2021.html @@ -0,0 +1,181 @@ + + + + + + + + + + + + + + + + Join Apache Cassandra for Google Summer of Code 2021 + + http://cassandra.apache.org/blog/2021/03/10/join_cassandra_gsoc_2021.html;> + + https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css; integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous"> + + + + + https://use.fontawesome.com/releases/v5.2.0/css/all.css; integrity="sha384-hWVjflwFxL6sNzntih27bfxkr27PmbbK/iSvJ+a4+0owXq79v+lsFkW54bOGbiDQ" crossorigin="anonymous"> + + http://cassandra.apache.org/feed.xml; title="Apache Cassandra Website" /> + + + + + + + + + + + Apache Software Foundation + +http://www.apache.org;>Apache Homepage +http://www.apache.org/licenses/;>License +http://www.apache.org/foundation/sponsorship.html;>Sponsorship +http://www.apache.org/foundation/thanks.html;>Thanks +http://www.apache.org/security/;>Security + + + + + + Apache Cassandra + + + + +Join Apache Cassandra for Google Summer of Code 2021 + + + + + + + + + + + + + + + Toggle navigation + + + + + + + + + + Home + Download + Documentation + Community + +Blog + + + + + + + + + + + + Join Apache Cassandra for Google Summer of Code 2021 +Posted on March 10, 2021 by Paulo Motta + Back to the Apache Cassandra Blog + + + https://developers.google.com/open-source/gsoc/resources/downloads/GSoC-logo-horizontal-800.png; style="width:40%;" /> + + +I have been involved with Apache Cassandra for the past eight years, and I’m very proud to mention that my open source journey started a little more than a decade ago during my participation at the Google Summer of Code (GSoC). + +GSoC is a program sponsored by Google to promote open source development, where post-secondary students submit project proposals to open source organizations. Selected students receive community mentorship and a stipend from Google to work on the project for ten weeks during the northern hemisphere summer. Over 16,000 students from 111
[cassandra-website] 01/02: Add Google Summer of Code 2021 blog post
This is an automated email from the ASF dual-hosted git repository. paulo pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git commit 7daffb6d23a1021fcf399756ca4332a59b819293 Author: Paulo Motta AuthorDate: Mon Mar 8 22:14:52 2021 -0300 Add Google Summer of Code 2021 blog post Patch by Paulo Motta; Reviewed by Brandon Williams for CASSANDRA-16504 --- .../2021-03-10-join_cassandra_gsoc_2021.markdown | 36 ++ 1 file changed, 36 insertions(+) diff --git a/src/_posts/2021-03-10-join_cassandra_gsoc_2021.markdown b/src/_posts/2021-03-10-join_cassandra_gsoc_2021.markdown new file mode 100644 index 000..64ad410 --- /dev/null +++ b/src/_posts/2021-03-10-join_cassandra_gsoc_2021.markdown @@ -0,0 +1,36 @@ +--- +layout: post +title: "Join Apache Cassandra for Google Summer of Code 2021" +date: 2021-03-10 +author: Paulo Motta +categories: blog +excerpt: Getting involved with Apache Cassandra @ Google Summer of Code 2021 +--- + + + https://developers.google.com/open-source/gsoc/resources/downloads/GSoC-logo-horizontal-800.png; style="width:40%;"/> + + +I have been involved with Apache Cassandra for the past eight years, and I'm very proud to mention that my open source journey started a little more than a decade ago during my participation at the Google Summer of Code (GSoC). + +GSoC is a program sponsored by Google to promote open source development, where post-secondary students submit project proposals to open source organizations. Selected students receive community mentorship and a stipend from Google to work on the project for ten weeks during the northern hemisphere summer. Over 16,000 students from 111 countries have participated so far! More details about the program can be found on the [official GSoC website](https://summerofcode.withgoogle.com/). + +The Apache Software Foundation (ASF) has been a [GSoC mentor organization](https://community.apache.org/gsoc.html) since the beginning of the program 17 years ago. The ASF acts as an "umbrella" organization, which means that students can submit project proposals to any subproject within the ASF. Apache Cassandra mentored a successful GSoC project in [2016](https://summerofcode.withgoogle.com/archive/2016/projects/5429448547500032/) and we are participating again this year. The applicatio [...] + +### How to Get Involved + + Getting Started + +The best way to get started if you're new to Apache Cassandra is to get acquainted by reading the [documentation](https://cassandra.apache.org/doc/latest/) and setting up a local [development environment](https://cassandra.apache.org/doc/latest/development/ide.html). Play around with a locally running instance via [cqlsh](https://cassandra.apache.org/doc/latest/tools/cqlsh.html?highlight=cqlsh) and [nodetool](https://cassandra.apache.org/doc/latest/tools/nodetool/nodetool.html) to get a [...] + + GSoC Project Ideas + +Once you have a basic understanding of how the project works, browse the [GSoC ideas list](https://issues.apache.org/jira/browse/CASSANDRA-16456?jql=labels%20%3D%20gsoc2021%20AND%20project%20%3D%20Cassandra) to select ideas that you are interested in working on. Browse the [codebase](https://github.com/apache/cassandra/) to identify components related to the idea you picked. You are welcome to propose other projects if they’re not on the ideas list. + + Writing a proposal + +Write a message on the JIRA ticket introducing yourself and demonstrating your interest in working on that particular idea. Sketch a quick proposal on how you plan to tackle the problem and share it with the community for feedback. If you don't know where to start, don't hesitate to ask for help! + + Useful Resources + +There are many good resources on the web on preparing for GSoC, particularly the [ASF GSoC Guide](https://community.apache.org/gsoc.html) and the Python community notes on [GSoC expectations](https://wiki.python.org/moin/SummerOfCode/Expectations). The best GSoC students are self-motivated and proactive, and following the tips above should increase your chances of getting selected and delivering your project successfully. Good luck! - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-site updated (00a0834 -> f8f3ae3)
This is an automated email from the ASF dual-hosted git repository. paulo pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git. from 00a0834 generate docs for 5795317e new 7daffb6 Add Google Summer of Code 2021 blog post new f8f3ae3 generate docs for 7daffb6d The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../blog/2021/03/10/join_cassandra_gsoc_2021.html | 181 + content/blog/index.html| 16 +- content/blog/page2/index.html | 18 +- content/blog/page3/index.html | 9 + .../configuration/cassandra_config_file.html | 4 +- content/doc/4.0-beta5/cql/definitions.html | 6 +- .../doc/4.0-beta5/getting_started/installing.html | 2 +- content/doc/4.0-beta5/new/streaming.html | 10 +- content/doc/4.0-beta5/searchindex.js | 2 +- content/doc/4.0-beta5/tools/cqlsh.html | 4 +- content/doc/4.0-beta5/tools/nodetool/import.html | 9 +- .../configuration/cassandra_config_file.html | 4 +- content/doc/latest/cql/definitions.html| 6 +- content/doc/latest/getting_started/installing.html | 2 +- content/doc/latest/new/streaming.html | 10 +- content/doc/latest/searchindex.js | 2 +- content/doc/latest/tools/cqlsh.html| 4 +- content/doc/latest/tools/nodetool/import.html | 9 +- content/feed.xml | 164 --- .../2021-03-10-join_cassandra_gsoc_2021.markdown | 36 .../configuration/cassandra_config_file.rst.txt| 4 +- src/doc/4.0-beta5/_sources/cql/definitions.rst.txt | 6 +- .../_sources/getting_started/installing.rst.txt| 2 +- src/doc/4.0-beta5/_sources/new/streaming.rst.txt | 13 +- src/doc/4.0-beta5/_sources/tools/cqlsh.rst.txt | 4 +- .../configuration/cassandra_config_file.html | 4 +- src/doc/4.0-beta5/cql/definitions.html | 6 +- src/doc/4.0-beta5/getting_started/installing.html | 2 +- src/doc/4.0-beta5/new/streaming.html | 10 +- src/doc/4.0-beta5/searchindex.js | 2 +- src/doc/4.0-beta5/tools/cqlsh.html | 4 +- src/doc/4.0-beta5/tools/nodetool/import.html | 9 +- 32 files changed, 357 insertions(+), 207 deletions(-) create mode 100644 content/blog/2021/03/10/join_cassandra_gsoc_2021.html create mode 100644 src/_posts/2021-03-10-join_cassandra_gsoc_2021.markdown - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated: Add Google Summer of Code 2021 blog post
This is an automated email from the ASF dual-hosted git repository. paulo pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 18f3266 Add Google Summer of Code 2021 blog post 18f3266 is described below commit 18f3266a5941ba49ed276bf3c8e7543c94d5a785 Author: Paulo Motta AuthorDate: Mon Mar 8 22:14:52 2021 -0300 Add Google Summer of Code 2021 blog post Patch by Paulo Motta; Reviewed by Brandon Williams for CASSANDRA-16504 --- .../2021-03-09-join_cassandra_gsoc_2021.markdown | 36 ++ 1 file changed, 36 insertions(+) diff --git a/src/_posts/2021-03-09-join_cassandra_gsoc_2021.markdown b/src/_posts/2021-03-09-join_cassandra_gsoc_2021.markdown new file mode 100644 index 000..c0b4192 --- /dev/null +++ b/src/_posts/2021-03-09-join_cassandra_gsoc_2021.markdown @@ -0,0 +1,36 @@ +--- +layout: post +title: "Join Apache Cassandra for Google Summer of Code 2021" +date: 2021-03-09 +author: Paulo Motta +categories: blog +excerpt: Getting involved with Apache Cassandra @ Google Summer of Code 2021 +--- + + + https://developers.google.com/open-source/gsoc/resources/downloads/GSoC-logo-horizontal-800.png; style="width:40%;"/> + + +I have been involved with Apache Cassandra for the past eight years, and I'm very proud to mention that my open source journey started a little more than a decade ago during my participation at the Google Summer of Code (GSoC). + +GSoC is a program sponsored by Google to promote open source development, where post-secondary students submit project proposals to open source organizations. Selected students receive community mentorship and a stipend from Google to work on the project for ten weeks during the northern hemisphere summer. Over 16,000 students from 111 countries have participated so far! More details about the program can be found on the [official GSoC website](https://summerofcode.withgoogle.com/). + +The Apache Software Foundation (ASF) has been a [GSoC mentor organization](https://community.apache.org/gsoc.html) since the beginning of the program 17 years ago. The ASF acts as an "umbrella" organization, which means that students can submit project proposals to any subproject within the ASF. Apache Cassandra mentored a successful GSoC project in [2016](https://summerofcode.withgoogle.com/archive/2016/projects/5429448547500032/) and we are participating again this year. The applicatio [...] + +### How to Get Involved + + Getting Started + +The best way to get started if you're new to Apache Cassandra is to get acquainted by reading the [documentation](https://cassandra.apache.org/doc/latest/) and setting up a local [development environment](https://cassandra.apache.org/doc/latest/development/ide.html). Play around with a locally running instance via [cqlsh](https://cassandra.apache.org/doc/latest/tools/cqlsh.html?highlight=cqlsh) and [nodetool](https://cassandra.apache.org/doc/latest/tools/nodetool/nodetool.html) to get a [...] + + GSoC Project Ideas + +Once you have a basic understanding of how the project works, browse the [GSoC ideas list](https://issues.apache.org/jira/browse/CASSANDRA-16456?jql=labels%20%3D%20gsoc2021%20AND%20project%20%3D%20Cassandra) to select ideas that you are interested in working on. Browse the [codebase](https://github.com/apache/cassandra/) to identify components related to the idea you picked. You are welcome to propose other projects if they’re not on the ideas list. + + Writing a proposal + +Write a message on the JIRA ticket introducing yourself and demonstrating your interest in working on that particular idea. Sketch a quick proposal on how you plan to tackle the problem and share it with the community for feedback. If you don't know where to start, don't hesitate to ask for help! + + Useful Resources + +There are many good resources on the web on preparing for GSoC, particularly the [ASF GSoC Guide](https://community.apache.org/gsoc.html) and the Python community notes on [GSoC expectations](https://wiki.python.org/moin/SummerOfCode/Expectations). The best GSoC students are self-motivated and proactive, and following the tips above should increase your chances of getting selected and delivering your project successfully. Good luck! - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16483) ColumnFilter::toString doesn't return a valid CQL fragment
[
https://issues.apache.org/jira/browse/CASSANDRA-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16483:
---
Fix Version/s: 4.0-beta5
> ColumnFilter::toString doesn't return a valid CQL fragment
> --
>
> Key: CASSANDRA-16483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16483
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Sam Tunnicliffe
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.25, 3.11.11, 4.0-beta5, 4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This was changed in CASSANDRA-16415 to include indications about queried vs
> fetched reagular & static columns. However, the result is used by
> {{AbstractReadQuery::toCQLString}}, which causes it to produce an illegal
> query string.
> This breaks a couple of dtests because they're looking for CQL strings in
> logs, which are no longer found:
> *
> {{upgrade_tests/paging_test.py::TestPagingWithDeletions::test_failure_threshold_deletions}}
> * {{cql_test.py::TestCQLSlowQuery}} has a couple of failing tests,
> {{test_local_query/test_remote_query}}
> We should also check audit and fql logs (and any other place where
> {{toCQLString}} is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16483) ColumnFilter::toString doesn't return a valid CQL fragment
[
https://issues.apache.org/jira/browse/CASSANDRA-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16483:
--
Fix Version/s: (was: 4.0-beta)
4.0
3.11.11
3.0.25
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/9c7b69e4229b6817b248bcf5270a783eed1f9930
https://github.com/apache/cassandra-dtest/commit/7d10bdd19765bc337ee7bbe516c3a1e76391f5c0
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> ColumnFilter::toString doesn't return a valid CQL fragment
> --
>
> Key: CASSANDRA-16483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16483
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Sam Tunnicliffe
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.25, 3.11.11, 4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This was changed in CASSANDRA-16415 to include indications about queried vs
> fetched reagular & static columns. However, the result is used by
> {{AbstractReadQuery::toCQLString}}, which causes it to produce an illegal
> query string.
> This breaks a couple of dtests because they're looking for CQL strings in
> logs, which are no longer found:
> *
> {{upgrade_tests/paging_test.py::TestPagingWithDeletions::test_failure_threshold_deletions}}
> * {{cql_test.py::TestCQLSlowQuery}} has a couple of failing tests,
> {{test_local_query/test_remote_query}}
> We should also check audit and fql logs (and any other place where
> {{toCQLString}} is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16483) ColumnFilter::toString doesn't return a valid CQL fragment
[
https://issues.apache.org/jira/browse/CASSANDRA-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299011#comment-17299011
]
Andres de la Peña commented on CASSANDRA-16483:
---
Committed to {{cassandra-3.0}} as
[{{9c7b69e4229b6817b248bcf5270a783eed1f9930}}|https://github.com/apache/cassandra/commit/9c7b69e4229b6817b248bcf5270a783eed1f9930]
and merged up to
[{{cassandra-3.11}}|https://github.com/apache/cassandra/commit/dc82d481d94e38b41dc3de1a07eb18c77d34ad1e]
and
[{{trunk}}|https://github.com/apache/cassandra/commit/0e990d7231f6152e95d5c21a6660594fc9a507bb].
Dtest changes committed as
[{{7d10bdd19765bc337ee7bbe516c3a1e76391f5c0}}|https://github.com/apache/cassandra-dtest/commit/7d10bdd19765bc337ee7bbe516c3a1e76391f5c0].
Thanks for the review.
> ColumnFilter::toString doesn't return a valid CQL fragment
> --
>
> Key: CASSANDRA-16483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16483
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Sam Tunnicliffe
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This was changed in CASSANDRA-16415 to include indications about queried vs
> fetched reagular & static columns. However, the result is used by
> {{AbstractReadQuery::toCQLString}}, which causes it to produce an illegal
> query string.
> This breaks a couple of dtests because they're looking for CQL strings in
> logs, which are no longer found:
> *
> {{upgrade_tests/paging_test.py::TestPagingWithDeletions::test_failure_threshold_deletions}}
> * {{cql_test.py::TestCQLSlowQuery}} has a couple of failing tests,
> {{test_local_query/test_remote_query}}
> We should also check audit and fql logs (and any other place where
> {{toCQLString}} is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Include column selections in TestCQLSlowQuery
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 7d10bdd Include column selections in TestCQLSlowQuery
7d10bdd is described below
commit 7d10bdd19765bc337ee7bbe516c3a1e76391f5c0
Author: Andrés de la Peña
AuthorDate: Wed Mar 10 17:39:07 2021 +
Include column selections in TestCQLSlowQuery
patch by Andrés de la Peña; reviewed by Benjamin Lerer for CASSANDRA-16483
---
cql_test.py | 107 ++--
1 file changed, 105 insertions(+), 2 deletions(-)
diff --git a/cql_test.py b/cql_test.py
index dde7b7d..360df42 100644
--- a/cql_test.py
+++ b/cql_test.py
@@ -1169,6 +1169,10 @@ class TestCQLSlowQuery(CQLTester):
CREATE TABLE {} (
k int,
v int,
+l list,
+s set,
+m map,
+"Escaped_Name" int,
PRIMARY KEY(k)
);
""".format(table))
@@ -1239,6 +1243,46 @@ class TestCQLSlowQuery(CQLTester):
query="SELECT * FROM {} WHERE v <= 2
ALLOW FILTERING",
logged_query="SELECT \* FROM ks.{} WHERE
v <= 2")
+# test logging of slow queries with column selections
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT v FROM {}",
+ logged_query="SELECT v FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT l FROM {}",
+ logged_query="SELECT l FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT s FROM {}",
+ logged_query="SELECT s FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT m FROM {}",
+ logged_query="SELECT m FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT \"Escaped_Name\" FROM {}",
+ logged_query="SELECT \"Escaped_Name\"
FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT v,s FROM {}",
+ logged_query="SELECT s, v FROM ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT k,v,s FROM {}",
+ logged_query="SELECT s, v FROM ks.{}")
+
+# test logging of slow queries with primary key-only column selections
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT k FROM {}",
+ logged_query="SELECT \* FROM ks.{}")
+
+# test logging of slow queries with sub-selections (only supported
since 4.0)
+if node.cluster.version() >= '4.0':
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT s[0] FROM {}",
+ logged_query="SELECT s\[0\] FROM
ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT m[0] FROM {}",
+ logged_query="SELECT m\[0\] FROM
ks.{}")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT k,v,s[0],m[1] FROM {}",
+ logged_query="SELECT m\[1\], s\[0\],
v FROM ks.{}")
+
@staticmethod
def _assert_logs_slow_queries_with_wide_table(node, session, asc=True):
create_ks(session, 'ks', 1)
@@ -1250,6 +1294,10 @@ class TestCQLSlowQuery(CQLTester):
c2 int,
v int,
s int STATIC,
+lt list,
+st set,
+mp map,
+"Escaped_Name" int,
PRIMARY KEY(k, c1, c2)
) WITH CLUSTERING ORDER BY (c1 {}, c2 {});
""".format(table, "ASC" if asc else "DESC", "ASC" if asc else "DESC"))
@@ -1316,8 +1364,8 @@ class TestCQLSlowQuery(CQLTester):
query="SELECT * FROM {} WHERE c1 IN (1)
ALLOW FILTERING",
logged_query="SELECT \* FROM ks.{} WHERE
c1 = 1")
TestCQLSlowQuery._assert_logs(node, session, table,
- query="SELECT * FROM {} WHERE c1 IN (1,
2) ALLOW FILTERING",
-
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 0e990d7231f6152e95d5c21a6660594fc9a507bb Merge: b70ae0e dc82d48 Author: Andrés de la Peña AuthorDate: Wed Mar 10 17:34:20 2021 + Merge branch 'cassandra-3.11' into trunk CHANGES.txt| 1 + .../org/apache/cassandra/db/AbstractReadQuery.java | 2 +- src/java/org/apache/cassandra/db/ReadCommand.java | 2 +- .../apache/cassandra/db/filter/ColumnFilter.java | 38 --- .../cassandra/db/filter/ColumnFilterTest.java | 44 ++ 5 files changed, 72 insertions(+), 15 deletions(-) diff --cc CHANGES.txt index a3a03c0,b959a43..ffec3cb --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,49 -1,9 +1,50 @@@ -3.11.11 +4.0-beta5 + * Send FAILED_SESSION_MSG on shutdown and on in-progress repairs during startup (CASSANDRA-16425) + * Reinstate removed ApplicationState padding (CASSANDRA-16484) + * Expose data dirs to ColumnFamilyStoreMBean (CASSANDRA-16335) + * Add possibility to copy SSTables in SSTableImporter instead of moving them (CASSANDRA-16407) + * Fix DESCRIBE statement for CUSTOM indices with options (CASSANDRA-16482) + * Fix cassandra-stress JMX connection (CASSANDRA-16473) + * Avoid processing redundant application states on endpoint changes (CASSANDRA-16381) + * Prevent parent repair sessions leak (CASSANDRA-16446) + * Fix timestamp issue in SinglePartitionSliceCommandTest testPartitionD…eletionRowDeletionTie (CASSANDRA-16443) + * Promote protocol V5 out of beta (CASSANDRA-14973) + * Fix incorrect encoding for strings can be UTF8 (CASSANDRA-16429) + * Fix node unable to join when RF > N in multi-DC with added warning (CASSANDRA-16296) + * Add an option to nodetool tablestats to check sstable location correctness (CASSANDRA-16344) + * Unable to ALTER KEYSPACE while decommissioned/assassinated nodes are in gossip (CASSANDRA-16422) + * Metrics backward compatibility restored after CASSANDRA-15066 (CASSANDRA-16083) + * Reduce new reserved keywords introduced since 3.0 (CASSANDRA-16439) + * Improve system tables handling in case of disk failures (CASSANDRA-14793) + * Add access and datacenters to unreserved keywords (CASSANDRA-16398) + * Fix nodetool ring, status output when DNS resolution or port printing are in use (CASSANDRA-16283) + * Upgrade Jacoco to 0.8.6 (for Java 11 support) (CASSANDRA-16365) + * Move excessive repair debug loggings to trace level (CASSANDRA-16406) + * Restore validation of each message's protocol version (CASSANDRA-16374) + * Upgrade netty and chronicle-queue dependencies to get Auditing and native library loading working on arm64 architectures (CASSANDRA-16384,CASSANDRA-16392) + * Release StreamingTombstoneHistogramBuilder spool when switching writers (CASSANDRA-14834) + * Correct memtable on-heap size calculations to match actual use (CASSANDRA-16318) + * Fix client notifications in CQL protocol v5 (CASSANDRA-16353) + * Too defensive check when picking sstables for preview repair (CASSANDRA-16284) + * Ensure pre-negotiation native protocol responses have correct stream id (CASSANDRA-16376) + * Fix check for -Xlog in cassandra-env.sh (CASSANDRA-16279) + * SSLFactory should initialize SSLContext before setting protocols (CASSANDRA-16362) + * Restore sasi dependencies jflex, snowball-stemmer, and concurrent-trees, in the cassandra-all pom (CASSANDRA-16303) + * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) +Merged from 3.11: * Upgrade jackson-databind to 2.9.10.8 (CASSANDRA-16462) + * Fix digest computation for queries with fetched but non queried columns (CASSANDRA-15962) + * Reduce amount of allocations during batch statement execution (CASSANDRA-16201) + * Update jflex-1.6.0.jar to match upstream (CASSANDRA-16393) Merged from 3.0: + * Fix ColumnFilter::toString not returning a valid CQL fragment (CASSANDRA-16483) * Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades (CASSANDRA-16415) + * Update debian packaging for python3 (CASSANDRA-16396) * Avoid pushing schema mutations when setting up distributed system keyspaces locally (CASSANDRA-16387) + * Prevent unbounded number of pending flushing tasks (CASSANDRA-16261) + * Improve empty hint file handling during startup (CASSANDRA-16162) + * Fix skipping on pre-3.0 created compact storage sstables due to missing primary key liveness (CASSANDRA-16226) + * Allow empty string in collections with COPY FROM in cqlsh (CASSANDRA-16372) Merged from 2.2: * Fix centos packaging for arm64, >=4.0 rpm's now require python3 (CASSANDRA-16477) * Make TokenMetadata's ring version increments atomic (CASSANDRA-16286) diff --cc src/java/org/apache/cassandra/db/AbstractReadQuery.java index c6ec329,000..ec1a6b1 mode 100644,00..100644 ---
[cassandra] branch cassandra-3.11 updated (4a804eb -> dc82d48)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 4a804eb Merge branch 'cassandra-3.0' into cassandra-3.11 (including c8d2394, 6524903, cf0be97) new 9c7b69e Fix ColumnFilter::toString not returning a valid CQL fragment new dc82d48 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ReadCommand.java | 4 +-- .../apache/cassandra/db/filter/ColumnFilter.java | 40 -- .../cassandra/db/filter/ColumnFilterTest.java | 40 ++ 4 files changed, 72 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Fix ColumnFilter::toString not returning a valid CQL fragment
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 9c7b69e Fix ColumnFilter::toString not returning a valid CQL fragment
9c7b69e is described below
commit 9c7b69e4229b6817b248bcf5270a783eed1f9930
Author: Andrés de la Peña
AuthorDate: Wed Mar 10 17:19:17 2021 +
Fix ColumnFilter::toString not returning a valid CQL fragment
patch by Andrés de la Peña; reviewed by Benjamin Lerer for CASSANDRA-16483
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/ReadCommand.java | 4 +-
.../apache/cassandra/db/filter/ColumnFilter.java | 43 +++---
.../cassandra/db/filter/ColumnFilterTest.java | 36 ++
4 files changed, 69 insertions(+), 15 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 567764e..25f938b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.25:
+ * Fix ColumnFilter::toString not returning a valid CQL fragment
(CASSANDRA-16483)
* Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades
(CASSANDRA-16415)
* Avoid pushing schema mutations when setting up distributed system keyspaces
locally (CASSANDRA-16387)
Merged from 2.2:
diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java
b/src/java/org/apache/cassandra/db/ReadCommand.java
index 925708e..4fd4b5f 100644
--- a/src/java/org/apache/cassandra/db/ReadCommand.java
+++ b/src/java/org/apache/cassandra/db/ReadCommand.java
@@ -595,8 +595,8 @@ public abstract class ReadCommand implements ReadQuery
public String toCQLString()
{
StringBuilder sb = new StringBuilder();
-sb.append("SELECT ").append(columnFilter());
-sb.append(" FROM
").append(metadata().ksName).append('.').append(metadata.cfName);
+sb.append("SELECT ").append(columnFilter().toCQLString());
+sb.append(" FROM
").append(metadata().ksName).append('.').append(metadata().cfName);
appendCQLWhereClause(sb);
if (limits() != DataLimits.NONE)
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
index 520c43c..cbc8871 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
@@ -134,6 +134,7 @@ public class ColumnFilter
/**
* Whether we can skip the value for the provided selected column.
*/
+@SuppressWarnings("unused")
public boolean canSkipValue(ColumnDefinition column)
{
// We don't use that currently, see #10655 for more details.
@@ -275,7 +276,7 @@ public class ColumnFilter
SortedSetMultimap s = null;
if (subSelections != null)
{
-s =
TreeMultimap.create(Comparator.naturalOrder(),
Comparator.naturalOrder());
+s = TreeMultimap.create(Comparator.naturalOrder(),
Comparator.naturalOrder());
for (ColumnSubselection subSelection : subSelections)
s.put(subSelection.column().name, subSelection);
}
@@ -305,27 +306,43 @@ public class ColumnFilter
@Override
public String toString()
{
-String prefix = "";
-
if (isFetchAll)
return "*/*";
if (queried.isEmpty())
-return prefix + "[]";
+return "[]";
+
+return toString(false);
+}
+
+public String toCQLString()
+{
+if (queried == null || queried.isEmpty())
+return "*";
-StringJoiner joiner = new StringJoiner(", ", "[", "]");
-Iterator it = queried.selectOrderIterator();
-while (it.hasNext())
+return toString(true);
+}
+
+private String toString(boolean cql)
+{
+Iterator columns = queried.selectOrderIterator();
+StringJoiner joiner = cql ? new StringJoiner(", ") : new
StringJoiner(", ", "[", "]");
+
+while (columns.hasNext())
{
-ColumnDefinition column = it.next();
-SortedSet s = subSelections != null ?
subSelections.get(column.name) : Collections.emptySortedSet();
+ColumnDefinition column = columns.next();
+String columnName = cql ? column.name.toCQLString() :
String.valueOf(column.name);
+
+SortedSet s = subSelections != null
+? subSelections.get(column.name)
+: Collections.emptySortedSet();
if (s.isEmpty())
-joiner.add(String.valueOf(column.name));
+joiner.add(columnName);
else
-s.forEach(subSel -> joiner.add(String.format("%s%s",
column.name, subSel)));
+s.forEach(subSel ->
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit dc82d481d94e38b41dc3de1a07eb18c77d34ad1e
Merge: 4a804eb 9c7b69e
Author: Andrés de la Peña
AuthorDate: Wed Mar 10 17:27:03 2021 +
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/ReadCommand.java | 4 +--
.../apache/cassandra/db/filter/ColumnFilter.java | 40 --
.../cassandra/db/filter/ColumnFilterTest.java | 40 ++
4 files changed, 72 insertions(+), 13 deletions(-)
diff --cc CHANGES.txt
index 3b4b8fd,25f938b..b959a43
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,6 -1,5 +1,7 @@@
-3.0.25:
+3.11.11
+ * Upgrade jackson-databind to 2.9.10.8 (CASSANDRA-16462)
+Merged from 3.0:
+ * Fix ColumnFilter::toString not returning a valid CQL fragment
(CASSANDRA-16483)
* Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades
(CASSANDRA-16415)
* Avoid pushing schema mutations when setting up distributed system
keyspaces locally (CASSANDRA-16387)
Merged from 2.2:
diff --cc src/java/org/apache/cassandra/db/filter/ColumnFilter.java
index f405431,cbc8871..dfbce4a
--- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
@@@ -405,29 -306,43 +405,47 @@@ public class ColumnFilte
@Override
public String toString()
{
- String prefix = "";
-if (isFetchAll)
+if (isFetchAll && queried == null)
return "*/*";
++String prefix = "";
+if (isFetchAll)
+prefix = "*/";
+
if (queried.isEmpty())
-return "[]";
+return prefix + "[]";
- StringJoiner joiner = new StringJoiner(", ", "[", "]");
- Iterator it = queried.selectOrderIterator();
- while (it.hasNext())
-return toString(false);
++return prefix + toString(false);
+ }
+
+ public String toCQLString()
+ {
+ if (queried == null || queried.isEmpty())
+ return "*";
+
+ return toString(true);
+ }
+
+ private String toString(boolean cql)
+ {
+ Iterator columns = queried.selectOrderIterator();
+ StringJoiner joiner = cql ? new StringJoiner(", ") : new
StringJoiner(", ", "[", "]");
+
+ while (columns.hasNext())
{
- ColumnDefinition column = it.next();
- SortedSet s = subSelections != null ?
subSelections.get(column.name) : Collections.emptySortedSet();
+ ColumnDefinition column = columns.next();
+ String columnName = cql ? column.name.toCQLString() :
String.valueOf(column.name);
+
+ SortedSet s = subSelections != null
+ ? subSelections.get(column.name)
+ : Collections.emptySortedSet();
if (s.isEmpty())
- joiner.add(String.valueOf(column.name));
+ joiner.add(columnName);
else
- s.forEach(subSel -> joiner.add(String.format("%s%s",
column.name , subSel)));
+ s.forEach(subSel -> joiner.add(String.format("%s%s",
columnName, subSel)));
}
- return prefix + joiner.toString();
+ return joiner.toString();
}
public static class Serializer
diff --cc test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
index 1ddf039,7977c6b..adf074f
--- a/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
+++ b/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
@@@ -33,7 -27,7 +33,8 @@@ import org.junit.runners.Parameterized
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.config.DatabaseDescriptor;
+ import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.db.PartitionColumns;
import org.apache.cassandra.db.marshal.Int32Type;
import org.apache.cassandra.db.marshal.SetType;
@@@ -287,20 -289,12 +319,22 @@@ public class ColumnFilterTes
Consumer check = filter -> {
testRoundTrips(filter);
assertFetchedQueried(true, true, filter, v1);
-
-assertEquals("*/*", filter.toString());
-assertEquals("v1", filter.toCQLString());
-assertFetchedQueried(true, true, filter, s1, s2, v2);
-assertCellFetchedQueried(true, true, filter, v2, path0, path1,
path2, path3, path4);
-assertCellFetchedQueried(true, true, filter, s2, path0, path1,
path2, path3, path4);
+if (anyNodeOn30)
+{
+assertEquals("*/*", filter.toString());
++assertEquals("*", filter.toCQLString());
[cassandra] branch trunk updated (b70ae0e -> 0e990d7)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b70ae0e Merge branch 'cassandra-3.11' into trunk new 9c7b69e Fix ColumnFilter::toString not returning a valid CQL fragment new dc82d48 Merge branch 'cassandra-3.0' into cassandra-3.11 new 0e990d7 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/db/AbstractReadQuery.java | 2 +- src/java/org/apache/cassandra/db/ReadCommand.java | 2 +- .../apache/cassandra/db/filter/ColumnFilter.java | 38 --- .../cassandra/db/filter/ColumnFilterTest.java | 44 ++ 5 files changed, 72 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12949) CFS.setCompressionParameters() method can affect schema globally
[
https://issues.apache.org/jira/browse/CASSANDRA-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-12949:
--
Status: Ready to Commit (was: Review In Progress)
+1
> CFS.setCompressionParameters() method can affect schema globally
>
>
> Key: CASSANDRA-12949
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12949
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> CFS.setCompressionParameters() method can affect schema globally, despite
> being intended as a way to change compression locally on just one node, for
> experimental purposes.
> CFS.setCompactionParamaters(), on the other hand, works correctly, by never
> modifying the {{CFMetaData}} object in-place.
> Its modification by CFS.setCompressionParameters() means that any subsequent
> ALTER that affects that table will pick up the change made by
> CFS.setCompressionParameters() and disseminate it to the rest of the cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16497) concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution broken
[
https://issues.apache.org/jira/browse/CASSANDRA-16497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16497:
---
Fix Version/s: (was: 4.0-beta)
(was: 4.0)
4.0-rc
> concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution broken
> --
>
> Key: CASSANDRA-16497
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16497
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Adam Holmberg
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-rc
>
>
> {noformat}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:169)
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:102)
> {noformat}
> https://ci-cassandra.apache.org/job/Cassandra-trunk/308/testReport/org.apache.cassandra.concurrent/LongSharedExecutorPoolTest/testPromptnessOfExecution/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298890#comment-17298890
]
Sam Tunnicliffe commented on CASSANDRA-16493:
-
Thanks, that issue is known (and fixed) in
[https://datastax-oss.atlassian.net/browse/JAVA-2922.]
I believe a release of the java-driver is imminent, at which point I'll update
the bundled lib for CASSANDRA-13951
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298873#comment-17298873
]
Ekaterina Dimitrova edited comment on CASSANDRA-16493 at 3/10/21, 2:37 PM:
---
I feel [these
failures|https://issues.apache.org/jira/browse/CASSANDRA-16494?focusedCommentId=17297810=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17297810]
might be useful to know about. The tests were run locally on my Mac, Java 8
The assertion error rooted in the V5 switch
was (Author: e.dimitrova):
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion error rooted in the V5 switch
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298873#comment-17298873
]
Ekaterina Dimitrova edited comment on CASSANDRA-16493 at 3/10/21, 2:37 PM:
---
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion error rooted in the V5 switch
was (Author: e.dimitrova):
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion error rooted in the V5 switch
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298873#comment-17298873
]
Ekaterina Dimitrova commented on CASSANDRA-16493:
-
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion rooted in the V5 switch
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298873#comment-17298873
]
Ekaterina Dimitrova edited comment on CASSANDRA-16493 at 3/10/21, 2:36 PM:
---
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion error rooted in the V5 switch
was (Author: e.dimitrova):
I feel these failures might be useful to know about. The tests were run locally
on my Mac, Java 8
The assertion rooted in the V5 switch
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16506) Drop unittest2 dependency since we no longer support python < 2.7
[ https://issues.apache.org/jira/browse/CASSANDRA-16506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16506: - Resolution: Later Status: Resolved (was: Triage Needed) While python2.7 is deprecated, it does still work (CASSANDRA-16414), so I don't see any compelling reason to purposefully break this test for it. When 2.7 is so dead it doesn't work at all this will become cruft we can remove. > Drop unittest2 dependency since we no longer support python < 2.7 > - > > Key: CASSANDRA-16506 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16506 > Project: Cassandra > Issue Type: Task >Reporter: Jeff Widman >Assignee: Jeff Widman >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > `cqlsh` has a dependency on `unittest2`. > But `unittest2` is simply a backport of python 2.7+ features to older pythons. > Given that support for Python 2.7 is itself now deprecated in favor of Python > 3 support, there's no need to include the `unittest2` import... > I opened a pull request fixing this here: > https://github.com/apache/cassandra/pull/921 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16509) Bump cqlsh version to 6.0.0
[ https://issues.apache.org/jira/browse/CASSANDRA-16509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16509: - Change Category: Operability Complexity: Low Hanging Fruit Component/s: CQL/Interpreter Fix Version/s: 4.0-rc 4.0 Status: Open (was: Triage Needed) > Bump cqlsh version to 6.0.0 > --- > > Key: CASSANDRA-16509 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16509 > Project: Cassandra > Issue Type: Task > Components: CQL/Interpreter >Reporter: Jeff Widman >Priority: Normal > Fix For: 4.0, 4.0-rc > > Time Spent: 20m > Remaining Estimate: 0h > > For the Cassandra 4.x release series, please should bump the cqlsh version... > This way users can report which version of cqlsh they're using, along with > the protocol/server version. > It's been 5.0.1 for the cassandra 3.x series, so 6.0.0 probably makes sense > because there are possible breaking changes depending on how CASSANDRA-16508 > is resolved. > Right now, if a user says they're using `cqlsh` 5.0.1, we have no idea which > release they're actually using because that version number hasn't been > incremented for several releases, even though the code is changing. > I maintain a standalone version of `cqlsh` on PyPI for lighter-weight > installs, and it'd be very helpful if the version was bumped. > I opened a PR here to fix it: https://github.com/apache/cassandra/pull/923 > Although I wasn't sure if this should be set to 6.0.0 because 6.0.0-beta5 > might be more appropriate if there's another beta release planned before GA. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16483) ColumnFilter::toString doesn't return a valid CQL fragment
[
https://issues.apache.org/jira/browse/CASSANDRA-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298847#comment-17298847
]
Andres de la Peña commented on CASSANDRA-16483:
---
Great, I have created CASSANDRA-16510 to deal with the problems in
{{ReadCommand::toCQLString}}, so this ticket is focused on
{{ColumnFilter::toCQLString}} and broken tests.
> ColumnFilter::toString doesn't return a valid CQL fragment
> --
>
> Key: CASSANDRA-16483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16483
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Sam Tunnicliffe
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This was changed in CASSANDRA-16415 to include indications about queried vs
> fetched reagular & static columns. However, the result is used by
> {{AbstractReadQuery::toCQLString}}, which causes it to produce an illegal
> query string.
> This breaks a couple of dtests because they're looking for CQL strings in
> logs, which are no longer found:
> *
> {{upgrade_tests/paging_test.py::TestPagingWithDeletions::test_failure_threshold_deletions}}
> * {{cql_test.py::TestCQLSlowQuery}} has a couple of failing tests,
> {{test_local_query/test_remote_query}}
> We should also check audit and fql logs (and any other place where
> {{toCQLString}} is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16510) Make ReadCommand::toCQLString return valid CQL
Andres de la Peña created CASSANDRA-16510:
-
Summary: Make ReadCommand::toCQLString return valid CQL
Key: CASSANDRA-16510
URL: https://issues.apache.org/jira/browse/CASSANDRA-16510
Project: Cassandra
Issue Type: Improvement
Components: CQL/Syntax, Observability/Logging
Reporter: Andres de la Peña
The method {{ReadCommand::toCQLString}} doesn't always return queries that are
valid CQL. For example, queries for a table without clustering columns always
add the {{WHERE}} keyword even if there isn't a restriction following it:
{code:java}
CREATE TABLE t (k int PRIMARY KEY, v int);
SELECT * FROM t;
-- toCQLString 3.0:SELECT * FROM k.t WHERE () = () LIMIT 100
-- toCQLString 3.11/trunk: SELECT * FROM k.t WHERE LIMIT 100
{code}
Also, case sensitive names and text values are not properly quoted:
{code:java}
CREATE TABLE "T" ("K" text, "C" text, "V" text, PRIMARY KEY("K", "C"));
SELECT * FROM "T" WHERE "K" = 'a' AND "C" = 'b' AND "V" = 'c' ALLOW FILTERING;
-- toCQLString 3.0/3.11/trunk: SELECT * FROM k.T WHERE K = a AND V = c AND (C)
= (b) LIMIT 100
{code}
Similar problems can be found on selections of collection items:
{code:java}
CREATE TABLE k.t (k int, c int, s set, m map, PRIMARY KEY(k,
c));
SELECT s['a'] FROM t;
-- toCQLString trunk: SELECT s[a] FROM k.t LIMIT 100
SELECT * FROM t WHERE m['a'] = 'b' ALLOW FILTERING;
-- toCQLString 3.0/3.11/trunk: SELECT * FROM k.t WHERE m[a] = b LIMIT 100
{code}
Some similar problems with more impact than the described above are already
being addressed in CASSANDRA-16483 and CASSANDRA-16479.
I think we should add more exhaustive JUnit tests for
{{ReadCommand::toCQLString}}, making sure that its output is parseable and
equivalent to the original command. Also, we probably should make sure that we
have {{toCQLString}} methods in all the query components that are used by
{{ReadCommand::toCQLString}} (selection, filters, etc.). This way we can use
them instead of generic {{toString}} methods, making it clear that they are
supposed to return valid CQL.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298825#comment-17298825 ] Benedict Elliott Smith commented on CASSANDRA-16499: Well, it's not entirely unknown as we tested on several Linux environments when developing this, and found significant improvement with the behaviour in this executor. I vaguely recall evidence that, on Windows, it was not advantageous. It's certainly worth revisiting, it's possible that the work stealing is the only benefit today, with the scheduling changes now a net cost on all platforms, however to do a proper analysis is not as simple as you suggest, and I probably won't have time to dig into such an endeavour for a while. > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: Cassandra Write trace 5; warmed up -- 02.txt, > MaybeStartSpinning Unpark fix on 4beta4; Cassandra Stress results -- 01.txt, > MaybeStartSpinning Unpark fix; Cassandra Stress results -- 02.txt, SEPWorker > trace 2 delay examples -- 01.txt, SEPWorker trace 2 delays.txt, SEPWorker > trace 3 delays; with proposed fix.txt, Single-thread Latencies report -- > 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- 01.txt, Stress Write > sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, proposed fix patches.zip, > tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern of threads waiting in > AbstractWriteResponseHandler while nothing else is busy > * checked out Cassandra 3.11.10 source, built it, debugged & stepped > through key areas to try and understand behavior. > * instrumented key areas with custom tracing code & timestamps to 0.01 > millisecond. > ** _see patch attached._ > * benchmarked Cassandra 4 beta 4 & verified delays also present > * shown & traced delays with my healthcare scenario benchmark > * shown & traced delays with the +Cassandra stress-test+ tool. > The configuration was: > * single-node Cassandra running locally, on a recent Dell laptop with NVmE > SSD. > * for the healthcare scenario: > ** Java client app running 1 or 10 threads; > ** trialled LOCAL_ONE and ANY consistency levels; > ** trialled unbatched, BatchType.UNLOGGED and BatchType.LOGGED. > * for 'cassandra-stress': > ** cassandra-stress.bat write n=1 -rate threads=1 > Without deeply understanding the code, I have considered a couple of possible > areas/ ideas as to improvement. I worked on the 3.11.10 codebase. I'd be > interested to understand whether or not these might be sound or not; note > that neither achieves as much improvement as might theoretically be hoped for. > My investigations based on the key observation of large delays between > StorageProxy.performLocally() being invoked and the LocalMutationRunnable > actually executing, for single-threaded workloads. > What I looked at: > * Without fully understanding
[cassandra-builds] branch trunk updated: added docs for Plausible deployment (CASSANDRA-16488)
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new 3659429 added docs for Plausible deployment (CASSANDRA-16488) 3659429 is described below commit 3659429f88b787b47db34e26601929434bd06e78 Author: Stefan Miklosovic AuthorDate: Tue Mar 9 21:14:55 2021 +0100 added docs for Plausible deployment (CASSANDRA-16488) Co-authored-by: mck --- plausible/README.adoc| 151 +++ plausible/app.conf | 31 + plausible/docker-compose.yml | 55 3 files changed, 237 insertions(+) diff --git a/plausible/README.adoc b/plausible/README.adoc new file mode 100644 index 000..84b12ca --- /dev/null +++ b/plausible/README.adoc @@ -0,0 +1,151 @@ +== Analytics for Cassandra Web + +This document serves as a guide for anybody who wants to re/deploy https://plausible.io/[Plausible]. + +=== Overview of the current deployment + +Plausible is currently deployed on t2.medium instance in AWS EC2, it runs on Debian 10 Buster and +it has Elastic IP 3.124.255.5 (stays after restarts). Plausible runs in Docker Compose and it is +secured by HTTPS. Certificates for HTTPS are obtained from Let's Encrypt and they are +automatically renewed after expiration. There is nginx reverse proxy in front of Plausible as +the other Docker Compose deployment. + +Plausible service is hosted under https://www.instaclustr.com[Instaclustr]'s account. Please contact +stefan dot miklosovic at instaclustr dot com for any related questions / issues. + +Domain `plausible.cassandra.apache.org` points to 3.124.255.5. Please contact gmcdonald at apache dot org for +any related questions / issues. + +For the access to all credentials (PEM to ssh to that box, logins to Plausible etc), please contact +someone on the PMC, as credentials are stored in the private subversion repository. + +=== Setup of Plausible from scratch + +Please be sure that before proceeding, your box has these services / software installed: + +* docker +* docker-compose +* git + +The deployment consists of these logical steps: + +* setup HTTPS for nginx proxy +* setup & run self-hosted Plausible +* setup & run reverse nginx proxy talking to Plausible + + Setup of HTTP for nginx proxy + +First we generate HTTPS certificates: + + +$ git clone https://github.com/wmnnd/nginx-certbot.git +$ cd nginx-certbox + + +Follow the https://github.com/wmnnd/nginx-certbot[readme] of nginx-certbox repo. You need to change +`domains` in `init-letsencrypt.sh` script to `domains=(plausible.cassandra.apache.org)`. + +Next, modify `data/nginx/app.conf`, change `server_name` to `plausible.cassandra.apache.org` for +both 80 and 443 case and change paths to certificates. + +`proxy_pass` field of `location /` for server listening to port 443 should be set to `http://plausible:8000/;`. +`plausible` in this address points to name of a service of Docker compose we will setup afterwards. + +This will generate HTTPS certificates, please be sure that you have 80 and 443 port open. + + +$ ./init-letsencrypt.sh +$ docker-compose up -d + + +`data/nginx/app.conf` file with described changes is also in the same directory as this document for reference. + + Setup of Plausible + + +$ git clone https://github.com/plausible/hosting +$ cd hosting + + +The main configuration file is `plausible-conf.env` + + +ADMIN_USER_EMAIL=stefan.mikloso...@instaclustr.com +ADMIN_USER_NAME=admin +ADMIN_USER_PWD=you will log in with this password under ADMIN_USER_EMAIL +BASE_URL=https://plausible.cassandra.apache.org/ +SECRET_KEY_BASE=see for the explanation below +PORT=8000 +DISABLE_AUTH=false +DISABLE_REGISTRATION=false +GEOLITE2_COUNTRY_DB=/geoip/GeoLite2-Country.mmdb + + +`SECRET_KEY_BASE` is an internal secret key used by Phoenix Framework which Plausible builds on. +Follow the https://hexdocs.pm/phoenix/Mix.Tasks.Phx.Gen.Secret.html#content[instructions] to generate one. +The setup to generate such a secret is very long as it requires you to setup Erlang OTP, Phoenix etc etc and it seems +to me that the value of `SECRET_KEY_BASE` needs to be some random string so https://github.com/plausible/analytics/discussions/824[anything sensible will do]. + +For GeoIP (seeing a world map with hits from countries), you need to https://dev.maxmind.com/geoip/geoip2/geolite2/[create an account here]. +Then provide credentials in `geopip/geoip.conf`. + +`DISABLE_REGISTRATION` might be set to `true` after all accounts are created as we will prevent everybody to +try to create accounts and potentially abuse this service. + +You do not need to do anything for SMTP to work, all default values are just fine in order to be able to +send transactional emails. Please consult SMTP setup in FAQ section to know
[jira] [Updated] (CASSANDRA-16500) Missing validation in AbstractType.writeValue():
[
https://issues.apache.org/jira/browse/CASSANDRA-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16500:
--
Reviewers: Andres de la Peña
> Missing validation in AbstractType.writeValue():
>
>
> Key: CASSANDRA-16500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16500
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Benjamin Lerer
>Assignee: Alexandre Dutra
>Priority: Normal
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Some validation checks present in {{AbstractType.writeValue()}} in
> {{cassandra-3.11}} are not there in {{trunk}}.
> In 3.11 the checks used assertion. It would make sense to use
> {{IOExceptions}} instead as used in {{AbstractType.readValue()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16483) ColumnFilter::toString doesn't return a valid CQL fragment
[
https://issues.apache.org/jira/browse/CASSANDRA-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16483:
--
Status: Ready to Commit (was: Review In Progress)
> ColumnFilter::toString doesn't return a valid CQL fragment
> --
>
> Key: CASSANDRA-16483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16483
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Sam Tunnicliffe
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This was changed in CASSANDRA-16415 to include indications about queried vs
> fetched reagular & static columns. However, the result is used by
> {{AbstractReadQuery::toCQLString}}, which causes it to produce an illegal
> query string.
> This breaks a couple of dtests because they're looking for CQL strings in
> logs, which are no longer found:
> *
> {{upgrade_tests/paging_test.py::TestPagingWithDeletions::test_failure_threshold_deletions}}
> * {{cql_test.py::TestCQLSlowQuery}} has a couple of failing tests,
> {{test_local_query/test_remote_query}}
> We should also check audit and fql logs (and any other place where
> {{toCQLString}} is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12949) CFS.setCompressionParameters() method can affect schema globally
[
https://issues.apache.org/jira/browse/CASSANDRA-12949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298751#comment-17298751
]
Alex Petrov commented on CASSANDRA-12949:
-
[~aleksey] made a change to reset local params after {{ALTER}} on all branches
(and added the test). {{trunk}} now is a bit different: there is now a concept
of "local-only" metadata, which we can use for local-only changes that should
not be propagated. I've avoided, at least for now, using it across the
codebase, since arguably it's best if we only use it on the code paths where we
actually need it. Let me know what you think about this change.
> CFS.setCompressionParameters() method can affect schema globally
>
>
> Key: CASSANDRA-12949
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12949
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> CFS.setCompressionParameters() method can affect schema globally, despite
> being intended as a way to change compression locally on just one node, for
> experimental purposes.
> CFS.setCompactionParamaters(), on the other hand, works correctly, by never
> modifying the {{CFMetaData}} object in-place.
> Its modification by CFS.setCompressionParameters() means that any subsequent
> ALTER that affects that table will pick up the change made by
> CFS.setCompressionParameters() and disseminate it to the rest of the cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16453) NPE in Slice#make on RT + partition deletion reconciliation on timestamp tie
[
https://issues.apache.org/jira/browse/CASSANDRA-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16453:
---
Fix Version/s: (was: 4.0-rc)
4.0
4.0-beta5
> NPE in Slice#make on RT + partition deletion reconciliation on timestamp tie
>
>
> Key: CASSANDRA-16453
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16453
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 4.0-beta5, 4.0
>
>
> There’s an NPE in Slice#make on RT + partition deletion reconciliation.
> Minimal repro:
> {code:java}
> try (Cluster cluster = init(builder().withNodes(3).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE
> distributed_test_keyspace.table_0 (pk0 bigint,ck0 bigint,regular0
> bigint,regular1 bigint,regular2 bigint, PRIMARY KEY (pk0, ck0)) WITH
> CLUSTERING ORDER BY (ck0 ASC);"));
> long pk = 0L;
> cluster.coordinator(1).execute("DELETE FROM
> distributed_test_keyspace.table_0 USING TIMESTAMP 100230 WHERE pk0=? AND
> ck0>?;", ConsistencyLevel.ALL, pk, 2L);
> cluster.get(3).executeInternal("DELETE FROM
> distributed_test_keyspace.table_0 USING TIMESTAMP 100230 WHERE pk0=?;", pk);
> cluster.coordinator(1).execute("SELECT * FROM
> distributed_test_keyspace.table_0 WHERE pk0=? AND ck0>=? AND ck0 ConsistencyLevel.ALL,
> pk, 1L, 3L);
> }
> {code}
> Details:
> {code:java}
> java.lang.AssertionError: Error merging RTs on
> distributed_test_keyspace.table_0: merged=null, versions=[Marker
> EXCL_START_BOUND(2)@100230/1613500432, Marker
> EXCL_START_BOUND(2)@100230/1613500432, null],
> sources={[Full(/127.0.0.1:7012,(-3074457345618258603,3074457345618258601]),
> Full(/127.0.0.2:7012,(-3074457345618258603,3074457345618258601]),
> Full(/127.0.0.3:7012,(-3074457345618258603,3074457345618258601])]}, debug
> info:
> /127.0.0.1:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> Marker EXCL_START_BOUND(2)@100230/1613500432
> Marker EXCL_END_BOUND(3)@100230/1613500432,
> /127.0.0.2:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> Marker EXCL_START_BOUND(2)@100230/1613500432
> Marker EXCL_END_BOUND(3)@100230/1613500432,
> /127.0.0.3:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=100230, localDeletion=1613500432 columns=[[] |
> [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> {code}
> Exception:
> {code:java}
> java.lang.NullPointerException
> at org.apache.cassandra.db.Slice.make(Slice.java:74)
> at
> org.apache.cassandra.service.reads.repair.RowIteratorMergeListener.closeOpenMarker(RowIteratorMergeListener.java:351)
> at
> org.apache.cassandra.service.reads.repair.RowIteratorMergeListener.onMergedRangeTombstoneMarkers(RowIteratorMergeListener.java:315)
> at
> org.apache.cassandra.service.reads.DataResolver$2$1.onMergedRangeTombstoneMarkers(DataResolver.java:378)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:592)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:541)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:219)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:523)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:391)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:50)
> at
>
[jira] [Updated] (CASSANDRA-16453) NPE in Slice#make on RT + partition deletion reconciliation on timestamp tie
[
https://issues.apache.org/jira/browse/CASSANDRA-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-16453:
Since Version: 4.0-beta4
Source Control Link:
https://github.com/apache/cassandra/commit/6f13c864a02b32daa7696eca27431f5385a306df
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thank you for review! Committed to trunk with
[6f13c864a02b32daa7696eca27431f5385a306df|https://github.com/apache/cassandra/commit/6f13c864a02b32daa7696eca27431f5385a306df]
> NPE in Slice#make on RT + partition deletion reconciliation on timestamp tie
>
>
> Key: CASSANDRA-16453
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16453
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 4.0-rc
>
>
> There’s an NPE in Slice#make on RT + partition deletion reconciliation.
> Minimal repro:
> {code:java}
> try (Cluster cluster = init(builder().withNodes(3).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE
> distributed_test_keyspace.table_0 (pk0 bigint,ck0 bigint,regular0
> bigint,regular1 bigint,regular2 bigint, PRIMARY KEY (pk0, ck0)) WITH
> CLUSTERING ORDER BY (ck0 ASC);"));
> long pk = 0L;
> cluster.coordinator(1).execute("DELETE FROM
> distributed_test_keyspace.table_0 USING TIMESTAMP 100230 WHERE pk0=? AND
> ck0>?;", ConsistencyLevel.ALL, pk, 2L);
> cluster.get(3).executeInternal("DELETE FROM
> distributed_test_keyspace.table_0 USING TIMESTAMP 100230 WHERE pk0=?;", pk);
> cluster.coordinator(1).execute("SELECT * FROM
> distributed_test_keyspace.table_0 WHERE pk0=? AND ck0>=? AND ck0 ConsistencyLevel.ALL,
> pk, 1L, 3L);
> }
> {code}
> Details:
> {code:java}
> java.lang.AssertionError: Error merging RTs on
> distributed_test_keyspace.table_0: merged=null, versions=[Marker
> EXCL_START_BOUND(2)@100230/1613500432, Marker
> EXCL_START_BOUND(2)@100230/1613500432, null],
> sources={[Full(/127.0.0.1:7012,(-3074457345618258603,3074457345618258601]),
> Full(/127.0.0.2:7012,(-3074457345618258603,3074457345618258601]),
> Full(/127.0.0.3:7012,(-3074457345618258603,3074457345618258601])]}, debug
> info:
> /127.0.0.1:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> Marker EXCL_START_BOUND(2)@100230/1613500432
> Marker EXCL_END_BOUND(3)@100230/1613500432,
> /127.0.0.2:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=-9223372036854775808, localDeletion=2147483647
> columns=[[] | [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> Marker EXCL_START_BOUND(2)@100230/1613500432
> Marker EXCL_END_BOUND(3)@100230/1613500432,
> /127.0.0.3:7012 => [distributed_test_keyspace.table_0] key=0
> partition_deletion=deletedAt=100230, localDeletion=1613500432 columns=[[] |
> [regular0 regular1 regular2]] repaired_digest=
> repaired_digest_conclusive==true
> {code}
> Exception:
> {code:java}
> java.lang.NullPointerException
> at org.apache.cassandra.db.Slice.make(Slice.java:74)
> at
> org.apache.cassandra.service.reads.repair.RowIteratorMergeListener.closeOpenMarker(RowIteratorMergeListener.java:351)
> at
> org.apache.cassandra.service.reads.repair.RowIteratorMergeListener.onMergedRangeTombstoneMarkers(RowIteratorMergeListener.java:315)
> at
> org.apache.cassandra.service.reads.DataResolver$2$1.onMergedRangeTombstoneMarkers(DataResolver.java:378)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:592)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:541)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:219)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:523)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:391)
> at
>
[jira] [Commented] (CASSANDRA-16477) Fix centos packaging for arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298724#comment-17298724 ] Michael Semb Wever commented on CASSANDRA-16477: The commit message is wrong: only 4.0 now *requires* python3. Other versions are compatible with python3 now (honouring the python you have installed). CHANGES.txt has been ninja-fixed. > Fix centos packaging for arm64 > -- > > Key: CASSANDRA-16477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16477 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.1.23, 3.0.25, 3.11.11, 4.0-beta5, 4.0 > > > The cryptography python library (needed by urllib3) does not install on arm64 > using centos 7 and python2. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (ce075b3 -> 4a804eb)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ce075b3 Merge branch 'cassandra-3.0' into cassandra-3.11 add c8d2394 Revert "Remove python3:Depends from Debian packaging" add 6524903 Revert "Ninja update debian packaging to python3 from python for CASSANDRA-16478" add cf0be97 Revert "Update debian packaging to python3" new a7b71eb ninja-fix CHANGES.txt (CASSANDRA-16477) new 81e4725 Merge branch 'cassandra-2.2' into cassandra-3.0 new 4a804eb Merge branch 'cassandra-3.0' into cassandra-3.11 (including c8d2394, 6524903, cf0be97) The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (814af71 -> b70ae0e)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 814af71 Merge branch 'cassandra-3.11' into trunk add c8d2394 Revert "Remove python3:Depends from Debian packaging" add 6524903 Revert "Ninja update debian packaging to python3 from python for CASSANDRA-16478" add cf0be97 Revert "Update debian packaging to python3" new a7b71eb ninja-fix CHANGES.txt (CASSANDRA-16477) new 81e4725 Merge branch 'cassandra-2.2' into cassandra-3.0 new 4a804eb Merge branch 'cassandra-3.0' into cassandra-3.11 (including c8d2394, 6524903, cf0be97) new b70ae0e Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b70ae0e6ac41e10b274e2af7afab1ebbdacffa83 Merge: 814af71 4a804eb Author: Mick Semb Wever AuthorDate: Wed Mar 10 11:01:15 2021 +0100 Merge branch 'cassandra-3.11' into trunk CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --cc CHANGES.txt index 5aa2757,3b4b8fd..a3a03c0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,86 -1,24 +1,86 @@@ -3.11.11 +4.0-beta5 + * Send FAILED_SESSION_MSG on shutdown and on in-progress repairs during startup (CASSANDRA-16425) + * Reinstate removed ApplicationState padding (CASSANDRA-16484) + * Expose data dirs to ColumnFamilyStoreMBean (CASSANDRA-16335) + * Add possibility to copy SSTables in SSTableImporter instead of moving them (CASSANDRA-16407) + * Fix DESCRIBE statement for CUSTOM indices with options (CASSANDRA-16482) + * Fix cassandra-stress JMX connection (CASSANDRA-16473) + * Avoid processing redundant application states on endpoint changes (CASSANDRA-16381) + * Prevent parent repair sessions leak (CASSANDRA-16446) + * Fix timestamp issue in SinglePartitionSliceCommandTest testPartitionD…eletionRowDeletionTie (CASSANDRA-16443) + * Promote protocol V5 out of beta (CASSANDRA-14973) + * Fix incorrect encoding for strings can be UTF8 (CASSANDRA-16429) + * Fix node unable to join when RF > N in multi-DC with added warning (CASSANDRA-16296) + * Add an option to nodetool tablestats to check sstable location correctness (CASSANDRA-16344) + * Unable to ALTER KEYSPACE while decommissioned/assassinated nodes are in gossip (CASSANDRA-16422) + * Metrics backward compatibility restored after CASSANDRA-15066 (CASSANDRA-16083) + * Reduce new reserved keywords introduced since 3.0 (CASSANDRA-16439) + * Improve system tables handling in case of disk failures (CASSANDRA-14793) + * Add access and datacenters to unreserved keywords (CASSANDRA-16398) + * Fix nodetool ring, status output when DNS resolution or port printing are in use (CASSANDRA-16283) + * Upgrade Jacoco to 0.8.6 (for Java 11 support) (CASSANDRA-16365) + * Move excessive repair debug loggings to trace level (CASSANDRA-16406) + * Restore validation of each message's protocol version (CASSANDRA-16374) + * Upgrade netty and chronicle-queue dependencies to get Auditing and native library loading working on arm64 architectures (CASSANDRA-16384,CASSANDRA-16392) + * Release StreamingTombstoneHistogramBuilder spool when switching writers (CASSANDRA-14834) + * Correct memtable on-heap size calculations to match actual use (CASSANDRA-16318) + * Fix client notifications in CQL protocol v5 (CASSANDRA-16353) + * Too defensive check when picking sstables for preview repair (CASSANDRA-16284) + * Ensure pre-negotiation native protocol responses have correct stream id (CASSANDRA-16376) + * Fix check for -Xlog in cassandra-env.sh (CASSANDRA-16279) + * SSLFactory should initialize SSLContext before setting protocols (CASSANDRA-16362) + * Restore sasi dependencies jflex, snowball-stemmer, and concurrent-trees, in the cassandra-all pom (CASSANDRA-16303) + * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) +Merged from 3.11: * Upgrade jackson-databind to 2.9.10.8 (CASSANDRA-16462) + * Fix digest computation for queries with fetched but non queried columns (CASSANDRA-15962) + * Reduce amount of allocations during batch statement execution (CASSANDRA-16201) + * Update jflex-1.6.0.jar to match upstream (CASSANDRA-16393) Merged from 3.0: * Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades (CASSANDRA-16415) + * Update debian packaging for python3 (CASSANDRA-16396) * Avoid pushing schema mutations when setting up distributed system keyspaces locally (CASSANDRA-16387) + * Prevent unbounded number of pending flushing tasks (CASSANDRA-16261) + * Improve empty hint file handling during startup (CASSANDRA-16162) + * Fix skipping on pre-3.0 created compact storage sstables due to missing primary key liveness (CASSANDRA-16226) + * Allow empty string in collections with COPY FROM in cqlsh (CASSANDRA-16372) Merged from 2.2: - * Fix centos packaging for arm64, >=3.0 rpm's now require python3 (CASSANDRA-16477) + * Fix centos packaging for arm64, >=4.0 rpm's now require python3 (CASSANDRA-16477) * Make TokenMetadata's ring version increments atomic (CASSANDRA-16286) -3.11.10 - * Fix digest computation for queries with fetched but non queried columns (CASSANDRA-15962) - * Reduce amount of allocations during batch statement execution (CASSANDRA-16201) - * Update jflex-1.6.0.jar to match upstream (CASSANDRA-16393) - * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) - * Rate limit validation compactions using compaction_throughput_mb_per_sec (CASSANDRA-16161) +4.0-beta4 + * DROP COMPACT STORAGE should invalidate prepared statements still
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 81e472571a6a6df935986b45b1575a0b8415172d Merge: cf0be97 a7b71eb Author: Mick Semb Wever AuthorDate: Wed Mar 10 10:27:16 2021 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --cc CHANGES.txt index e015772,7d1d27b..567764e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,29 -1,6 +1,29 @@@ -2.2.20 +3.0.25: + * Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades (CASSANDRA-16415) + * Avoid pushing schema mutations when setting up distributed system keyspaces locally (CASSANDRA-16387) +Merged from 2.2: - * Fix centos packaging for arm64, >=3.0 rpm's now require python3 (CASSANDRA-16477) + * Fix centos packaging for arm64, >=4.0 rpm's now require python3 (CASSANDRA-16477) * Make TokenMetadata's ring version increments atomic (CASSANDRA-16286) + + +3.0.24: + * Prevent unbounded number of pending flushing tasks; Add PendingFlushTasks metric (CASSANDRA-16261) + * Improve empty hint file handling during startup (CASSANDRA-16162) + * Allow empty string in collections with COPY FROM in cqlsh (CASSANDRA-16372) + * Fix skipping on pre-3.0 created compact storage sstables due to missing primary key liveness (CASSANDRA-16226) + * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) + * Extend the exclusion of replica filtering protection to other indices instead of just SASI (CASSANDRA-16311) + * Synchronize transaction logs for JBOD (CASSANDRA-16225) + * Fix the counting of cells per partition (CASSANDRA-16259) + * Fix serial read/non-applying CAS linearizability (CASSANDRA-12126) + * Avoid potential NPE in JVMStabilityInspector (CASSANDRA-16294) + * Improved check of num_tokens against the length of initial_token (CASSANDRA-14477) + * Fix a race condition on ColumnFamilyStore and TableMetrics (CASSANDRA-16228) + * Remove the SEPExecutor blocking behavior (CASSANDRA-16186) + * Wait for schema agreement when bootstrapping (CASSANDRA-15158) + * Fix invalid cell value skipping when reading from disk (CASSANDRA-16223) + * Prevent invoking enable/disable gossip when not in NORMAL (CASSANDRA-16146) +Merged from 2.2: * Remove OpenJDK log warning (CASSANDRA-15563) * Fix the histogram merge of the table metrics (CASSANDRA-16259) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (cf0be97 -> 81e4725)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from cf0be97 Revert "Update debian packaging to python3" new a7b71eb ninja-fix CHANGES.txt (CASSANDRA-16477) new 81e4725 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11 (including c8d2394, 6524903, cf0be97)
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 4a804eb807a1bd10ad30f2fac08db3217fe26e3a Merge: ce075b3 81e4725 Author: Mick Semb Wever AuthorDate: Wed Mar 10 10:58:47 2021 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 (including c8d2394, 6524903, cf0be97) CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --cc CHANGES.txt index 26f07fe,567764e..3b4b8fd --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -4,18 -2,12 +4,18 @@@ Merged from 3.0 * Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades (CASSANDRA-16415) * Avoid pushing schema mutations when setting up distributed system keyspaces locally (CASSANDRA-16387) Merged from 2.2: - * Fix centos packaging for arm64, >=3.0 rpm's now require python3 (CASSANDRA-16477) + * Fix centos packaging for arm64, >=4.0 rpm's now require python3 (CASSANDRA-16477) * Make TokenMetadata's ring version increments atomic (CASSANDRA-16286) - -3.0.24: - * Prevent unbounded number of pending flushing tasks; Add PendingFlushTasks metric (CASSANDRA-16261) +3.11.10 + * Fix digest computation for queries with fetched but non queried columns (CASSANDRA-15962) + * Reduce amount of allocations during batch statement execution (CASSANDRA-16201) + * Update jflex-1.6.0.jar to match upstream (CASSANDRA-16393) + * Fix DecimalDeserializer#toString OOM (CASSANDRA-14925) + * Rate limit validation compactions using compaction_throughput_mb_per_sec (CASSANDRA-16161) + * SASI's `max_compaction_flush_memory_in_mb` settings over 100GB revert to default of 1GB (CASSANDRA-16071) +Merged from 3.0: + * Prevent unbounded number of pending flushing tasks (CASSANDRA-16261) * Improve empty hint file handling during startup (CASSANDRA-16162) * Allow empty string in collections with COPY FROM in cqlsh (CASSANDRA-16372) * Fix skipping on pre-3.0 created compact storage sstables due to missing primary key liveness (CASSANDRA-16226) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: ninja-fix CHANGES.txt (CASSANDRA-16477)
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-2.2 by this push: new a7b71eb ninja-fix CHANGES.txt (CASSANDRA-16477) a7b71eb is described below commit a7b71eb63eecde97e8ecb70827b09a621040147d Author: Mick Semb Wever AuthorDate: Wed Mar 10 10:26:25 2021 +0100 ninja-fix CHANGES.txt (CASSANDRA-16477) --- CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.txt b/CHANGES.txt index cdb2def..7d1d27b 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,5 @@ 2.2.20 - * Fix centos packaging for arm64, >=3.0 rpm's now require python3 (CASSANDRA-16477) + * Fix centos packaging for arm64, >=4.0 rpm's now require python3 (CASSANDRA-16477) * Make TokenMetadata's ring version increments atomic (CASSANDRA-16286) * Remove OpenJDK log warning (CASSANDRA-15563) * Fix the histogram merge of the table metrics (CASSANDRA-16259) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16500) Missing validation in AbstractType.writeValue():
[
https://issues.apache.org/jira/browse/CASSANDRA-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Dutra updated CASSANDRA-16500:
Test and Documentation Plan: PR:
https://github.com/apache/cassandra/pull/924
Status: Patch Available (was: In Progress)
> Missing validation in AbstractType.writeValue():
>
>
> Key: CASSANDRA-16500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16500
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Benjamin Lerer
>Assignee: Alexandre Dutra
>Priority: Normal
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Some validation checks present in {{AbstractType.writeValue()}} in
> {{cassandra-3.11}} are not there in {{trunk}}.
> In 3.11 the checks used assertion. It would make sense to use
> {{IOExceptions}} instead as used in {{AbstractType.readValue()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298715#comment-17298715 ] Tom Whitmore commented on CASSANDRA-16499: -- Hi [~benedict], thanks very much for your response. I was actually very curious to find out how this change affects performance on a Linux environment. At this stage it's not known, but given the improvements on Windows it seems like it might be prudent to benchmark whether perhaps some benefit could be realized on Linux too. It's two commits and a before/after with two 20-second stress tests – any chance you or another contributor ([~clohfink] perhaps) could give this a try? Patch is here: * [^proposed fix patches.zip] Let me know your results. > single-threaded write workloads can spend ~70% time waiting on SEPExecutor > -- > > Key: CASSANDRA-16499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16499 > Project: Cassandra > Issue Type: Bug >Reporter: Tom Whitmore >Priority: Normal > Attachments: Cassandra Write trace 5; warmed up -- 02.txt, > MaybeStartSpinning Unpark fix on 4beta4; Cassandra Stress results -- 01.txt, > MaybeStartSpinning Unpark fix; Cassandra Stress results -- 02.txt, SEPWorker > trace 2 delay examples -- 01.txt, SEPWorker trace 2 delays.txt, SEPWorker > trace 3 delays; with proposed fix.txt, Single-thread Latencies report -- > 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- 01.txt, Stress Write > sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, proposed fix patches.zip, > tracing & experimental change patches.zip > > > Hi all! While conducting benchmarking of Cassandra against other databases > for a particular healthcare solution, I found some surprising anomalies in > single-threaded write performance. > Analysis & tracing suggest there might be some inefficiencies in inter-thread > execution in Cassandra; > * Tracing showed an average delay of 1.52 ms between > StorageProxy.performLocally() being called, and the LocalMutationRunnable > actually executing. > * Total operation time averaged 2.06 ms (measured at Message.Dispatcher > processRequest()). This suggested ~72% of the +total operation time+ being > lost waiting for thread scheduling in SEPExecutor. > * With I tested with multiple threads, performance with 10 threads was 27x > higher. This supports a hypothesis that scheduling delays may be hindering > single-threaded progress. > * Transaction throughput for Cassandra with a single-threaded workload, > measured far lower than that of PostgreSQL on the same hardware. Postgres > achieved ~200k committed transactions/ minute including fsync; Cassandra > achieves ~37k per minute. Given they are both essentially writing to a commit > log, it may be informative to understand why the differences are arising. > Cassandra's architecture seems in theory like it might be aligned for our > usecase, given the Commit Log and Log Structured Merge design. Some of our > customers have configurations posing high single-threaded loads. Therefore I > spent some time trying to understand why efficiency for such loads seemed > less than expected. > My investigation so far: > * benchmarked Cassandra 3.11.10 > * stack-dumped it under load & identified a pattern of threads waiting in > AbstractWriteResponseHandler while nothing else is busy > * checked out Cassandra 3.11.10 source, built it, debugged & stepped > through key areas to try and understand behavior. > * instrumented key areas with custom tracing code & timestamps to 0.01 > millisecond. > ** _see patch attached._ > * benchmarked Cassandra 4 beta 4 & verified delays also present > * shown & traced delays with my healthcare scenario benchmark > * shown & traced delays with the +Cassandra stress-test+ tool. > The configuration was: > * single-node Cassandra running locally, on a recent Dell laptop with NVmE > SSD. > * for the healthcare scenario: > ** Java client app running 1 or 10 threads; > ** trialled LOCAL_ONE and ANY consistency levels; > ** trialled unbatched, BatchType.UNLOGGED and BatchType.LOGGED. > * for 'cassandra-stress': > ** cassandra-stress.bat write n=1 -rate threads=1 > Without deeply understanding the code, I have considered a couple of possible > areas/ ideas as to improvement. I worked on the 3.11.10 codebase. I'd be > interested to understand whether or not these might be sound or not; note > that neither achieves as much improvement as might theoretically be hoped for. > My investigations based on the key observation of large delays between > StorageProxy.performLocally() being invoked and the LocalMutationRunnable > actually executing, for single-threaded workloads. > What I looked at: > * Without fully understanding
[jira] [Commented] (CASSANDRA-16499) single-threaded write workloads can spend ~70% time waiting on SEPExecutor
[
https://issues.apache.org/jira/browse/CASSANDRA-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298706#comment-17298706
]
Benedict Elliott Smith commented on CASSANDRA-16499:
I'm afraid I don't have much time to look into this, and have no way of
repeating your tests as I do not have a Windows environment. However, this
suggests that on your platform the benefits of {{maybeExecuteImmediately}}
might ideally be combined with {{ThreadPoolExecutor}} scheduling behaviour.
Since we mostly target Linux (and in fact have recently voted explicitly to
remove official Windows support), this isn't likely to find a huge amount of
interest from contributors, but if you want to produce a suitable patch for
Windows that does not modify the existing behaviour, it would be considered for
inclusion.
If you want to pursue that, I would propose first looking into whether spinning
is even necessary for your platform (hopefully unlikely - if spinning makes it
faster to explicitly unpark, that would be a surprising feature of the
scheduler). If not, copying the SEP code to remove all of the unnecessary
spinning logic should be straightforward and produce a
{{ThreadPoolExecutor}}-like class that supports {{maybeExecuteImmediately}}.
It's possible this could be useful for other platforms too.
Please submit proposals via GitHub links, and follow the [style
guide.|https://cassandra.apache.org/doc/latest/development/code_style.html]
> single-threaded write workloads can spend ~70% time waiting on SEPExecutor
> --
>
> Key: CASSANDRA-16499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16499
> Project: Cassandra
> Issue Type: Bug
>Reporter: Tom Whitmore
>Priority: Normal
> Attachments: Cassandra Write trace 5; warmed up -- 02.txt,
> MaybeStartSpinning Unpark fix on 4beta4; Cassandra Stress results -- 01.txt,
> MaybeStartSpinning Unpark fix; Cassandra Stress results -- 02.txt, SEPWorker
> trace 2 delay examples -- 01.txt, SEPWorker trace 2 delays.txt, SEPWorker
> trace 3 delays; with proposed fix.txt, Single-thread Latencies report --
> 01.xlsx, Stress Write 2 sgl-thread vs 10 threads -- 01.txt, Stress Write
> sgl-thread 1 -- 01.txt, Stress Write trace 1.txt, proposed fix patches.zip,
> tracing & experimental change patches.zip
>
>
> Hi all! While conducting benchmarking of Cassandra against other databases
> for a particular healthcare solution, I found some surprising anomalies in
> single-threaded write performance.
> Analysis & tracing suggest there might be some inefficiencies in inter-thread
> execution in Cassandra;
> * Tracing showed an average delay of 1.52 ms between
> StorageProxy.performLocally() being called, and the LocalMutationRunnable
> actually executing.
> * Total operation time averaged 2.06 ms (measured at Message.Dispatcher
> processRequest()). This suggested ~72% of the +total operation time+ being
> lost waiting for thread scheduling in SEPExecutor.
> * With I tested with multiple threads, performance with 10 threads was 27x
> higher. This supports a hypothesis that scheduling delays may be hindering
> single-threaded progress.
> * Transaction throughput for Cassandra with a single-threaded workload,
> measured far lower than that of PostgreSQL on the same hardware. Postgres
> achieved ~200k committed transactions/ minute including fsync; Cassandra
> achieves ~37k per minute. Given they are both essentially writing to a commit
> log, it may be informative to understand why the differences are arising.
> Cassandra's architecture seems in theory like it might be aligned for our
> usecase, given the Commit Log and Log Structured Merge design. Some of our
> customers have configurations posing high single-threaded loads. Therefore I
> spent some time trying to understand why efficiency for such loads seemed
> less than expected.
> My investigation so far:
> * benchmarked Cassandra 3.11.10
> * stack-dumped it under load & identified a pattern of threads waiting in
> AbstractWriteResponseHandler while nothing else is busy
> * checked out Cassandra 3.11.10 source, built it, debugged & stepped
> through key areas to try and understand behavior.
> * instrumented key areas with custom tracing code & timestamps to 0.01
> millisecond.
> ** _see patch attached._
> * benchmarked Cassandra 4 beta 4 & verified delays also present
> * shown & traced delays with my healthcare scenario benchmark
> * shown & traced delays with the +Cassandra stress-test+ tool.
> The configuration was:
> * single-node Cassandra running locally, on a recent Dell laptop with NVmE
> SSD.
> * for the healthcare scenario:
> ** Java client app running 1 or 10 threads;
> ** trialled LOCAL_ONE and ANY consistency levels;
> ** trialled
[jira] [Assigned] (CASSANDRA-16493) IPMembershipTest.sameIPFailWithoutReplace fails with timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-16493:
---
Assignee: Sam Tunnicliffe (was: Brandon Williams)
> IPMembershipTest.sameIPFailWithoutReplace fails with timeout
>
>
> Key: CASSANDRA-16493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16493
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/java
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: ci-failures.png
>
>
> https://ci-cassandra.apache.org/job/Cassandra-trunk/307/testReport/junit/org.apache.cassandra.distributed.test/IPMembershipTest/sameIPFailWithoutReplace/
> {noformat}
> java.lang.RuntimeException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/0:0:0:0:0:0:0:1:9042
> (com.datastax.driver.core.exceptions.TransportException:
> [localhost/0:0:0:0:0:0:0:1:9042] Cannot connect), localhost/127.0.0.1:9042
> (com.datastax.driver.core.exceptions.OperationTimedOutException:
> [localhost/127.0.0.1:9042] Operation timed out))
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:146)
> at
> org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:114)
> at
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:66)
> at
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:154)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:56)
> at org.apache.cassandra.stress.Stress.run(Stress.java:155)
> at org.apache.cassandra.stress.Stress.main(Stress.java:63)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
