[jira] [Commented] (CASSANDRA-13701) Lower default num_tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072398#comment-17072398 ] Jeremy Hanna commented on CASSANDRA-13701: -- [~mshuler] What would we need to do to update testing to 16 so that it coincides with the new defaults? > Lower default num_tokens > > > Key: CASSANDRA-13701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13701 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Chris Lohfink >Assignee: Jeremy Hanna >Priority: Low > > For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not > necessary. It is very expensive for operations processes and scanning. Its > come up a lot and its pretty standard and known now to always reduce the > num_tokens within the community. We should just lower the defaults. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13701) Lower default num_tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072390#comment-17072390 ] Jeremy Hanna commented on CASSANDRA-13701: -- As discussed in [this thread|https://lists.apache.org/thread.html/r164d8a4143551b5ef774734afdce0ef31a0e461d71276f8446be%40%3Cdev.cassandra.apache.org%3E] the community decided on a default of 16 for now. I'll assign to myself and put in some release notes about it. At the same time I'll add some documentation for the topic. > Lower default num_tokens > > > Key: CASSANDRA-13701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13701 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Chris Lohfink >Priority: Low > > For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not > necessary. It is very expensive for operations processes and scanning. Its > come up a lot and its pretty standard and known now to always reduce the > num_tokens within the community. We should just lower the defaults. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13701) Lower default num_tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna reassigned CASSANDRA-13701: Assignee: Jeremy Hanna > Lower default num_tokens > > > Key: CASSANDRA-13701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13701 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Chris Lohfink >Assignee: Jeremy Hanna >Priority: Low > > For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not > necessary. It is very expensive for operations processes and scanning. Its > come up a lot and its pretty standard and known now to always reduce the > num_tokens within the community. We should just lower the defaults. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-8629) Exceptions in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072307#comment-17072307
]
miranda zhang edited comment on CASSANDRA-8629 at 4/1/20, 2:47 AM:
---
Still hitting this exception, even with no-warmup, like
{code:java}
cassandra-stress user no-warmup ops(insert=10) profile=stressSpec.yaml
cl=QUORUM duration=30m -mode native cql3 maxPending=32768 connectionsPerHost=10
-rate threads=1500 -node file=node_list.txt{code}
only see this when "threads" is set to a large number.
{code:java}
Exception in thread "Thread-1182" java.lang.AssertionErrorException in thread
"Thread-1182" java.lang.AssertionError at
org.apache.cassandra.utils.DynamicList.remove(DynamicList.java:141) at
org.apache.cassandra.utils.LockedDynamicList.remove(LockedDynamicList.java:57)
at org.apache.cassandra.stress.generate.Seed.remove(Seed.java:83) at
org.apache.cassandra.stress.generate.SeedManager.markLastWrite(SeedManager.java:115)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.setHasNext(PartitionIterator.java:710)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seekToCurrentRow(PartitionIterator.java:473)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seek(PartitionIterator.java:452)
at
org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.reset(PartitionIterator.java:285)
at
org.apache.cassandra.stress.generate.PartitionIterator.reset(PartitionIterator.java:107)
at
org.apache.cassandra.stress.operations.PartitionOperation.reset(PartitionOperation.java:115)
at
org.apache.cassandra.stress.operations.PartitionOperation.ready(PartitionOperation.java:101)
at
org.apache.cassandra.stress.StressAction$StreamOfOperations.nextOp(StressAction.java:357)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:463){code}
was (Author: miranda):
Still hitting this exception, even with no-warmup, like
{code:java}
cassandra-stress user no-warmup ops(insert=10) profile=stressSpec.yaml
cl=QUORUM duration=30m -mode native cql3 maxPending=32768 connectionsPerHost=10
-rate threads=1500 -node file=node_list.txt{code}
only see this when "threads" is set to a large number.
> Exceptions in cassandra-stress
> --
>
> Key: CASSANDRA-8629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8629
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/stress
>Reporter: Aleksey Yeschenko
>Priority: Low
> Labels: stress
>
> cassandra-stress when run with tiny n, throws
> org.apache.commons.math3.exception.NotStrictlyPositiveException.
> Now, n=1 doesn't really make any sense, w/ 50k writes used just for warmup,
> but an exception is still an exception. Labeled w/ priority: Trivial.
> Profile used: http://pastebin.com/raw.php?i=9U5EMdVq
> {noformat}
> tools/bin/cassandra-stress user profile=partition.yaml ops\(insert=1\) n=1
> -rate threads=50
> INFO 18:21:59 Using data-center name 'datacenter1' for
> DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
> datacenter name with DCAwareRoundRobinPolicy constructor)
> Connected to cluster: Test Cluster
> Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
> INFO 18:21:59 New Cassandra host localhost/127.0.0.1:9042 added
> Created schema. Sleeping 1s for propagation.
> Exception in thread "main"
> org.apache.commons.math3.exception.NotStrictlyPositiveException: standard
> deviation (0)
> at
> org.apache.commons.math3.distribution.NormalDistribution.(NormalDistribution.java:108)
> at
> org.apache.cassandra.stress.settings.OptionDistribution$GaussianFactory.get(OptionDistribution.java:418)
> at
> org.apache.cassandra.stress.generate.SeedManager.(SeedManager.java:59)
> at
> org.apache.cassandra.stress.settings.SettingsCommandUser.getFactory(SettingsCommandUser.java:78)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:61)
> at org.apache.cassandra.stress.Stress.main(Stress.java:109)
> {noformat}
> On cassandra-2.1 HEAD, I cannot reproduce it, but get a different exception,
> with n=10:
> {noformat}
> Exception in thread "Thread-13" java.lang.AssertionError
> at
> org.apache.cassandra.stress.util.DynamicList.remove(DynamicList.java:156)
> at org.apache.cassandra.stress.generate.Seed.remove(Seed.java:83)
> at
> org.apache.cassandra.stress.generate.SeedManager.markLastWrite(SeedManager.java:115)
> at
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.setHasNext(PartitionIterator.java:561)
> at
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seek(PartitionIterator.java:333)
> at
>
[jira] [Commented] (CASSANDRA-8629) Exceptions in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072307#comment-17072307
]
miranda zhang commented on CASSANDRA-8629:
--
Still hitting this exception, even with no-warmup, like
{code:java}
cassandra-stress user no-warmup ops(insert=10) profile=stressSpec.yaml
cl=QUORUM duration=30m -mode native cql3 maxPending=32768 connectionsPerHost=10
-rate threads=1500 -node file=node_list.txt{code}
only see this when "threads" is set to a large number.
> Exceptions in cassandra-stress
> --
>
> Key: CASSANDRA-8629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8629
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/stress
>Reporter: Aleksey Yeschenko
>Priority: Low
> Labels: stress
>
> cassandra-stress when run with tiny n, throws
> org.apache.commons.math3.exception.NotStrictlyPositiveException.
> Now, n=1 doesn't really make any sense, w/ 50k writes used just for warmup,
> but an exception is still an exception. Labeled w/ priority: Trivial.
> Profile used: http://pastebin.com/raw.php?i=9U5EMdVq
> {noformat}
> tools/bin/cassandra-stress user profile=partition.yaml ops\(insert=1\) n=1
> -rate threads=50
> INFO 18:21:59 Using data-center name 'datacenter1' for
> DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
> datacenter name with DCAwareRoundRobinPolicy constructor)
> Connected to cluster: Test Cluster
> Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
> INFO 18:21:59 New Cassandra host localhost/127.0.0.1:9042 added
> Created schema. Sleeping 1s for propagation.
> Exception in thread "main"
> org.apache.commons.math3.exception.NotStrictlyPositiveException: standard
> deviation (0)
> at
> org.apache.commons.math3.distribution.NormalDistribution.(NormalDistribution.java:108)
> at
> org.apache.cassandra.stress.settings.OptionDistribution$GaussianFactory.get(OptionDistribution.java:418)
> at
> org.apache.cassandra.stress.generate.SeedManager.(SeedManager.java:59)
> at
> org.apache.cassandra.stress.settings.SettingsCommandUser.getFactory(SettingsCommandUser.java:78)
> at org.apache.cassandra.stress.StressAction.run(StressAction.java:61)
> at org.apache.cassandra.stress.Stress.main(Stress.java:109)
> {noformat}
> On cassandra-2.1 HEAD, I cannot reproduce it, but get a different exception,
> with n=10:
> {noformat}
> Exception in thread "Thread-13" java.lang.AssertionError
> at
> org.apache.cassandra.stress.util.DynamicList.remove(DynamicList.java:156)
> at org.apache.cassandra.stress.generate.Seed.remove(Seed.java:83)
> at
> org.apache.cassandra.stress.generate.SeedManager.markLastWrite(SeedManager.java:115)
> at
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.setHasNext(PartitionIterator.java:561)
> at
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seek(PartitionIterator.java:333)
> at
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.reset(PartitionIterator.java:242)
> at
> org.apache.cassandra.stress.generate.PartitionIterator.reset(PartitionIterator.java:99)
> at org.apache.cassandra.stress.Operation.ready(Operation.java:110)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:288)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15631) Add AssertJ test dependency
[ https://issues.apache.org/jira/browse/CASSANDRA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072296#comment-17072296 ] David Capwell commented on CASSANDRA-15631: --- moving to commits rather than diff so the links are stable bq. Not sure which assert you were referring to since the code has changed since your comment [I believe it was this|https://github.com/newkek/cassandra/commit/e10c0da352c8dc7c362f36fd081eb9970f9fd2d9#diff-1dd73505fd4b8d16fb40a4aebb68e5eeL251], you comment that it is [now here|https://github.com/newkek/cassandra/commit/e10c0da352c8dc7c362f36fd081eb9970f9fd2d9#diff-1dd73505fd4b8d16fb40a4aebb68e5eeR245]. LGTM. bq. Latest run (#17) had a failure that didn't seem related to these changes, and the test passed when I ran it locally. Yeah, that test is flakey, CASSANDRA-15650 fixes it. For the most part LGTM. I would +1 but I want to look closer at the Junit 5 thing, ill make sure to do this tomorrow. > Add AssertJ test dependency > --- > > Key: CASSANDRA-15631 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15631 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest, Test/unit >Reporter: Kevin Gallardo >Assignee: Kevin Gallardo >Priority: Normal > Fix For: 4.0-beta > > > See > [proposal|https://lists.apache.org/thread.html/rc562ec47578d0ae6f346ba9e3d7469c1cd3f8b521a72ddcb2accc47b%40%3Cdev.cassandra.apache.org%3E]. > The goal is to add [AssertJ|https://assertj.github.io/doc/] to the test > framework to allow for more comprehensible and easier to write tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15661) Improve logging by using more appropriate levels
[ https://issues.apache.org/jira/browse/CASSANDRA-15661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072240#comment-17072240 ] Jon Haddad commented on CASSANDRA-15661: Got my circle ci results. [Unit tests|https://app.circleci.com/pipelines/github/rustyrazorblade/cassandra/17/workflows/82684478-5d63-4ab2-8db4-b6c05d05edca/jobs/236] passing [JVM DTest|https://app.circleci.com/pipelines/github/rustyrazorblade/cassandra/17/workflows/82684478-5d63-4ab2-8db4-b6c05d05edca/jobs/235] passing [DTests|https://app.circleci.com/pipelines/github/rustyrazorblade/cassandra/17/workflows/82684478-5d63-4ab2-8db4-b6c05d05edca/jobs/237] have a single failure that appears to be unrelated (failure with repair due to a node not starting) > Improve logging by using more appropriate levels > - > > Key: CASSANDRA-15661 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15661 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Logging >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Normal > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > There are a number of log statements using logging levels that are a bit too > conservative. For example: > * Flushing memtables is currently at debug. This is a relatively rare event > that is important enough to be INFO > * When compaction finishes we log the progress at debug > * Different steps in incremental repair are logged as debug, should be INFO > * when reaching connection limits in ConnectionLimitHandler.java we log at > warn rather than error. Since this is a client disconnect it’s more than a > warning, we’re taking action and disconnecting. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15406) Add command to show the progress of data streaming and index build
[ https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072227#comment-17072227 ] Stefan Miklosovic commented on CASSANDRA-15406: --- Hi [~dcapwell], I think that this ticket will be more or less transformed just into a bug fix. I do not thing that there is a need for a "command" for nodetool and having a lot of work before release anyway, it is not a good idea to introduce something completely new. As I see it, this ticket will be just about the introduction of global percentage into netstats output + I think I hit a bug when we stream entire SSTables and netstats is displaying it weirdly so that should be fixed too. > Add command to show the progress of data streaming and index build > --- > > Key: CASSANDRA-15406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15406 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Streaming, Legacy/Streaming and Messaging, > Tool/nodetool >Reporter: maxwellguo >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0, 4.x > > > I found that we should supply a command to show the progress of streaming > when we do the operation of bootstrap/move/decommission/removenode. For when > do data streaming , noboday knows which steps there program are in , so I > think a command to show the joing/leaving node's is needed . -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15582) 4.0 quality testing: metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072226#comment-17072226 ] David Capwell commented on CASSANDRA-15582: --- agree. I recently sent a patch for CASSANDRA-15674, this is a regression in 3.0. Metrics are under tested but super important for operating a cluster, so we need to improve our testing of metrics. bq. ideally ones that tested edge cases Yeah thats where they are easier to regress. CASSANDRA-15674 only happens during compaction interruption (or disk failure)... > 4.0 quality testing: metrics > > > Key: CASSANDRA-15582 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15582 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Josh McKenzie >Assignee: Romain Hardouin >Priority: Normal > Fix For: 4.0-rc > > > In past releases we've unknowingly broken metrics integrations and introduced > performance regressions in metrics collection and reporting. We strive in 4.0 > to not do that. Metrics should work well! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15631) Add AssertJ test dependency
[ https://issues.apache.org/jira/browse/CASSANDRA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072223#comment-17072223 ] David Capwell commented on CASSANDRA-15631: --- sorry, life has been all over the place recently, I will likely get to this tomorrow. > Add AssertJ test dependency > --- > > Key: CASSANDRA-15631 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15631 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest, Test/unit >Reporter: Kevin Gallardo >Assignee: Kevin Gallardo >Priority: Normal > Fix For: 4.0-beta > > > See > [proposal|https://lists.apache.org/thread.html/rc562ec47578d0ae6f346ba9e3d7469c1cd3f8b521a72ddcb2accc47b%40%3Cdev.cassandra.apache.org%3E]. > The goal is to add [AssertJ|https://assertj.github.io/doc/] to the test > framework to allow for more comprehensible and easier to write tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15406) Add command to show the progress of data streaming and index build
[ https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072221#comment-17072221 ] David Capwell commented on CASSANDRA-15406: --- bq. David Capwell where do we stand with this? I was reading your pull request for 15399 and do I understand it right that one is postponed indefinitely? (at least until 4.0 is out) Yes, since the repair work are improvements and new features, its best they are not in 4.0.0 as 4.0.0 is is under freeze right now. I want early feedback on the tables so different tools could provide feedback or feature requests, but still should not go into 4.0.0. bq. I would like to work on this but I am not sure if there is some reasonable probability it will eventually and actually end up being merged before 4.0 is out. Best to ask on the dev mailing list. bq. Could you elaborate more about what parts of this code would be similar with your issue which ended up postponed so we would not duplicate things? I left a comment https://issues.apache.org/jira/browse/CASSANDRA-15399?focusedCommentId=17072219=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17072219 that shows what part of the code. How I understand this JIRA is not that they conflict, but that this JIRA benefits 15399; I may be wrong in this understanding. bq. query it via JMX or is it true that virtual tables would work too? The trend is to kill off JMX in favor of virtual tables. I have not looked at your code, but many prefer virtual tables over JMX. > Add command to show the progress of data streaming and index build > --- > > Key: CASSANDRA-15406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15406 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Streaming, Legacy/Streaming and Messaging, > Tool/nodetool >Reporter: maxwellguo >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0, 4.x > > > I found that we should supply a command to show the progress of streaming > when we do the operation of bootstrap/move/decommission/removenode. For when > do data streaming , noboday knows which steps there program are in , so I > think a command to show the joing/leaving node's is needed . -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15399) Add ability to track state in repair
[
https://issues.apache.org/jira/browse/CASSANDRA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072219#comment-17072219
]
David Capwell commented on CASSANDRA-15399:
---
bq. does it mean that feature is not going to happen either or I am free do
dig into it?
I argue that CASSANDRA-15566 is not a regression, so I have been meaning to
bring it up on the ML to not have it in 4.0. This JIRA is a feature so with
the freeze in mind its not right to go in, which is why I stopped mostly.
CASSANDRA-15406 I linked these two since repair coordinator would benefit from
it, and being able to expose that state more clearly would help operators. I
only now see your comments in CASSANDRA-15406, sorry for not replying before; I
will reply there.
bq. I do not want to reinvent the wheel so you telling me what parts of the
code are common (as you say in 15406) would be very beneficial and I would be
grateful for that a lot.
Repair has 3 main phases: setup, validate, sync/streaming. In case there are
issues or slownesses, it is desirable to know where and how fast progress is
being made. In this ticket I added validation tracking which tracks the rate
of progress and % completed, I had not worked on sync as I am less familiar and
had more tools to monitor (there are stats, not tied to a repair but you could
detect if streaming wasn't happening, you can't do that with validate).
Repair has two main code paths for interacting with streaming
* org.apache.cassandra.repair.RepairMessageVerbHandler#doVerb - if the
coordinator needs to ask another node to stream data; look for *SYNC_* verbs
* org.apache.cassandra.repair.RepairJob#standardSyncing - an individual job (a
single repair is 1+ jobs). This delegates to LocalSyncTask,
AsymmetricRemoteSyncTask, and SymmetricRemoteSyncTask to perform the sync.
What would be desirable is that we could track which streams were happening on
each node for a specific repair job, and how the progress is going for
streaming (which is what CASSANDRA-15406 looks to offer). This ticket makes it
so the state is in-memory only (so rebooting the node doesn't cause the
metadata to be out of sync), but also makes sure the data lives far past the
actual events. The main reasons were to allow operators to see historic data,
but would also allow repair coordination to be able to get access so it could
learn what other nodes think (for example, if validation is done on a node but
the coordinator has not seen it yet, it could re-ask for the tree to be sent).
> Add ability to track state in repair
>
>
> Key: CASSANDRA-15399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15399
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> To enhance the visibility in repair, we should expose internal state via
> virtual tables; the state should include coordinator as well as participant
> state (validation, sync, etc.)
> I propose the following tables:
> repairs - high level summary of the global state of repair; this should be
> called on the coordinator.
> {code:sql}
> CREATE TABLE repairs (
> id uuid,
> keyspace_name text,
> table_names frozen>,
> ranges frozen>,
> coordinator text,
> participants frozen>,
> state text,
> progress_percentage float,
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id) )
> )
> {code}
> repair_tasks - represents RepairJob and participants state. This will show
> if validations are running on participants and the progress they are making;
> this should be called on the coordinator.
> {code:sql}
> CREATE TABLE repair_tasks (
> id uuid,
> session_id uuid,
> keyspace_name text,
> table_name text,
> ranges frozen>,
> coordinator text,
> participant text,
> state text,
> state_description text,
> progress_percentage float, -- between 0.0 and 100.0
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name, participant )
> )
> {code}
> repair_validations - shows the state of the validation task and updated
> periodically while validation is running; this should be called on the
> participants.
> {code:sql}
> CREATE TABLE repair_validations (
> id uuid,
> session_id uuid,
> ranges frozen>,
> keyspace_name text,
> table_name text,
> initiator text,
> state text,
> progress_percentage float,
> queue_duration_ms bigint,
> runtime_duration_ms bigint,
> total_duration_ms bigint,
> estimated_partitions bigint,
> partitions_processed bigint,
>
[jira] [Commented] (CASSANDRA-15564) Refactor repair coordinator so errors are consistent
[ https://issues.apache.org/jira/browse/CASSANDRA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072175#comment-17072175 ] David Capwell commented on CASSANDRA-15564: --- [~ifesdjeen] I believe the dtest stuff is done, safe to close this? > Refactor repair coordinator so errors are consistent > > > Key: CASSANDRA-15564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15564 > Project: Cassandra > Issue Type: Sub-task > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 17h 20m > Remaining Estimate: 0h > > This is to split the change in CASSANDRA-15399 so the refactor is isolated > out. > Currently the repair coordinator special cases the exit cases at each call > site; this makes it so that errors can be inconsistent and there are cases > where proper complete isn't done (proper notifications, and forgetting to > update ActiveRepairService). > [Circle > CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorJmxConsistency] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072118#comment-17072118 ] Alan Boudreault commented on CASSANDRA-15659: - Hello, That makes sense Patrick. I think Python 3.6 is a good choice for the official python3 supported version. And it´s correct to only run dtests with this version. However, I think also it would be better to relax the requirement for the different distributions (removing the strong check). All Python3 versions can be supported at *best effort*. So my suggestion: 1. Remove the strong python 3.6 requirement check 2. Do some manual testing to ensure we can at least connect to a cluster and execute queries with all py3 runtimes. 3. Fix obvious issues if there are. 4. (optional) add a simple smoke test in dtest that is run for all runtimes. I am going to check when I could try to help with the python3 support. > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-alpha > > > From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072099#comment-17072099
]
Ekaterina Dimitrova edited comment on CASSANDRA-13994 at 3/31/20, 7:19 PM:
---
Thanks [~djoshi], as this one is marked to land at rc not alpha/beta I haven't
started it yet.
Please let me know if it is a blocker for something and requires faster
attention.
was (Author: e.dimitrova):
Thanks @Dinesh, as this one is marked to land at rc not alpha/beta I haven't
started it yet.
Please let me know if it is a blocker for something and requires faster
attention.
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072099#comment-17072099
]
Ekaterina Dimitrova commented on CASSANDRA-13994:
-
Thanks @Dinesh, as this one is marked to land at rc not alpha/beta I haven't
started it yet.
Please let me know if it is a blocker for something and requires faster
attention.
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15674) liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if IndexSummaryRedistribution gets interrupted
[ https://issues.apache.org/jira/browse/CASSANDRA-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072028#comment-17072028 ] David Capwell commented on CASSANDRA-15674: --- to handle this I added 2 hooks into the transaction: onCommit, onRollback. Both the hooks apply the update, but apply different updates. onCommit deletes the whole file since the transaction will add it back with the latest size, onRollback removes the diff (the disk mutation is done outside of the transaction, so still need to apply). > liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if > IndexSummaryRedistribution gets interrupted > - > > Key: CASSANDRA-15674 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15674 > Project: Cassandra > Issue Type: Bug > Components: Local/Compaction, Observability/Metrics >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-alpha > > > IndexSummaryRedistribution is a compaction task and as such extends Holder > and supports cancelation by throwing a CompactionInterruptedException. The > issue is that IndexSummaryRedistribution tries to use transactions, but > mutates the sstable in-place; transaction is unable to roll back. > This would be fine (only updates summary) if it wasn’t for the fact the task > attempts to also mutate the two metrics liveDiskSpaceUsed and > totalDiskSpaceUsed, since these can’t be rolled back any cancelation could > corrupt these metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15637) CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference between thrift and the new system.size_estimates table when dealing with multiple dc
[
https://issues.apache.org/jira/browse/CASSANDRA-15637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072027#comment-17072027
]
David Capwell commented on CASSANDRA-15637:
---
updated to fix the above.
> CqlInputFormat regression going from 2.1 to 3.x caused by semantic difference
> between thrift and the new system.size_estimates table when dealing with
> multiple dc deployments
> --
>
> Key: CASSANDRA-15637
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15637
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Tools
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In 3.0 CqlInputFormat switched away from thrift in favor of a new
> system.size_estimates table, but the semantics changed when dealing with
> multiple DCs or when Cassandra is not collocated with Hadoop.
> The core issues are:
> * system.size_estimates uses the primary range, in a multi-dc setup this
> could lead to uneven ranges
> example:
> {code}
> DC1: [0, 10, 20, 30]
> DC2: [1, 11, 21, 31]
> DC3: [2, 12, 22, 32]
> {code}
> Using NetworkTopologyStrategy the primary ranges are: [0, 1), [1, 2), [2,
> 10), [10, 11), [11, 12), [12, 20), [20, 21), [21, 22), [22, 30), [30, 31),
> [31, 32), [32, 0).
> Given this the only ranges that are more than one token are: [2, 10), [12,
> 20), [22, 30).
> * system.size_estimates is not replicated so need to hit every node in the
> cluster to get estimates, if nodes are down in the DC with non-size-1 ranges
> there is no way to get a estimate.
> * CqlInputFormat used to call describe_local_ring so all interactions were
> with a single DC, the java driver doesn't filter the DC so looks to allow
> cross DC traffic and includes nodes from other DCs in the replica set; in the
> example above, the amount of splits went from 4 to 12.
> * CqlInputFormat used to call describe_splits_ex to dynamically calculate the
> estimates, this was on the "local primary range" and was able to hit replicas
> to create estimates if the primary was down. With system.size_estimates we no
> longer have backup and no longer expose the "local primary range" in multi-dc.
> * CqlInputFormat had a config cassandra.input.keyRange which let you define
> your own range. If the range doesn't perfectly match the local range then
> the intersectWith calls will produce ranges with no estimates. Example: [0,
> 10, 20], cassandra.input.keyRange=5,15. This won't find any estimates so
> will produce 2 splits with 128 estimate (default when not found).
> * CqlInputFormat special cases Cassandra being collocated with Hadoop and
> assumes this when querying system.size_estimates as it doesn't filter to the
> specific host, this means that non-collocated deployments randomly select the
> nodes and create splits with ranges the hosts do not have locally.
> The problems are deterministic to replicate, the following test will show it
> 1) deploy a 3 DC cluster with 3 nodes each
> 2) create DC2 tokens are +1 of DC1 and DC3 are +1 of DC2
> 3) CREATE KEYSPACE simpleuniform0 WITH replication = {‘class’:
> ‘NetworkTopologyStrategy’, ‘DC1’: 3, ‘DC2’: 3, ‘DC3’: 3};
> 4) CREATE TABLE simpletable0 (pk bigint, ck bigint, value blob, PRIMARY KEY
> (pk, ck))
> 5) insert 500k partitions uniformly: [0, 500,000)
> 6) wait until estimates catch up to writes
> 7) for all nodes, SELECT * FROM system.size_estimates
> You will get the following
> {code}
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
> simpleuniform0 | simpletable0 | -9223372036854775808 | -6148914691236517206
> | 87 | 122240
> simpleuniform0 | simpletable0 | 6148914691236517207 | -9223372036854775808
> | 87 | 121472
> (2 rows)
> keyspace_name | table_name | range_start | range_end |
> mean_partition_size | partitions_count
> +--+-+-+-+--
> simpleuniform0 | simpletable0 | 2 | 6148914691236517205 |
> 87 | 243072
> (1 rows)
> keyspace_name | table_name | range_start | range_end
> | mean_partition_size | partitions_count
> +--+--+--+-+--
>
[jira] [Updated] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Joshi updated CASSANDRA-13994:
-
Reviewers: Dinesh Joshi
Making myself as a reviewer. [~ifesdjeen] please feel free to add yourself if
you also want to review this.
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071980#comment-17071980 ] Patrick Bannister commented on CASSANDRA-15659: --- I don't plan to work this ticket, but I'm posting to share some relevant context that may be useful. Testing was the main reason we limited Python 3.x support by minor version. The cqlsh dtests are costly in terms of time and compute resources. Unfortunately, most of the substantive testing of cqlsh is in the dtests. In the past, the thinking was to expand the Python 3.x minor versions supported by cqlsh after dropping Python 2 support. The idea was to limit the number of test environments needed to properly test cqlsh. [SaferScanner|https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/saferscanner.py] is another problem. It depends on implementation details of the Python re module, and it's sensitive to the Python minor version. We had to rewrite it twice in the past two years. > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-alpha > > > From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15650) Fix flaky test org.apache.cassandra.distributed.test.*RepairCoordinatorFastTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071973#comment-17071973
]
David Capwell commented on CASSANDRA-15650:
---
[~blerer] fixed all feedback, can you review again?
> Fix flaky test
> org.apache.cassandra.distributed.test.*RepairCoordinatorFastTest
> ---
>
> Key: CASSANDRA-15650
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15650
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Test failure:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/177/workflows/3dff37a5-9bf4-40e2-8d5b-f127b416dc79/jobs/862
> {code}
> [junit-timeout] Testcase:
> onlyCoordinator[SEQUENTIAL/true](org.apache.cassandra.distributed.test.FullRepairCoordinatorFastTest):
> FAILED
> [junit-timeout] nodetool command repair was successful but not expected to
> be. Actual: 0
> [junit-timeout] junit.framework.AssertionFailedError: nodetool command repair
> was successful but not expected to be. Actual: 0
> [junit-timeout] at
> org.apache.cassandra.distributed.api.NodeToolResult$Asserts.failure(NodeToolResult.java:76)
> [junit-timeout] at
> org.apache.cassandra.distributed.test.RepairCoordinatorFast.onlyCoordinator(RepairCoordinatorFast.java:255)
> [junit-timeout] at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [junit-timeout] at java.lang.Thread.run(Thread.java:748)
> {code}
> [Circle CI
> LOWER|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FrepairCoordinatorTestFlaky]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071950#comment-17071950
]
ZhaoYang edited comment on CASSANDRA-15660 at 3/31/20, 4:36 PM:
[patch|https://github.com/apache/cassandra/pull/502/files]
[dtest|https://github.com/apache/cassandra-dtest/pull/60/files]
Marking it patch-available.. I have added a regression test and
{{cqlsh_tests/test_cqlsh.py}} works fine locally, but I am not able to run
circle-ci because queue is full..
was (Author: jasonstack):
Marking it patch-available.. I have added a regression test and
{{cqlsh_tests/test_cqlsh.py}} works fine locally, but I am not able to run
circle-ci because queue is full..
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15660:
-
Test and Documentation Plan:
CI pending: https://circleci.com/gh/jasonstack/cassandra/1114
Added regression cqlsh dtest.
was:CI pending: https://circleci.com/gh/jasonstack/cassandra/1114
Status: Patch Available (was: Open)
Marking it patch-available.. I have added a regression test and
{{cqlsh_tests/test_cqlsh.py}} works fine locally, but I am not able to run
circle-ci because queue is full..
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071940#comment-17071940
]
Stefan Miklosovic commented on CASSANDRA-15660:
---
tested and works as expected
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[
https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071922#comment-17071922
]
Josh McKenzie commented on CASSANDRA-15358:
---
[~benedict] - anything I or others can do to help get this merged in? We're
testing things and hitting this bug quite frequently.
> Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
>
>
> Key: CASSANDRA-15358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15358
> Project: Cassandra
> Issue Type: Bug
> Components: Test/benchmark
>Reporter: Santhosh Kumar Ramalingam
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Labels: 4.0, alpha
> Fix For: 4.0, 4.0-beta
>
> Attachments: all_errors.txt, debug_logs_during_repair.txt,
> repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt
>
>
> Hitting a bug with cassandra 4 alpha version. The same bug is repeated with
> difefrent version of Java(8,11 &12) [~benedict]
>
> Stack trace:
> {code:java}
> INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for
> CQL clients on /1.3.0.6:9042 (unencrypted)...
> WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 -
> 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message
> of type PING_REQ whose timeout expired before reaching the network
> WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 -
> 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..)
> failed: Connection refused: /1.3.0.2:7000
> Caused by: java.net.ConnectException: finishConnect(..) failed: Connection
> refused
> at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124)
> at io.netty.channel.unix.Socket.finishConnect(Socket.java:243)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 -
> 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of
> type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network
> INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91
> - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect
>
> ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407
> InboundMessageHandler.java:657 -
> 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception
> caught while processing inbound messages; terminating connection
> java.lang.IllegalArgumentException: initialBuffer is not a direct buffer.
> at io.netty.buffer.UnpooledDirectByteBuf.(UnpooledDirectByteBuf.java:87)
> at
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:59)
> at
> org.apache.cassandra.net.BufferPoolAllocator$Wrapped.(BufferPoolAllocator.java:95)
> at
> org.apache.cassandra.net.BufferPoolAllocator.newDirectBuffer(BufferPoolAllocator.java:56)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
> at
> io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
> at
>
[jira] [Comment Edited] (CASSANDRA-15657) Improve zero-copy-streaming containment check by using file sections
[ https://issues.apache.org/jira/browse/CASSANDRA-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066128#comment-17066128 ] Stefan Miklosovic edited comment on CASSANDRA-15657 at 3/31/20, 3:29 PM: - [~djoshi] I think this is the duplicate of Jira of yours https://issues.apache.org/jira/browse/CASSANDRA-14586 was (Author: stefan.miklosovic): [~bhagat_dineshbe2006] I think this is the duplicate of Jira of yours https://issues.apache.org/jira/browse/CASSANDRA-14586 > Improve zero-copy-streaming containment check by using file sections > > > Key: CASSANDRA-15657 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15657 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0 > > > Currently zero copy streaming is only enabled for leveled-compaction strategy > and it checks if all keys in the sstables are included in the transferred > ranges. > This is very inefficient. The containment check can be improved by checking > if transferred sections (the transferred file positions) cover entire sstable. > I also enabled ZCS for all compaction strategies since the new containment > check is very fast.. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15406) Add command to show the progress of data streaming and index build
[
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071884#comment-17071884
]
Stefan Miklosovic commented on CASSANDRA-15406:
---
I have added "global percentage" functionality into netstats both for sending
and receiving. It looks like this:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.1
Receiving 133 files, 27664559 bytes total. Already received 21 files
(15.79%), 918834 bytes total (3.32%)
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
{code}
On sending:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.2
Sending 133 files, 27664559 bytes total. Already sent 42 files
(31.58%), 1841015 bytes total (6.65%)
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
{code}
There is a bug in the current code as if we are streaming entire SSTables via
CassandraEntireSSTableStreamWriter and CassandraOutgoingFile respectively,
there is not any update on particular components of a SSTable as it counts only
"db" file to be there. That introduces this bug:
{code:java}
Mode: NORMAL
Rebuild 2c0b43f0-735d-11ea-9346-fb0ffe238736
/127.0.0.2 Sending 19 files, 27664559 bytes total. Already sent 133 files,
27664559 bytes total
/tmp/dtests15682026295742741219/node2/data/distributed_test_keyspace/cf-196b3...
{code}
Basically, number of files to be sent is lower than files sent.
Maybe something for [~djoshi] as food for thought.
I have fixed it here
[https://github.com/apache/cassandra/compare/trunk...smiklosovic:CASSANDRA-15406]
(forget the test, will be finished).
[~jasonstack], this fix would be way better if we manage to merge yours
CASSANDRA-15657 because mine is checking if SSTable is meant to be streamed
entirely and this is not performance-friendly. With your patch it would be just
one call to already cached value because you have introduced a field for that.
Could somebody please give me some feedback here if this is what we want?
Thanks
@
> Add command to show the progress of data streaming and index build
> ---
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Streaming, Legacy/Streaming and Messaging,
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>
> I found that we should supply a command to show the progress of streaming
> when we do the operation of bootstrap/move/decommission/removenode. For when
> do data streaming , noboday knows which steps there program are in , so I
> think a command to show the joing/leaving node's is needed .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15670) Transient Replication: unable to insert data when the keyspace is configured with the SimpleStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Fernandez reassigned CASSANDRA-15670:
---
Assignee: Francisco Fernandez
> Transient Replication: unable to insert data when the keyspace is configured
> with the SimpleStrategy
>
>
> Key: CASSANDRA-15670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15670
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Transient Replication
>Reporter: Alan Boudreault
>Assignee: Francisco Fernandez
>Priority: Normal
> Fix For: 4.0-rc
>
>
> An error is thrown then trying to insert data with the transient replication
> + SimpleStrategy configured.
> Test case:
> {code:java}
> CREATE KEYSPACE test_tr WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '3/1'};
> CREATE TABLE test_tr.users (id int PRIMARY KEY, username text) with
> read_repair ='NONE';
> INSERT INTO test_tr.users (id, username) VALUES (1, 'alan');{code}
>
> traceback:
> {code:java}
> ERROR [Native-Transport-Requests-8] 2020-03-27 10:27:17,188
> ErrorMessage.java:450 - Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at
> org.apache.cassandra.db.ConsistencyLevel.eachQuorumForRead(ConsistencyLevel.java:103)
> at
> org.apache.cassandra.db.ConsistencyLevel.eachQuorumForWrite(ConsistencyLevel.java:112)
> at
> org.apache.cassandra.locator.ReplicaPlans$2.select(ReplicaPlans.java:409)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:353)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:348)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:341)
> at
> org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:330)
> at
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:1171)
> at
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:713)
> at
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:951)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:475)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:453)
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233)
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
> -->
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ConsistencyLevel.java#L103
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Fernandez reassigned CASSANDRA-15676:
---
Assignee: Francisco Fernandez
> flaky test testWriteUnknownResult-
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: Francisco Fernandez
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Failure observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be
> CasWriteUncertainException
> at
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15631) Add AssertJ test dependency
[ https://issues.apache.org/jira/browse/CASSANDRA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071854#comment-17071854 ] Kevin Gallardo commented on CASSANDRA-15631: [~dcapwell] I have added a new commit on the same branch that fixes the issues you mentioned. Thanks > Add AssertJ test dependency > --- > > Key: CASSANDRA-15631 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15631 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest, Test/unit >Reporter: Kevin Gallardo >Assignee: Kevin Gallardo >Priority: Normal > Fix For: 4.0-beta > > > See > [proposal|https://lists.apache.org/thread.html/rc562ec47578d0ae6f346ba9e3d7469c1cd3f8b521a72ddcb2accc47b%40%3Cdev.cassandra.apache.org%3E]. > The goal is to add [AssertJ|https://assertj.github.io/doc/] to the test > framework to allow for more comprehensible and easier to write tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15671) Testcase: testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): FAILED
[ https://issues.apache.org/jira/browse/CASSANDRA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071834#comment-17071834 ] Francisco Fernandez commented on CASSANDRA-15671: - [~e.dimitrova] you might want to review this when you have some free cycles? Thanks > Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > -- > > Key: CASSANDRA-15671 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15671 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Francisco Fernandez >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > The following test failure was observed: > [junit-timeout] Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > [junit-timeout] expected:<4> but was:<5> > [junit-timeout] junit.framework.AssertionFailedError: expected:<4> but was:<5> > [junit-timeout] at > org.apache.cassandra.db.compaction.CancelCompactionsTest.testSubrangeCompaction(CancelCompactionsTest.java:190) > Java 8 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15671) Testcase: testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): FAILED
[ https://issues.apache.org/jira/browse/CASSANDRA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Fernandez updated CASSANDRA-15671: Test and Documentation Plan: This is just a flaky test fix Status: Patch Available (was: In Progress) I've modified the tests to take into account only test triggered compactions for assertions. [https://github.com/apache/cassandra/pull/503] > Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > -- > > Key: CASSANDRA-15671 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15671 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Francisco Fernandez >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > The following test failure was observed: > [junit-timeout] Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > [junit-timeout] expected:<4> but was:<5> > [junit-timeout] junit.framework.AssertionFailedError: expected:<4> but was:<5> > [junit-timeout] at > org.apache.cassandra.db.compaction.CancelCompactionsTest.testSubrangeCompaction(CancelCompactionsTest.java:190) > Java 8 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15671) Testcase: testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): FAILED
[ https://issues.apache.org/jira/browse/CASSANDRA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15671: --- Labels: pull-request-available (was: ) > Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > -- > > Key: CASSANDRA-15671 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15671 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Francisco Fernandez >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 4.0-beta > > > The following test failure was observed: > [junit-timeout] Testcase: > testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): >FAILED > [junit-timeout] expected:<4> but was:<5> > [junit-timeout] junit.framework.AssertionFailedError: expected:<4> but was:<5> > [junit-timeout] at > org.apache.cassandra.db.compaction.CancelCompactionsTest.testSubrangeCompaction(CancelCompactionsTest.java:190) > Java 8 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15665:
-
Bug Category: Parent values: Code(13163)
Complexity: Normal
Discovered By: Code Inspection
Severity: Normal
Status: Open (was: Triage Needed)
> StreamManager should clearly differentiate between "initiator" and "receiver"
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between
> stream sessions (in form of {{StreamResultFuture}}) which have been either
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}} assumes we should first looking into
> "initiator" sessions, then into "receiver" ones: this is a dangerous
> assumptions, in particular for test environments where the same process could
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message
> comes from an initiator or follower session, in order to correctly
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve
> clarity and save from potential bugs/headaches at testing time, and doing
> such changes now that we're revamping streaming for 4.0 seems the right time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance
[ https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071776#comment-17071776 ] Stefan Miklosovic commented on CASSANDRA-15586: --- I contacted FreeBSD guy and he said that he is already tracking 4.0 branch and he follows this quite closely and he is planning to port it as soon as it is out. I asked him to approach us if he ever encounters bugs and issues which make Cassandra 4 on that system buggy. > 4.0 quality testing: Cluster Setup and Maintenance > -- > > Key: CASSANDRA-15586 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15586 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Josh McKenzie >Assignee: Ekaterina Dimitrova >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-rc > > > We want 4.0 to be easy for users to setup out of the box and just work. This > means having low friction when users download the Cassandra package and start > running it. For example, users should be able to easily configure and start > new 4.0 clusters and have tokens distributed evenly. Another example is > packaging, it should be easy to install Cassandra on all supported platforms > (e.g. packaging) and have Cassandra use standard platform integrations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance
[ https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071776#comment-17071776 ] Stefan Miklosovic edited comment on CASSANDRA-15586 at 3/31/20, 1:20 PM: - I contacted FreeBSD guy and he said that he is already tracking 4.0 branch and he follows this quite closely and he is planning to port it as soon as it is out. I asked him to approach us if he ever encountered bugs and issues which would make Cassandra 4 on that system buggy. was (Author: stefan.miklosovic): I contacted FreeBSD guy and he said that he is already tracking 4.0 branch and he follows this quite closely and he is planning to port it as soon as it is out. I asked him to approach us if he ever encounters bugs and issues which make Cassandra 4 on that system buggy. > 4.0 quality testing: Cluster Setup and Maintenance > -- > > Key: CASSANDRA-15586 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15586 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Josh McKenzie >Assignee: Ekaterina Dimitrova >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-rc > > > We want 4.0 to be easy for users to setup out of the box and just work. This > means having low friction when users download the Cassandra package and start > running it. For example, users should be able to easily configure and start > new 4.0 clusters and have tokens distributed evenly. Another example is > packaging, it should be easy to install Cassandra on all supported platforms > (e.g. packaging) and have Cassandra use standard platform integrations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-8272:
Reviewers: Benjamin Lerer, ZhaoYang (was: Benjamin Lerer)
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 3.0.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15660:
-
Source Control Link:
[patch|https://github.com/apache/cassandra/pull/502/files]
[dtest|https://github.com/apache/cassandra-dtest/pull/60/files] (was:
https://github.com/apache/cassandra/pull/502/files)
Test and Documentation Plan: CI pending:
https://circleci.com/gh/jasonstack/cassandra/1114
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15660:
-
Source Control Link: https://github.com/apache/cassandra/pull/502/files
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15660:
---
Labels: pull-request-available (was: )
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15679) cqlsh COPY FROM of map of blobs fails with parse error "unhashable type: 'bytearray'"
[
https://issues.apache.org/jira/browse/CASSANDRA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Sorokoumov reassigned CASSANDRA-15679:
Assignee: (was: Aleksandr Sorokoumov)
> cqlsh COPY FROM of map of blobs fails with parse error "unhashable type:
> 'bytearray'"
> -
>
> Key: CASSANDRA-15679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15679
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Erick Ramirez
>Priority: Normal
>
> h2. Background
> A user was having issues loading CSV data with the {{COPY FROM}} command into
> a {{map}} column with {{blob}} values.
> h2. Replication steps
> I can easily replicate the problem with this simple table:
> {noformat}
> CREATE TABLE community.blobmaptable (
> id text PRIMARY KEY,
> blobmapcol map
> )
> {noformat}
> I have this CSV file that contains just 1 row:
> {noformat}
> $ cat blobmap.csv
> c3,{3: 0x74776f}
> {noformat}
> And here's the error when I try to load it:
> {noformat}
> cqlsh:community> COPY blobmaptable (id, blobmapcol) FROM '~/blobmap.csv' ;
> Using 1 child processes
> Starting copy of community.blobmaptable with columns [id, blobmapcol].
> Failed to import 1 rows: ParseError - Failed to parse {3: 0x74776f} :
> unhashable type: 'bytearray', given up without retries
> Failed to process 1 rows; failed rows written to
> import_community_blobmaptable.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.389 seconds (0 skipped).
> {noformat}
> I've also logged
> [PYTHON-1234|https://datastax-oss.atlassian.net/browse/PYTHON-1234] because I
> wasn't sure if it was a Python driver issue. Cheers!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15679) cqlsh COPY FROM of map of blobs fails with parse error "unhashable type: 'bytearray'"
[
https://issues.apache.org/jira/browse/CASSANDRA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Sorokoumov reassigned CASSANDRA-15679:
Assignee: Aleksandr Sorokoumov
> cqlsh COPY FROM of map of blobs fails with parse error "unhashable type:
> 'bytearray'"
> -
>
> Key: CASSANDRA-15679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15679
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Erick Ramirez
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
>
> h2. Background
> A user was having issues loading CSV data with the {{COPY FROM}} command into
> a {{map}} column with {{blob}} values.
> h2. Replication steps
> I can easily replicate the problem with this simple table:
> {noformat}
> CREATE TABLE community.blobmaptable (
> id text PRIMARY KEY,
> blobmapcol map
> )
> {noformat}
> I have this CSV file that contains just 1 row:
> {noformat}
> $ cat blobmap.csv
> c3,{3: 0x74776f}
> {noformat}
> And here's the error when I try to load it:
> {noformat}
> cqlsh:community> COPY blobmaptable (id, blobmapcol) FROM '~/blobmap.csv' ;
> Using 1 child processes
> Starting copy of community.blobmaptable with columns [id, blobmapcol].
> Failed to import 1 rows: ParseError - Failed to parse {3: 0x74776f} :
> unhashable type: 'bytearray', given up without retries
> Failed to process 1 rows; failed rows written to
> import_community_blobmaptable.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.389 seconds (0 skipped).
> {noformat}
> I've also logged
> [PYTHON-1234|https://datastax-oss.atlassian.net/browse/PYTHON-1234] because I
> wasn't sure if it was a Python driver issue. Cheers!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15630) Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15630:
---
Fix Version/s: (was: 4.0-beta)
4.0-alpha
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/e08053b77cac4ec91fd398d7bad65bba1394f45f
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed into trunk at e08053b77cac4ec91fd398d7bad65bba1394f45f
> Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
> ---
>
> Key: CASSANDRA-15630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15630
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: CASS-15630-TEST-DOCKER.zip
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The test fails sometimes with the following error message and trace.
> {code:java}
> processed count values don't match expected:<90> but was:<89>
> junit.framework.AssertionFailedError: processed count values don't match
> expected:<90> but was:<89>
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.doCheck(ConnectionUtils.java:217)
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.check(ConnectionUtils.java:200)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$testSerializeError$24(ConnectionTest.java:494)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$doTest$8(ConnectionTest.java:240)
> at
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:260)
> at org.apache.cassandra.net.ConnectionTest.doTest(ConnectionTest.java:238)
> at org.apache.cassandra.net.ConnectionTest.test(ConnectionTest.java:227)
> at
> org.apache.cassandra.net.ConnectionTest.testSerializeError(ConnectionTest.java:435){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15630) Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15630:
---
Status: Ready to Commit (was: Review In Progress)
> Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
> ---
>
> Key: CASSANDRA-15630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15630
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: CASS-15630-TEST-DOCKER.zip
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The test fails sometimes with the following error message and trace.
> {code:java}
> processed count values don't match expected:<90> but was:<89>
> junit.framework.AssertionFailedError: processed count values don't match
> expected:<90> but was:<89>
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.doCheck(ConnectionUtils.java:217)
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.check(ConnectionUtils.java:200)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$testSerializeError$24(ConnectionTest.java:494)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$doTest$8(ConnectionTest.java:240)
> at
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:260)
> at org.apache.cassandra.net.ConnectionTest.doTest(ConnectionTest.java:238)
> at org.apache.cassandra.net.ConnectionTest.test(ConnectionTest.java:227)
> at
> org.apache.cassandra.net.ConnectionTest.testSerializeError(ConnectionTest.java:435){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-8272:
-
Fix Version/s: 4.0
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Sylvain Lebresne
>Assignee: Andres de la Peña
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 3.0.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix race-conditions in ConnectionTesti
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new e08053b Fix race-conditions in ConnectionTesti
e08053b is described below
commit e08053b77cac4ec91fd398d7bad65bba1394f45f
Author: yifan-c
AuthorDate: Fri Mar 13 11:30:43 2020 -0700
Fix race-conditions in ConnectionTesti
patch by Yifan Cai; reviewed by Benjamin Lerer for CASSANDRA-15630
---
test/unit/org/apache/cassandra/Util.java | 22 +---
.../org/apache/cassandra/net/ConnectionUtils.java | 65 ++
2 files changed, 44 insertions(+), 43 deletions(-)
diff --git a/test/unit/org/apache/cassandra/Util.java
b/test/unit/org/apache/cassandra/Util.java
index 3dcaff7..c989407 100644
--- a/test/unit/org/apache/cassandra/Util.java
+++ b/test/unit/org/apache/cassandra/Util.java
@@ -28,6 +28,7 @@ import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Supplier;
import java.util.stream.Collectors;
@@ -41,6 +42,7 @@ import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import afu.org.checkerframework.checker.oigj.qual.O;
import org.apache.cassandra.db.compaction.ActiveCompactionsTracker;
import org.apache.cassandra.db.compaction.CompactionTasks;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
@@ -579,18 +581,24 @@ public class Util
}
}
-public static void spinAssertEquals(Object expected, Supplier s,
int timeoutInSeconds)
+public static void spinAssertEquals(Object expected, Supplier
actualSupplier, int timeoutInSeconds)
{
-long start = System.currentTimeMillis();
-Object lastValue = null;
-while (System.currentTimeMillis() < start + (1000 * timeoutInSeconds))
+spinAssertEquals(null, expected, actualSupplier, timeoutInSeconds,
TimeUnit.SECONDS);
+}
+
+public static void spinAssertEquals(String message, T expected,
Supplier actualSupplier, long timeout, TimeUnit timeUnit)
+{
+long startNano = System.nanoTime();
+long expireAtNano = startNano + timeUnit.toNanos(timeout);
+T actual = null;
+while (System.nanoTime() < expireAtNano)
{
-lastValue = s.get();
-if (lastValue.equals(expected))
+actual = actualSupplier.get();
+if (actual.equals(expected))
break;
Thread.yield();
}
-assertEquals(expected, lastValue);
+assertEquals(message, expected, actual);
}
public static void joinThread(Thread thread) throws InterruptedException
diff --git a/test/unit/org/apache/cassandra/net/ConnectionUtils.java
b/test/unit/org/apache/cassandra/net/ConnectionUtils.java
index e391785..5aff390 100644
--- a/test/unit/org/apache/cassandra/net/ConnectionUtils.java
+++ b/test/unit/org/apache/cassandra/net/ConnectionUtils.java
@@ -18,19 +18,17 @@
package org.apache.cassandra.net;
+import java.util.Objects;
import java.util.concurrent.TimeUnit;
+import java.util.function.Supplier;
-import com.google.common.util.concurrent.Uninterruptibles;
-import org.junit.Assert;
-
-import org.apache.cassandra.net.InboundMessageHandlers;
-import org.apache.cassandra.net.OutboundConnection;
+import static org.apache.cassandra.Util.spinAssertEquals;
public class ConnectionUtils
{
public interface FailCheck
{
-public void accept(String message, long expected, long actual);
+public void accept(String message, Long expected, Supplier
actualSupplier);
}
public static class OutboundCountChecker
@@ -98,44 +96,44 @@ public class ConnectionUtils
public void check()
{
-doCheck(Assert::assertEquals);
+doCheck((message, expected, actual) -> spinAssertEquals(message,
expected, actual, 5, TimeUnit.SECONDS));
}
public void check(FailCheck failCheck)
{
-doCheck((message, expect, actual) -> { if (expect != actual)
failCheck.accept(message, expect, actual); });
+doCheck((message, expect, actual) -> { if (!Objects.equals(expect,
actual.get())) failCheck.accept(message, expect, actual); });
}
private void doCheck(FailCheck testAndFailCheck)
{
if (checkSubmitted)
{
-testAndFailCheck.accept("submitted count values don't match",
submitted, connection.submittedCount());
+testAndFailCheck.accept("submitted count values don't match",
submitted, connection::submittedCount);
}
if (checkPending)
{
-
[jira] [Commented] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071634#comment-17071634
]
ZhaoYang commented on CASSANDRA-15660:
--
The issue is that arguments are being evaluated in shell script...
{{{$VAR}}} will be evaluated and causing \{{*}} to list all files, we should
use {{"{$VAR}"}} instead.
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang reassigned CASSANDRA-15660:
Assignee: ZhaoYang
> Unable to specify -e/--execute flag in cqlsh
>
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file
> and use "-f" flag.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15626) Need microsecond precision for dropped columns so we can avoid timestamp issues
[ https://issues.apache.org/jira/browse/CASSANDRA-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071585#comment-17071585 ] Benjamin Lerer commented on CASSANDRA-15626: I agree that considering the impact of the change and the usecase, this ticket is a super low priority thing. >From a correctness point of view, I still believe that it is valid issue. > Need microsecond precision for dropped columns so we can avoid timestamp > issues > --- > > Key: CASSANDRA-15626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15626 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Ryan Svihla >Priority: Normal > > In CASSANDRA-15557 the fix for the flaky test is reimplementing the logic > from CASSANDRA-12997 which was removed as part of CASSANDRA-13426 > However, since dropped columns are stored at a millisecond precision instead > of a microsecond precision and ClientState.getTimestamp adds microseconds on > each call we will lose the precision on save and some writes that should be > dropped could reappear. > Note views affected as well > > [https://github.com/apache/cassandra/blob/cb83fbff479bb90e9abeaade9e0f8843634c974d/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L712-L716] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15678:
--
Reviewers: Jeff Jirsa
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
> Attachments: 15678-trunk.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help stop output (CASSANDRA-15401)
> Merged from 3.0:
> * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506)
> * Include updates to static column in mutation size calculations
> (CASSANDRA-15293)
> * Fix point-in-time recoevery ignoring timestamp of updates to static
> columns (CASSANDRA-15292)
> * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306)
> * Fix sstabledump's position key value when partitions have multiple rows
> (CASSANDRA-14721)
> * Avoid over-scanning data directories in LogFile.verify() (CASSANDRA-15364)
> * Bump generations and document changes to system_distributed and
> system_traces in 3.0, 3.11
>(CASSANDRA-15441)
> * Fix system_traces creation timestamp; optimise system keyspace upgrades
> (CASSANDRA-15398)
[jira] [Updated] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15678:
--
Test and Documentation Plan: Republished overwritten entries for 3.11.6 in
NEWS.txt, CHANGES.txt
Status: Patch Available (was: In Progress)
Patch available as follows:
* PR #501 - https://github.com/apache/cassandra/pull/501
* Attached - [^15678-trunk.txt]
Ready for review. (y)
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
> Attachments: 15678-trunk.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help stop output (CASSANDRA-15401)
> Merged from 3.0:
> * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506)
> * Include updates to static column in mutation size calculations
> (CASSANDRA-15293)
> * Fix point-in-time recoevery ignoring timestamp of updates to static
> columns (CASSANDRA-15292)
> * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306)
> * Fix sstabledump's position key value when partitions have multiple rows
> (CASSANDRA-14721)
> * Avoid
[jira] [Updated] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15678:
--
Attachment: 15678-trunk.txt
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
> Attachments: 15678-trunk.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help stop output (CASSANDRA-15401)
> Merged from 3.0:
> * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506)
> * Include updates to static column in mutation size calculations
> (CASSANDRA-15293)
> * Fix point-in-time recoevery ignoring timestamp of updates to static
> columns (CASSANDRA-15292)
> * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306)
> * Fix sstabledump's position key value when partitions have multiple rows
> (CASSANDRA-14721)
> * Avoid over-scanning data directories in LogFile.verify() (CASSANDRA-15364)
> * Bump generations and document changes to system_distributed and
> system_traces in 3.0, 3.11
>(CASSANDRA-15441)
> * Fix system_traces creation timestamp; optimise system keyspace upgrades
>
[jira] [Updated] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15678:
---
Labels: pull-request-available (was: )
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help stop output (CASSANDRA-15401)
> Merged from 3.0:
> * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506)
> * Include updates to static column in mutation size calculations
> (CASSANDRA-15293)
> * Fix point-in-time recoevery ignoring timestamp of updates to static
> columns (CASSANDRA-15292)
> * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306)
> * Fix sstabledump's position key value when partitions have multiple rows
> (CASSANDRA-14721)
> * Avoid over-scanning data directories in LogFile.verify() (CASSANDRA-15364)
> * Bump generations and document changes to system_distributed and
> system_traces in 3.0, 3.11
>(CASSANDRA-15441)
> * Fix system_traces creation timestamp; optimise system keyspace upgrades
> (CASSANDRA-15398)
> * Fix various data directory prefix matching issues (CASSANDRA-13974)
[jira] [Updated] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15678:
--
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help stop output (CASSANDRA-15401)
> Merged from 3.0:
> * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506)
> * Include updates to static column in mutation size calculations
> (CASSANDRA-15293)
> * Fix point-in-time recoevery ignoring timestamp of updates to static
> columns (CASSANDRA-15292)
> * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306)
> * Fix sstabledump's position key value when partitions have multiple rows
> (CASSANDRA-14721)
> * Avoid over-scanning data directories in LogFile.verify() (CASSANDRA-15364)
> * Bump generations and document changes to system_distributed and
> system_traces in 3.0, 3.11
>(CASSANDRA-15441)
> * Fix system_traces creation timestamp; optimise system keyspace upgrades
>
[jira] [Commented] (CASSANDRA-15678) Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
[
https://issues.apache.org/jira/browse/CASSANDRA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071489#comment-17071489
]
Erick Ramirez commented on CASSANDRA-15678:
---
Noting here that a {{diff}} shows some lines unrelated to 3.11.6 that's also
missing from {{trunk}}:
{noformat}
$ git diff cassandra-3.11.5..cassandra-3.11.6 -- NEWS.txt
{noformat}
{noformat}
@@ -580,6 +607,8 @@ Upgrading
- The 'index_interval' option for 'CREATE TABLE' statements, which has been
deprecated
since 2.1 and replaced with the 'min_index_interval' and
'max_index_interval' options,
has now been removed.
+ - The 'replicate_on_write' and 'populate_io_cache_on_flush' options for
'CREATE TABLE' statements,
+ which have been deprecated since 2.1, have also been removed.
- Batchlog entries are now stored in a new table - system.batches.
The old one has been deprecated.
- JMX methods set/getCompactionStrategyClass have been removed, use
{noformat}
It will need to be dealt with separately which I'll discuss on Slack at a later
stage.
> Updates for 3.11.6 got overwritten for NEWS.txt, CHANGES.txt
>
>
> Key: CASSANDRA-15678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15678
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>
> h2. Background
> I discovered by accident that the C* 3.11.6 sections are missing from the
> {{trunk}} version of
> [NEWS.txt|[https://github.com/apache/cassandra/blob/trunk/NEWS.txt]] and
> [CHANGES.txt|https://github.com/apache/cassandra/blob/trunk/CHANGES.txt].
> I've posted the missing text below.
> h2.
> [NEWS.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/NEWS.txt]
> {noformat}
> PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
> --
> QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
> SocketServer and ServerSocketReceiver components.Logback has not been
> upgraded to avoid breaking deployments and customizations
> based on older versions. If you are using vulnerable components you will need
> to upgrade to a newer version of Logback or stop using the vulnerable
> components. {noformat}
> {noformat}
> 3.11.6
> ==
> Upgrading
> -
> - Sstables for tables using with a frozen UDT written by C* 3.0 appear as
> corrupted.
> Background: The serialization-header in the -Statistics.db sstable
> component contains the type information
> of the table columns. C* 3.0 write incorrect type information for
> frozen UDTs by omitting the
> "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423
> in C* 3.6. Since then, the missing
> "frozen" information leads to deserialization issues that result in
> CorruptSSTableExceptions, potentially other
> exceptions as well.
> As a mitigation, the sstable serialization-headers are rewritten to
> contain the missing "frozen" information for
> UDTs once, when an upgrade from C* 3.0 is detected. This migration does
> not touch snapshots or backups.
> The sstablescrub tool now performs a check of the sstable
> serialization-header against the schema. A mismatch of
> the types in the serialization-header and the schema will cause
> sstablescrub to error out and stop by default.
> See the new `-e` option. `-e off` disables the new validation code. `-e
> fix` or `-e fix-only`, e.g.
> `sstablescrub -e fix keyspace table`, will validate the
> serialization-header, rewrite the non-frozen UDTs
> in the serialzation-header to frozen UDTs, if that matches the schema,
> and continue with scrub.
> See `sstablescrub -h`.
> (CASSANDRA-15035)
> - repair_session_max_tree_depth setting has been added to
> cassandra.yaml to allow operators to reduce
> merkle tree size if repair is creating too much heap pressure. See
> CASSANDRA-14096 for details.
> - Nothing specific to this release, but please see previous upgrading
> sections,
> especially if you are upgrading from 3.0.
> {noformat}
> h2.
> [CHANGES.txt|https://github.com/apache/cassandra/blob/cassandra-3.11.6/CHANGES.txt]
> {noformat}
> 3.11.6
> * Fix bad UDT sstable metadata serialization headers written by C* 3.0 on
> upgrade and in sstablescrub (CASSANDRA-15035)
> * Fix nodetool compactionstats showing extra pending task for TWCS - patch
> implemented (CASSANDRA-15409)
> * Fix SELECT JSON formatting for the "duration" type (CASSANDRA-15075)
> * Fix LegacyLayout to have same behavior as 2.x when handling unknown column
> names (CASSANDRA-15081)
> * Update nodetool help
