[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958337#comment-15958337
 ] 

Corentin Chary commented on CASSANDRA-13418:


What do you think about provide_overlapping_tombstones = "ignore" ? This is a 
little would integrate nicely with the code and does not add yet another 
compaction option (but sounds a little weird).


> Allow TWCS to ignore overlaps
> -
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12213:
-
Status: Patch Available  (was: In Progress)

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.x, 3.11.x
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958179#comment-15958179
 ] 

Stefania commented on CASSANDRA-12213:
--

Thanks for the review. I've updated the patch to use a single list and 
relaunched the tests.

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.x, 3.11.x
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13419) Relax limit on number of pending endpoints during CAS

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957793#comment-15957793
 ] 

Ariel Weisberg commented on CASSANDRA-13419:


[~slebresne] [~pauloricardomg]

Sylvain I don't follow your suggestion.

So the starting point is that we need the quorum for the condition check or 
serial read to include at least one replica that responded to PREPARE. This 
fixes the stale read issue from CASSANDRA-8346.

So we might only consider a node pending for CAS for the timeout of a Paxos 
round. Because if it's been pending longer than that amount of time it must 
have been part of the quorum of the PREPARE? What drives that guarantee?

Could we do something very simple like remember who was in the QUORUM for 
PREPARE and require a response from at least one of them when doing the 
condition check or read?

I don't see having the pending node be in a different state as being super hard 
either. We can record a timestamp when it first joins and then compare how long 
it has been when deciding whether it is pending for the purposes of Paxos. We 
are measuring time since what though? Since the coordinator first learned about 
the pending node via Gossip?

> Relax limit on number of pending endpoints during CAS
> -
>
> Key: CASSANDRA-13419
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13419
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, CQL
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> CASSANDRA-8346 avoids stale reads during CAS when checking the condition or 
> doing serial reads by disallowing more than one pending endpoint.
> It seems like it should be possible to allow more than one pending endpoint 
> by being smarter about who we read from during the QUORUM read or about the 
> state of pending nodes that are there for host replacement.
> Sylvain suggested 
> bq. Well, I guess things are working as they do for decently good reason 
> here. That said, thinking about it, it could be that the solution from 
> CASSANDRA-8346 is a bit of a big hammer: I believe it's enough to ensure that 
> we read from at least one replica that responded to PREPARE 'in the same 
> Paxos round' But we have timeouts on the paxos round, so it could be it is 
> possible to reduce drastically the time we consider a node pending for CAS so 
> that it's not a real problem in practice. Something like having pending node 
> move to a "almost there" state before becoming true replica, and staying in 
> that state for basically the max time of a paxos round, and then Paxos might 
> be able to replace "pending" nodes by those "almost there" for PREPARE.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13419) Relax limit on number of pending endpoints during CAS

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-13419:
--

 Summary: Relax limit on number of pending endpoints during CAS
 Key: CASSANDRA-13419
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13419
 Project: Cassandra
  Issue Type: Improvement
  Components: Coordination, CQL
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg


CASSANDRA-8346 avoids stale reads during CAS when checking the condition or 
doing serial reads by disallowing more than one pending endpoint.

It seems like it should be possible to allow more than one pending endpoint by 
being smarter about who we read from during the QUORUM read or about the state 
of pending nodes that are there for host replacement.

Sylvain suggested 

bq. Well, I guess things are working as they do for decently good reason here. 
That said, thinking about it, it could be that the solution from CASSANDRA-8346 
is a bit of a big hammer: I believe it's enough to ensure that we read from at 
least one replica that responded to PREPARE 'in the same Paxos round' But we 
have timeouts on the paxos round, so it could be it is possible to reduce 
drastically the time we consider a node pending for CAS so that it's not a real 
problem in practice. Something like having pending node move to a "almost 
there" state before becoming true replica, and staying in that state for 
basically the max time of a paxos round, and then Paxos might be able to 
replace "pending" nodes by those "almost there" for PREPARE.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg resolved CASSANDRA-13327.

Resolution: Not A Problem

Closing to open a ticket specific to relaxing the limit on number of pending 
endpoints and CAS.

> Pending endpoints size check for CAS doesn't play nicely with 
> writes-on-replacement
> ---
>
> Key: CASSANDRA-13327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13327
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> Consider this ring:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.4   MR DOWN NORMAL -7148113328562451251
> where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to 
> the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and 
> making no progress.
> Then the down node was replaced so we had:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.5   MR UP JOINING -7148113328562451251
> It’s confusing in the ring - the first JOINING is a genuine bootstrap, the 
> second is a replacement. We now had CAS unavailables (but no non-CAS 
> unvailables). I think it’s because the pending endpoints check thinks that 
> 127.0.0.5 is gaining a range when it’s just replacing.
> The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t 
> unnecessarily fail these requests.
> It also appears like required participants is bumped by 1 during a host 
> replacement so if the replacing host fails you will get unavailables and 
> timeouts.
> This is related to the check added in CASSANDRA-8346



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957722#comment-15957722
 ] 

Jeff Jirsa commented on CASSANDRA-13418:


Agreed. Most time series use cases would certainly benefit from 
{{getFullyExpiredSSTables()}}  ignoring overlaps for expiration.

> Allow TWCS to ignore overlaps
> -
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13418) Allow TWCS to ignore overlaps

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13418:
---
Labels: twcs  (was: )

> Allow TWCS to ignore overlaps
> -
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13418) Allow TWCS to ignore overlaps

[Corentin Chary (JIRA)]

Corentin Chary created CASSANDRA-13418:
--

 Summary: Allow TWCS to ignore overlaps
 Key: CASSANDRA-13418
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
 Project: Cassandra
  Issue Type: Improvement
  Components: Compaction
Reporter: Corentin Chary


http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
you really want read-repairs you're going to have sstables blocking the 
expiration of other fully expired SSTables because they overlap.

You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
very low value and that will purge the blockers of old data that should already 
have expired, thus removing the overlaps and allowing the other SSTables to 
expire.

The thing is that this is rather CPU intensive and not optimal. If you have 
time series, you might not care if all your data doesn't exactly expire at the 
right time, or if data re-appears for some time, as long as it gets deleted as 
soon as it can. And in this situation I believe it would be really beneficial 
to allow users to simply ignore overlapping SSTables when looking for fully 
expired ones.

To the question: why would you need read-repairs ?
- Full repairs basically take longer than the TTL of the data on my dataset, so 
this isn't really effective.
- Even with a 10% chances of doing a repair, we found out that this would be 
enough to greatly reduce entropy of the most used data (and if you have 
timeseries, you're likely to have a dashboard doing the same important queries 
over and over again).
- LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.

I'll try to come up with a patch demonstrating how this would work, try it on 
our system and report the effects.

cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13403) nodetool repair breaks SASI index

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957588#comment-15957588
 ] 

Igor Novgorodov edited comment on CASSANDRA-13403 at 4/5/17 8:33 PM:
-

-The problem does not appear to be present in *trunk* branch.-
-Just built latest trunk - and no matter what i can't reproduce it.-
-Do not know which commit fixed it...-

-Should we close?-

Nope, it's still there, but now the index breaks *only* with *incremental 
repair*.
With --full it's ok.



was (Author: blind_oracle):
-The problem does not appear to be present in *trunk* branch.
Just built latest trunk - and no matter what i can't reproduce it.
Do not know which commit fixed it...

Should we close?-

Nope, it's still there, but now the index breaks *only* with *incremental 
repair*.
With --full it's ok.


> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13403) nodetool repair breaks SASI index

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957588#comment-15957588
 ] 

Igor Novgorodov edited comment on CASSANDRA-13403 at 4/5/17 8:32 PM:
-

-The problem does not appear to be present in *trunk* branch.
Just built latest trunk - and no matter what i can't reproduce it.
Do not know which commit fixed it...

Should we close?-

Nope, it's still there, but now the index breaks *only* with *incremental 
repair*.
With --full it's ok.



was (Author: blind_oracle):
The problem does not appear to be present in *trunk* branch.
Just built latest trunk - and no matter what i can't reproduce it.
Do not know which commit fixed it...

Should we close?

> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13403) nodetool repair breaks SASI index

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957588#comment-15957588
 ] 

Igor Novgorodov commented on CASSANDRA-13403:
-

The problem does not appear to be present in *trunk* branch.
Just built latest trunk - and no matter what i can't reproduce it.
Do not know which commit fixed it...

Should we close?

> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12982) Customizable hint ttl has been removed

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957554#comment-15957554
 ] 

Aleksey Yeschenko commented on CASSANDRA-12982:
---

bq. Someone reading HintsReader wouldn’t expect the method to behave that way, 
and there doesn’t seem to be any benefit to doing it that way. It’s slightly 
riskier, no faster, and approximately as understandable as just copying the 
bytes.

It might not be faster (though I expect it is), but it does create less garbage 
(one extra array, one extra byte buffer, and one {{DataInputBuffer}} in the 
original version) - and it is a regression for the common case (hint is live). 
If you reencode timestamp and gcgs, you are no worse than before, 
allocation-wise. I might open a JIRA later to address this, but I guess it'll 
do for now - hint replay is not on the hot path.

I made some very minor cosmetic changes on top of you branch - which aren't 
really necessary, but if you don't mind, I'll be a tiny bit happier if they 
make it in. Links:

||branch||testall||dtest||
|[12982-3.11|https://github.com/iamaleksey/cassandra/tree/12982-3.11]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12982-3.11-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12982-3.11-dtest]|
|[12982-trunk|https://github.com/iamaleksey/cassandra/tree/12982-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12982-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-12982-trunk-dtest]|

With or without these, LGTM.

> Customizable hint ttl has been removed
> --
>
> Key: CASSANDRA-12982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12982
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 3.11.x
>
>
> The cassandra.maxHintTTL property added in CASSANDRA-5988 was removed in the 
> hint system rewrite for 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: Fix typo in TokenAllocation logging

[dikang]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 305de286b -> 4d15f2d06


Fix typo in TokenAllocation logging


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d15f2d0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d15f2d0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d15f2d0

Branch: refs/heads/trunk
Commit: 4d15f2d065649c62b55625f739f61eadb0fac174
Parents: 305de28
Author: Dikang Gu 
Authored: Wed Apr 5 12:01:23 2017 -0700
Committer: Dikang Gu 
Committed: Wed Apr 5 12:01:23 2017 -0700

--
 .../org/apache/cassandra/dht/tokenallocator/TokenAllocation.java | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d15f2d0/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
--
diff --git 
a/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java 
b/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
index 1d06b11..7c9d22c 100644
--- a/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
+++ b/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
@@ -62,8 +62,8 @@ public class TokenAllocation
 SummaryStatistics os = replicatedOwnershipStats(tokenMetadataCopy, 
rs, endpoint);
 tokenMetadataCopy.updateNormalTokens(tokens, endpoint);
 SummaryStatistics ns = replicatedOwnershipStats(tokenMetadataCopy, 
rs, endpoint);
-logger.warn("Replicated node load in datacentre before allocation 
{}", statToString(os));
-logger.warn("Replicated node load in datacentre after allocation 
{}", statToString(ns));
+logger.warn("Replicated node load in datacenter before allocation 
{}", statToString(os));
+logger.warn("Replicated node load in datacenter after allocation 
{}", statToString(ns));
 
 // TODO: Is it worth doing the replicated ownership calculation 
always to be able to raise this alarm?
 if (ns.getStandardDeviation() > os.getStandardDeviation())

[jira] [Updated] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10130:

Summary: Node failure during 2i update after streaming can have incomplete 
2i when restarted  (was: Node failure during MV/2i update after streaming can 
have incomplete MV/2i when restarted)

> Node failure during 2i update after streaming can have incomplete 2i when 
> restarted
> ---
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Yuki Morishita
>Assignee: T Jake Luciani
>Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during 
> MV/2i update can leave received SSTables live when restarted while MV/2i are 
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the 
> startup, or at least warn user when the node restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta reopened CASSANDRA-10130:
-

> Node failure during 2i update after streaming can have incomplete 2i when 
> restarted
> ---
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Yuki Morishita
>Assignee: T Jake Luciani
>Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during 
> MV/2i update can leave received SSTables live when restarted while MV/2i are 
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the 
> startup, or at least warn user when the node restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957404#comment-15957404
 ] 

Paulo Motta commented on CASSANDRA-10130:
-

I'm afraid this is still an issue for 2is though, since if a node fails [during 
2i rebuild after 
stream|https://github.com/apache/cassandra/blob/af3fe39dcabd9ef77a00309ce6741268423206df/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java#L212]
 the sstables are already live in the indexed table so 2is will not be fixed by 
a subsequent repair. Reopening and updating title.

> Node failure during 2i update after streaming can have incomplete 2i when 
> restarted
> ---
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Yuki Morishita
>Assignee: T Jake Luciani
>Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during 
> MV/2i update can leave received SSTables live when restarted while MV/2i are 
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the 
> startup, or at least warn user when the node restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-10145) Change protocol to allow sending key space independent of query string

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-10145:
-
Reviewer: Robert Stupp  (was: Tyler Hobbs)

> Change protocol to allow sending key space independent of query string
> --
>
> Key: CASSANDRA-10145
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10145
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Vishy Kasar
>Assignee: Sandeep Tamhankar
> Fix For: 4.0
>
> Attachments: 10145-trunk.txt
>
>
> Currently keyspace is either embedded in the query string or set through "use 
> keyspace" on a connection by client driver. 
> There are practical use cases where client user has query and keyspace 
> independently. In order for that scenario to work, they will have to create 
> one client session per keyspace or have to resort to some string replace 
> hackery.
> It will be nice if protocol allowed sending keyspace separately from the 
> query. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13417) Illegal unicode character breaks compilation on Chinese env OS

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa resolved CASSANDRA-13417.

Resolution: Fixed

Committed as {{bfdc1e0fdb3e4adad8d044203feaab8350dfdee8}}


> Illegal unicode character breaks compilation on Chinese env OS
> --
>
> Key: CASSANDRA-13417
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13417
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.11.0, 4.0
>
>
> Creating JIRA for tracking GH issue 
> https://github.com/apache/cassandra/pull/104
> Fix is contained within a comment block, so skipping CI.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[1/3] cassandra git commit: Delete illegal character from StandardTokenizerImpl.jflex, which will cause compilation error in Chinese environment OS

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 2c1504cd7 -> bfdc1e0fd
  refs/heads/trunk 201dd6f05 -> 305de286b


Delete illegal character from StandardTokenizerImpl.jflex, which will cause 
compilation error in Chinese environment OS

Closes #104

Patch by hxd; Reviewed by Jeff Jirsa for CASSANDRA-13417


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bfdc1e0f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bfdc1e0f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bfdc1e0f

Branch: refs/heads/cassandra-3.11
Commit: bfdc1e0fdb3e4adad8d044203feaab8350dfdee8
Parents: 2c1504c
Author: hxd 
Authored: Wed Apr 5 09:46:34 2017 +0800
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:24:37 2017 -0700

--
 CHANGES.txt| 1 +
 .../cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfdc1e0f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 28606d6..d3460d8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -18,6 +18,7 @@
  * More fixes to the TokenAllocator (CASSANDRA-12990)
  * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
  * Address message coalescing regression (CASSANDRA-12676)
+ * Delete illegal character from StandardTokenizerImpl.jflex (CASSANDRA-13417)
 Merged from 3.0:
  * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
  * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfdc1e0f/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex 
b/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
index d0270ff..bdc35eb 100644
--- 
a/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
+++ 
b/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
@@ -22,7 +22,7 @@ import java.util.Arrays;
 /**
  * This class implements Word Break rules from the Unicode Text Segmentation 
  * algorithm, as specified in 
- * http://unicode.org/reports/tr29/;>Unicode Standard Annex #29. 
∂
+ * http://unicode.org/reports/tr29/;>Unicode Standard Annex #29. 
  * 
  * Tokens produced are of the following types:
  * 

[2/3] cassandra git commit: Delete illegal character from StandardTokenizerImpl.jflex, which will cause compilation error in Chinese environment OS

[jjirsa]

Delete illegal character from StandardTokenizerImpl.jflex, which will cause 
compilation error in Chinese environment OS

Closes #104

Patch by hxd; Reviewed by Jeff Jirsa for CASSANDRA-13417


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bfdc1e0f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bfdc1e0f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bfdc1e0f

Branch: refs/heads/trunk
Commit: bfdc1e0fdb3e4adad8d044203feaab8350dfdee8
Parents: 2c1504c
Author: hxd 
Authored: Wed Apr 5 09:46:34 2017 +0800
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:24:37 2017 -0700

--
 CHANGES.txt| 1 +
 .../cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfdc1e0f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 28606d6..d3460d8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -18,6 +18,7 @@
  * More fixes to the TokenAllocator (CASSANDRA-12990)
  * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
  * Address message coalescing regression (CASSANDRA-12676)
+ * Delete illegal character from StandardTokenizerImpl.jflex (CASSANDRA-13417)
 Merged from 3.0:
  * Fix view builder bug that can filter out data on restart (CASSANDRA-13405)
  * Fix 2i page size calculation when there are no regular columns 
(CASSANDRA-13400)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfdc1e0f/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex 
b/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
index d0270ff..bdc35eb 100644
--- 
a/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
+++ 
b/src/java/org/apache/cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex
@@ -22,7 +22,7 @@ import java.util.Arrays;
 /**
  * This class implements Word Break rules from the Unicode Text Segmentation 
  * algorithm, as specified in 
- * http://unicode.org/reports/tr29/;>Unicode Standard Annex #29. 
∂
+ * http://unicode.org/reports/tr29/;>Unicode Standard Annex #29. 
  * 
  * Tokens produced are of the following types:
  * 

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jjirsa]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/305de286
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/305de286
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/305de286

Branch: refs/heads/trunk
Commit: 305de286b557a71e8b38f7a7a87ad4c4faa0f23d
Parents: 201dd6f bfdc1e0
Author: Jeff Jirsa 
Authored: Wed Apr 5 10:24:52 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:26:30 2017 -0700

--
 CHANGES.txt| 1 +
 .../cassandra/index/sasi/analyzer/StandardTokenizerImpl.jflex  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/305de286/CHANGES.txt
--

[jira] [Created] (CASSANDRA-13417) Illegal unicode character breaks compilation on Chinese env OS

[Jeff Jirsa (JIRA)]

Jeff Jirsa created CASSANDRA-13417:
--

 Summary: Illegal unicode character breaks compilation on Chinese 
env OS
 Key: CASSANDRA-13417
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13417
 Project: Cassandra
  Issue Type: Bug
Reporter: Jeff Jirsa
Assignee: Jeff Jirsa
Priority: Minor
 Fix For: 3.11.0, 4.0


Creating JIRA for tracking GH issue https://github.com/apache/cassandra/pull/104

Fix is contained within a comment block, so skipping CI.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957251#comment-15957251
 ] 

Marcus Eriksson edited comment on CASSANDRA-13413 at 4/5/17 5:12 PM:
-

running a new test with wamerican installed and it also collects the 
compression xmls, thanks guys: https://circleci.com/gh/krummas/cassandra/8


was (Author: krummas):
running a new test with wamerican installed and it also collects the 
compression xmls, thanks guys

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13410) nodetool upgradesstables/scrub/compact ignores system tables

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13410:
---
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 4.x)
   (was: 3.0.x)
   4.0
   3.11.0
   3.0.13
   Status: Resolved  (was: Ready to Commit)

Committed as {{6efb44b3ab5816cb7c2b007dba68b3224f899ac8}} to 3.0 and merged up. 

Agreeing with [~pauloricardomg] , skipping 2.2 as it's not critical.



> nodetool upgradesstables/scrub/compact ignores system tables
> 
>
> Key: CASSANDRA-13410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13410
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.13, 3.11.0, 4.0
>
>
> CASSANDRA-11627 changed the behavior of nodetool commands that work across 
> all keyspaces. Sometimes it's OK (not compacting system.peers when you call 
> compact probably isn't going to anger anyone), but sometimes it's not 
> (disableautocompaction, flush, upgradesstables, etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957250#comment-15957250
 ] 

Michael Shuler commented on CASSANDRA-13414:


[~jjirsa] we run {{-Dtest-runners=4}} everywhere else for our ant targets.

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957251#comment-15957251
 ] 

Marcus Eriksson commented on CASSANDRA-13413:
-

running a new test with wamerican installed and it also collects the 
compression xmls, thanks guys

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[3/6] cassandra git commit: Nodetool upgradesstables doesnt upgrade system tables

[jjirsa]

Nodetool upgradesstables doesnt upgrade system tables

Patch by Jeff Jirsa; Reviewed by Marcus Eriksson for CASSANDRA-13410


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6efb44b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6efb44b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6efb44b3

Branch: refs/heads/trunk
Commit: 6efb44b3ab5816cb7c2b007dba68b3224f899ac8
Parents: 0efb392
Author: Jeff Jirsa 
Authored: Mon Apr 3 17:41:54 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:09:41 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d9a97ad..8bd21bc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -15,6 +15,7 @@
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
  * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
  * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384)
+ * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 Merged from 2.2:
  * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
  * Discard in-flight shadow round responses (CASSANDRA-12653)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 7d125ad..6d060f4 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -322,7 +322,7 @@ public class NodeTool
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe)
 {
-return parseOptionalKeyspace(cmdArgs, nodeProbe, 
KeyspaceSet.NON_SYSTEM);
+return parseOptionalKeyspace(cmdArgs, nodeProbe, KeyspaceSet.ALL);
 }
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe, KeyspaceSet defaultKeyspaceSet)

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jjirsa]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/201dd6f0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/201dd6f0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/201dd6f0

Branch: refs/heads/trunk
Commit: 201dd6f05cfc9ae98aded4f658a41d8c924e342f
Parents: 22acb00 2c1504c
Author: Jeff Jirsa 
Authored: Wed Apr 5 10:11:15 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:11:43 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/201dd6f0/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/201dd6f0/src/java/org/apache/cassandra/tools/NodeTool.java
--

[2/6] cassandra git commit: Nodetool upgradesstables doesnt upgrade system tables

[jjirsa]

Nodetool upgradesstables doesnt upgrade system tables

Patch by Jeff Jirsa; Reviewed by Marcus Eriksson for CASSANDRA-13410


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6efb44b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6efb44b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6efb44b3

Branch: refs/heads/cassandra-3.11
Commit: 6efb44b3ab5816cb7c2b007dba68b3224f899ac8
Parents: 0efb392
Author: Jeff Jirsa 
Authored: Mon Apr 3 17:41:54 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:09:41 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d9a97ad..8bd21bc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -15,6 +15,7 @@
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
  * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
  * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384)
+ * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 Merged from 2.2:
  * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
  * Discard in-flight shadow round responses (CASSANDRA-12653)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 7d125ad..6d060f4 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -322,7 +322,7 @@ public class NodeTool
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe)
 {
-return parseOptionalKeyspace(cmdArgs, nodeProbe, 
KeyspaceSet.NON_SYSTEM);
+return parseOptionalKeyspace(cmdArgs, nodeProbe, KeyspaceSet.ALL);
 }
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe, KeyspaceSet defaultKeyspaceSet)

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c1504cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c1504cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c1504cd

Branch: refs/heads/cassandra-3.11
Commit: 2c1504cd78d74430c83fdca66850f920c0b6ed17
Parents: 1a3466e 6efb44b
Author: Jeff Jirsa 
Authored: Wed Apr 5 10:10:36 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:11:06 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c1504cd/CHANGES.txt
--
diff --cc CHANGES.txt
index 1ca8733,8bd21bc..28606d6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -56,146 -62,6 +56,147 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
++ * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 +Merged from 2.2:
 + * Fix JVM metric names (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + 

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c1504cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c1504cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c1504cd

Branch: refs/heads/trunk
Commit: 2c1504cd78d74430c83fdca66850f920c0b6ed17
Parents: 1a3466e 6efb44b
Author: Jeff Jirsa 
Authored: Wed Apr 5 10:10:36 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:11:06 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c1504cd/CHANGES.txt
--
diff --cc CHANGES.txt
index 1ca8733,8bd21bc..28606d6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -56,146 -62,6 +56,147 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
++ * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 +Merged from 2.2:
 + * Fix JVM metric names (CASSANDRA-13103)
 + * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
 + * Discard in-flight shadow round responses (CASSANDRA-12653)
 + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153)
 + * Wrong logger name in AnticompactionTask (CASSANDRA-13343)
 + * Commitlog replay may fail if last mutation is within 4 bytes of end of 
segment (CASSANDRA-13282)
 + * Fix queries updating multiple time the same list (CASSANDRA-13130)
 + * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053)
 + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 + * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 + * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 + * Fix negative mean latency metric (CASSANDRA-12876)
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Remove unused repositories (CASSANDRA-13278)
 + * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 + * Use portable stderr for java error in startup (CASSANDRA-13211)
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make 

[1/6] cassandra git commit: Nodetool upgradesstables doesnt upgrade system tables

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 0efb392bc -> 6efb44b3a
  refs/heads/cassandra-3.11 1a3466e43 -> 2c1504cd7
  refs/heads/trunk 22acb0023 -> 201dd6f05


Nodetool upgradesstables doesnt upgrade system tables

Patch by Jeff Jirsa; Reviewed by Marcus Eriksson for CASSANDRA-13410


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6efb44b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6efb44b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6efb44b3

Branch: refs/heads/cassandra-3.0
Commit: 6efb44b3ab5816cb7c2b007dba68b3224f899ac8
Parents: 0efb392
Author: Jeff Jirsa 
Authored: Mon Apr 3 17:41:54 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Apr 5 10:09:41 2017 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d9a97ad..8bd21bc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -15,6 +15,7 @@
  * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305)
  * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238)
  * Legacy caching options can prevent 3.0 upgrade (CASSANDRA-13384)
+ * Nodetool upgradesstables/scrub/compact ignores system tables 
(CASSANDRA-13410)
 Merged from 2.2:
  * Honor truststore-password parameter in cassandra-stress (CASSANDRA-12773)
  * Discard in-flight shadow round responses (CASSANDRA-12653)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6efb44b3/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 7d125ad..6d060f4 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -322,7 +322,7 @@ public class NodeTool
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe)
 {
-return parseOptionalKeyspace(cmdArgs, nodeProbe, 
KeyspaceSet.NON_SYSTEM);
+return parseOptionalKeyspace(cmdArgs, nodeProbe, KeyspaceSet.ALL);
 }
 
 protected List parseOptionalKeyspace(List cmdArgs, 
NodeProbe nodeProbe, KeyspaceSet defaultKeyspaceSet)

[jira] [Resolved] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-13414.

Resolution: Duplicate

Closed as dupe contained by CASSANDRA-13413

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957246#comment-15957246
 ] 

Marcus Eriksson commented on CASSANDRA-13413:
-

sure, will include!

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957245#comment-15957245
 ] 

Michael Shuler commented on CASSANDRA-13413:


I was working on multiple runners and fixing a broken test in CASSANDRA-13414. 
It looks like we OOM the container with more than one runner, so since y'all 
are here, can you include this one-liner to fix 
o.a.c.utils.BitSetTest.compareBitSets?
{code}
diff --git a/circle.yml b/circle.yml
index f0df31265a..66e752f367 100644
--- a/circle.yml
+++ b/circle.yml
@@ -8,6 +8,7 @@ test:
 - sudo ifconfig lo:2 127.0.0.3 up
 - sudo ifconfig lo:3 127.0.0.4 up
 - sudo ifconfig lo:4 127.0.0.5 up
+- sudo apt-get update; sudo apt-get install wamerican
 - ant build
 
   override:
{code}

Since you're removing the pre: section, I'm not sure how to include that in the 
parallel bits, yet. Thanks!

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957231#comment-15957231
 ] 

Jeff Jirsa commented on CASSANDRA-13414:


Is {{test.runners=2}} something that works well elsewhere? 


> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12805) Website documentation for commitlog

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12805:
---
Resolution: Fixed
  Reviewer: Jon Haddad
Status: Resolved  (was: Patch Available)

Committed in {{22acb00235ee081d3555cb1ff2780805e0268b07}} , added 
[~rustyrazorblade] and [~eprothro] as reviewers. Thanks to all three of you!


> Website documentation for commitlog
> ---
>
> Key: CASSANDRA-12805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12805
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Hau Phan
>Assignee: Hau Phan
>Priority: Minor
>  Labels: documentation
> Attachments: 12805-trunk.txt
>
>
> Updated Storage Engine page for commitlogs
> Commit: 
> https://github.com/nothau/cassandra/commit/f90038e9f35281bdd58dabb25f21836a690e56f5



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-12805) Website documentation for commitlog

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-12805:
--

Assignee: Hau Phan

> Website documentation for commitlog
> ---
>
> Key: CASSANDRA-12805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12805
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Hau Phan
>Assignee: Hau Phan
>Priority: Minor
>  Labels: documentation
> Attachments: 12805-trunk.txt
>
>
> Updated Storage Engine page for commitlogs
> Commit: 
> https://github.com/nothau/cassandra/commit/f90038e9f35281bdd58dabb25f21836a690e56f5



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: Updated commitlog documentation storage_engine.rst

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2ba06310d -> 22acb0023


Updated commitlog documentation storage_engine.rst

Patch by Hau Phan; Reviewed by Jon Haddad and Evan Prothro for CASSANDRA-12805


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/22acb002
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/22acb002
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/22acb002

Branch: refs/heads/trunk
Commit: 22acb00235ee081d3555cb1ff2780805e0268b07
Parents: 2ba0631
Author: nothau 
Authored: Thu Oct 27 16:20:32 2016 -0500
Committer: Jeff Jirsa 
Committed: Wed Apr 5 09:44:20 2017 -0700

--
 doc/source/architecture/storage_engine.rst | 49 -
 1 file changed, 48 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/22acb002/doc/source/architecture/storage_engine.rst
--
diff --git a/doc/source/architecture/storage_engine.rst 
b/doc/source/architecture/storage_engine.rst
index e4114e5..72d5802 100644
--- a/doc/source/architecture/storage_engine.rst
+++ b/doc/source/architecture/storage_engine.rst
@@ -22,7 +22,54 @@ Storage Engine
 CommitLog
 ^
 
-.. todo:: todo
+Commitlogs are an append only log of all mutations local to a Cassandra node. 
Any data written to Cassandra will first be written to a commit log before 
being written to a memtable. This provides durability in the case of unexpected 
shutdown. On startup, any mutations in the commit log will be applied to 
memtables.
+
+All mutations write optimized by storing in commitlog segments, reducing the 
number of seeks needed to write to disk. Commitlog Segments are limited by the 
"commitlog_segment_size_in_mb" option, once the size is reached, a new 
commitlog segment is created. Commitlog segments can be archived, deleted, or 
recycled once all its data has been flushed to SSTables.  Commitlog segments 
are truncated when Cassandra has written data older than a certain point to the 
SSTables. Running "nodetool drain" before stopping Cassandra will write 
everything in the memtables to SSTables and remove the need to sync with the 
commitlogs on startup.
+
+- ``commitlog_segment_size_in_mb``: The default size is 32, which is almost 
always fine, but if you are archiving commitlog segments (see 
commitlog_archiving.properties), then you probably want a finer granularity of 
archiving; 8 or 16 MB is reasonable. Max mutation size is also configurable via 
max_mutation_size_in_kb setting in cassandra.yaml. The default is half the size 
commitlog_segment_size_in_mb * 1024.
+
+***NOTE: If max_mutation_size_in_kb is set explicitly then 
commitlog_segment_size_in_mb must be set to at least twice the size of 
max_mutation_size_in_kb / 1024***
+
+*Default Value:* 32
+
+Commitlogs are an append only log of all mutations local to a Cassandra node. 
Any data written to Cassandra will first be written to a commit log before 
being written to a memtable. This provides durability in the case of unexpected 
shutdown. On startup, any mutations in the commit log will be applied.
+
+- ``commitlog_sync``: may be either “periodic” or “batch.”
+
+  - ``batch``: In batch mode, Cassandra won’t ack writes until the commit 
log has been fsynced to disk. It will wait "commitlog_sync_batch_window_in_ms" 
milliseconds between fsyncs. This window should be kept short because the 
writer threads will be unable to do extra work while waiting. You may need to 
increase concurrent_writes for the same reason.
+
+- ``commitlog_sync_batch_window_in_ms``: Time to wait between "batch" 
fsyncs
+*Default Value:* 2
+
+  - ``periodic``: In periodic mode, writes are immediately ack'ed, and the 
CommitLog is simply synced every "commitlog_sync_period_in_ms" milliseconds.
+
+- ``commitlog_sync_period_in_ms``: Time to wait between "periodic" fsyncs
+*Default Value:* 1
+
+*Default Value:* batch
+
+*** NOTE: In the event of an unexpected shutdown, Cassandra can lose up to the 
sync period or more if the sync is delayed. If using "batch" mode, it is 
recommended to store commitlogs in a separate, dedicated device.**
+
+
+- ``commitlog_directory``: This option is commented out by default When 
running on magnetic HDD, this should be a separate spindle than the data 
directories. If not set, the default directory is 
$CASSANDRA_HOME/data/commitlog.
+
+*Default Value:* /var/lib/cassandra/commitlog
+
+- ``commitlog_compression``: Compression to apply to the commitlog. If 
omitted, the commit log will be written uncompressed.  LZ4, Snappy, and Deflate 
compressors are supported.
+
+(Default Value: (complex option)::
+
+#   - class_name: LZ4Compressor
+# 

[jira] [Updated] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13414:
---
Fix Version/s: (was: 4.0)
   4.x

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13413) Run more test targets on CircleCI

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957103#comment-15957103
 ] 

Jeff Jirsa edited comment on CASSANDRA-13413 at 4/5/17 4:16 PM:


Build #6 looks better, but it's somehow not gathering the junit result properly 

{quote}
Your build ran 2980 tests in junit with 0 failures
Slowest test: org.apache.cassandra.cql3.PagingQueryTest pagingOnRegularColumn 
(took 105.24 seconds).
{quote}

{quote}
[junit] Testsuite: org.apache.cassandra.net.MessagingServiceTest-compression
[junit] Testsuite: 
org.apache.cassandra.net.MessagingServiceTest-compression Tests run: 12, 
Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.835 sec
[junit] 
[junit] Testcase: 
testDroppedMessages(org.apache.cassandra.net.MessagingServiceTest)-compression: 
  FAILED
[junit] expected:<...dropped latency: 227[8] ms> but was:<...dropped 
latency: 227[7] ms>
[junit] junit.framework.AssertionFailedError: expected:<...dropped latency: 
227[8] ms> but was:<...dropped latency: 227[7] ms>
[junit] at 
org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:114)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.net.MessagingServiceTest FAILED
{quote}

Passed on 3 of the 4 containers/suites, though, so it's pretty close. 
{{test-compression}} puts results into {{build/test/output/compression/}} so 
you'll need to copy those into the results directory in the post step.


was (Author: jjirsa):
Build #6 looks better, but it's somehow not gathering the junit result properly 

{quote}
Your build ran 2980 tests in junit with 0 failures
Slowest test: org.apache.cassandra.cql3.PagingQueryTest pagingOnRegularColumn 
(took 105.24 seconds).
{quote}

{quote}
[junit] Testsuite: org.apache.cassandra.net.MessagingServiceTest-compression
[junit] Testsuite: 
org.apache.cassandra.net.MessagingServiceTest-compression Tests run: 12, 
Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.835 sec
[junit] 
[junit] Testcase: 
testDroppedMessages(org.apache.cassandra.net.MessagingServiceTest)-compression: 
  FAILED
[junit] expected:<...dropped latency: 227[8] ms> but was:<...dropped 
latency: 227[7] ms>
[junit] junit.framework.AssertionFailedError: expected:<...dropped latency: 
227[8] ms> but was:<...dropped latency: 227[7] ms>
[junit] at 
org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:114)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.net.MessagingServiceTest FAILED
{quote}

Passed on 3 of the 4 containers/suites, though, so it's pretty close.

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957110#comment-15957110
 ] 

Michael Shuler commented on CASSANDRA-13414:


My trunk test failed with 2 runners:
{noformat}
Failing command: ant test -Dtest.runners=2
Exit code: -1
Output:

   [junit] 
[junit] Testsuite: org.apache.cassandra.dht.RangeTest
[junit] Testsuite: org.apache.cassandra.dht.SplitterTest
[junit] Testsuite: org.apache.cassandra.dht.RangeTest Tests run: 28, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.559 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.dht.StreamStateStoreTest
[junit] Testsuite: org.apache.cassandra.dht.StreamStateStoreTest Tests run: 
1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.841 sec
[junit] 
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/commitlog:210
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/data:210
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/saved_caches:210
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/hints:210
[junit] Testsuite: org.apache.cassandra.gms.ArrayBackedBoundedStatsTest
[junit] Testsuite: org.apache.cassandra.gms.ArrayBackedBoundedStatsTest 
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.351 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.dht.SplitterTest Tests run: 4, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.766 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.gms.ArrivalWindowTest
[junit] Testsuite: org.apache.cassandra.gms.ArrivalWindowTest Tests run: 1, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.939 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.gms.EndpointStateTest
[junit] Testsuite: org.apache.cassandra.gms.FailureDetectorTest
[junit] Testsuite: org.apache.cassandra.gms.FailureDetectorTest Tests run: 
1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.804 sec
[junit] 
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/commitlog:214
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/data:214
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/saved_caches:214
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/hints:214
[junit] Testsuite: org.apache.cassandra.gms.EndpointStateTest Tests run: 2, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.87 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.gms.GossipDigestTest
[junit] Testsuite: org.apache.cassandra.gms.GossipDigestTest Tests run: 1, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.378 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.gms.PendingRangeCalculatorServiceTest
[junit] Testsuite: org.apache.cassandra.gms.GossiperTest
[junit] Testsuite: org.apache.cassandra.gms.GossiperTest Tests run: 1, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.59 sec
[junit] 
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/commitlog:216
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/data:216
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/saved_caches:216
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/hints:216
[junit] Testsuite: org.apache.cassandra.gms.SerializationsTest
[junit] Testsuite: org.apache.cassandra.gms.SerializationsTest Tests run: 
2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.865 sec
[junit] 
[junit] Testsuite: 
org.apache.cassandra.gms.PendingRangeCalculatorServiceTest Tests run: 1, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.558 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.gms.ShadowRoundTest
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/commitlog:217
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/data:217
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/saved_caches:217
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/hints:217
[junit] Testsuite: org.apache.cassandra.hints.ChecksummedDataInputTest
[junit] Testsuite: org.apache.cassandra.hints.ChecksummedDataInputTest 
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.175 sec
[junit] 
[junit] Testsuite: org.apache.cassandra.hints.HintMessageTest
[junit] Testsuite: org.apache.cassandra.gms.ShadowRoundTest Tests run: 1, 
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.446 sec
[junit] 
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/commitlog:219
   [delete] Deleting directory 
/home/ubuntu/cassandra/build/test/cassandra/data:219
   [delete] Deleting directory 

[jira] [Comment Edited] (CASSANDRA-13416) Multiple test failures with NPE in org.apache.cassandra.streaming.StreamSession

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957104#comment-15957104
 ] 

Michael Shuler edited comment on CASSANDRA-13416 at 4/5/17 3:59 PM:


It looks like this was fixed in 
https://github.com/apache/cassandra/commit/633babf0f02fac56cad7bff03a4ff415feb38f39


was (Author: mshuler):
I looks like this was fixed in 
https://github.com/apache/cassandra/commit/633babf0f02fac56cad7bff03a4ff415feb38f39

> Multiple test failures with NPE in 
> org.apache.cassandra.streaming.StreamSession
> ---
>
> Key: CASSANDRA-13416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13416
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Critical
>  Labels: test-failure
> Attachments: jenkins-trunk_testall-1491_logs.tar.gz
>
>
> example failures:
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences_compression
> one example stack trace:
> {code}
> Stacktrace
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.streaming.StreamSession.isKeepAliveSupported(StreamSession.java:244)
>   at 
> org.apache.cassandra.streaming.StreamSession.(StreamSession.java:196)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.getOrCreateNextSession(StreamCoordinator.java:293)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator.getOrCreateNextSession(StreamCoordinator.java:158)
>   at 
> org.apache.cassandra.streaming.StreamPlan.requestRanges(StreamPlan.java:94)
>   at 
> org.apache.cassandra.repair.LocalSyncTask.startSync(LocalSyncTask.java:85)
>   at org.apache.cassandra.repair.SyncTask.run(SyncTask.java:75)
>   at 
> org.apache.cassandra.repair.LocalSyncTaskTest.testDifference(LocalSyncTaskTest.java:117)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957103#comment-15957103
 ] 

Jeff Jirsa commented on CASSANDRA-13413:


Build #6 looks better, but it's somehow not gathering the junit result properly 

{quote}
Your build ran 2980 tests in junit with 0 failures
Slowest test: org.apache.cassandra.cql3.PagingQueryTest pagingOnRegularColumn 
(took 105.24 seconds).
{quote}

{quote}
[junit] Testsuite: org.apache.cassandra.net.MessagingServiceTest-compression
[junit] Testsuite: 
org.apache.cassandra.net.MessagingServiceTest-compression Tests run: 12, 
Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.835 sec
[junit] 
[junit] Testcase: 
testDroppedMessages(org.apache.cassandra.net.MessagingServiceTest)-compression: 
  FAILED
[junit] expected:<...dropped latency: 227[8] ms> but was:<...dropped 
latency: 227[7] ms>
[junit] junit.framework.AssertionFailedError: expected:<...dropped latency: 
227[8] ms> but was:<...dropped latency: 227[7] ms>
[junit] at 
org.apache.cassandra.net.MessagingServiceTest.testDroppedMessages(MessagingServiceTest.java:114)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.net.MessagingServiceTest FAILED
{quote}

Passed on 3 of the 4 containers/suites, though, so it's pretty close.

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13416) Multiple test failures with NPE in org.apache.cassandra.streaming.StreamSession

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-13416.

Resolution: Fixed

I looks like this was fixed in 
https://github.com/apache/cassandra/commit/633babf0f02fac56cad7bff03a4ff415feb38f39

> Multiple test failures with NPE in 
> org.apache.cassandra.streaming.StreamSession
> ---
>
> Key: CASSANDRA-13416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13416
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Critical
>  Labels: test-failure
> Attachments: jenkins-trunk_testall-1491_logs.tar.gz
>
>
> example failures:
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences_compression
> one example stack trace:
> {code}
> Stacktrace
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.streaming.StreamSession.isKeepAliveSupported(StreamSession.java:244)
>   at 
> org.apache.cassandra.streaming.StreamSession.(StreamSession.java:196)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.getOrCreateNextSession(StreamCoordinator.java:293)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator.getOrCreateNextSession(StreamCoordinator.java:158)
>   at 
> org.apache.cassandra.streaming.StreamPlan.requestRanges(StreamPlan.java:94)
>   at 
> org.apache.cassandra.repair.LocalSyncTask.startSync(LocalSyncTask.java:85)
>   at org.apache.cassandra.repair.SyncTask.run(SyncTask.java:75)
>   at 
> org.apache.cassandra.repair.LocalSyncTaskTest.testDifference(LocalSyncTaskTest.java:117)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13416) Multiple test failures with NPE in org.apache.cassandra.streaming.StreamSession

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler reassigned CASSANDRA-13416:
--

Assignee: Michael Shuler

> Multiple test failures with NPE in 
> org.apache.cassandra.streaming.StreamSession
> ---
>
> Key: CASSANDRA-13416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13416
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Critical
>  Labels: test-failure
> Attachments: jenkins-trunk_testall-1491_logs.tar.gz
>
>
> example failures:
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout_compression
> http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences_compression
> one example stack trace:
> {code}
> Stacktrace
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.streaming.StreamSession.isKeepAliveSupported(StreamSession.java:244)
>   at 
> org.apache.cassandra.streaming.StreamSession.(StreamSession.java:196)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.getOrCreateNextSession(StreamCoordinator.java:293)
>   at 
> org.apache.cassandra.streaming.StreamCoordinator.getOrCreateNextSession(StreamCoordinator.java:158)
>   at 
> org.apache.cassandra.streaming.StreamPlan.requestRanges(StreamPlan.java:94)
>   at 
> org.apache.cassandra.repair.LocalSyncTask.startSync(LocalSyncTask.java:85)
>   at org.apache.cassandra.repair.SyncTask.run(SyncTask.java:75)
>   at 
> org.apache.cassandra.repair.LocalSyncTaskTest.testDifference(LocalSyncTaskTest.java:117)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13416) Multiple test failures with NPE in org.apache.cassandra.streaming.StreamSession

[Michael Shuler (JIRA)]

Michael Shuler created CASSANDRA-13416:
--

 Summary: Multiple test failures with NPE in 
org.apache.cassandra.streaming.StreamSession
 Key: CASSANDRA-13416
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13416
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Shuler
Priority: Critical
 Attachments: jenkins-trunk_testall-1491_logs.tar.gz

example failures:

http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.dht/StreamStateStoreTest/testUpdateAndQueryAvailableRanges_compression
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/LocalSyncTaskTest/testDifference_compression
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/incrementalStreamPlan_compression
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.repair/StreamingRepairTaskTest/fullStreamPlan_compression
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testScheduleTimeout_compression
http://cassci.datastax.com/job/trunk_testall/1491/testReport/org.apache.cassandra.streaming/StreamTransferTaskTest/testFailSessionDuringTransferShouldNotReleaseReferences_compression

one example stack trace:
{code}
Stacktrace

java.lang.NullPointerException
at 
org.apache.cassandra.streaming.StreamSession.isKeepAliveSupported(StreamSession.java:244)
at 
org.apache.cassandra.streaming.StreamSession.(StreamSession.java:196)
at 
org.apache.cassandra.streaming.StreamCoordinator$HostStreamingData.getOrCreateNextSession(StreamCoordinator.java:293)
at 
org.apache.cassandra.streaming.StreamCoordinator.getOrCreateNextSession(StreamCoordinator.java:158)
at 
org.apache.cassandra.streaming.StreamPlan.requestRanges(StreamPlan.java:94)
at 
org.apache.cassandra.repair.LocalSyncTask.startSync(LocalSyncTask.java:85)
at org.apache.cassandra.repair.SyncTask.run(SyncTask.java:75)
at 
org.apache.cassandra.repair.LocalSyncTaskTest.testDifference(LocalSyncTaskTest.java:117)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13415) test failure in org.apache.cassandra.db.compaction.PendingRepairManagerTest.getNextBackgroundTask-compression

[Michael Shuler (JIRA)]

Michael Shuler created CASSANDRA-13415:
--

 Summary: test failure in 
org.apache.cassandra.db.compaction.PendingRepairManagerTest.getNextBackgroundTask-compression
 Key: CASSANDRA-13415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13415
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Shuler
 Attachments: 
TEST-org.apache.cassandra.db.compaction.PendingRepairManagerTest.log

example failure:

http://cassci.datastax.com/job/trunk_testall/1490/testReport/org.apache.cassandra.db.compaction/PendingRepairManagerTest/getNextBackgroundTask_compression

{code}
Stacktrace

junit.framework.AssertionFailedError
at 
org.apache.cassandra.db.compaction.PendingRepairManagerTest.getNextBackgroundTask(PendingRepairManagerTest.java:155)
Standard Output

ERROR [main] 2017-04-03 20:02:09,109 SubstituteLogger.java:250 - SLF4J: stderr
INFO  [main] 2017-04-03 20:02:09,474 YamlConfigurationLoader.java:89 - 
Configuration location: file:/home/automaton/cassandra/test/conf/cassandra.yaml
DEBUG [main] 2017-04-03 20:02:09,477 YamlConfigurationLoader.java:108 - Loading 
settings from file:/home/automaton/cassandra/test/conf/cassandra.yaml
INFO  [main] 2017-04-03 20:02:11,007 Config.java:454 - Node 
configuration:[allocate_tokens_for_keyspace=null; authentica
...[truncated 84651 chars]...
 position=21545)
DEBUG [MemtableFlushWriter:2] 2017-04-03 20:02:19,371 
ColumnFamilyStore.java:1226 - Flushed to 
[BigTableReader(path='/home/automaton/cassandra/build/test/cassandra/data:165/ks_1491249739206/tbl-7145c66018a811e7b0c3bf9fab0791de/na-1-big-Data.db')]
 (1 sstables, 4.760KiB), biggest 4.760KiB, smallest 4.760KiB
DEBUG [main] 2017-04-03 20:02:19,374 PendingRepairManager.java:101 - Creating 
ks_1491249739206.tbl compaction strategy for pending repair: 
715a11b0-18a8-11e7-b0c3-bf9fab0791de
{code}

possibly related to CASSANDRA-13224



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13414:
---
Component/s: Testing

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 4.0, 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13414:
---
Fix Version/s: 3.11.x
   3.0.x
   2.2.x
   2.1.x
   4.0

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Fix For: 4.0, 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13414:
---
Attachment: 0001-Parallelize-tests-and-install-word-list-in-circleci.patch

> circle.yml: parallelize tests and install word list
> ---
>
> Key: CASSANDRA-13414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>Priority: Minor
> Attachments: 
> 0001-Parallelize-tests-and-install-word-list-in-circleci.patch
>
>
> - Run 2 test runners, since circleci container has 2 vCPUs.
> - o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
> branches when test word list is unavailable.
> I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
> test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13414) circle.yml: parallelize tests and install word list

[Michael Shuler (JIRA)]

Michael Shuler created CASSANDRA-13414:
--

 Summary: circle.yml: parallelize tests and install word list
 Key: CASSANDRA-13414
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13414
 Project: Cassandra
  Issue Type: Improvement
Reporter: Michael Shuler
Assignee: Michael Shuler
Priority: Minor


- Run 2 test runners, since circleci container has 2 vCPUs.
- o.a.c.utils.BitSetTest.compareBitSets fails in 2.1 and is skipped in later 
branches when test word list is unavailable.

I verified that compareBitSets passes again in a 2.1 run, and I have a trunk 
test with 2 runners in progress. Will check run time, when that has completed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13224) testall failure in org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956967#comment-15956967
 ] 

Blake Eggleston commented on CASSANDRA-13224:
-

Awesome, thank you Paulo. +1

> testall failure in 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized
> ---
>
> Key: CASSANDRA-13224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13224
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Blake Eggleston
>  Labels: test-failure, testall
> Attachments: 
> TEST-org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_testall/1407/testReport/org.apache.cassandra.db.compaction/CompactionStrategyManagerPendingRepairTest/cleanupCompactionFinalized
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: 
>   at 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized(CompactionStrategyManagerPendingRepairTest.java:235)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13224) testall failure in org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13224:

Reviewer: Blake Eggleston

> testall failure in 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized
> ---
>
> Key: CASSANDRA-13224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13224
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Blake Eggleston
>  Labels: test-failure, testall
> Attachments: 
> TEST-org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_testall/1407/testReport/org.apache.cassandra.db.compaction/CompactionStrategyManagerPendingRepairTest/cleanupCompactionFinalized
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: 
>   at 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized(CompactionStrategyManagerPendingRepairTest.java:235)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13404) Hostname verification for client-to-node encryption

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956937#comment-15956937
 ] 

Stefan Podkowinski commented on CASSANDRA-13404:


I think it was mentioned somewhere that reusing SSLContext instances would be 
preferable in the future due to performance reasons. We'd have to change the 
code to either return a shared or a newly created instance if we would add this 
feature. 

The main motivation for CASSANDRA-9220 and related client tickets was to 
prevent men-in-the-middle attacks. If you send your login credentials, you have 
to make sure that the connection hasn't been compromised and therefor it's 
important to verify that the peer is really the server you think you're talking 
to. This can be done by verifying the trust chain of the certificate and the 
hostname for which the certificate has been issued for.

Once the connection has been verified, the connection confidentiality has been 
established and there's no point for the server to in turn verify the client 
certificate again to prevent MiM. The only scenario where it would make sense 
to verify clients is when you're not able to verify server certificates 
correctly on the client side. At least the Java and Python driver should now do 
this correctly (incl. hostnames), but there could be other clients where you'd 
prefer to verify from server side. But given operational implications (there 
are usually much more client nodes than cluster nodes in the network) of having 
to manage a lot of certificates for a potentially elastic number of clients, 
this would be a quite heavy handed way to address this issue for most users. In 
this case you probably would want to spend the effort fixing the clients to 
correctly verify the servers. 

This doesn't mean I'm -1 here as long as code changes are small, but just 
wanted to share my thoughts why this hasn't been implemented yet.


> Hostname verification for client-to-node encryption
> ---
>
> Key: CASSANDRA-13404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13404
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Fix For: 4.x
>
> Attachments: 13404-trunk.txt
>
>
> Similarily to CASSANDRA-9220, Cassandra should support hostname verification 
> for client-node connections.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12811) testall failure in org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns-compression

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-12811:
---
Status: Ready to Commit  (was: Patch Available)

> testall failure in 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns-compression
> 
>
> Key: CASSANDRA-12811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12811
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
>  Labels: test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.X_testall/34/testReport/org.apache.cassandra.cql3.validation.operations/DeleteTest/testDeleteWithOneClusteringColumns_compression/
> {code}
> Error Message
> Expected empty result but got 1 rows
> {code}
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: Expected empty result but got 1 rows
>   at org.apache.cassandra.cql3.CQLTester.assertEmpty(CQLTester.java:1089)
>   at 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:463)
>   at 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:427)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13276) Regression on CASSANDRA-11416: can't load snapshots of tables with dropped columns

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13276:

Reviewer: Alex Petrov

> Regression on CASSANDRA-11416: can't load snapshots of tables with dropped 
> columns
> --
>
> Key: CASSANDRA-13276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13276
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Matt Kopit
>Assignee: Andrés de la Peña
>
> I'm running Cassandra 3.10 and running into the exact same issue described in 
> CASSANDRA-11416: 
> 1. A table is created with columns 'a' and 'b'
> 2. Data is written to the table
> 3. Drop column 'b'
> 4. Take a snapshot
> 5. Drop the table
> 6. Run the snapshot schema.cql to recreate the table and the run the alter
> 7. Try to restore the snapshot data using sstableloader
> sstableloader yields the error:
> java.lang.RuntimeException: Unknown column b during deserialization



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-8272) 2ndary indexes can return stale data

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-8272:
--

Assignee: Andrés de la Peña

> 2ndary indexes can return stale data
> 
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
> Fix For: 2.1.x
>
>
> When replica return 2ndary index results, it's possible for a single replica 
> to return a stale result and that result will be sent back to the user, 
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are 
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before 
> having applied the insert, then the now stale result will be returned (since 
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and 
> provided we make the index inherit the gcGrace of it's parent CF), instead of 
> skipping that tombstone, we'd insert in the result a corresponding range 
> tombstone.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13276) Regression on CASSANDRA-11416: can't load snapshots of tables with dropped columns

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-13276:
---

Assignee: Andrés de la Peña  (was: Alex Petrov)

> Regression on CASSANDRA-11416: can't load snapshots of tables with dropped 
> columns
> --
>
> Key: CASSANDRA-13276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13276
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Matt Kopit
>Assignee: Andrés de la Peña
>
> I'm running Cassandra 3.10 and running into the exact same issue described in 
> CASSANDRA-11416: 
> 1. A table is created with columns 'a' and 'b'
> 2. Data is written to the table
> 3. Drop column 'b'
> 4. Take a snapshot
> 5. Drop the table
> 6. Run the snapshot schema.cql to recreate the table and the run the alter
> 7. Try to restore the snapshot data using sstableloader
> sstableloader yields the error:
> java.lang.RuntimeException: Unknown column b during deserialization



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12811) testall failure in org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns-compression

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956921#comment-15956921
 ] 

Benjamin Lerer commented on CASSANDRA-12811:


Nice work. +1

> testall failure in 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns-compression
> 
>
> Key: CASSANDRA-12811
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12811
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
>  Labels: test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.X_testall/34/testReport/org.apache.cassandra.cql3.validation.operations/DeleteTest/testDeleteWithOneClusteringColumns_compression/
> {code}
> Error Message
> Expected empty result but got 1 rows
> {code}
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: Expected empty result but got 1 rows
>   at org.apache.cassandra.cql3.CQLTester.assertEmpty(CQLTester.java:1089)
>   at 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:463)
>   at 
> org.apache.cassandra.cql3.validation.operations.DeleteTest.testDeleteWithOneClusteringColumns(DeleteTest.java:427)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13307) The specification of protocol version in cqlsh means the python driver doesn't automatically downgrade protocol version.

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956912#comment-15956912
 ] 

Joel Knighton commented on CASSANDRA-13307:
---

There's one you can set through the Edit button, if you scroll down. If you 
don't have permissions to access/edit that somehow, come complain in 
#cassandra-dev on IRC.

Thanks for volunteering to review!

> The specification of protocol version in cqlsh means the python driver 
> doesn't automatically downgrade protocol version.
> 
>
> Key: CASSANDRA-13307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Matt Byrd
>Assignee: Matt Byrd
>Priority: Minor
> Fix For: 3.11.x
>
>
> Hi,
> Looks like we've regressed on the issue described in:
> https://issues.apache.org/jira/browse/CASSANDRA-9467
> In that we're no longer able to connect from newer cqlsh versions
> (e.g trunk) to older versions of Cassandra with a lower version of the 
> protocol (e.g 2.1 with protocol version 3)
> The problem seems to be that we're relying on the ability for the client to 
> automatically downgrade protocol version implemented in Cassandra here:
> https://issues.apache.org/jira/browse/CASSANDRA-12838
> and utilised in the python client here:
> https://datastax-oss.atlassian.net/browse/PYTHON-240
> The problem however comes when we implemented:
> https://datastax-oss.atlassian.net/browse/PYTHON-537
> "Don't downgrade protocol version if explicitly set" 
> (included when we bumped from 3.5.0 to 3.7.0 of the python driver as part of 
> fixing: https://issues.apache.org/jira/browse/CASSANDRA-11534)
> Since we do explicitly specify the protocol version in the bin/cqlsh.py.
> I've got a patch which just adds an option to explicitly specify the protocol 
> version (for those who want to do that) and then otherwise defaults to not 
> setting the protocol version, i.e using the protocol version from the client 
> which we ship, which should by default be the same protocol as the server.
> Then it should downgrade gracefully as was intended. 
> Let me know if that seems reasonable.
> Thanks,
> Matt



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13307) The specification of protocol version in cqlsh means the python driver doesn't automatically downgrade protocol version.

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-13307:
--
Reviewer: mck

> The specification of protocol version in cqlsh means the python driver 
> doesn't automatically downgrade protocol version.
> 
>
> Key: CASSANDRA-13307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Matt Byrd
>Assignee: Matt Byrd
>Priority: Minor
> Fix For: 3.11.x
>
>
> Hi,
> Looks like we've regressed on the issue described in:
> https://issues.apache.org/jira/browse/CASSANDRA-9467
> In that we're no longer able to connect from newer cqlsh versions
> (e.g trunk) to older versions of Cassandra with a lower version of the 
> protocol (e.g 2.1 with protocol version 3)
> The problem seems to be that we're relying on the ability for the client to 
> automatically downgrade protocol version implemented in Cassandra here:
> https://issues.apache.org/jira/browse/CASSANDRA-12838
> and utilised in the python client here:
> https://datastax-oss.atlassian.net/browse/PYTHON-240
> The problem however comes when we implemented:
> https://datastax-oss.atlassian.net/browse/PYTHON-537
> "Don't downgrade protocol version if explicitly set" 
> (included when we bumped from 3.5.0 to 3.7.0 of the python driver as part of 
> fixing: https://issues.apache.org/jira/browse/CASSANDRA-11534)
> Since we do explicitly specify the protocol version in the bin/cqlsh.py.
> I've got a patch which just adds an option to explicitly specify the protocol 
> version (for those who want to do that) and then otherwise defaults to not 
> setting the protocol version, i.e using the protocol version from the client 
> which we ship, which should by default be the same protocol as the server.
> Then it should downgrade gracefully as was intended. 
> Let me know if that seems reasonable.
> Thanks,
> Matt



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12728) Handling partially written hint files

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956880#comment-15956880
 ] 

Aleksey Yeschenko commented on CASSANDRA-12728:
---

[~jjirsa] Looks good to me, though can you please extend the log message to 
include the name of the file that caused EOF? Thanks.

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Garvit Juniwal
>  Labels: lhf
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12728) Handling partially written hint files

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12728:
--
Status: Ready to Commit  (was: Patch Available)

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Garvit Juniwal
>  Labels: lhf
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12728) Handling partially written hint files

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12728:
--
Status: Patch Available  (was: Open)

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Garvit Juniwal
>  Labels: lhf
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-11830) "nodetool flush" not flushing system keyspace

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-11830.
-
Resolution: Duplicate

Closing as duplicate of CASSANDRA-13410.

> "nodetool flush" not flushing system keyspace
> -
>
> Key: CASSANDRA-11830
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11830
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Ubuntu 14.04, Oracle Java 8, Cassandra 2.2.6
>Reporter: Dominik Keil
>
> I'm regularly splitting off maintenance systems from our QA cluster by adding 
> a new node to a new "datacenter", joining it, then stopping and removing it 
> (adapting the schema before and after accordingly).
> In order to not mix up the systems I rename the cluster in the newly created 
> maintenance system by updating cluster_name in system.local
> In the past, when running "nodetool flush" then restarting Cassandra, this 
> worked as expected.
> However, this time it did not. After restarting Cassandra the old cluster 
> name was in place every time. However, explicitly flushing the system 
> keyspace using "nodetool flush system" did work as expected.
> Is this a bug or did the default behaviour of "nodetool flush" change at some 
> point?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13410) nodetool upgradesstables/scrub/compact ignores system tables

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956868#comment-15956868
 ] 

Paulo Motta commented on CASSANDRA-13410:
-

For the record this also affects 2.2 and was actually introduced by 
CASSANDRA-5483, as reported by CASSANDRA-11830, so I'm closing that as 
duplicate of this. But given 2.2 is nearly in critical-fixes-only mode, and 
would require a different patch I think it's fine to keep this 3.0+ only.

> nodetool upgradesstables/scrub/compact ignores system tables
> 
>
> Key: CASSANDRA-13410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13410
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-11627 changed the behavior of nodetool commands that work across 
> all keyspaces. Sometimes it's OK (not compacting system.peers when you call 
> compact probably isn't going to anger anyone), but sometimes it's not 
> (disableautocompaction, flush, upgradesstables, etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956832#comment-15956832
 ] 

Marcus Eriksson commented on CASSANDRA-13413:
-

my test failed quite badly: https://circleci.com/gh/krummas/cassandra/5

rerunning

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956829#comment-15956829
 ] 

Jason Brown commented on CASSANDRA-13413:
-

bq. not sure how predictable the performance is

Yeah, you are correct about that, and that's the best reason for not including 
it. I'm still +1 on the current patch on your branch

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956813#comment-15956813
 ] 

Marcus Eriksson commented on CASSANDRA-13413:
-

bq. Maybe add 'microbench' target somewhere?
would be nice, but not sure the place is in CI - we would need jmh to output 
some sort of junit xml test report, and fail builds based on the benchmark 
times I guess? And not sure how predictable the performance is in CircleCI if 
it actually gives us something (other than we make sure that the benchmarks can 
actually build/execute)

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13413) Run more test targets on CircleCI

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956786#comment-15956786
 ] 

Jason Brown commented on CASSANDRA-13413:
-

this is awesome! +1.

Maybe add 'microbench' target somewhere? Not sure how long it takes to execute, 
though...

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13413:

Description: Currently we only run {{ant test}} on circleci, we should use 
all the (free) containers we have and run more targets in parallel.  (was: 
Currently we only run `ant test` on circleci, we should use all the (free) 
containers we have and run more targets in parallel.)

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run {{ant test}} on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13413:

Status: Patch Available  (was: Open)

https://github.com/krummas/cassandra/commits/marcuse/testcircle

It runs {{ant eclipse-warnings; ant test}} if we only have one container 
configured, if we have 4 it also runs {{ant long-test}}, {{ant 
test-compression}} and {{ant stress-test}} on the different containers

https://circleci.com/gh/krummas/cassandra/5 (not finished when posting this, so 
might be broken)

> Run more test targets on CircleCI
> -
>
> Key: CASSANDRA-13413
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>
> Currently we only run `ant test` on circleci, we should use all the (free) 
> containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13339) java.nio.BufferOverflowException: null

[Chris Richards (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956733#comment-15956733
 ] 

Chris Richards edited comment on CASSANDRA-13339 at 4/5/17 12:33 PM:
-

Although I do have an occasional "nodetool repair" running, I've not observed 
any correlation between the occurrences of this exception and the running of 
the repair.

I've seen this on two boxes in a cluster - all but one instance was on one 
node, and this node is by far the busier of the two.  This is perhaps not 
surprising as it appears to be affected by timing, and the apply() function is 
being run in a worker thread.


was (Author: crichards):
Although I do have an occasional "nodetool repair" running, I've not observed 
any correlation between the occurrences of this exception and the running of 
the repair.

> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated 
> to 4.1.8.Final) running on a 2 node cluster.  It would have been processing 
> around 50 queries/second at the time (mixture of 
> inserts/updates/selects/deletes) : there's a collection of tables (some with 
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_121]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.9.jar:3.9]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  

[jira] [Created] (CASSANDRA-13413) Run more test targets on CircleCI

[Marcus Eriksson (JIRA)]

Marcus Eriksson created CASSANDRA-13413:
---

 Summary: Run more test targets on CircleCI
 Key: CASSANDRA-13413
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13413
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson


Currently we only run `ant test` on circleci, we should use all the (free) 
containers we have and run more targets in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13412) Update of column with TTL results in secondary index not returning row

[Enrique Bautista Barahona (JIRA)]

Enrique Bautista Barahona created CASSANDRA-13412:
-

 Summary: Update of column with TTL results in secondary index not 
returning row
 Key: CASSANDRA-13412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13412
 Project: Cassandra
  Issue Type: Bug
Reporter: Enrique Bautista Barahona


Cassandra versions: 2.2.3, 3.0.11
1 datacenter, keyspace has RF 3. Default consistency level.

Steps:
1. I create these table and index.
{code}
CREATE TABLE my_table (
a text,
b text,
c text,
d set,
e float,
f text,
g int,
h double,
j set,
k float,
m set,
PRIMARY KEY (a, b, c)
) WITH read_repair_chance = 0.0
   AND dclocal_read_repair_chance = 0.1
   AND gc_grace_seconds = 864000
   AND bloom_filter_fp_chance = 0.01
   AND caching = { 'keys' : 'ALL', 'rows_per_partition' : 'NONE' }
   AND comment = ''
   AND compaction = { 'class' : 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
   AND compression = { 'sstable_compression' : 
'org.apache.cassandra.io.compress.LZ4Compressor' }
   AND default_time_to_live = 0
   AND speculative_retry = '99.0PERCENTILE'
   AND min_index_interval = 128
   AND max_index_interval = 2048;
CREATE INDEX my_index ON my_table (c);
{code}
2. I have 9951 INSERT statements in a file and I run the following command to 
execute them. The INSERT statements have no TTL and no consistency level is 
specified.
{code}
cqlsh   -u  -f 
{code}
3. I update a column filtering by the whole primary key, and setting a TTL. For 
example:
{code}
UPDATE my_table USING TTL 30 SET h = 10 WHERE a = 'test_a' AND b = 'test_b' AND 
c = 'test_c';
{code}
4. After the time specified in the TTL I run the following queries:
{code}
SELECT * FROM my_table WHERE a = 'test_a' AND b = 'test_b' AND c = 'test_c';
SELECT * FROM my_table WHERE c = 'test_c';
{code}
The first one returns the correct row with an empty h column (as it has 
expired). However, the second query (which uses the secondary index on column 
c) returns nothing.

I've done the query through my app which uses the Java driver v3.0.4 and reads 
with CL local_one, from the cql shell and from DBeaver 3.8.5. All display the 
same behaviour. The queries are performed minutes after the writes and the 
servers don't have a high load, so I think it's unlikely to be a consistency 
issue.

I've tried to reproduce the issue in ccm and cqlsh by creating a new keyspace 
and table, and inserting just 1 row, and the bug doesn't manifest. This leads 
me to think that it's an issue only present with not trivially small amounts of 
data, or maybe present only after Cassandra compacts or performs whatever 
maintenance it needs to do.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956759#comment-15956759
 ] 

Jason Brown commented on CASSANDRA-8457:


belatedly adding support for ssl hostname verification (CASSANDRA-9220). thanks 
to the sslnodetonode_test.py for finding this (that dtest will need an update, 
as well)

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13339) java.nio.BufferOverflowException: null

[Chris Richards (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956733#comment-15956733
 ] 

Chris Richards commented on CASSANDRA-13339:


Although I do have an occasional "nodetool repair" running, I've not observed 
any correlation between the occurrences of this exception and the running of 
the repair.

> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated 
> to 4.1.8.Final) running on a 2 node cluster.  It would have been processing 
> around 50 queries/second at the time (mixture of 
> inserts/updates/selects/deletes) : there's a collection of tables (some with 
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_121]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.9.jar:3.9]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> 

[jira] [Commented] (CASSANDRA-13339) java.nio.BufferOverflowException: null

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956719#comment-15956719
 ] 

Jonas Borgström commented on CASSANDRA-13339:
-

Some further observations. So far this only seems to happen on nodes I run 
"nodetool repair --full" on. Even though the repair increases load on all nodes 
I only see this issue on the node I started the repair command on.

> java.nio.BufferOverflowException: null
> --
>
> Key: CASSANDRA-13339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13339
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Chris Richards
>
> I'm seeing the following exception running Cassandra 3.9 (with Netty updated 
> to 4.1.8.Final) running on a 2 node cluster.  It would have been processing 
> around 50 queries/second at the time (mixture of 
> inserts/updates/selects/deletes) : there's a collection of tables (some with 
> counters some without) and a single materialized view.
> ERROR [MutationStage-4] 2017-03-15 22:50:33,052 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:380)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:122)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serialize(PartitionUpdate.java:790)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.serialize(Mutation.java:393)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:279) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:241) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1347)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2539)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_121]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
>  [apache-cassandra-3.9.jar:3.9]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.9.jar:3.9]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> and then again shortly afterwards
> ERROR [MutationStage-3] 2017-03-15 23:27:36,198 StorageProxy.java:1353 - 
> Failed to apply mutation locally : {}
> java.nio.BufferOverflowException: null
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.writeUnsignedVInt(BufferedDataOutputStreamPlus.java:262)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.rows.EncodingStats$Serializer.serialize(EncodingStats.java:233)
>  

[jira] [Created] (CASSANDRA-13411) CQL query using the MAX function returns resultset with Row(null, null ...) if data is not found

[Andrew Efimov (JIRA)]

Andrew Efimov created CASSANDRA-13411:
-

 Summary: CQL query using the MAX function returns resultset with 
Row(null, null ...) if data is not found
 Key: CASSANDRA-13411
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13411
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: 3.10
Reporter: Andrew Efimov
Priority: Minor


CQL query using the MAX function returns resultset with rows.size=1 if data is 
not found. And Row has only null values.
{{SELECT id, value, MAX(date) FROM table WHERE id = "1"}}
If table does not have row by {{id = "1"}} then session returns ResultSet with 
Rows.size = 1 and Row(null, null, null).
This is a problem to determine whether or not a data has actually been exist.
I did not check other aggregation functions.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13411) CQL query using the MAX function returns resultset with Row(null, null ...) if data is not found

[Andrew Efimov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Efimov updated CASSANDRA-13411:
--
Description: 
CQL query using the MAX function returns resultset with rows.size=1 if data is 
not found. And Row has only null values.
{{SELECT id, value, MAX(date) FROM table WHERE id = "13411"}}
If table does not have row by {{id = "13411"}} then session returns ResultSet 
with Rows.size = 1 and Row(null, null, null).
This is a problem to determine whether or not a data has actually been exist.
I did not check other aggregation functions.


  was:
CQL query using the MAX function returns resultset with rows.size=1 if data is 
not found. And Row has only null values.
{{SELECT id, value, MAX(date) FROM table WHERE id = "1"}}
If table does not have row by {{id = "1"}} then session returns ResultSet with 
Rows.size = 1 and Row(null, null, null).
This is a problem to determine whether or not a data has actually been exist.
I did not check other aggregation functions.



> CQL query using the MAX function returns resultset with Row(null, null ...) 
> if data is not found
> 
>
> Key: CASSANDRA-13411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13411
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.10
>Reporter: Andrew Efimov
>Priority: Minor
>
> CQL query using the MAX function returns resultset with rows.size=1 if data 
> is not found. And Row has only null values.
> {{SELECT id, value, MAX(date) FROM table WHERE id = "13411"}}
> If table does not have row by {{id = "13411"}} then session returns ResultSet 
> with Rows.size = 1 and Row(null, null, null).
> This is a problem to determine whether or not a data has actually been exist.
> I did not check other aggregation functions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13410) nodetool upgradesstables/scrub/compact ignores system tables

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956635#comment-15956635
 ] 

Marcus Eriksson commented on CASSANDRA-13410:
-

+1

> nodetool upgradesstables/scrub/compact ignores system tables
> 
>
> Key: CASSANDRA-13410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13410
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-11627 changed the behavior of nodetool commands that work across 
> all keyspaces. Sometimes it's OK (not compacting system.peers when you call 
> compact probably isn't going to anger anyone), but sometimes it's not 
> (disableautocompaction, flush, upgradesstables, etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13410) nodetool upgradesstables/scrub/compact ignores system tables

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13410:

Status: Ready to Commit  (was: Patch Available)

> nodetool upgradesstables/scrub/compact ignores system tables
> 
>
> Key: CASSANDRA-13410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13410
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-11627 changed the behavior of nodetool commands that work across 
> all keyspaces. Sometimes it's OK (not compacting system.peers when you call 
> compact probably isn't going to anger anyone), but sometimes it's not 
> (disableautocompaction, flush, upgradesstables, etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13410) nodetool upgradesstables/scrub/compact ignores system tables

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13410:

Reviewer: Marcus Eriksson

> nodetool upgradesstables/scrub/compact ignores system tables
> 
>
> Key: CASSANDRA-13410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13410
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> CASSANDRA-11627 changed the behavior of nodetool commands that work across 
> all keyspaces. Sometimes it's OK (not compacting system.peers when you call 
> compact probably isn't going to anger anyone), but sometimes it's not 
> (disableautocompaction, flush, upgradesstables, etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13404) Hostname verification for client-to-node encryption

[Jan Karlsson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956604#comment-15956604
 ] 

Jan Karlsson commented on CASSANDRA-13404:
--

{quote}
To back up and add a bit more context (for myself, if anything), where do you 
want to add the additional hostname verification? Can you explain the specific 
behaviour you're looking to add? 
{quote}
The behaviour I am trying to add is that the server validates that the client 
certificate is issued to the IP address/host that the client connects from. You 
are correct that this would require require_client_auth to be set as this will 
ensure that the server validates the client to begin with. Disabling 
require_client_auth while enabling hostname verification will actually not do 
anything. We wont validate anything. Do you think we should add a warning 
during startup that you cannot have hostname validation without requiring 
validation?
{quote}
Further, this would require the database server to know all of the possible 
peers that would want to connect to it, before the process starts.
{quote}
Not necessarily, I take the incoming connection, extract the IP, then the 
identification algorithm checks whether the SAN in the certificate holds this 
IP address.
{quote}
Also, I've spoken with the netty developers, and they said netty currently does 
not support (in either netty 4.0 or 4.1) the ability to perform hostname 
verification on the server side (either openssl or jdk ssl). Thus, I'm not sure 
how you verified your patch behaves correctly.
{quote}
I used the java driver and added [Netty Options | 
http://docs.datastax.com/en/drivers/java/2.1/com/datastax/driver/core/NettyOptions.html#afterBootstrapInitialized-io.netty.bootstrap.Bootstrap-]
 to change the local address in afterBootstrapInitialized. This allows me to 
change what interface I use to connect to C*. Then I used a certificate I had 
forged for a different interface and tested to connect to a node. Worked like a 
charm. I then applied my patch and got a exception on both the server and the 
client side. Lastly i switched which IP address I connected from to the 
interface that was specified in the certificate and the exceptions disappeared.

> Hostname verification for client-to-node encryption
> ---
>
> Key: CASSANDRA-13404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13404
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
> Fix For: 4.x
>
> Attachments: 13404-trunk.txt
>
>
> Similarily to CASSANDRA-9220, Cassandra should support hostname verification 
> for client-node connections.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13321) Add a checksum component for the sstable metadata (-Statistics.db) file

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956443#comment-15956443
 ] 

Marcus Eriksson edited comment on CASSANDRA-13321 at 4/5/17 7:42 AM:
-

I made an alternative implementation of this: 
https://github.com/krummas/cassandra/commits/marcuse/appendedchecksum which 
just appends the checksum within the same file, much simpler but does not solve 
all the issues that the versioned component patch above solves, but feels much 
safer. We still need to rename the file on top of the old file using this 
patch, so not sure if the risk of the old patch is worth it, wdyt [~jasobrown]?

Also note that we don't read the whole -Statistics.db file, instead we read the 
individual components within the file (validation metadata, compaction metadata 
etc), so each component has its own checksum in this patch


was (Author: krummas):
I made an alternative implementation of this: 
https://github.com/krummas/cassandra/commits/marcuse/appendedchecksum which 
just appends the checksum within the same file, much simpler but does not solve 
all the issues that the versioned component patch above solves, but feels much 
safer. We still need to rename the file on top of the old file using this 
patch, so not sure if the risk of the old patch is worth it, wdyt [~jasobrown]?

> Add a checksum component for the sstable metadata (-Statistics.db) file
> ---
>
> Key: CASSANDRA-13321
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13321
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> Since we keep important information in the sstable metadata file now, we 
> should add a checksum component for it. One danger being if a bit gets 
> flipped in repairedAt we could consider the sstable repaired when it is not.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13307) The specification of protocol version in cqlsh means the python driver doesn't automatically downgrade protocol version.

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956455#comment-15956455
 ] 

mck commented on CASSANDRA-13307:
-

Am taking a look at this ([~tjake] you're too slow and the Caribbean is no 
excuse :-)

I'm not sure if there's a jira field i'm supposed put my name in as a/the 
reviewer, [~jjirsa]?

> The specification of protocol version in cqlsh means the python driver 
> doesn't automatically downgrade protocol version.
> 
>
> Key: CASSANDRA-13307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13307
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Matt Byrd
>Assignee: Matt Byrd
>Priority: Minor
> Fix For: 3.11.x
>
>
> Hi,
> Looks like we've regressed on the issue described in:
> https://issues.apache.org/jira/browse/CASSANDRA-9467
> In that we're no longer able to connect from newer cqlsh versions
> (e.g trunk) to older versions of Cassandra with a lower version of the 
> protocol (e.g 2.1 with protocol version 3)
> The problem seems to be that we're relying on the ability for the client to 
> automatically downgrade protocol version implemented in Cassandra here:
> https://issues.apache.org/jira/browse/CASSANDRA-12838
> and utilised in the python client here:
> https://datastax-oss.atlassian.net/browse/PYTHON-240
> The problem however comes when we implemented:
> https://datastax-oss.atlassian.net/browse/PYTHON-537
> "Don't downgrade protocol version if explicitly set" 
> (included when we bumped from 3.5.0 to 3.7.0 of the python driver as part of 
> fixing: https://issues.apache.org/jira/browse/CASSANDRA-11534)
> Since we do explicitly specify the protocol version in the bin/cqlsh.py.
> I've got a patch which just adds an option to explicitly specify the protocol 
> version (for those who want to do that) and then otherwise defaults to not 
> setting the protocol version, i.e using the protocol version from the client 
> which we ship, which should by default be the same protocol as the server.
> Then it should downgrade gracefully as was intended. 
> Let me know if that seems reasonable.
> Thanks,
> Matt



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13321) Add a checksum component for the sstable metadata (-Statistics.db) file

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956443#comment-15956443
 ] 

Marcus Eriksson commented on CASSANDRA-13321:
-

I made an alternative implementation of this: 
https://github.com/krummas/cassandra/commits/marcuse/appendedchecksum which 
just appends the checksum within the same file, much simpler but does not solve 
all the issues that the versioned component patch above solves, but feels much 
safer. We still need to rename the file on top of the old file using this 
patch, so not sure if the risk of the old patch is worth it, wdyt [~jasobrown]?

> Add a checksum component for the sstable metadata (-Statistics.db) file
> ---
>
> Key: CASSANDRA-13321
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13321
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> Since we keep important information in the sstable metadata file now, we 
> should add a checksum component for it. One danger being if a bit gets 
> flipped in repairedAt we could consider the sstable repaired when it is not.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956442#comment-15956442
 ] 

Hao Zhong commented on CASSANDRA-13358:
---

I modified the code according to the Cassandra style. Although the original 
code has an if check, it can avoid only NPE, not InvalidRequestException. As a 
result, the patch is still necessary. 

> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 2.1.18
>
> Attachments: cassandra.patch
>
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Zhong updated CASSANDRA-13358:
--
Attachment: cassandra.patch

> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 2.1.18
>
> Attachments: cassandra.patch
>
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Issue Comment Deleted] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Zhong updated CASSANDRA-13358:
--
Comment: was deleted

(was: diff --git 
a/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
index fbfc54c..e0ad630 100644
--- a/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
@@ -46,8 +46,15 @@
 public void checkAccess(ClientState state) throws UnauthorizedException, 
InvalidRequestException
 {
 TableMetadataRef baseTable = View.findBaseTable(keyspace(), 
columnFamily());
-if (baseTable != null)
-state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
Permission.ALTER);
+try
+{
+   if (baseTable != null)
+   state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
Permission.ALTER);
+}
+catch (InvalidRequestException e)
+{
+throw e;
+}
 }
 
 public void validate(ClientState state)
)

> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 2.1.18
>
> Attachments: cassandra.patch
>
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Zhong updated CASSANDRA-13358:
--
Status: Patch Available  (was: Open)

diff --git 
a/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
index fbfc54c..e0ad630 100644
--- a/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/AlterViewStatement.java
@@ -46,8 +46,15 @@
 public void checkAccess(ClientState state) throws UnauthorizedException, 
InvalidRequestException
 {
 TableMetadataRef baseTable = View.findBaseTable(keyspace(), 
columnFamily());
-if (baseTable != null)
-state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
Permission.ALTER);
+try
+{
+   if (baseTable != null)
+   state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
Permission.ALTER);
+}
+catch (InvalidRequestException e)
+{
+throw e;
+}
 }
 
 public void validate(ClientState state)


> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 2.1.18
>
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Zhong updated CASSANDRA-13358:
--
Fix Version/s: 2.1.18
   Status: Open  (was: Patch Available)

> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
> Fix For: 2.1.18
>
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13358) AlterViewStatement.checkAccess can throw exceptions

[Hao Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hao Zhong updated CASSANDRA-13358:
--
Attachment: (was: cassandra.patch)

> AlterViewStatement.checkAccess can throw exceptions
> ---
>
> Key: CASSANDRA-13358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13358
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hao Zhong
>Assignee: Hao Zhong
>
> The AlterViewStatement.checkAccess method has code lines as follow:
> {code:title=AlterViewStatement.java|borderStyle=solid}
>   if (baseTable != null)
> state.hasColumnFamilyAccess(keyspace(), baseTable.name, 
> Permission.ALTER);
> {code}
> These lines can throw InvalidRequestException. Indeed, 
> DropTableStatement.checkAccess has a similar problem, and was fixed in 
> CASSANDRA-6687. The fixed code is as follow:
> {code:title=DropTableStatement.java|borderStyle=solid}
>  try
> {
> state.hasColumnFamilyAccess(keyspace(), columnFamily(), 
> Permission.DROP);
> }
> catch (InvalidRequestException e)
> {
> if (!ifExists)
> throw e;
> }
> {code}
> Please fix the problem as CASSANDRA-6687 did.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)