[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392525#comment-16392525
 ] 

mck commented on CASSANDRA-11163:
-


|| branch || testall || dtest ||
| [14166-3.0|https://github.com/kgreav/cassandra/tree/14166-3.0]| 
[testall|https://circleci.com/gh/kgreav/cassandra/tree/14166-3.0] | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/506/]
 |
| [14166-3.11|https://github.com/kgreav/cassandra/tree/14166-3.11]  | 
[testall|https://circleci.com/gh/kgreav/cassandra/tree/14166-3.11]| 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/507]
 |
| [14166-trunk|https://github.com/kgreav/cassandra/tree/14166-trunk]| 
[testall|https://circleci.com/gh/kgreav/cassandra/tree/14166-trunk]   | 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/508]
 |

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> 

[jira] [Commented] (CASSANDRA-13121) repair progress message breaks legacy JMX support

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392356#comment-16392356
 ] 

Patrick Bannister commented on CASSANDRA-13121:
---

Would you please assign this issue to me? I'm finishing up a fix and a dtest 
for this issue, and I should have a patch soon for 3.11.2.

> repair progress message breaks legacy JMX support
> -
>
> Key: CASSANDRA-13121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13121
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Scott Bale
>Priority: Minor
>  Labels: lhf
>
> The error progress message in {{RepairRunnable}} is not compliant with the 
> {{LegacyJMXProgressSupport}} class, which uses a regex to match on the text 
> of a progress event. Therefore, actual failures slip through as successes if 
> using legacy JMX for repairs.
> In {{RepairRunnable}}
> {code}
> protected void fireErrorAndComplete(String tag, int progressCount, int 
> totalProgress, String message)
> {
> fireProgressEvent(tag, new ProgressEvent(ProgressEventType.ERROR, 
> progressCount, totalProgress, message));
> fireProgressEvent(tag, new ProgressEvent(ProgressEventType.COMPLETE, 
> progressCount, totalProgress, String.format("Repair command #%d finished with 
> error", cmd)));
> }
> {code}
> Note the {{"Repair command #%d finished with error"}}
> See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairRunnable.java#L109
> In {{LegacyJMXProgressSupport}}:
> {code}
> protected static final Pattern SESSION_FAILED_MATCHER = 
> Pattern.compile("Repair session .* for range .* failed with error .*");
> protected static final Pattern SESSION_SUCCESS_MATCHER = 
> Pattern.compile("Repair session .* for range .* finished");
> {code}
> See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/progress/jmx/LegacyJMXProgressSupport.java#L38
> Legacy JMX support was introduced for CASSANDRA-11430 (version 2.2.6) and the 
> bug was introduced as part of CASSANDRA-12279 (version 2.2.8).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14303) NetworkTopologyStrategy could have a "default replication" option

[Joseph Lynch (JIRA)]

Joseph Lynch created CASSANDRA-14303:


 Summary: NetworkTopologyStrategy could have a "default 
replication" option
 Key: CASSANDRA-14303
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14303
 Project: Cassandra
  Issue Type: Improvement
  Components: Configuration
Reporter: Joseph Lynch


Right now when creating a keyspace with {{NetworkTopologyStrategy}} the user 
has to manually specify the datacenters they want their data replicated to with 
parameters, e.g.:
{noformat}
 CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'dc1': 3, 'dc2': 3}{noformat}
This is a poor user interface because it requires the creator of the keyspace 
(typically a developer) to know the layout of the Cassandra cluster (which may 
or may not be controlled by them). Also, at least in my experience, folks typo 
the datacenters _all_ the time. To work around this I see a number of users 
creating automation around this where the automation describes the Cassandra 
cluster and automatically expands out to all the dcs that Cassandra knows 
about. Why can't Cassandra just do this for us, re-using the previously 
forbidden {{replication_factor}} option (for backwards compatibility):
{noformat}
 CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'replication_factor': 3}{noformat}
This would automatically replicate this Keyspace to all datacenters that are 
present in the cluster. If you need to _override_ the default you could supply 
a datacenter name, e.g.:
{noformat}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'replication_factor': 3, 'dc1': 0}
{noformat}
On the implementation side I think this may be reasonably straightforward to do 
an auto-expansion at the time of keyspace creation (or alter), where the above 
would automatically expand to list out the datacenters. We could allow this to 
be recomputed whenever an AlterKeyspaceStatement runs so that to add 
datacenters you would just run:
{noformat}
ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'replication_factor': 3}{noformat}
and this would check that if the dc's in the current schema are different you 
add in the new ones. Removing a datacenter becomes a two step process, e.g. if 
we wanted to remove {{dc1}} we would do:
{noformat}
// First tell it not to replicate to dc1
ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'replication_factor': 3, 'dc1': 0}
// Remove all nodes from dc1
ALTER KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'replication_factor': 3}{noformat}
I think the only issue with this would be that I think {{EACH_QUORUM}} doesn't 
handle DCs with 0 replicas very well, but I think that is tractable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392190#comment-16392190
 ] 

Jon Haddad edited comment on CASSANDRA-8460 at 3/9/18 12:56 AM:


I'll be honest, whenever I need to do performance testing, the last thing I 
reach for is stress, because I can't wrap my head around configuring it right.  
It's probably easier to create a ~50 line program to do the inserts.  I'll try 
to throw something together tomorrow.

Ultimately if this is going to benefit TWCS, it *has* to be tested with it, so 
we might as well do that up front.

It's the end of the day, and I'm not an expert in setting up lvmcache, so I'll 
have to try it out tomorrow as well. 

It'll be helpful to do some normal disk benchmarks as well to be sure it's set 
up right, probably using fio.  I'll include that.


was (Author: rustyrazorblade):
I'll be honest, whenever I need to do performance testing, the last thing I 
reach for is stress, because I can't wrap my head around configuring it right.  
It's probably easier to create a ~50 line program to do the inserts.  I'll try 
to throw something together tomorrow.

Ultimately if this is going to benefit TWCS, it *has* to be tested with it, so 
we might as well do that up front.

It's the end of the day, and I'm not an expert in setting up lvmcache, so I'll 
have to try it out tomorrow as well. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392190#comment-16392190
 ] 

Jon Haddad commented on CASSANDRA-8460:
---

I'll be honest, whenever I need to do performance testing, the last thing I 
reach for is stress, because I can't wrap my head around configuring it right.  
It's probably easier to create a ~50 line program to do the inserts.  I'll try 
to throw something together tomorrow.

Ultimately if this is going to benefit TWCS, it *has* to be tested with it, so 
we might as well do that up front.

It's the end of the day, and I'm not an expert in setting up lvmcache, so I'll 
have to try it out tomorrow as well. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13529) cassandra-stress light-weight transaction support

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392161#comment-16392161
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13529:
---

Hi [~djoshi3]

Thank You! for reviewing this code.
I've incorporated most of the comments. 
For the following comment:
{quote}
There is a lot that was added to SchemaQuery. It would be beneficial if you add 
two classes CASQueryStatic and CASQueryDynamic inheriting from a new class 
called CASQuery.

{quote}
I didn't create base-class {{CasQuery}} derive-class {{CASQueryStatic, 
CASQueryDynamic}} because {{CASQueryStatic}} is same as any other query as it 
doesn't require special treatment. Only for {{CASQueryDynamic}} we need to 
fetch current db values and then replace before doing actual operation. But I 
got your concern hence I have created a new class for {{CASQueryDynamic}} and 
moved majority of code from {{SchemaQuery}} -->  {{CASQueryDynamic}} to keep 
mainline simple. Please review this update patch and let me know your feedback.
|| trunk ||
| [patch| 
https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:CASSANDRA-13529?expand=1]
 |
| [uTest| https://circleci.com/gh/jaydeepkumar1984/cassandra/37] |

Jaydeep

> cassandra-stress light-weight transaction support
> -
>
> Key: CASSANDRA-13529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13529
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Stress
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 13529.txt, lwttest.yaml
>
>
> It would be nice to have a light-weight transaction support in 
> cassandra-stress.
> Although currently in cassandra-stress we can achieve light-weight 
> transaction partially by using static conditions like "IF col1 != null" or 
> "IF not EXIST". 
> If would be ideal to have full fledged light-weight transaction support like 
> "IF col1 = ? and col2 = ?". One way to implement is to read values from 
> Cassandra and use that in the condition so it will execute all the paxos 
> phases in Cassandra.
> Please find git link for the patch: 
> https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13529-trunk?expand=1
> ||trunk|
> |[branch|https://github.com/jaydeepkumar1984/cassandra/tree/13529-trunk]|
> |[utests|https://circleci.com/gh/jaydeepkumar1984/cassandra/8]|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13457) Diag. Events: Add base classes

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392139#comment-16392139
 ] 

mck edited comment on CASSANDRA-13457 at 3/9/18 12:07 AM:
--

[~spo...@gmail.com],
{quote}`diagnostic_events_enabled: true` … 
[13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb]
{quote}
+1
{quote}`DiagnosticEvent.addIfNotNull(..)` … 
[5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30]
{quote}
+1
{quote}What's the latest trend handling this?
{quote}
I see more constructor IoC happening in places, the new  streaming patches in 
4.0 for example.
 I guess it's mainly about avoiding the static functionality, keeping static 
down to the `instance` field.

`MessagingService.instance()` does it well, in that it also does the 
lazy-singleton-initialisation approach via the use of the `MSHandle` inner 
static class.

I had a question about whether we can/should go further and separate the 
concerns (bean vs function) in the events classes, in this 
[comment|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488].
{quote}The {{GossiperEvent}} will access these members from the constructor to 
collect relevant details for the event.
{quote}
Gotcha, I missed that, thanks.
{quote}I'd not serialize any internal classes, as we don't have any versioning 
in place and classes could change quickly enough to cause deserialization 
issues, even on bugfix releases.
{quote}
To clarify, I didn't mean serialising classes. I was only thinking that the 
event classes could pair the `Map toMap(..)` method with a 
'MyEvent fromString(..)` as a convenience to anyone in the java space. But I 
agree, it's premature and would only cause compatibility headaches.
{quote} That's also the reason I'd prefer to keep classes public, so they can 
be used for unit testing.
{quote}
So long as the unit test is also in the same package the class can be 
package-protected. 
{quote}After some basic testing I quickly realized that way too much events 
would be produced, if you happen to subscribe to them. 
{quote}
You're quite right. The frequency of calls increases diagnostics, to tracing, 
to metrics. What's written in that paragraph Stefan would make good dev 
documentation about when and how to use each throughout the codebase.

The point that "would enable/disable repair tracing through the 
{{diagnostic_events_enabled}} flag" is enough to warrant not worrying about any 
attempts at pairing events at this point in time. Maybe it's part of a bigger 
exercise later looking for what existing trace events should also be diagnostic 
events.


was (Author: michaelsembwever):
[~spo...@gmail.com],

 
{quote}`diagnostic_events_enabled: true`

[13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb]
{quote}
+1

 
{quote}`DiagnosticEvent.addIfNotNull(..)`

[5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30]
{quote}
+1

 
{quote}What's the latest trend handling this?
{quote}
I see more constructor IoC happening in places, the new netty streaming patches 
in 4.0 for example.
I guess it's mainly about avoiding the static functionality, keeping static 
down to the `instance` field.

`MessagingService.instance()` does it well, in that it also does the 
lazy-singleton-initialisation approach via the use of the `MSHandle` inner 
static class.

I had a question about whether we can/should go further and separate the 
concerns (bean vs function) in the events classes, in this 
[comment|[https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488]|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488].]
 

 
{quote}The {{GossiperEvent}} will access these members from the constructor to 
collect relevant details for the event.
{quote}
Gotcha, I missed that, thanks.

 
{quote}I'd not serialize any internal classes, as we don't have any versioning 
in place and classes could change quickly enough to cause deserialization 
issues, even on bugfix releases.
{quote}
To clarify, I didn't mean serialising classes. I was only thinking that the 
event classes could pair the `Map toMap(..)` method with a 
'MyEvent fromString(..)` as a convenience to anyone in the java space. But I 
agree, it's premature and would only cause compatibility headaches.

 
{quote} That's also the reason I'd prefer to keep classes public, so they can 
be used for unit testing.
{quote}
So long as the unit test is also in the same package the class can be 
package-protected.

 
{quote}After some basic testing I quickly realized that way too much events 
would be produced, if you happen to subscribe to them. 
{quote}
You're quite right. The frequency of calls increases 

[jira] [Comment Edited] (CASSANDRA-13457) Diag. Events: Add base classes

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392139#comment-16392139
 ] 

mck edited comment on CASSANDRA-13457 at 3/9/18 12:07 AM:
--

[~spo...@gmail.com],
{quote}`diagnostic_events_enabled: true` … 
[13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb]
{quote}
+1
{quote}`DiagnosticEvent.addIfNotNull(..)` … 
[5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30]
{quote}
+1
{quote}What's the latest trend handling this?
{quote}
I see more constructor IoC happening in places, the new  streaming patches in 
4.0 for example.
 I guess it's mainly about avoiding the static functionality, keeping static 
down to the `instance` field.

`MessagingService.instance()` does it well, in that it also does the 
lazy-singleton-initialisation approach via the use of the `MSHandle` inner 
static class.

I had a question about whether we can/should go further and separate the 
concerns (bean vs function) in the events classes, in this 
[comment|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488].
{quote}The {{GossiperEvent}} will access these members from the constructor to 
collect relevant details for the event.
{quote}
Gotcha, I missed that, thanks.
{quote}I'd not serialize any internal classes, as we don't have any versioning 
in place and classes could change quickly enough to cause deserialization 
issues, even on bugfix releases.
{quote}
To clarify, I didn't mean serialising classes. I was only thinking that the 
event classes could pair the `Map toMap(..)` method with a 
'MyEvent fromString(..)` as a convenience to anyone in the java space. But I 
agree, it's premature and would only cause compatibility headaches.
{quote} That's also the reason I'd prefer to keep classes public, so they can 
be used for unit testing.
{quote}
So long as the unit test is also in the same package the class can be 
package-protected. 
{quote}After some basic testing I quickly realized that way too much events 
would be produced, if you happen to subscribe to them. 
{quote}
You're quite right. The frequency of calls increases diagnostics, to tracing, 
to metrics. What's written in that paragraph Stefan would make good dev 
documentation about when and how to use each throughout the codebase.

The point you raise that "would enable/disable repair tracing through the 
{{diagnostic_events_enabled}} flag" is enough imo to warrant not worrying about 
any attempts at pairing events at this point in time. Maybe it's part of a 
bigger exercise later looking for what existing trace events should also be 
diagnostic events.


was (Author: michaelsembwever):
[~spo...@gmail.com],
{quote}`diagnostic_events_enabled: true` … 
[13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb]
{quote}
+1
{quote}`DiagnosticEvent.addIfNotNull(..)` … 
[5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30]
{quote}
+1
{quote}What's the latest trend handling this?
{quote}
I see more constructor IoC happening in places, the new  streaming patches in 
4.0 for example.
 I guess it's mainly about avoiding the static functionality, keeping static 
down to the `instance` field.

`MessagingService.instance()` does it well, in that it also does the 
lazy-singleton-initialisation approach via the use of the `MSHandle` inner 
static class.

I had a question about whether we can/should go further and separate the 
concerns (bean vs function) in the events classes, in this 
[comment|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488].
{quote}The {{GossiperEvent}} will access these members from the constructor to 
collect relevant details for the event.
{quote}
Gotcha, I missed that, thanks.
{quote}I'd not serialize any internal classes, as we don't have any versioning 
in place and classes could change quickly enough to cause deserialization 
issues, even on bugfix releases.
{quote}
To clarify, I didn't mean serialising classes. I was only thinking that the 
event classes could pair the `Map toMap(..)` method with a 
'MyEvent fromString(..)` as a convenience to anyone in the java space. But I 
agree, it's premature and would only cause compatibility headaches.
{quote} That's also the reason I'd prefer to keep classes public, so they can 
be used for unit testing.
{quote}
So long as the unit test is also in the same package the class can be 
package-protected. 
{quote}After some basic testing I quickly realized that way too much events 
would be produced, if you happen to subscribe to them. 
{quote}
You're quite right. The frequency of calls increases diagnostics, to tracing, 
to metrics. What's written in that paragraph Stefan would make good dev 
documentation 

[jira] [Commented] (CASSANDRA-13457) Diag. Events: Add base classes

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392139#comment-16392139
 ] 

mck commented on CASSANDRA-13457:
-

[~spo...@gmail.com],

 
{quote}`diagnostic_events_enabled: true`

[13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb]
{quote}
+1

 
{quote}`DiagnosticEvent.addIfNotNull(..)`

[5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30]
{quote}
+1

 
{quote}What's the latest trend handling this?
{quote}
I see more constructor IoC happening in places, the new netty streaming patches 
in 4.0 for example.
I guess it's mainly about avoiding the static functionality, keeping static 
down to the `instance` field.

`MessagingService.instance()` does it well, in that it also does the 
lazy-singleton-initialisation approach via the use of the `MSHandle` inner 
static class.

I had a question about whether we can/should go further and separate the 
concerns (bean vs function) in the events classes, in this 
[comment|[https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488]|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21#r28005488].]
 

 
{quote}The {{GossiperEvent}} will access these members from the constructor to 
collect relevant details for the event.
{quote}
Gotcha, I missed that, thanks.

 
{quote}I'd not serialize any internal classes, as we don't have any versioning 
in place and classes could change quickly enough to cause deserialization 
issues, even on bugfix releases.
{quote}
To clarify, I didn't mean serialising classes. I was only thinking that the 
event classes could pair the `Map toMap(..)` method with a 
'MyEvent fromString(..)` as a convenience to anyone in the java space. But I 
agree, it's premature and would only cause compatibility headaches.

 
{quote} That's also the reason I'd prefer to keep classes public, so they can 
be used for unit testing.
{quote}
So long as the unit test is also in the same package the class can be 
package-protected.

 
{quote}After some basic testing I quickly realized that way too much events 
would be produced, if you happen to subscribe to them. 
{quote}
You're quite right. The frequency of calls increases diagnostics, to tracing, 
to metrics. What's written in that paragraph Stefan would make good dev 
documentation about when and how to use each throughout the codebase.

The point that "would enable/disable repair tracing through the 
{{diagnostic_events_enabled}} flag" is enough to warrant not worrying about any 
attempts at pairing events at this point in time. Maybe it's part of a bigger 
exercise later looking for what existing trace events should also be diagnostic 
events.

> Diag. Events: Add base classes
> --
>
> Key: CASSANDRA-13457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13457
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core, Observability
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> Base ticket for adding classes that will allow you to implement and subscribe 
> to events.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13459) Diag. Events: Native transport integration

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392133#comment-16392133
 ] 

mck edited comment on CASSANDRA-13459 at 3/8/18 11:59 PM:
--

BTW the patch is 
[here|https://github.com/spodkowinski/cassandra/compare/WIP-13457...spodkowinski:WIP-13459].


was (Author: michaelsembwever):
BTW the patch is 
[here|https://github.com/spodkowinski/cassandra/blob/WIP-13459/].

> Diag. Events: Native transport integration
> --
>
> Key: CASSANDRA-13459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13459
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: client-impacting
>
> Events should be consumable by clients that would received subscribed events 
> from the connected node. This functionality is designed to work on top of 
> native transport with minor modifications to the protocol standard (see 
> [original 
> proposal|https://docs.google.com/document/d/1uEk7KYgxjNA0ybC9fOuegHTcK3Yi0hCQN5nTp5cNFyQ/edit?usp=sharing]
>  for further considered options). First we have to add another value for 
> existing event types. Also, we have to extend the protocol a bit to be able 
> to specify a sub-class and sub-type value. E.g. 
> {{DIAGNOSTIC_EVENT(GossiperEvent, MAJOR_STATE_CHANGE_HANDLED)}}. This still 
> has to be worked out and I'd appreciate any feedback.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13459) Diag. Events: Native transport integration

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392133#comment-16392133
 ] 

mck commented on CASSANDRA-13459:
-

BTW the patch is 
[here|https://github.com/spodkowinski/cassandra/blob/WIP-13459/].

> Diag. Events: Native transport integration
> --
>
> Key: CASSANDRA-13459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13459
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: client-impacting
>
> Events should be consumable by clients that would received subscribed events 
> from the connected node. This functionality is designed to work on top of 
> native transport with minor modifications to the protocol standard (see 
> [original 
> proposal|https://docs.google.com/document/d/1uEk7KYgxjNA0ybC9fOuegHTcK3Yi0hCQN5nTp5cNFyQ/edit?usp=sharing]
>  for further considered options). First we have to add another value for 
> existing event types. Also, we have to extend the protocol a bit to be able 
> to specify a sub-class and sub-type value. E.g. 
> {{DIAGNOSTIC_EVENT(GossiperEvent, MAJOR_STATE_CHANGE_HANDLED)}}. This still 
> has to be worked out and I'd appreciate any feedback.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/620f37c5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/620f37c5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/620f37c5

Branch: refs/heads/trunk
Commit: 620f37c54e63db1f651c5dc58cdf1c6824f88069
Parents: c87dc1a 2c15098
Author: Blake Eggleston 
Authored: Thu Mar 8 15:38:02 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:39:11 2018 -0800

--
 CHANGES.txt   |  1 +
 .../apache/cassandra/cql3/ColumnCondition.java|  4 
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 3 files changed, 23 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/CHANGES.txt
--
diff --cc CHANGES.txt
index 5b8ee0c,9c6a853..0e9ef6d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 -3.0.17
 +3.11.3
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
   * Log when SSTables are deleted (CASSANDRA-14302)
   * Fix batch commitlog sync regression (CASSANDRA-14292)
   * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[bdeggleston]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd409898
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd409898
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd409898

Branch: refs/heads/trunk
Commit: dd409898c2fbf9735ee4a93875c59d44352dcbb5
Parents: a7141e6 620f37c
Author: Blake Eggleston 
Authored: Thu Mar 8 15:55:32 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:55:32 2018 -0800

--
 CHANGES.txt   |  1 +
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 2 files changed, 19 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd409898/CHANGES.txt
--
diff --cc CHANGES.txt
index 9af46c3,0e9ef6d..c791e47
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,208 -1,8 +1,209 @@@
 +4.0
 + * Use Murmur3 for validation compactions (CASSANDRA-14002)
 + * Comma at the end of the seed list is interpretated as localhost 
(CASSANDRA-14285)
 + * Refactor read executor and response resolver, abstract read repair 
(CASSANDRA-14058)
 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993)
 + * Add a few options to nodetool verify (CASSANDRA-14201)
 + * CVE-2017-5929 Security vulnerability and redefine default log rotation 
policy (CASSANDRA-14183)
 + * Use JVM default SSL validation algorithm instead of custom default 
(CASSANDRA-13259)
 + * Better document in code InetAddressAndPort usage post 7544, incorporate 
port into UUIDGen node (CASSANDRA-14226)
 + * Fix sstablemetadata date string for minLocalDeletionTime (CASSANDRA-14132)
 + * Make it possible to change neverPurgeTombstones during runtime 
(CASSANDRA-14214)
 + * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174)
 + * Add nodetool clientlist (CASSANDRA-13665)
 + * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211)
 + * Non-disruptive seed node list reload (CASSANDRA-14190)
 + * Nodetool tablehistograms to print statics for all the tables 
(CASSANDRA-14185)
 + * Migrate dtests to use pytest and python3 (CASSANDRA-14134)
 + * Allow storage port to be configurable per node (CASSANDRA-7544)
 + * Make sub-range selection for non-frozen collections return null instead of 
empty (CASSANDRA-14182)
 + * BloomFilter serialization format should not change byte ordering 
(CASSANDRA-9067)
 + * Remove unused on-heap BloomFilter implementation (CASSANDRA-14152)
 + * Delete temp test files on exit (CASSANDRA-14153)
 + * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
 + * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
 + * Fix cassandra-stress startup failure (CASSANDRA-14106)
 + * Remove initialDirectories from CFS (CASSANDRA-13928)
 + * Fix trivial log format error (CASSANDRA-14015)
 + * Allow sstabledump to do a json object per partition (CASSANDRA-13848)
 + * Add option to optimise merkle tree comparison across replicas 
(CASSANDRA-3200)
 + * Remove unused and deprecated methods from AbstractCompactionStrategy 
(CASSANDRA-14081)
 + * Fix Distribution.average in cassandra-stress (CASSANDRA-14090)
 + * Support a means of logging all queries as they were invoked 
(CASSANDRA-13983)
 + * Presize collections (CASSANDRA-13760)
 + * Add GroupCommitLogService (CASSANDRA-13530)
 + * Parallelize initial materialized view build (CASSANDRA-12245)
 + * Fix flaky SecondaryIndexManagerTest.assert[Not]MarkedAsBuilt 
(CASSANDRA-13965)
 + * Make LWTs send resultset metadata on every request (CASSANDRA-13992)
 + * Fix flaky indexWithFailedInitializationIsNotQueryableAfterPartialRebuild 
(CASSANDRA-13963)
 + * Introduce leaf-only iterator (CASSANDRA-9988)
 + * Upgrade Guava to 23.3 and Airline to 0.8 (CASSANDRA-13997)
 + * Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
 + * Refactoring to specialised functional interfaces (CASSANDRA-13982)
 + * Speculative retry should allow more friendly params (CASSANDRA-13876)
 + * Throw exception if we send/receive repair messages to incompatible nodes 
(CASSANDRA-13944)
 + * Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
 + * Add nodetool cmd to print hinted handoff window (CASSANDRA-13728)
 + * Fix some alerts raised by static analysis (CASSANDRA-13799)
 + * Checksum sstable metadata (CASSANDRA-13321, CASSANDRA-13593)
 + * Add result set metadata to prepared statement MD5 hash calculation 
(CASSANDRA-10786)
 + * Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
 + * Expose recent histograms in JmxHistograms (CASSANDRA-13642)
 + * Fix buffer length comparison when decompressing in netty-based streaming 
(CASSANDRA-13899)
 + * Properly close 

[jira] [Updated] (CASSANDRA-14087) NPE when CAS encounters empty frozen collection

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-14087:

   Resolution: Fixed
Reproduced In: 3.11.1, 3.11.0, 3.0.0  (was: 3.0.0, 3.11.0, 3.11.1)
   Status: Resolved  (was: Patch Available)

All the test failures that were already failing in their respective branches, 
so +1.

Committed as {{2c150980cc1bfea81fd039f304e74fc2fb30fb45}}. Thanks, and sorry 
for the delay getting this in

> NPE when CAS encounters empty frozen collection
> ---
>
> Key: CASSANDRA-14087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14087
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jens Bannmann
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When a compare-and-set operation specifying an equality criterion with a 
> non-{{null}} value encounters an empty collection ({{null}} cell), the server 
> throws a {{NullPointerException}} and the query fails.
> This does not happen for non-frozen collections.
> There's a self-contained test case at 
> [github|https://github.com/incub8/cassandra-npe-in-cas].
> The stack trace for 3.11.0 is:
> {code}
> ERROR [Native-Transport-Requests-1] 2017-11-27 12:59:26,924 
> QueryMessage.java:129 - Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.cql3.ColumnCondition$CollectionBound.appliesTo(ColumnCondition.java:546)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.statements.CQL3CasRequest$ColumnsConditions.appliesTo(CQL3CasRequest.java:324)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.statements.CQL3CasRequest.appliesTo(CQL3CasRequest.java:210)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:265) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:441)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:416)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:217)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:233) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517)
>  [apache-cassandra-3.11.0.jar:3.11.0]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
>  [apache-cassandra-3.11.0.jar:3.11.0]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_151]
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  [apache-cassandra-3.11.0.jar:3.11.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.11.0.jar:3.11.0]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Fix NPE when performing comparison against a null frozen in LWT

[bdeggleston]

Fix NPE when performing comparison against a null frozen in LWT

Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14087


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c150980
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c150980
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c150980

Branch: refs/heads/cassandra-3.11
Commit: 2c150980cc1bfea81fd039f304e74fc2fb30fb45
Parents: d774005
Author: kurt 
Authored: Tue Dec 5 04:14:14 2017 +
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:34:13 2018 -0800

--
 CHANGES.txt   |  1 +
 .../apache/cassandra/cql3/ColumnCondition.java|  4 
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 3 files changed, 23 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b49a349..9c6a853 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
  * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 99e243c..c3a3af7 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -513,6 +513,10 @@ public class ColumnCondition
 else
 throw new InvalidRequestException(String.format("Invalid 
comparison with null for operator \"%s\"", operator));
 }
+else if (cell == null) // cell is null but condition has a value
+{
+return false;
+}
 
 // make sure we use v3 serialization format for comparison
 ByteBuffer conditionValue;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
index 8adce7a..a47691a 100644
--- 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
@@ -871,6 +871,24 @@ public class InsertUpdateIfConditionTest extends CQLTester
 }
 }
 
+@Test
+public void testFrozenWithNullValues() throws Throwable
+{
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
list("test"), list("comparison")), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen>"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
map("test", 3), map("comparison", 2)), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
set("test"), set("comparison")), row(false, null));
+}
 /**
  * Test expanded functionality from CASSANDRA-6839,
  * migrated from cql_tests.py:TestCQL.expanded_map_item_conditional_test()


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Fix NPE when performing comparison against a null frozen in LWT

[bdeggleston]

Fix NPE when performing comparison against a null frozen in LWT

Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14087


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c150980
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c150980
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c150980

Branch: refs/heads/trunk
Commit: 2c150980cc1bfea81fd039f304e74fc2fb30fb45
Parents: d774005
Author: kurt 
Authored: Tue Dec 5 04:14:14 2017 +
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:34:13 2018 -0800

--
 CHANGES.txt   |  1 +
 .../apache/cassandra/cql3/ColumnCondition.java|  4 
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 3 files changed, 23 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b49a349..9c6a853 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
  * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 99e243c..c3a3af7 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -513,6 +513,10 @@ public class ColumnCondition
 else
 throw new InvalidRequestException(String.format("Invalid 
comparison with null for operator \"%s\"", operator));
 }
+else if (cell == null) // cell is null but condition has a value
+{
+return false;
+}
 
 // make sure we use v3 serialization format for comparison
 ByteBuffer conditionValue;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
index 8adce7a..a47691a 100644
--- 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
@@ -871,6 +871,24 @@ public class InsertUpdateIfConditionTest extends CQLTester
 }
 }
 
+@Test
+public void testFrozenWithNullValues() throws Throwable
+{
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
list("test"), list("comparison")), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen>"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
map("test", 3), map("comparison", 2)), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
set("test"), set("comparison")), row(false, null));
+}
 /**
  * Test expanded functionality from CASSANDRA-6839,
  * migrated from cql_tests.py:TestCQL.expanded_map_item_conditional_test()


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/620f37c5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/620f37c5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/620f37c5

Branch: refs/heads/cassandra-3.11
Commit: 620f37c54e63db1f651c5dc58cdf1c6824f88069
Parents: c87dc1a 2c15098
Author: Blake Eggleston 
Authored: Thu Mar 8 15:38:02 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:39:11 2018 -0800

--
 CHANGES.txt   |  1 +
 .../apache/cassandra/cql3/ColumnCondition.java|  4 
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 3 files changed, 23 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/CHANGES.txt
--
diff --cc CHANGES.txt
index 5b8ee0c,9c6a853..0e9ef6d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 -3.0.17
 +3.11.3
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
   * Log when SSTables are deleted (CASSANDRA-14302)
   * Fix batch commitlog sync regression (CASSANDRA-14292)
   * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/620f37c5/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Fix NPE when performing comparison against a null frozen in LWT

[bdeggleston]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 d7740052f -> 2c150980c
  refs/heads/cassandra-3.11 c87dc1ae7 -> 620f37c54
  refs/heads/trunk a7141e6c9 -> dd409898c


Fix NPE when performing comparison against a null frozen in LWT

Patch by Kurt Greaves; Reviewed by Blake Eggleston for CASSANDRA-14087


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c150980
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c150980
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c150980

Branch: refs/heads/cassandra-3.0
Commit: 2c150980cc1bfea81fd039f304e74fc2fb30fb45
Parents: d774005
Author: kurt 
Authored: Tue Dec 5 04:14:14 2017 +
Committer: Blake Eggleston 
Committed: Thu Mar 8 15:34:13 2018 -0800

--
 CHANGES.txt   |  1 +
 .../apache/cassandra/cql3/ColumnCondition.java|  4 
 .../operations/InsertUpdateIfConditionTest.java   | 18 ++
 3 files changed, 23 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b49a349..9c6a853 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix NPE when performing comparison against a null frozen in LWT 
(CASSANDRA-14087)
  * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 99e243c..c3a3af7 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -513,6 +513,10 @@ public class ColumnCondition
 else
 throw new InvalidRequestException(String.format("Invalid 
comparison with null for operator \"%s\"", operator));
 }
+else if (cell == null) // cell is null but condition has a value
+{
+return false;
+}
 
 // make sure we use v3 serialization format for comparison
 ByteBuffer conditionValue;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c150980/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
index 8adce7a..a47691a 100644
--- 
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
+++ 
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
@@ -871,6 +871,24 @@ public class InsertUpdateIfConditionTest extends CQLTester
 }
 }
 
+@Test
+public void testFrozenWithNullValues() throws Throwable
+{
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
list("test"), list("comparison")), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen>"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
map("test", 3), map("comparison", 2)), row(false, null));
+
+createTable(String.format("CREATE TABLE %%s (k int PRIMARY KEY, m 
%s)", "frozen"));
+execute("INSERT INTO %s (k, m) VALUES (0, null)");
+
+assertRows(execute("UPDATE %s SET m = ? WHERE k = 0 IF m = ?", 
set("test"), set("comparison")), row(false, null));
+}
 /**
  * Test expanded functionality from CASSANDRA-6839,
  * migrated from cql_tests.py:TestCQL.expanded_map_item_conditional_test()


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-5836) Seed nodes should be able to bootstrap without manual intervention

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392115#comment-16392115
 ] 

Kurt Greaves commented on CASSANDRA-5836:
-

bq. Hm, I've tested the procedure quite a number of times and every time I 
forgot to change the replication to NTS or to extend the replication to the new 
DC I was getting a complaint from nodetool rebuild.
If N>RF it becomes less likely that you'll have one replica in each DC for 
every range. But it's irrelevant anyway, {{nodetool rebuild}} should probably 
avoid rebuilding SimpleStrategy keyspaces and you shouldn't get an error for 
them. Bootstrapping SimpleStrategy across DC's is still relevant as long as 
SimpleStrategy exists. The only solutions here are either to (1) remove 
SimpleStrategy, (2) bootstrap new nodes within a new DC, or (3) disallow 
addition of a DC when SimpleStrategy is being used. 1 is hard + unreasonable 
and 3 is just unreasonable. 2 makes the most sense, and with my patch we could 
forget about the instructions telling people to set auto_bootstrap=false when 
adding a new DC.

bq. Do you mean it is more common to see the error with a small cluster or 
other way round: more common that it will work with a small cluster?
It's more common that people do this with small clusters and it works/they 
don't realise that they didn't change to NTS.

Well that makes sense. But there's a few issues:
# You still need code to handle the case where a seed starts with 
auto_bootstrap=true but it's a new cluster. You could potentially know when to 
fail by checking your seeds list and seeing if you are the only seed (then 
create a cluster, else fail). But I still don't see this as terribly necessary.
# Seems a bit silly to have a new cluster procedure where the first step is to 
"set this to false in the yaml... because we said so". Especially when we can 
avoid that situation.

Note that when I say special case I mean a special case in the code, not for 
the user. My patch (maybe with some tweaks) should be able to decide 
automatically every case where a seed should bootstrap versus when it 
shouldn't. If we can do that in the code, there's no reason to worry about 
changing any procedures or behaviours, and we don't need to worry about 
explaining the intricacies of why a seed can't bootstrap.

> Seed nodes should be able to bootstrap without manual intervention
> --
>
> Key: CASSANDRA-5836
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5836
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Bill Hathaway
>Priority: Minor
>
> The current logic doesn't allow a seed node to be bootstrapped.  If a user 
> wants to bootstrap a node configured as a seed (for example to replace a seed 
> node via replace_token), they first need to remove the node's own IP from the 
> seed list, and then start the bootstrap process.  This seems like an 
> unnecessary step since a node never uses itself as a seed.
> I think it would be a better experience if the logic was changed to allow a 
> seed node to bootstrap without manual intervention when there are other seed 
> nodes up in a ring.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14061) trunk eclipse-warnings

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14061:
---
Fix Version/s: 4.0

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 4.0
>
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14061) trunk eclipse-warnings

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14061:
---
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14061) trunk eclipse-warnings

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392113#comment-16392113
 ] 

Jay Zhuang commented on CASSANDRA-14061:


Thanks for the review.
Committed as 
[a7141e6|https://github.com/apache/cassandra/commit/a7141e6c9df03287567c22c76372e166fc83d18e].

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Suppress 2 eclipse-warnings

[jzhuang]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5c67a7852 -> a7141e6c9


Suppress 2 eclipse-warnings

patch by Jay Zhuang; reviewed by Stefan Podkowinski for CASSANDRA-14061


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7141e6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7141e6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7141e6c

Branch: refs/heads/trunk
Commit: a7141e6c9df03287567c22c76372e166fc83d18e
Parents: 5c67a78
Author: Jay Zhuang 
Authored: Sun Nov 19 22:13:42 2017 -0800
Committer: Jay Zhuang 
Committed: Thu Mar 8 15:32:46 2018 -0800

--
 .../org/apache/cassandra/io/sstable/SSTableIdentityIterator.java   | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a7141e6c/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 3ade9ff..f9c6e82 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@ -49,6 +49,7 @@ public class SSTableIdentityIterator implements 
Comparable indexEntry, DecoratedKey key, boolean 
tombstoneOnly)
 {
 try


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392035#comment-16392035
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 3/8/18 10:45 PM:


Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}

Regarding using a TWCS workload, C* stress doesn't play very well with 
timestamps and TWCS in general unless I'm missing something. I've currently got 
to a point of using UDFs (minutesAgo is my UDF) because arithmetic operators 
are not supported until 4.0, and using a query spec something along those lines

{code}
queries:
  putindata:
cql: insert into twcstest (id, time, metric) VALUES (?, toTimestamp(now()), 
?)
  simple1:
cql: select * from twcstest where id = ? and time <= toTimestamp(now()) and 
time >= minutesAgo(5) LIMIT 288
fields: samerow 
{code}

I would have thought though, that using a very pointy Gaussian such as this 
{{gaussian(1..500M, 25000, 1000)}} would be a fair enough reflection of the 
right load because it will still predominantly select the same data due to an 
extremely low stdev of 1000 (Mean of 250M, stdev of 1000). I can re-run the 
benchmarks just to get more samples. 



was (Author: lerh low):
Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}



> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392035#comment-16392035
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

Sure. In the commands below {{/dev/md0}} is my RAID and {{/dev/xvdf}} is my SSD 
volume. 

{code}
sudo pvcreate /dev/md0 
sudo vgcreate VolGroupArray /dev/md0 /dev/xvdf 
sudo lvcreate -n SadOldCache -L 99900M VolGroupArray /dev/xvdf
sudo lvcreate -n SadOldCacheMeta -L 100M VolGroupArray /dev/xvdf
sudo lvconvert --type cache-pool --poolmetadata VolGroupArray/SadOldCacheMeta 
VolGroupArray/SadOldCache
sudo lvcreate -l 100%FREE -n RaidHDD VolGroupArray /dev/md0
sudo lvs -a vg
sudo lvs -a VolGroupArray
sudo lvconvert --type cache --cachepool VolGroupArray/SadOldCache 
VolGroupArray/RaidHDD
sudo vgchange -ay
{code}



> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392030#comment-16392030
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} Operation type.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392030#comment-16392030
 ] 

Lerh Chuan Low edited comment on CASSANDRA-8460 at 3/8/18 10:35 PM:


On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} OperationType.


was (Author: lerh low):
On other thoughts, maybe it isn't ideal to bundle it together with compactions 
and make a totally new {{Archiving}} Operation type.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14302) Log when sstables are deleted

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-14302:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Thanks, committed to 3.0 as {{d7740052ff6cee74fb6ab973f1052422180d75bc}}

> Log when sstables are deleted
> -
>
> Key: CASSANDRA-14302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[bdeggleston]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5c67a785
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5c67a785
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5c67a785

Branch: refs/heads/trunk
Commit: 5c67a7852c903a2891951ff52f07b77fb2756412
Parents: 87f50f6 c87dc1a
Author: Blake Eggleston 
Authored: Thu Mar 8 14:29:13 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:29:13 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c67a785/CHANGES.txt
--
diff --cc CHANGES.txt
index da66c14,5b8ee0c..9af46c3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,208 -1,8 +1,209 @@@
 +4.0
 + * Use Murmur3 for validation compactions (CASSANDRA-14002)
 + * Comma at the end of the seed list is interpretated as localhost 
(CASSANDRA-14285)
 + * Refactor read executor and response resolver, abstract read repair 
(CASSANDRA-14058)
 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993)
 + * Add a few options to nodetool verify (CASSANDRA-14201)
 + * CVE-2017-5929 Security vulnerability and redefine default log rotation 
policy (CASSANDRA-14183)
 + * Use JVM default SSL validation algorithm instead of custom default 
(CASSANDRA-13259)
 + * Better document in code InetAddressAndPort usage post 7544, incorporate 
port into UUIDGen node (CASSANDRA-14226)
 + * Fix sstablemetadata date string for minLocalDeletionTime (CASSANDRA-14132)
 + * Make it possible to change neverPurgeTombstones during runtime 
(CASSANDRA-14214)
 + * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174)
 + * Add nodetool clientlist (CASSANDRA-13665)
 + * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211)
 + * Non-disruptive seed node list reload (CASSANDRA-14190)
 + * Nodetool tablehistograms to print statics for all the tables 
(CASSANDRA-14185)
 + * Migrate dtests to use pytest and python3 (CASSANDRA-14134)
 + * Allow storage port to be configurable per node (CASSANDRA-7544)
 + * Make sub-range selection for non-frozen collections return null instead of 
empty (CASSANDRA-14182)
 + * BloomFilter serialization format should not change byte ordering 
(CASSANDRA-9067)
 + * Remove unused on-heap BloomFilter implementation (CASSANDRA-14152)
 + * Delete temp test files on exit (CASSANDRA-14153)
 + * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
 + * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
 + * Fix cassandra-stress startup failure (CASSANDRA-14106)
 + * Remove initialDirectories from CFS (CASSANDRA-13928)
 + * Fix trivial log format error (CASSANDRA-14015)
 + * Allow sstabledump to do a json object per partition (CASSANDRA-13848)
 + * Add option to optimise merkle tree comparison across replicas 
(CASSANDRA-3200)
 + * Remove unused and deprecated methods from AbstractCompactionStrategy 
(CASSANDRA-14081)
 + * Fix Distribution.average in cassandra-stress (CASSANDRA-14090)
 + * Support a means of logging all queries as they were invoked 
(CASSANDRA-13983)
 + * Presize collections (CASSANDRA-13760)
 + * Add GroupCommitLogService (CASSANDRA-13530)
 + * Parallelize initial materialized view build (CASSANDRA-12245)
 + * Fix flaky SecondaryIndexManagerTest.assert[Not]MarkedAsBuilt 
(CASSANDRA-13965)
 + * Make LWTs send resultset metadata on every request (CASSANDRA-13992)
 + * Fix flaky indexWithFailedInitializationIsNotQueryableAfterPartialRebuild 
(CASSANDRA-13963)
 + * Introduce leaf-only iterator (CASSANDRA-9988)
 + * Upgrade Guava to 23.3 and Airline to 0.8 (CASSANDRA-13997)
 + * Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
 + * Refactoring to specialised functional interfaces (CASSANDRA-13982)
 + * Speculative retry should allow more friendly params (CASSANDRA-13876)
 + * Throw exception if we send/receive repair messages to incompatible nodes 
(CASSANDRA-13944)
 + * Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
 + * Add nodetool cmd to print hinted handoff window (CASSANDRA-13728)
 + * Fix some alerts raised by static analysis (CASSANDRA-13799)
 + * Checksum sstable metadata (CASSANDRA-13321, CASSANDRA-13593)
 + * Add result set metadata to prepared statement MD5 hash calculation 
(CASSANDRA-10786)
 + * Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
 + * Expose recent histograms in JmxHistograms (CASSANDRA-13642)
 + * Fix buffer length comparison when decompressing in netty-based streaming 
(CASSANDRA-13899)
 + * Properly close 

[3/6] cassandra git commit: Log when SSTables are deleted

[bdeggleston]

Log when SSTables are deleted

Patch by Blake Eggleston; Reviewed by Chris Lohfink for CASSANDRA-14302


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d7740052
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d7740052
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d7740052

Branch: refs/heads/trunk
Commit: d7740052ff6cee74fb6ab973f1052422180d75bc
Parents: 00c90c1
Author: Blake Eggleston 
Authored: Thu Mar 8 13:46:15 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:27:39 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6596602..b49a349 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
  * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/src/java/org/apache/cassandra/io/sstable/SSTable.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTable.java 
b/src/java/org/apache/cassandra/io/sstable/SSTable.java
index 1e4488c..a54e7e0 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTable.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTable.java
@@ -104,6 +104,7 @@ public abstract class SSTable
  */
 public static boolean delete(Descriptor desc, Set components)
 {
+logger.info("Deleting sstable: {}", desc);
 // remove the DATA component first if it exists
 if (components.contains(Component.DATA))
 FileUtils.deleteWithConfirm(desc.filenameFor(Component.DATA));
@@ -118,7 +119,6 @@ public abstract class SSTable
 if (components.contains(Component.SUMMARY))
 FileUtils.delete(desc.filenameFor(Component.SUMMARY));
 
-logger.trace("Deleted {}", desc);
 return true;
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Log when SSTables are deleted

[bdeggleston]

Log when SSTables are deleted

Patch by Blake Eggleston; Reviewed by Chris Lohfink for CASSANDRA-14302


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d7740052
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d7740052
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d7740052

Branch: refs/heads/cassandra-3.11
Commit: d7740052ff6cee74fb6ab973f1052422180d75bc
Parents: 00c90c1
Author: Blake Eggleston 
Authored: Thu Mar 8 13:46:15 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:27:39 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6596602..b49a349 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
  * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/src/java/org/apache/cassandra/io/sstable/SSTable.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTable.java 
b/src/java/org/apache/cassandra/io/sstable/SSTable.java
index 1e4488c..a54e7e0 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTable.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTable.java
@@ -104,6 +104,7 @@ public abstract class SSTable
  */
 public static boolean delete(Descriptor desc, Set components)
 {
+logger.info("Deleting sstable: {}", desc);
 // remove the DATA component first if it exists
 if (components.contains(Component.DATA))
 FileUtils.deleteWithConfirm(desc.filenameFor(Component.DATA));
@@ -118,7 +119,6 @@ public abstract class SSTable
 if (components.contains(Component.SUMMARY))
 FileUtils.delete(desc.filenameFor(Component.SUMMARY));
 
-logger.trace("Deleted {}", desc);
 return true;
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Log when SSTables are deleted

[bdeggleston]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 00c90c16e -> d7740052f
  refs/heads/cassandra-3.11 61edf6f2f -> c87dc1ae7
  refs/heads/trunk 87f50f6a6 -> 5c67a7852


Log when SSTables are deleted

Patch by Blake Eggleston; Reviewed by Chris Lohfink for CASSANDRA-14302


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d7740052
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d7740052
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d7740052

Branch: refs/heads/cassandra-3.0
Commit: d7740052ff6cee74fb6ab973f1052422180d75bc
Parents: 00c90c1
Author: Blake Eggleston 
Authored: Thu Mar 8 13:46:15 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:27:39 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6596602..b49a349 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Log when SSTables are deleted (CASSANDRA-14302)
  * Fix batch commitlog sync regression (CASSANDRA-14292)
  * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
  * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7740052/src/java/org/apache/cassandra/io/sstable/SSTable.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTable.java 
b/src/java/org/apache/cassandra/io/sstable/SSTable.java
index 1e4488c..a54e7e0 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTable.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTable.java
@@ -104,6 +104,7 @@ public abstract class SSTable
  */
 public static boolean delete(Descriptor desc, Set components)
 {
+logger.info("Deleting sstable: {}", desc);
 // remove the DATA component first if it exists
 if (components.contains(Component.DATA))
 FileUtils.deleteWithConfirm(desc.filenameFor(Component.DATA));
@@ -118,7 +119,6 @@ public abstract class SSTable
 if (components.contains(Component.SUMMARY))
 FileUtils.delete(desc.filenameFor(Component.SUMMARY));
 
-logger.trace("Deleted {}", desc);
 return true;
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c87dc1ae
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c87dc1ae
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c87dc1ae

Branch: refs/heads/cassandra-3.11
Commit: c87dc1ae7fc0df0649e06a4c611d1cc5488dbb25
Parents: 61edf6f d774005
Author: Blake Eggleston 
Authored: Thu Mar 8 14:28:58 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:28:58 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c87dc1ae/CHANGES.txt
--
diff --cc CHANGES.txt
index bb652f3,b49a349..5b8ee0c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 -3.0.17
 +3.11.3
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Log when SSTables are deleted (CASSANDRA-14302)
   * Fix batch commitlog sync regression (CASSANDRA-14292)
   * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
   * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c87dc1ae/src/java/org/apache/cassandra/io/sstable/SSTable.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c87dc1ae
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c87dc1ae
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c87dc1ae

Branch: refs/heads/trunk
Commit: c87dc1ae7fc0df0649e06a4c611d1cc5488dbb25
Parents: 61edf6f d774005
Author: Blake Eggleston 
Authored: Thu Mar 8 14:28:58 2018 -0800
Committer: Blake Eggleston 
Committed: Thu Mar 8 14:28:58 2018 -0800

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/io/sstable/SSTable.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c87dc1ae/CHANGES.txt
--
diff --cc CHANGES.txt
index bb652f3,b49a349..5b8ee0c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,5 +1,8 @@@
 -3.0.17
 +3.11.3
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Log when SSTables are deleted (CASSANDRA-14302)
   * Fix batch commitlog sync regression (CASSANDRA-14292)
   * Write to pending endpoint when view replica is also base replica 
(CASSANDRA-14251)
   * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c87dc1ae/src/java/org/apache/cassandra/io/sstable/SSTable.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392024#comment-16392024
 ] 

Jon Haddad commented on CASSANDRA-8460:
---

Would you mind sharing the commands you used to set up the LVM Cache?  It's 
pretty easy to accidentally set up the pool as a balance of the 2 drives rather 
than using the SSD as a cache.

Both implementations are going to work best with TWCS, so I think that testing 
the workload with TWCS using time series writes is going to be a lot more 
productive than random writes, as you've noted.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14160) maxPurgeableTimestamp should traverse tables in order of minTimestamp

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392018#comment-16392018
 ] 

Jeff Jirsa commented on CASSANDRA-14160:


[~krummas] can you review?

 

> maxPurgeableTimestamp should traverse tables in order of minTimestamp
> -
>
> Key: CASSANDRA-14160
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14160
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Josh Snyder
>Assignee: Josh Snyder
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
>
> In maxPurgeableTimestamp, we iterate over the bloom filters of each 
> overlapping SSTable. Of the bloom filter hits, we take the SSTable with the 
> lowest minTimestamp. If we kept the SSTables in sorted order of minTimestamp, 
> then we could short-circuit the operation at the first bloom filter hit, 
> reducing cache pressure (or worse, I/O) and CPU time.
> I've written (but not yet benchmarked) [some 
> code|https://github.com/hashbrowncipher/cassandra/commit/29859a4a2e617f6775be49448858bc59fdafab44]
>  to demonstrate this possibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392001#comment-16392001
 ] 

Lerh Chuan Low commented on CASSANDRA-8460:
---

[~rustyrazorblade], 

About {{disk_failure_policy}}, that I am not aware so I'll have to look into 
it. With my current patch up to where it is now, there are also other things 
like Scrubber, streaming which I have yet to get to. Thanks for the heads up! 
If such an implication is necessary then maybe we will have to enforce it in 
the code. 

About LVM Cache, I spent some time following the man page and trying it out 
with Cassandra stress. I had spun up a few EC2 clusters. They were all using a 
raid array of 800GB each; one was SSD backed, another was magnetic HDD backed, 
and the final one was magnetic HDD backed with 100GB of LVM writethrough cache. 
I inserted ~200GB of data using cassandra-stress, waited for compactions to 
finish and then attempted a mixed (random) workload...the LVM performed even 
worse than HDD. I guess this was to be expected because the cache works best 
for hot data that is frequently read. 

I did briefly attempt a mixed workload where the queries are always trying to 
select the same data as much as possible (so {{gaussian(1..500M, 25000, 
1000)}}), and there wasn't any noticeable difference between the LVM and HDD 
backed cluster. 

Not sure if you have used LVMCache with a workload before that worked out for 
you and you'd be willing to share details about it...? 

Just thinking about it further, the cache is also very slightly different than 
the original proposal. The cache duplicates the data; making Cassandra 
understand archiving does not. There's also a slight bonus at least from the 
scenario for AWS, the cache consumes the IOPS of the volumes due to the 
duplication (or amplifying) of read and writes back and from the cache.

Any thoughts? (And thank you for your input once again :)) My clusters are 
still running so happy to try a few configurations if you have any to suggest, 
for now I'm just going to refresh myself on the code and look into getting it 
more presentable if someone else swings by and is willing to give their 
thoughts. 

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> 
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>Priority: Major
>  Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data 
> directories where we move the sstables once they are older than 
> max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big 
> spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14302) Log when sstables are deleted

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391973#comment-16391973
 ] 

Chris Lohfink commented on CASSANDRA-14302:
---

+1

> Log when sstables are deleted
> -
>
> Key: CASSANDRA-14302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14302) Log when sstables are deleted

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-14302:

 Reviewer: Chris Lohfink
Fix Version/s: 3.11.x
   3.0.x
   4.0
   Status: Patch Available  (was: Open)

[3.0|https://github.com/bdeggleston/cassandra/tree/14302-3.0]
[3.11|https://github.com/bdeggleston/cassandra/tree/14302-3.11]
[trunk|https://github.com/bdeggleston/cassandra/tree/14302-trunk]

> Log when sstables are deleted
> -
>
> Key: CASSANDRA-14302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14302) Log when sstables are deleted

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink reassigned CASSANDRA-14302:
-

Assignee: Blake Eggleston  (was: Chris Lohfink)

> Log when sstables are deleted
> -
>
> Key: CASSANDRA-14302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
>
> This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14302) Log when sstables are deleted

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink reassigned CASSANDRA-14302:
-

Assignee: Chris Lohfink  (was: Blake Eggleston)

> Log when sstables are deleted
> -
>
> Key: CASSANDRA-14302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Chris Lohfink
>Priority: Minor
>
> This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14302) Log when sstables are deleted

[Blake Eggleston (JIRA)]

Blake Eggleston created CASSANDRA-14302:
---

 Summary: Log when sstables are deleted
 Key: CASSANDRA-14302
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14302
 Project: Cassandra
  Issue Type: Improvement
Reporter: Blake Eggleston
Assignee: Blake Eggleston


This was removed in 3.0 and is super helpful for debugging issues in prod



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14242) Indexed static column returns inconsistent results

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391729#comment-16391729
 ] 

Andrés de la Peña commented on CASSANDRA-14242:
---

Thanks for reporting. I've reproduced the first case in 3.11 and trunk branches 
with [this unit 
test|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:14242-3.11]:

{code:java}
@Test
public void pagingWithIndexOnStaticColumn() throws Throwable
{
createTable("CREATE TABLE %s (k int, c int, s int static, PRIMARY KEY (k, 
c));");
execute("CREATE INDEX ON %s (s);");

int numRows = 100;
for (int id = 0; id < numRows; id++)
{
execute("INSERT INTO %s (k, c, s) VALUES (?, 1, 2);", id);
}

String query = String.format("SELECT * FROM %s.%s WHERE s=2", KEYSPACE, 
currentTable());
SimpleStatement stmt = new SimpleStatement(query);

try (Session session = sessionNet())
{
assertEquals(numRows, 
session.execute(stmt.setFetchSize(1000)).all().size());
assertEquals(numRows, 
session.execute(stmt.setFetchSize(100)).all().size());
assertEquals(numRows, 
session.execute(stmt.setFetchSize(10)).all().size());
assertEquals(numRows, 
session.execute(stmt.setFetchSize(1)).all().size());
}
}
{code}
Still trying to reproduce the case returning fewer results.

> Indexed static column returns inconsistent results
> --
>
> Key: CASSANDRA-14242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14242
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 3.11.2
> Java driver 3.4.0
> Ubuntu - 4.4.0-112-generic
>Reporter: Ross Black
>Assignee: Andrés de la Peña
>Priority: Major
>
> I am using Cassandra 3.11.2, and the Java driver 3.4.0
> I have a table that has a static column, where the static column has a 
> secondary index.
> When querying the table I get incomplete or duplicated results, depending on 
> the fetch size.
> e.g.
> {code:java}
> CREATE KEYSPACE hack WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> CREATE TABLE hack.stuff (id int, kind text, chunk int static, val1 int, 
> PRIMARY KEY (id, kind));
> CREATE INDEX stuff_chunk_index ON hack.stuff (chunk);{code}
> -- repeat with thousands of values for id =>
> {code:java}
>   INSERT INTO hack.stuff (id, chunk, kind, val1 ) VALUES (${id}, 777, 'A', 
> 123);{code}
> Querying from Java:
> {code:java}
>     final SimpleStatement statement = new SimpleStatement("SELECT id, kind, 
> val1 FROM hack.stuff WHERE chunk = " + chunk); 
>     statement.setFetchSize(fetchSize);
>     statement.setConsistencyLevel(ConsistencyLevel.ALL);
>     final ResultSet resultSet = connection.getSession().execute(statement);
>     for (Row row : resultSet) {
>     final int id = row.getInt("id");
>     }{code}
> *The number of results returned depends on the fetch-size.*
> e.g. For 30k values inserted, I get the following:
> ||fetch-size||result-size||
> |4|3|
> |2|30001|
> |5000|30006|
> |100|30303|
> In production, I have a much larger table where the correct result size for a 
> specific chunk is 20019, but some fetch sizes will return _significantly 
> fewer_ results.
> ||fetch-size||result-size|| ||
> |25000|20019| |
> |5000||*<== this one is has far fewer results*|
> |5001|20026| |
> (so far been unable to reproduce this with the simpler test table)
> Thanks,
> Ross



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14301) Error running cqlsh: unexpected keyword argument 'no_compact'

[M. Justin (JIRA)]

M. Justin created CASSANDRA-14301:
-

 Summary: Error running cqlsh: unexpected keyword argument 
'no_compact'
 Key: CASSANDRA-14301
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14301
 Project: Cassandra
  Issue Type: Bug
Reporter: M. Justin


I recently installed Cassandra 3.11.2 on my Mac using Homebrew.  When I run the 
"cqlsh" command, I get the following error:
{code:none}
$ cqlsh
Traceback (most recent call last):
  File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2443, in 

    main(*read_options(sys.argv[1:], os.environ))
  File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 2421, in 
main
    encoding=options.encoding)
  File "/usr/local/Cellar/cassandra/3.11.2/libexec/bin/cqlsh.py", line 488, in 
__init__
    **kwargs)
  File "cassandra/cluster.py", line 735, in cassandra.cluster.Cluster.__init__ 
(cassandra/cluster.c:10935)
TypeError: __init__() got an unexpected keyword argument 'no_compact'
{code}
Commenting out [line 483 of 
cqlsh.py|https://github.com/apache/cassandra/blob/cassandra-3.11.2/bin/cqlsh.py#L483]
 works around the issue:
{code}
# no_compact=no_compact
{code}
 I am not the only person impacted, as evidenced by [this existing Stack 
Overflow 
post|https://stackoverflow.com/questions/48885984/was-cqlsh-5-0-1-broken-in-cassandra-3-11-2-release]
 from 2/20/2018.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14300) Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 3.11.1)

[Bhanu M. Gandikota (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391517#comment-16391517
 ] 

Bhanu M. Gandikota commented on CASSANDRA-14300:


-bash-4.2$ java -version

openjdk version "1.8.0_151"

OpenJDK Runtime Environment (build 1.8.0_151-b12)

OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

 

 

-bash-4.2$ cat /proc/version

Linux version 3.10.0-693.11.1.el7.x86_64 (buil...@kbuilder.dev.centos.org) (gcc 
version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Mon Dec 4 23:52:40 UTC 
2017

-bash-4.2$

> Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 
> 3.11.1)
> ---
>
> Key: CASSANDRA-14300
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14300
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Bhanu M. Gandikota
>Priority: Blocker
>
> -bash-4.2$ nodetool upgradesstables
>  
> WARN  11:28:28,430 Small cdc volume detected at /cdc_raw; setting 
> cdc_total_space_in_mb to 1982.  You can override this in cassandra.yaml
>  
> error: null
> -- StackTrace --
> java.lang.AssertionError
>    at org.apache.cassandra.db.rows.BufferCell.(BufferCell.java:43)
>    at 
> org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1242)
>    at 
> org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1185)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:495)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:472)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:306)
>    at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:176)
>    at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:49)
>    at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.create(SSTableIdentityIterator.java:59)
>    at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:384)
>    at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
>    at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:122)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:113)
>    at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:466)
>    at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:163)
>    at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92)
>    at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:233)
>    at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:196)
>    at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>    at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85)
>    at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>    at 
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:428)
>    at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:315)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>    at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
>    at java.lang.Thread.run(Thread.java:745)
>  
> -bash-4.2$ 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14300) Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 3.11.1)

[Bhanu M. Gandikota (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391510#comment-16391510
 ] 

Bhanu M. Gandikota commented on CASSANDRA-14300:


As part of upgrading Cassandra from 2.2.5 to 3.11.1, I tried running "nodetool 
upgradesstables" on one node at a time , and it has been error-ing out. First 
run of "nodetool upgrades stables" took about 30 minutes before it failed... 
subsequent runs are failing immediately.

 

 

 

> Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 
> 3.11.1)
> ---
>
> Key: CASSANDRA-14300
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14300
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Bhanu M. Gandikota
>Priority: Blocker
>
> -bash-4.2$ nodetool upgradesstables
>  
> WARN  11:28:28,430 Small cdc volume detected at /cdc_raw; setting 
> cdc_total_space_in_mb to 1982.  You can override this in cassandra.yaml
>  
> error: null
> -- StackTrace --
> java.lang.AssertionError
>    at org.apache.cassandra.db.rows.BufferCell.(BufferCell.java:43)
>    at 
> org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1242)
>    at 
> org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1185)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:495)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:472)
>    at 
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:306)
>    at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:176)
>    at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:49)
>    at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.create(SSTableIdentityIterator.java:59)
>    at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:384)
>    at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
>    at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:122)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:113)
>    at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:466)
>    at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>    at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:163)
>    at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92)
>    at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:233)
>    at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:196)
>    at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>    at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85)
>    at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>    at 
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:428)
>    at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:315)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>    at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
>    at java.lang.Thread.run(Thread.java:745)
>  
> -bash-4.2$ 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14300) Nodetool upgradesstables erring out with Null assertion error (2.2.5 to 3.11.1)

[Bhanu M. Gandikota (JIRA)]

Bhanu M. Gandikota created CASSANDRA-14300:
--

 Summary: Nodetool upgradesstables erring out with Null assertion 
error (2.2.5 to 3.11.1)
 Key: CASSANDRA-14300
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14300
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction
Reporter: Bhanu M. Gandikota


-bash-4.2$ nodetool upgradesstables

 

WARN  11:28:28,430 Small cdc volume detected at /cdc_raw; setting 
cdc_total_space_in_mb to 1982.  You can override this in cassandra.yaml

 

error: null

-- StackTrace --

java.lang.AssertionError

   at org.apache.cassandra.db.rows.BufferCell.(BufferCell.java:43)

   at 
org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1242)

   at 
org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1185)

   at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:495)

   at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:472)

   at 
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:306)

   at 
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:176)

   at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:49)

   at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.create(SSTableIdentityIterator.java:59)

   at 
org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:384)

   at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)

   at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70)

   at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:122)

   at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:113)

   at 
org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:466)

   at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)

   at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:163)

   at 
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92)

   at 
org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:233)

   at 
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:196)

   at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)

   at 
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85)

   at 
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)

   at 
org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:428)

   at 
org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:315)

   at java.util.concurrent.FutureTask.run(FutureTask.java:266)

   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

   at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)

   at java.lang.Thread.run(Thread.java:745)

 

-bash-4.2$ 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Anonymous (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anonymous updated CASSANDRA-14281:
--
Status: Ready to Commit  (was: Patch Available)

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14299) cqlsh: ssl setting not read from cqlshrc in 3.11

[Christian Becker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Becker updated CASSANDRA-14299:
-
Summary: cqlsh: ssl setting not read from cqlshrc in 3.11   (was: cqlsh: 
ssl setting not read in 3.11 from cqlshrc)

> cqlsh: ssl setting not read from cqlshrc in 3.11 
> -
>
> Key: CASSANDRA-14299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14299
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Christian Becker
>Priority: Major
>
> With CASSANDRA-10458 an option was added to read the {{--ssl}} flag from 
> cqlshrc, however the commit seems to have been incorrectly merged or the 
> changes were dropped somehow.
> Currently adding the following has no effect:
> {code:java}
> [connection]
> ssl = true{code}
> When looking at the current tree it's obvious that the flag is not read: 
> [https://github.com/apache/cassandra/blame/cassandra-3.11/bin/cqlsh.py#L2247]
> However it should have been added with 
> [https://github.com/apache/cassandra/commit/70649a8d65825144fcdbde136d9b6354ef1fb911]
> The values like {{DEFAULT_SSL = False}}  are present, but the 
> {{option_with_default()}} call is missing.
> Git blame also shows no change to that line which would have reverted the 
> change.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14299) cqlsh: ssl setting not read in 3.11 from cqlshrc

[Christian Becker (JIRA)]

Christian Becker created CASSANDRA-14299:


 Summary: cqlsh: ssl setting not read in 3.11 from cqlshrc
 Key: CASSANDRA-14299
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14299
 Project: Cassandra
  Issue Type: Bug
Reporter: Christian Becker


With CASSANDRA-10458 an option was added to read the {{--ssl}} flag from 
cqlshrc, however the commit seems to have been incorrectly merged or the 
changes were dropped somehow.

Currently adding the following has no effect:
{code:java}
[connection]
ssl = true{code}
When looking at the current tree it's obvious that the flag is not read: 
[https://github.com/apache/cassandra/blame/cassandra-3.11/bin/cqlsh.py#L2247]

However it should have been added with 
[https://github.com/apache/cassandra/commit/70649a8d65825144fcdbde136d9b6354ef1fb911]

The values like {{DEFAULT_SSL = False}}  are present, but the 
{{option_with_default()}} call is missing.

Git blame also shows no change to that line which would have reverted the 
change.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Michael Burman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Burman updated CASSANDRA-14281:
---
Status: Patch Available  (was: Open)

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing

[Michael Burman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Burman updated CASSANDRA-14281:
---
Description: 
Currently for each write/read/rangequery/CAS touching the CFS we write a 
latency metric which takes a lot of processing time (up to 66% of the total 
processing time if the update was empty). 

The way latencies are recorded is to use both a dropwizard "Timer" as well as 
"Counter". Latter is used for totalLatency and the previous is decaying metric 
for rates and certain percentile metrics. We then replicate all of these CFS 
writes to the KeyspaceMetrics and globalWriteLatencies. 

Instead of doing this on the write phase we should merge the metrics when 
they're read. This is much less common occurrence and thus we save a lot of CPU 
time in total. This also speeds up the write path.

Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
update operation, which causes a contention if there are more than one thread 
updating the histogram. This impacts scalability when using larger machines. We 
should make it lock-free as much as possible and also avoid a single CAS-update 
from blocking all the concurrent threads from making an update.

  was:Currently, the DecayingEstimatedHistogramReservoir acquires a lock for 
each update operation, which causes a contention if there are more than one 
thread updating the histogram. This impacts scalability when using larger 
machines. We should make it lock-free as much as possible and also avoid a 
single CAS-update from blocking all the concurrent threads from making an 
update.

Summary: Improve LatencyMetrics performance by reducing write path 
processing  (was: Reduce contention on DecayingEstimatedHistogramReservoir)

> Improve LatencyMetrics performance by reducing write path processing
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a 
> latency metric which takes a lot of processing time (up to 66% of the total 
> processing time if the update was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as 
> "Counter". Latter is used for totalLatency and the previous is decaying 
> metric for rates and certain percentile metrics. We then replicate all of 
> these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when 
> they're read. This is much less common occurrence and thus we save a lot of 
> CPU time in total. This also speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14281) Reduce contention on DecayingEstimatedHistogramReservoir

[Michael Burman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391327#comment-16391327
 ] 

Michael Burman commented on CASSANDRA-14281:


Here's a branch for these updates:

[https://github.com/burmanm/cassandra/tree/latency_metrics]

Benchmark results (on my laptop, which doesn't have a lot of contention)

trunk:

[java] LatencyTrackingBench.benchInsertToDEHR thrpt 5 5509872.821 ± 523366.638 
ops/s
 [java] LatencyTrackingBench.benchLatencyMetricsWrite thrpt 5 2148740.763 ± 
839102.027 ops/s

branch:

[java] LatencyTrackingBench.benchInsertToDEHR thrpt 5 16387458.999 ± 
9339490.676 ops/s
 [java] LatencyTrackingBench.benchLatencyMetricsWrite thrpt 5 6789997.113 ± 
1778364.870 ops/s

So around 3 times the performance. 13% of the time is only spent in DEHR in the 
benchmark, benchmark-related cleaning takes ~10% and GC around 22%. Around 42% 
in the Dropwizard inherited (Counter & Timer's other work) classes. Significant 
improvements to these would require removing Dropwizard classes from 
LatencyMetrics (and use of ThreadLocal etc to reduce volatile reads, CAS writes 
etc)

> Reduce contention on DecayingEstimatedHistogramReservoir
> 
>
> Key: CASSANDRA-14281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Michael Burman
>Assignee: Michael Burman
>Priority: Major
>
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each 
> update operation, which causes a contention if there are more than one thread 
> updating the histogram. This impacts scalability when using larger machines. 
> We should make it lock-free as much as possible and also avoid a single 
> CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14298) cqlshlib tests broken on b.a.o

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-14298:
---
Summary: cqlshlib tests broken on b.a.o  (was: cqlsh tests broken on b.a.o)

> cqlshlib tests broken on b.a.o
> --
>
> Key: CASSANDRA-14298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Testing
>Reporter: Stefan Podkowinski
>Priority: Major
>
> It appears that cqlsh-tests on builds.apache.org on all branches stopped 
> working since we removed nosetests from the system environment. See e.g. 
> [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
>  Looks like we either have to make nosetests available again or migrate to 
> pytest as we did with dtests. Giving pytest a quick try resulted in many 
> errors locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14298) cqlsh tests broken on b.a.o

[Stefan Podkowinski (JIRA)]

Stefan Podkowinski created CASSANDRA-14298:
--

 Summary: cqlsh tests broken on b.a.o
 Key: CASSANDRA-14298
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
 Project: Cassandra
  Issue Type: Bug
  Components: Build, Testing
Reporter: Stefan Podkowinski


It appears that cqlsh-tests on builds.apache.org on all branches stopped 
working since we removed nosetests from the system environment. See e.g. 
[here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
 Looks like we either have to make nosetests available again or migrate to 
pytest as we did with dtests. Giving pytest a quick try resulted in many errors 
locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14061) trunk eclipse-warnings

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391177#comment-16391177
 ] 

ZhaoYang commented on CASSANDRA-14061:
--

Actually, I am not sure how the warning is introduced in 13299.. Thanks for 
fixing it!

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14061) trunk eclipse-warnings

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-14061:
---
Status: Ready to Commit  (was: Patch Available)

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14061) trunk eclipse-warnings

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391173#comment-16391173
 ] 

Stefan Podkowinski commented on CASSANDRA-14061:


+1

> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-14061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14061
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> {noformat}
> eclipse-warnings:
> [mkdir] Created dir: /home/ubuntu/cassandra/build/ecj
>  [echo] Running Eclipse Code Analysis.  Output logged to 
> /home/ubuntu/cassandra/build/ecj/eclipse_compiler_checks.txt
>  [java] --
>  [java] 1. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 59)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, file.getPath(), iterator);
>  [java]   
> ^^^
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2. ERROR in 
> /home/ubuntu/cassandra/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
>  (at line 79)
>  [java]   return new SSTableIdentityIterator(sstable, key, 
> partitionLevelDeletion, dfile.getPath(), iterator);
>  [java]   
> 
>  [java] Potential resource leak: 'iterator' may not be closed at this 
> location
>  [java] --
>  [java] 2 problems (2 errors)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14242) Indexed static column returns inconsistent results

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña reassigned CASSANDRA-14242:
-

Assignee: Andrés de la Peña

> Indexed static column returns inconsistent results
> --
>
> Key: CASSANDRA-14242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14242
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 3.11.2
> Java driver 3.4.0
> Ubuntu - 4.4.0-112-generic
>Reporter: Ross Black
>Assignee: Andrés de la Peña
>Priority: Major
>
> I am using Cassandra 3.11.2, and the Java driver 3.4.0
> I have a table that has a static column, where the static column has a 
> secondary index.
> When querying the table I get incomplete or duplicated results, depending on 
> the fetch size.
> e.g.
> {code:java}
> CREATE KEYSPACE hack WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> CREATE TABLE hack.stuff (id int, kind text, chunk int static, val1 int, 
> PRIMARY KEY (id, kind));
> CREATE INDEX stuff_chunk_index ON hack.stuff (chunk);{code}
> -- repeat with thousands of values for id =>
> {code:java}
>   INSERT INTO hack.stuff (id, chunk, kind, val1 ) VALUES (${id}, 777, 'A', 
> 123);{code}
> Querying from Java:
> {code:java}
>     final SimpleStatement statement = new SimpleStatement("SELECT id, kind, 
> val1 FROM hack.stuff WHERE chunk = " + chunk); 
>     statement.setFetchSize(fetchSize);
>     statement.setConsistencyLevel(ConsistencyLevel.ALL);
>     final ResultSet resultSet = connection.getSession().execute(statement);
>     for (Row row : resultSet) {
>     final int id = row.getInt("id");
>     }{code}
> *The number of results returned depends on the fetch-size.*
> e.g. For 30k values inserted, I get the following:
> ||fetch-size||result-size||
> |4|3|
> |2|30001|
> |5000|30006|
> |100|30303|
> In production, I have a much larger table where the correct result size for a 
> specific chunk is 20019, but some fetch sizes will return _significantly 
> fewer_ results.
> ||fetch-size||result-size|| ||
> |25000|20019| |
> |5000||*<== this one is has far fewer results*|
> |5001|20026| |
> (so far been unable to reproduce this with the simpler test table)
> Thanks,
> Ross



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Romain Hardouin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391077#comment-16391077
 ] 

Romain Hardouin commented on CASSANDRA-14293:
-

Useful feature, thanks!

Just a reminder (because the ticket doesn't mention it): NEWS.txt should 
mention that NONE is deprecated in favor of NEVER.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14292) Fix batch commitlog sync regression

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391034#comment-16391034
 ] 

ZhaoYang commented on CASSANDRA-14292:
--

Thanks for the review.

> Fix batch commitlog sync regression
> ---
>
> Key: CASSANDRA-14292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14292
> Project: Cassandra
>  Issue Type: Bug
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: 14292-3.0-dtest.png, 14292-3.0-unittest.png
>
>
> Prior to CASSANDRA-13987, in batch commitlog mode, commitlog will be synced 
> to disk right after mutation comes.
>  * haveWork semaphore is released in BatchCommitLogService.maybeWaitForSync
>  * AbstractCommitlogService will continue and sync to disk
> After C-13987, it makes a branch for chain maker flush more frequently in 
> periodic mode. To make sure in batch mode CL still flushes immediately, it 
> added {{syncRequested}} flag.
>  Unfortunately, in 3.0 branch, this flag is not being set to true when 
> mutation is waiting.
> So in AbstractCommitlogService, it will not execute the CL sync branch until 
> it reaches sync window(2ms)..
> {code:java|title=AbstractCommitLogService.java}
> if (lastSyncedAt + syncIntervalMillis <= pollStarted || shutdown || 
> syncRequested)
> {
> // in this branch, we want to flush the commit log to disk
> syncRequested = false;
> commitLog.sync(shutdown, true);
> lastSyncedAt = pollStarted;
> syncComplete.signalAll();
> }
> else
> {
> // in this branch, just update the commit log sync headers
> commitLog.sync(false, false);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13457) Diag. Events: Add base classes

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391026#comment-16391026
 ] 

Stefan Podkowinski commented on CASSANDRA-13457:


{quote}diagnostic_events_enabled: true

After recent discussions on the dev ML around the use of experimental flags, eg 
on MV, would it make more sense that this was false by default?
{quote}
Publishing events without any subscribers should be noop anyways. I intended 
the flag rather as a kill switch, in case the feature is causing any issues in 
production. But you're right, it's a new feature and it should probably not be 
enabled by default. It's not a big deal to change it to true if you're going to 
use it. 
([13b1e8adbd|https://github.com/apache/cassandra/commit/13b1e8adbd3f597311b7f67d71318dc939dd54fb])
{quote}DiagnosticEvent.addIfNotNull(..)

this seems it could be a bit trivial… Can we just enforce toString 
serialisation in the subclasses instead? (like Hint does it)
{quote}
I'm not really a fan of such helper methods either. I guess my intention was to 
keep null values absent from the return mapped and also prevent NPEs. But we 
should catch any exception from {{toMap}} anyways, so I'm fine removing it in 
favor of {{String.valueOf}} on the caller side. 
([5770638d6|https://github.com/apache/cassandra/commit/5770638d689333d59f57a9ef977dbb1fa6964e30])
{quote}Can we avoid the static fields? So to be avoiding adding to the 
CASSANDRA-7837 problems… I don't think C* has a better habit in place for this? 
But a singleton would be one better than all static fields…
{quote}
I don't mind as long as we try to keep things consistent. What's the latest 
trend handling this? {{instance()}}?
 But I don't think swapping out the implementation like in {{Trace}} is a use 
cases that we have to support here. Basically the "service" is just a 
broadcaster that implements publish/subscribe semantics and I can't really 
think of any reason someone wants to change that and use their own 
implementation. 
([ba09523d|https://github.com/apache/cassandra/commit/ba09523de7cad3b3732c0e7e60b072f84e809e21])
{quote}Not too sure why a number of fields in existing classes were changed 
from private to package-protected, for example in Gossiper. If it's for tests 
in latter branches should they deserve the @VisibleForTesting annotation? And 
should that change also happen in the latter branches
{quote}
The {{GossiperEvent}} will access these members from the constructor to collect 
relevant details for the event. Passing these values directly from {{Gossiper}} 
to {{GossiperEvent.*}} would mean that {{Gossiper}} needs to generate some 
garbage by creating immutable copies of some of the values, while at the same 
time events might not even be enabled and would be discarded anyways.
{quote}Should the event classes be included in the client jarfile (or some 
'type and deserialisation' part of the event classes). This would then 
introduce issues of compatibility, (eg enums). If event classes are not exposed 
client-side, could they then be package private? (they're not used outside 
their package)
{quote}
The implementation in CASSANDRA-13459 would only expose events serialized as 
strings. I'd not serialize any internal classes, as we don't have any 
versioning in place and classes could change quickly enough to cause 
deserialization issues, even on bugfix releases. The only reason for keeping 
complex objects as event members would be to make them accessible in unit 
tests, see CASSANDRA-13458. That's also the reason I'd prefer to keep classes 
public, so they can be used for unit testing.
{quote}What about conglomeration between metrics, diag events, and tracing 
events? For example i wonder if the latter two will often pair?, good example 
in HintsDispatcher
{quote}
There's probably some overlap, but I'd expect to rather see metrics on hot 
paths where you want histograms and need to do local buffering and sampling to 
cope with the load. Diag. events should be emitted key events only. As you 
mentioned the HintsDispatcher, one of the first versions had events for 
dispatched hints. After some basic testing I quickly realized that way too much 
events would be produced, if you happen to subscribe to them. You could easily 
maintain a histogram for that, but not emit diag events. So it makes sense to 
limit emitted events for the hints service to dispatcher and maybe pages. 
Although there's some overlap when it comes to looking at the outcome, yes. But 
other diag events such as dispatcherCreated, abortRequested, would not make 
much sense to have as a metric, as you need a bit more context to make them 
useful for further analyzis and the HintEvent has that (hostId, address, 
dispatcher,..).

As for tracing events, I'd expect only some pairing between them and diag 
events for repairs at this point. Avoiding redundancy should be possible, e.g. 
by replacing 

[jira] [Updated] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14215:
-
Since Version: 3.0.0  (was: 3.0.9)

> Cassandra does not seem to be respecting max hint window
> 
>
> Key: CASSANDRA-14215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14215
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hints, Streaming and Messaging
>Reporter: Arijit Banerjee
>Assignee: Kurt Greaves
>Priority: Major
>
> On Cassandra 3.0.9, it was observed that Cassandra continues to write hints 
> even though a node remains down (and does not come up) for longer than the 
> default 3 hour window.
>  
> After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the 
> following log line in cassandra (debug) logs:
>  StorageProxy.java:2625 - Adding hints for [/10.0.100.84]
>  
> One possible code path seems to be:
> cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in 
> StorageProxy.java)
>  
> The "true" parameter above explicitly states that a hint should be recorded 
> and ignores the time window calculation performed by the shouldHint method 
> invoked in other code paths. Is there a reason for this behavior?
>  
> Edit: There are actually two stacks that seem to be producing hints, the 
> "cas" and "syncWriteBatchedMutations" methods. I have posted them below.
>  
> A third issue seems to be that Cassandra seems to reset the timer which 
> counts how long a node has been down after a restart. Thus if Cassandra is 
> restarted on a good node, it continues to accumulate hints for a down node 
> over the next three hours.
>  
> WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - 
> Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) 
> at 
> org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) 
> at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432)
>  at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407)
>  at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) 
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
>  at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at 
> java.lang.Thread.run(Thread.java:748) WARN
>  
>  
> [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding 
> hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) 
> at 
> org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:1247)
>  at 
> org.apache.cassandra.service.StorageProxy.syncWriteBatchedMutations(StorageProxy.java:1014)
>  at 
> org.apache.cassandra.service.StorageProxy.mutateAtomically(StorageProxy.java:899)
>  at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:834)
>  at 
> org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:365)
>  at 
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:343)
>  at 
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:329)
>  at 
> 

[jira] [Updated] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14215:
-
Reproduced In: 3.11.2, 3.0.16  (was: 3.0.9)
   Status: Patch Available  (was: In Progress)

> Cassandra does not seem to be respecting max hint window
> 
>
> Key: CASSANDRA-14215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14215
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hints, Streaming and Messaging
>Reporter: Arijit Banerjee
>Assignee: Kurt Greaves
>Priority: Major
>
> On Cassandra 3.0.9, it was observed that Cassandra continues to write hints 
> even though a node remains down (and does not come up) for longer than the 
> default 3 hour window.
>  
> After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the 
> following log line in cassandra (debug) logs:
>  StorageProxy.java:2625 - Adding hints for [/10.0.100.84]
>  
> One possible code path seems to be:
> cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in 
> StorageProxy.java)
>  
> The "true" parameter above explicitly states that a hint should be recorded 
> and ignores the time window calculation performed by the shouldHint method 
> invoked in other code paths. Is there a reason for this behavior?
>  
> Edit: There are actually two stacks that seem to be producing hints, the 
> "cas" and "syncWriteBatchedMutations" methods. I have posted them below.
>  
> A third issue seems to be that Cassandra seems to reset the timer which 
> counts how long a node has been down after a restart. Thus if Cassandra is 
> restarted on a good node, it continues to accumulate hints for a down node 
> over the next three hours.
>  
> WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - 
> Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) 
> at 
> org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) 
> at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432)
>  at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407)
>  at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) 
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
>  at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at 
> java.lang.Thread.run(Thread.java:748) WARN
>  
>  
> [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding 
> hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) 
> at 
> org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:1247)
>  at 
> org.apache.cassandra.service.StorageProxy.syncWriteBatchedMutations(StorageProxy.java:1014)
>  at 
> org.apache.cassandra.service.StorageProxy.mutateAtomically(StorageProxy.java:899)
>  at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:834)
>  at 
> org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:365)
>  at 
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:343)
>  at 
> 

[jira] [Commented] (CASSANDRA-14215) Cassandra does not seem to be respecting max hint window

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390946#comment-16390946
 ] 

Kurt Greaves commented on CASSANDRA-14215:
--

So I know I said today... but then I realised there were some issues with the 
tests I'd written and instead got distracted by another ticket.

I've got two patches, one for the CAS problem (which should go to 3.0, 3.11, 
and trunk), and a patch for the non-persistent hint window problem which is 
aimed only at trunk. 

CAS patches:
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...kgreav:14215-3.0]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...kgreav:14215-3.11]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...kgreav:14215-trunk-1]|

Hint window patch:
|[trunk|https://github.com/apache/cassandra/compare/trunk...kgreav:14215-trunk-2]|

There's a dtest for each patch, at the moment both are in a single branch.
|[dtest|https://github.com/apache/cassandra-dtest/compare/master...kgreav:14215]|


> Cassandra does not seem to be respecting max hint window
> 
>
> Key: CASSANDRA-14215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14215
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hints, Streaming and Messaging
>Reporter: Arijit Banerjee
>Assignee: Kurt Greaves
>Priority: Major
>
> On Cassandra 3.0.9, it was observed that Cassandra continues to write hints 
> even though a node remains down (and does not come up) for longer than the 
> default 3 hour window.
>  
> After doing "nodetool setlogginglevel org.apache.cassandra TRACE", we see the 
> following log line in cassandra (debug) logs:
>  StorageProxy.java:2625 - Adding hints for [/10.0.100.84]
>  
> One possible code path seems to be:
> cas -> commitPaxos(proposal, consistencyForCommit, true); -> submitHint (in 
> StorageProxy.java)
>  
> The "true" parameter above explicitly states that a hint should be recorded 
> and ignores the time window calculation performed by the shouldHint method 
> invoked in other code paths. Is there a reason for this behavior?
>  
> Edit: There are actually two stacks that seem to be producing hints, the 
> "cas" and "syncWriteBatchedMutations" methods. I have posted them below.
>  
> A third issue seems to be that Cassandra seems to reset the timer which 
> counts how long a node has been down after a restart. Thus if Cassandra is 
> restarted on a good node, it continues to accumulate hints for a down node 
> over the next three hours.
>  
> WARN [SharedPool-Worker-14] 2018-02-06 22:15:51,136 StorageProxy.java:2636 - 
> Adding hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> org.apache.cassandra.service.StorageProxy.stackTrace(StorageProxy.java:2608) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2617) 
> at 
> org.apache.cassandra.service.StorageProxy.submitHint(StorageProxy.java:2603) 
> at 
> org.apache.cassandra.service.StorageProxy.commitPaxos(StorageProxy.java:540) 
> at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:282) at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:432)
>  at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407)
>  at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) 
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
>  at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
>  at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) at 
> java.lang.Thread.run(Thread.java:748) WARN
>  
>  
> [SharedPool-Worker-8] 2018-02-06 22:15:51,153 StorageProxy.java:2636 - Adding 
> hints for [/10.0.100.84] with stack trace: java.lang.Throwable: at 
> 

[jira] [Commented] (CASSANDRA-5836) Seed nodes should be able to bootstrap without manual intervention

[Oleksandr Shulgin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390903#comment-16390903
 ] 

Oleksandr Shulgin commented on CASSANDRA-5836:
--

{quote}As long as there is at least 1 replica (for every range) in the source 
DC it will work.{quote}

Hm, I've tested the procedure quite a number of times and every time I forgot 
to change the replication to NTS or to extend the replication to the new DC I 
was getting a complaint from nodetool rebuild.

{quote}This is quite common for the case where you're adding a DC to a small 
cluster. Personally I'd prefer we get rid of SimpleStrategy altogether... but 
that's likely problematic/controversial.{quote}

Do you mean it is more common to see the error with a small cluster or other 
way round: more common that it will work with a small cluster?

{quote}If we make auto_bootstrap false by default...{quote}

This not what I was suggesting.  Maybe I didn't express myself clear enough.  
What I suggest is to:
1) Allow seed nodes to bootstrap in presence of {{auto_boostrap=true}}, this 
setting still being the default one.
2) Update documented procedure for setting up a new cluster to manually set 
{{auto_boostrap=false}} before starting the nodes for the first time, then 
remove the setting or change it to {{true}}.

This removes all special cases:
a) Setting up the first DC is then not different from setting up an additional 
one w.r.t. {{auto_bootstrap}} setting.
b) Seed nodes are not different from non-seeds w.r.t. bootstrap behavior.
c) The very first seed node is not different from the rest.


> Seed nodes should be able to bootstrap without manual intervention
> --
>
> Key: CASSANDRA-5836
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5836
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Bill Hathaway
>Priority: Minor
>
> The current logic doesn't allow a seed node to be bootstrapped.  If a user 
> wants to bootstrap a node configured as a seed (for example to replace a seed 
> node via replace_token), they first need to remove the node's own IP from the 
> seed list, and then start the bootstrap process.  This seems like an 
> unnecessary step since a node never uses itself as a seed.
> I think it would be a better experience if the logic was changed to allow a 
> seed node to bootstrap without manual intervention when there are other seed 
> nodes up in a ring.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13237) Legacy deserializer can create unexpected boundary range tombstones

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13237:

Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.0.12
   3.11.0

> Legacy deserializer can create unexpected boundary range tombstones
> ---
>
> Key: CASSANDRA-13237
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13237
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Major
> Fix For: 3.0.12, 3.11.0
>
>
> Most of the code don't generate a range tombstone boundary with the same 
> deletion time on both side as this is basically useless, and there is some 
> assertion in {{DataResolver}} that actually expect this. However, the 
> deserializer for legacy sstable doesn't always properly avoid their creation 
> and we can thus generate them (and break the {{DataResolver}} assertion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org