[jira] [Commented] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533153#comment-16533153
 ] 

Benedict commented on CASSANDRA-7282:
-

I actually wonder if we should make this ticket depend on a new ticket to 
separate all memtables by token range.  By doing this at a slightly higher 
level, we can flush data for each token range on an independent schedule.  This 
might permit more even use our disks, and reduce overall compaction work by 
permitting us to flush larger sstables on average.

It's also an easier level to integrate decisions about token range changes, and 
avoids the unsightly slicing (on flush) of already sliced data.

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533127#comment-16533127
 ] 

Benedict edited comment on CASSANDRA-7282 at 7/4/18 11:35 PM:
--

Hi Michael,

I'll try to review this properly, as I find time.  A couple of things leap out 
to me though:
 # While we certainly need to handle out of range tokens, it seems to me that 
we should anyway try hard to minimise their occurrence, by e.g. switching 
memtable on a token range change.  Ideally we would switch before this new 
token range actually takes effect in the case of an expansion; whether or not 
it makes sense to defer a token range removal probably needs some thought.
 # A range tree is probably overkill.  Why not binary search over an array?


was (Author: benedict):
Hi Michael,

I'll try to review this properly, as I find time.  A couple of things leap out 
to me though:
 # While we certainly need to handle out of range tokens, we should try harder 
to minimise their occurrence, by switching memtable on a token range change.  
Ideally we would switch before this new token range actually takes effect in 
the case of an expansion; whether or not it makes sense to defer a token range 
removal probably needs some thought.
 # A range tree is probably overkill.  Why not binary search over an array?

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533127#comment-16533127
 ] 

Benedict commented on CASSANDRA-7282:
-

Hi Michael,

I'll try to review this properly, as I find time.  A couple of things leap out 
to me though:
 # While we certainly need to handle out of range tokens, we should try harder 
to minimise their occurrence, by switching memtable on a token range change.  
Ideally we would switch before this new token range actually takes effect in 
the case of an expansion; whether or not it makes sense to defer a token range 
removal probably needs some thought.
 # A range tree is probably overkill.  Why not binary search over an array?

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7282:

Tester:   (was: Ryan McGuire)

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reassigned CASSANDRA-7282:
---

Assignee: Michael Burman

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Benedict (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7282:

Reviewer: Benedict

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Michael Burman
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14557) Consider adding default and required keyspace replication options

2018-07-04 Thread Vinay Chella (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinay Chella reassigned CASSANDRA-14557:


Assignee: Sumanth Pasupuleti

> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.0
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right 
> now - the system_auth table for example is created with RF=1 (to take into 
> account single node setups afaict from CASSANDRA-5112), and a user can 
> further create a keyspace with RF=1 posing availability and streaming risks 
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
>  # {{default_keyspace_rf}} (default: 1) - If replication factors are not 
> specified, use this number.
>  # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from 
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
>  * Provide defaults for new keyspaces created with SimpleStrategy or 
> NetworkTopologyStrategy (CASSANDRA-14303)
>  * Make the automatic token allocation algorithm interface more intuitive 
> allowing easy use of the new token allocation algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply 
> don’t set the setting. For backwards compatibility the default remains 1 and 
> C* would create with RF=1, and would default to current behavior of allowing 
> any RF on keyspaces.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14555) Verify effect of CASSANDRA-14252 on streaming endpoint selection

2018-07-04 Thread Aleksey Yeschenko (JIRA) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532946#comment-16532946
 ] 

Aleksey Yeschenko commented on CASSANDRA-14555:
---

That might be true, but I share [~beobal]'s concern here. I don't think it was 
perfectly right to introduce the default change so late in the minor cycle in 
3.0.17.

Can we revert from 3.0 and 3.11 please? And apply the suggested mitigations to 
trunk?

Thanks.

> Verify effect of CASSANDRA-14252 on streaming endpoint selection
> 
>
> Key: CASSANDRA-14555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14555
> Project: Cassandra
>  Issue Type: Task
>  Components: Streaming and Messaging
>Reporter: Sam Tunnicliffe
>Priority: Major
> Fix For: 4.x
>
>
> CASSANDRA-14252 makes a slight change to {{DynamicEndpointSnitch}} so that it 
> is somewhat more likely a replica in a remote DC is contacted when replicas 
> in the local DC are considered degraded. This seems reasonable on the read 
> path, but it could also affect selection of endpoints for streaming and cross 
> DC streaming is probably something that operators want to control more 
> tightly. To be clear, I’m not 100% sure that this is actually an issue, but 
> I’d like to have some investigation into it before we ship a change to 
> default behaviour.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-7282) Faster Memtable map

2018-07-04 Thread Michael Burman (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Burman updated CASSANDRA-7282:
--
Status: Patch Available  (was: Open)

> Faster Memtable map
> ---
>
> Key: CASSANDRA-7282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Priority: Major
>  Labels: performance
> Fix For: 4.x
>
> Attachments: jasobrown-sample-run.txt, profile.yaml, reads.svg, 
> run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in 
> our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
> majority of users use a hash partitioner, it occurs to me we could maintain a 
> hybrid ordered list / hash map. The list would impose the normal order on the 
> collection, but a hash index would live alongside as part of the same data 
> structure, simply mapping into the list and permitting O(1) lookups and 
> inserts.
> I've chosen to implement this initial version as a linked-list node per item, 
> but we can optimise this in future by storing fatter nodes that permit a 
> cache-line's worth of hashes to be checked at once,  further reducing the 
> constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14558) dtest: log-watching thread leak and thread starvation

2018-07-04 Thread Stefan Podkowinski (JIRA) 
Stefan Podkowinski created CASSANDRA-14558:
--

 Summary: dtest: log-watching thread leak and thread starvation
 Key: CASSANDRA-14558
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14558
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Stefan Podkowinski
Assignee: Stefan Podkowinski


We get occasional build timeouts on b.a.o after pytest becomes unresponsive for 
over 20 minutes. This will result in a thread dump like this one:

[https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-2.2-dtest/117/consoleFull]

If you look for "Log-watching thread starting" messages and the dump, it 
becomes quite obvious whats the issue here.

I had a quick look at the python3 / pytest related changes in CASSANDRA-14134 
and it seems that some of the handling around dtest_setup's 
{{log_watch_thread}} var has been changed in a way that would prevent 
eventually yielding the allocated thread.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14167) IndexOutOfBoundsException when selecting column counter and consistency quorum

2018-07-04 Thread Sylvain Lebresne (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-14167:
-
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.3
   3.0.17
   Status: Resolved  (was: Patch Available)

Sorry, it slipped to the cracks. CI was clean so committed. As the vote for 
3.0.17 and 3.11.3 got -1ed yesterday, this should make the next releases, 
whenever those happen.

> IndexOutOfBoundsException when selecting column counter and consistency quorum
> --
>
> Key: CASSANDRA-14167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14167
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
> Environment: Cassandra 3.11.1
> Ubuntu 14-04
>Reporter: Tristan Last
>Assignee: Francisco Fernandez
>Priority: Major
> Fix For: 3.0.17, 3.11.3
>
>
> This morning I upgraded my cluster from 3.11.0 to 3.11.1 and it appears when 
> I perform a query on a counter specifying the column name cassandra throws 
> the following exception:
> {code:java}
> WARN [ReadStage-1] 2018-01-15 10:58:30,121 
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread 
> Thread[ReadStage-1,5,main]: {}
> java.lang.IndexOutOfBoundsException: null
> java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_144]
> java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) ~[na:1.8.0_144]
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:173)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.context.CounterContext.updateDigest(CounterContext.java:696)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.AbstractCell.digest(AbstractCell.java:126) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.AbstractRow.digest(AbstractRow.java:73) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:181)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:263)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:120) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadResponse.createDigestResponse(ReadResponse.java:87)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:345) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:50)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_144]
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
>  [apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.11.1.jar:3.11.1]
> java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
> {code}
> Query works completely find on consistency level ONE but not on QUORUM. 
> Is this possibly related to CASSANDRA-11726?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-07-04 Thread slebresne 
Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49e63c27
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49e63c27
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49e63c27

Branch: refs/heads/cassandra-3.11
Commit: 49e63c273e973c0f8ab3769caae6f82c18449f44
Parents: c3a66ab 93012e4
Author: Sylvain Lebresne 
Authored: Wed Jul 4 11:51:34 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 11:51:34 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 23 
 3 files changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49e63c27/CHANGES.txt
--
diff --cc CHANGES.txt
index 951b0c0,00944dc..32ccda3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,6 +1,24 @@@
 -3.0.17
 +3.11.3
 + * Validate supported column type with SASI analyzer (CASSANDRA-13669)
 + * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929)
 + * Reduce nodetool GC thread count (CASSANDRA-14475)
 + * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055)
 + * Remove string formatting lines from BufferPool hot path (CASSANDRA-14416)
 + * Update metrics to 3.1.5 (CASSANDRA-12924)
 + * Detect OpenJDK jvm type and architecture (CASSANDRA-12793)
 + * Don't use guava collections in the non-system keyspace jmx attributes 
(CASSANDRA-12271)
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
 - * Restore resumable hints delivery, backport CASSANDRA-11960 
(CASSANDRA-14419)
   * Always close RT markers returned by ReadCommand#executeLocally() 
(CASSANDRA-14515)
   * Reverse order queries with range tombstones can cause data loss 
(CASSANDRA-14513)
   * Fix regression of lagging commitlog flush log message (CASSANDRA-14451)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49e63c27/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --cc test/unit/org/apache/cassandra/db/CounterCellTest.java
index 74599c3,8859fc6..5208cb2
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@@ -29,12 -29,16 +29,15 @@@ import org.junit.Test
  
  import org.apache.cassandra.SchemaLoader;
  import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.db.rows.BTreeRow;
  import org.apache.cassandra.db.rows.BufferCell;
  import org.apache.cassandra.db.rows.Cell;
 -import org.apache.cassandra.db.rows.CellPath;
  import org.apache.cassandra.db.rows.Cells;
  import org.apache.cassandra.db.context.CounterContext;
+ import org.apache.cassandra.db.rows.Row;
  import org.apache.cassandra.exceptions.ConfigurationException;
  import org.apache.cassandra.schema.KeyspaceParams;
+ import org.apache.cassandra.serializers.AsciiSerializer;
  import org.apache.cassandra.utils.*;
  
  import static org.junit.Assert.*;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Ignore empty Counter cells on digest calculation.

2018-07-04 Thread slebresne 
Ignore empty Counter cells on digest calculation.

patch by Francisco Fernández Castaño; reviewed by Sylvain Lebresne for 
CASSANDRA-14167


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93012e43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93012e43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93012e43

Branch: refs/heads/trunk
Commit: 93012e43ea03411456c20e7cec1ede73ff46b7d2
Parents: c498258
Author: Fransisco Fernandez Castano 
Authored: Sun Apr 22 13:08:26 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 11:50:41 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 23 
 3 files changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee95718..00944dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
  * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419)
  * Always close RT markers returned by ReadCommand#executeLocally() 
(CASSANDRA-14515)
  * Reverse order queries with range tombstones can cause data loss 
(CASSANDRA-14513)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/src/java/org/apache/cassandra/db/context/CounterContext.java
--
diff --git a/src/java/org/apache/cassandra/db/context/CounterContext.java 
b/src/java/org/apache/cassandra/db/context/CounterContext.java
index 29e5cfc..b402464 100644
--- a/src/java/org/apache/cassandra/db/context/CounterContext.java
+++ b/src/java/org/apache/cassandra/db/context/CounterContext.java
@@ -692,6 +692,9 @@ public class CounterContext
  */
 public void updateDigest(MessageDigest message, ByteBuffer context)
 {
+// context can be empty due to the optimization from CASSANDRA-10657
+if (!context.hasRemaining())
+return;
 ByteBuffer dup = context.duplicate();
 dup.position(context.position() + headerLength(context));
 message.update(dup);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/CounterCellTest.java 
b/test/unit/org/apache/cassandra/db/CounterCellTest.java
index a8ddfcc..8859fc6 100644
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@ -29,13 +29,16 @@ import org.junit.Test;
 
 import org.apache.cassandra.SchemaLoader;
 import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.db.rows.BTreeRow;
 import org.apache.cassandra.db.rows.BufferCell;
 import org.apache.cassandra.db.rows.Cell;
 import org.apache.cassandra.db.rows.CellPath;
 import org.apache.cassandra.db.rows.Cells;
 import org.apache.cassandra.db.context.CounterContext;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.schema.KeyspaceParams;
+import org.apache.cassandra.serializers.AsciiSerializer;
 import org.apache.cassandra.utils.*;
 
 import static org.junit.Assert.*;
@@ -281,4 +284,24 @@ public class CounterCellTest
 
 assert Arrays.equals(digest1.digest(), digest2.digest());
 }
+
+@Test
+public void testDigestWithEmptyCells() throws Exception
+{
+// For DB-1881
+ColumnFamilyStore cfs = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(COUNTER1);
+
+ColumnDefinition emptyColDef = 
cfs.metadata.getColumnDefinition(ByteBufferUtil.bytes("val2"));
+BufferCell emptyCell = BufferCell.live(emptyColDef, 0, 
ByteBuffer.allocate(0));
+
+Row.Builder builder = BTreeRow.unsortedBuilder(0);
+
builder.newRow(Clustering.make(AsciiSerializer.instance.serialize("test")));
+builder.addCell(emptyCell);
+Row row = builder.build();
+
+MessageDigest digest = MessageDigest.getInstance("md5");
+row.digest(digest);
+assertNotNull(digest.digest());
+}
+
 }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Ignore empty Counter cells on digest calculation.

2018-07-04 Thread slebresne 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 c4982587b -> 93012e43e
  refs/heads/cassandra-3.11 c3a66ab1e -> 49e63c273
  refs/heads/trunk d500100a4 -> 243c371f4


Ignore empty Counter cells on digest calculation.

patch by Francisco Fernández Castaño; reviewed by Sylvain Lebresne for 
CASSANDRA-14167


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93012e43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93012e43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93012e43

Branch: refs/heads/cassandra-3.0
Commit: 93012e43ea03411456c20e7cec1ede73ff46b7d2
Parents: c498258
Author: Fransisco Fernandez Castano 
Authored: Sun Apr 22 13:08:26 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 11:50:41 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 23 
 3 files changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee95718..00944dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
  * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419)
  * Always close RT markers returned by ReadCommand#executeLocally() 
(CASSANDRA-14515)
  * Reverse order queries with range tombstones can cause data loss 
(CASSANDRA-14513)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/src/java/org/apache/cassandra/db/context/CounterContext.java
--
diff --git a/src/java/org/apache/cassandra/db/context/CounterContext.java 
b/src/java/org/apache/cassandra/db/context/CounterContext.java
index 29e5cfc..b402464 100644
--- a/src/java/org/apache/cassandra/db/context/CounterContext.java
+++ b/src/java/org/apache/cassandra/db/context/CounterContext.java
@@ -692,6 +692,9 @@ public class CounterContext
  */
 public void updateDigest(MessageDigest message, ByteBuffer context)
 {
+// context can be empty due to the optimization from CASSANDRA-10657
+if (!context.hasRemaining())
+return;
 ByteBuffer dup = context.duplicate();
 dup.position(context.position() + headerLength(context));
 message.update(dup);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/CounterCellTest.java 
b/test/unit/org/apache/cassandra/db/CounterCellTest.java
index a8ddfcc..8859fc6 100644
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@ -29,13 +29,16 @@ import org.junit.Test;
 
 import org.apache.cassandra.SchemaLoader;
 import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.db.rows.BTreeRow;
 import org.apache.cassandra.db.rows.BufferCell;
 import org.apache.cassandra.db.rows.Cell;
 import org.apache.cassandra.db.rows.CellPath;
 import org.apache.cassandra.db.rows.Cells;
 import org.apache.cassandra.db.context.CounterContext;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.schema.KeyspaceParams;
+import org.apache.cassandra.serializers.AsciiSerializer;
 import org.apache.cassandra.utils.*;
 
 import static org.junit.Assert.*;
@@ -281,4 +284,24 @@ public class CounterCellTest
 
 assert Arrays.equals(digest1.digest(), digest2.digest());
 }
+
+@Test
+public void testDigestWithEmptyCells() throws Exception
+{
+// For DB-1881
+ColumnFamilyStore cfs = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(COUNTER1);
+
+ColumnDefinition emptyColDef = 
cfs.metadata.getColumnDefinition(ByteBufferUtil.bytes("val2"));
+BufferCell emptyCell = BufferCell.live(emptyColDef, 0, 
ByteBuffer.allocate(0));
+
+Row.Builder builder = BTreeRow.unsortedBuilder(0);
+
builder.newRow(Clustering.make(AsciiSerializer.instance.serialize("test")));
+builder.addCell(emptyCell);
+Row row = builder.build();
+
+MessageDigest digest = MessageDigest.getInstance("md5");
+row.digest(digest);
+assertNotNull(digest.digest());
+}
+
 }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail:

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-07-04 Thread slebresne 
Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49e63c27
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49e63c27
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49e63c27

Branch: refs/heads/trunk
Commit: 49e63c273e973c0f8ab3769caae6f82c18449f44
Parents: c3a66ab 93012e4
Author: Sylvain Lebresne 
Authored: Wed Jul 4 11:51:34 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 11:51:34 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 23 
 3 files changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49e63c27/CHANGES.txt
--
diff --cc CHANGES.txt
index 951b0c0,00944dc..32ccda3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,23 -1,6 +1,24 @@@
 -3.0.17
 +3.11.3
 + * Validate supported column type with SASI analyzer (CASSANDRA-13669)
 + * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929)
 + * Reduce nodetool GC thread count (CASSANDRA-14475)
 + * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055)
 + * Remove string formatting lines from BufferPool hot path (CASSANDRA-14416)
 + * Update metrics to 3.1.5 (CASSANDRA-12924)
 + * Detect OpenJDK jvm type and architecture (CASSANDRA-12793)
 + * Don't use guava collections in the non-system keyspace jmx attributes 
(CASSANDRA-12271)
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
 - * Restore resumable hints delivery, backport CASSANDRA-11960 
(CASSANDRA-14419)
   * Always close RT markers returned by ReadCommand#executeLocally() 
(CASSANDRA-14515)
   * Reverse order queries with range tombstones can cause data loss 
(CASSANDRA-14513)
   * Fix regression of lagging commitlog flush log message (CASSANDRA-14451)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49e63c27/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --cc test/unit/org/apache/cassandra/db/CounterCellTest.java
index 74599c3,8859fc6..5208cb2
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@@ -29,12 -29,16 +29,15 @@@ import org.junit.Test
  
  import org.apache.cassandra.SchemaLoader;
  import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.db.rows.BTreeRow;
  import org.apache.cassandra.db.rows.BufferCell;
  import org.apache.cassandra.db.rows.Cell;
 -import org.apache.cassandra.db.rows.CellPath;
  import org.apache.cassandra.db.rows.Cells;
  import org.apache.cassandra.db.context.CounterContext;
+ import org.apache.cassandra.db.rows.Row;
  import org.apache.cassandra.exceptions.ConfigurationException;
  import org.apache.cassandra.schema.KeyspaceParams;
+ import org.apache.cassandra.serializers.AsciiSerializer;
  import org.apache.cassandra.utils.*;
  
  import static org.junit.Assert.*;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

2018-07-04 Thread slebresne 
Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/243c371f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/243c371f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/243c371f

Branch: refs/heads/trunk
Commit: 243c371f4881be554439731cb809680a37c875ac
Parents: d500100 49e63c2
Author: Sylvain Lebresne 
Authored: Wed Jul 4 12:10:05 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 12:10:05 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 24 +++-
 3 files changed, 27 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/243c371f/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/243c371f/src/java/org/apache/cassandra/db/context/CounterContext.java
--
diff --cc src/java/org/apache/cassandra/db/context/CounterContext.java
index 9b34231,b402464..29dc3f0
--- a/src/java/org/apache/cassandra/db/context/CounterContext.java
+++ b/src/java/org/apache/cassandra/db/context/CounterContext.java
@@@ -690,11 -690,14 +690,14 @@@ public class CounterContex
   * nodes. This means in particular that we always have:
   *  updateDigest(ctx) == updateDigest(clearAllLocal(ctx))
   */
 -public void updateDigest(MessageDigest message, ByteBuffer context)
 +public void updateDigest(Hasher hasher, ByteBuffer context)
  {
+ // context can be empty due to the optimization from CASSANDRA-10657
+ if (!context.hasRemaining())
+ return;
  ByteBuffer dup = context.duplicate();
  dup.position(context.position() + headerLength(context));
 -message.update(dup);
 +HashingUtils.updateBytes(hasher, dup);
  }
  
  /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/243c371f/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --cc test/unit/org/apache/cassandra/db/CounterCellTest.java
index b410427,5208cb2..0ee59e1
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@@ -27,8 -28,8 +27,8 @@@ import org.junit.BeforeClass
  import org.junit.Test;
  
  import org.apache.cassandra.SchemaLoader;
- import org.apache.cassandra.net.MessagingService;
 -import org.apache.cassandra.config.ColumnDefinition;
 +import org.apache.cassandra.schema.ColumnMetadata;
+ import org.apache.cassandra.db.rows.BTreeRow;
  import org.apache.cassandra.db.rows.BufferCell;
  import org.apache.cassandra.db.rows.Cell;
  import org.apache.cassandra.db.rows.Cells;
@@@ -272,12 -275,32 +274,32 @@@ public class CounterCellTes
  
  Cell original = createCounterCellFromContext(cfs, col, state, 5);
  
 -ColumnDefinition cDef = cfs.metadata.getColumnDefinition(col);
 +ColumnMetadata cDef = cfs.metadata().getColumn(col);
  Cell cleared = BufferCell.live(cDef, 5, 
CounterContext.instance().clearAllLocal(state.context));
  
 -original.digest(digest1);
 -cleared.digest(digest2);
 +original.digest(hasher1);
 +cleared.digest(hasher2);
  
 -assert Arrays.equals(digest1.digest(), digest2.digest());
 +Assert.assertEquals(hasher1.hash(), hasher2.hash());
  }
+ 
+ @Test
+ public void testDigestWithEmptyCells() throws Exception
+ {
+ // For DB-1881
+ ColumnFamilyStore cfs = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(COUNTER1);
+ 
 -ColumnDefinition emptyColDef = 
cfs.metadata.getColumnDefinition(ByteBufferUtil.bytes("val2"));
++ColumnMetadata emptyColDef = 
cfs.metadata().getColumn(ByteBufferUtil.bytes("val2"));
+ BufferCell emptyCell = BufferCell.live(emptyColDef, 0, 
ByteBuffer.allocate(0));
+ 
+ Row.Builder builder = BTreeRow.unsortedBuilder(0);
+ 
builder.newRow(Clustering.make(AsciiSerializer.instance.serialize("test")));
+ builder.addCell(emptyCell);
+ Row row = builder.build();
+ 
 -MessageDigest digest = MessageDigest.getInstance("md5");
 -row.digest(digest);
 -assertNotNull(digest.digest());
++Hasher hasher = HashingUtils.CURRENT_HASH_FUNCTION.newHasher();
++row.digest(hasher);
++assertNotNull(hasher.hash());
+ }
+ 
  }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail:

[2/6] cassandra git commit: Ignore empty Counter cells on digest calculation.

2018-07-04 Thread slebresne 
Ignore empty Counter cells on digest calculation.

patch by Francisco Fernández Castaño; reviewed by Sylvain Lebresne for 
CASSANDRA-14167


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93012e43
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93012e43
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93012e43

Branch: refs/heads/cassandra-3.11
Commit: 93012e43ea03411456c20e7cec1ede73ff46b7d2
Parents: c498258
Author: Fransisco Fernandez Castano 
Authored: Sun Apr 22 13:08:26 2018 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jul 4 11:50:41 2018 +0200

--
 CHANGES.txt |  1 +
 .../cassandra/db/context/CounterContext.java|  3 +++
 .../apache/cassandra/db/CounterCellTest.java| 23 
 3 files changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee95718..00944dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
  * Restore resumable hints delivery, backport CASSANDRA-11960 (CASSANDRA-14419)
  * Always close RT markers returned by ReadCommand#executeLocally() 
(CASSANDRA-14515)
  * Reverse order queries with range tombstones can cause data loss 
(CASSANDRA-14513)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/src/java/org/apache/cassandra/db/context/CounterContext.java
--
diff --git a/src/java/org/apache/cassandra/db/context/CounterContext.java 
b/src/java/org/apache/cassandra/db/context/CounterContext.java
index 29e5cfc..b402464 100644
--- a/src/java/org/apache/cassandra/db/context/CounterContext.java
+++ b/src/java/org/apache/cassandra/db/context/CounterContext.java
@@ -692,6 +692,9 @@ public class CounterContext
  */
 public void updateDigest(MessageDigest message, ByteBuffer context)
 {
+// context can be empty due to the optimization from CASSANDRA-10657
+if (!context.hasRemaining())
+return;
 ByteBuffer dup = context.duplicate();
 dup.position(context.position() + headerLength(context));
 message.update(dup);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/93012e43/test/unit/org/apache/cassandra/db/CounterCellTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/CounterCellTest.java 
b/test/unit/org/apache/cassandra/db/CounterCellTest.java
index a8ddfcc..8859fc6 100644
--- a/test/unit/org/apache/cassandra/db/CounterCellTest.java
+++ b/test/unit/org/apache/cassandra/db/CounterCellTest.java
@@ -29,13 +29,16 @@ import org.junit.Test;
 
 import org.apache.cassandra.SchemaLoader;
 import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.db.rows.BTreeRow;
 import org.apache.cassandra.db.rows.BufferCell;
 import org.apache.cassandra.db.rows.Cell;
 import org.apache.cassandra.db.rows.CellPath;
 import org.apache.cassandra.db.rows.Cells;
 import org.apache.cassandra.db.context.CounterContext;
+import org.apache.cassandra.db.rows.Row;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.schema.KeyspaceParams;
+import org.apache.cassandra.serializers.AsciiSerializer;
 import org.apache.cassandra.utils.*;
 
 import static org.junit.Assert.*;
@@ -281,4 +284,24 @@ public class CounterCellTest
 
 assert Arrays.equals(digest1.digest(), digest2.digest());
 }
+
+@Test
+public void testDigestWithEmptyCells() throws Exception
+{
+// For DB-1881
+ColumnFamilyStore cfs = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(COUNTER1);
+
+ColumnDefinition emptyColDef = 
cfs.metadata.getColumnDefinition(ByteBufferUtil.bytes("val2"));
+BufferCell emptyCell = BufferCell.live(emptyColDef, 0, 
ByteBuffer.allocate(0));
+
+Row.Builder builder = BTreeRow.unsortedBuilder(0);
+
builder.newRow(Clustering.make(AsciiSerializer.instance.serialize("test")));
+builder.addCell(emptyCell);
+Row row = builder.build();
+
+MessageDigest digest = MessageDigest.getInstance("md5");
+row.digest(digest);
+assertNotNull(digest.digest());
+}
+
 }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13971) Automatic certificate management using Vault

2018-07-04 Thread Stefan Podkowinski (JIRA) 


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13971:
---
Reviewer:   (was: Jason Brown)

> Automatic certificate management using Vault
> 
>
> Key: CASSANDRA-13971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: security
> Fix For: 4.x
>
> Attachments: start_vault_ssl.sh
>
>
> We've been adding security features during the last years to enable users to 
> secure their clusters, if they are willing to use them and do so correctly. 
> Some features are powerful and easy to work with, such as role based 
> authorization. Other features that require to manage a local keystore are 
> rather painful to deal with. Think about setting up SSL..
> To be fair, keystore related issues and certificate handling hasn't been 
> invented by us. We're just following Java standards there. But that doesn't 
> mean that we absolutely have to, if there are better options. I'd like to 
> give it a shoot and find out if we can automate certificate/key handling 
> (PKI) by using external APIs. In this case, the implementation will be based 
> on [Vault|https://vaultproject.io]. But certificate management services 
> offered by cloud providers may also be able to handle the use-case and I 
> intend to create a generic, pluggable API for that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14167) IndexOutOfBoundsException when selecting column counter and consistency quorum

2018-07-04 Thread Gzegoz Jurgo (JIRA) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529469#comment-16529469
 ] 

Gzegoz Jurgo edited comment on CASSANDRA-14167 at 7/4/18 6:34 AM:
--

[~slebresne] [~fcofdezc] what is the status of this issue ? It is going to be 
fixed with next release ? 


was (Author: gzeska):
[~slebresne] [~fcofdezc] what it the status with this issue ? It is going to be 
fixed with next release ? 

> IndexOutOfBoundsException when selecting column counter and consistency quorum
> --
>
> Key: CASSANDRA-14167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14167
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
> Environment: Cassandra 3.11.1
> Ubuntu 14-04
>Reporter: Tristan Last
>Assignee: Francisco Fernandez
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> This morning I upgraded my cluster from 3.11.0 to 3.11.1 and it appears when 
> I perform a query on a counter specifying the column name cassandra throws 
> the following exception:
> {code:java}
> WARN [ReadStage-1] 2018-01-15 10:58:30,121 
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread 
> Thread[ReadStage-1,5,main]: {}
> java.lang.IndexOutOfBoundsException: null
> java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_144]
> java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) ~[na:1.8.0_144]
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:173)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.context.CounterContext.updateDigest(CounterContext.java:696)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.AbstractCell.digest(AbstractCell.java:126) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.AbstractRow.digest(AbstractRow.java:73) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:181)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:263)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:120) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadResponse.createDigestResponse(ReadResponse.java:87)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:345) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:50)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[apache-cassandra-3.11.1.jar:3.11.1]
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_144]
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
>  [apache-cassandra-3.11.1.jar:3.11.1]
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.11.1.jar:3.11.1]
> java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
> {code}
> Query works completely find on consistency level ONE but not on QUORUM. 
> Is this possibly related to CASSANDRA-11726?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org