[jira] [Commented] (CASSANDRA-10993) Make read and write requests paths fully non-blocking, eliminate related stages

2016-07-10 Thread Norman Maurer (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370184#comment-15370184
 ] 

Norman Maurer commented on CASSANDRA-10993:
---

[~thobbs] so all good ? If not just ping me...

> Make read and write requests paths fully non-blocking, eliminate related 
> stages
> ---
>
> Key: CASSANDRA-10993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10993
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.x
>
>
> Building on work done by [~tjake] (CASSANDRA-10528), [~slebresne] 
> (CASSANDRA-5239), and others, convert read and write request paths to be 
> fully non-blocking, to enable the eventual transition from SEDA to TPC 
> (CASSANDRA-10989)
> Eliminate {{MUTATION}}, {{COUNTER_MUTATION}}, {{VIEW_MUTATION}}, {{READ}}, 
> and {{READ_REPAIR}} stages, move read and write execution directly to Netty 
> context.
> For lack of decent async I/O options on Linux, we’ll still have to retain an 
> extra thread pool for serving read requests for data not residing in our page 
> cache (CASSANDRA-5863), however.
> Implementation-wise, we only have two options available to us: explicit FSMs 
> and chained futures. Fibers would be the third, and easiest option, but 
> aren’t feasible in Java without resorting to direct bytecode manipulation 
> (ourselves or using [quasar|https://github.com/puniverse/quasar]).
> I have seen 4 implementations bases on chained futures/promises now - three 
> in Java and one in C++ - and I’m not convinced that it’s the optimal (or 
> sane) choice for representing our complex logic - think 2i quorum read 
> requests with timeouts at all levels, read repair (blocking and 
> non-blocking), and speculative retries in the mix, {{SERIAL}} reads and 
> writes.
> I’m currently leaning towards an implementation based on explicit FSMs, and 
> intend to provide a prototype - soonish - for comparison with 
> {{CompletableFuture}}-like variants.
> Either way the transition is a relatively boring straightforward refactoring.
> There are, however, some extension points on both write and read paths that 
> we do not control:
> - authorisation implementations will have to be non-blocking. We have control 
> over built-in ones, but for any custom implementation we will have to execute 
> them in a separate thread pool
> - 2i hooks on the write path will need to be non-blocking
> - any trigger implementations will not be allowed to block
> - UDFs and UDAs
> We are further limited by API compatibility restrictions in the 3.x line, 
> forbidding us to alter, or add any non-{{default}} interface methods to those 
> extension points, so these pose a problem.
> Depending on logistics, expecting to get this done in time for 3.4 or 3.6 
> feature release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-8457) nio MessagingService

2016-07-10 Thread Norman Maurer (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370179#comment-15370179
 ] 

Norman Maurer commented on CASSANDRA-8457:
--

[~jasobrown] I did a first review-round and left some comments. All in all it 
looks solid from a Netty perspective, can't say a lot about cassandra 
perspective here :) Let me know if you have any questions etc.

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370159#comment-15370159
 ] 

Stefania commented on CASSANDRA-9318:
-

bq. I've pushed a few more commits to address your concerns.

The new commits look good, the code is clearly cleaner with an abstract 
BackPressureState created by the strategy, and the back-pressure window equal 
to the write timeout. Some things left to do:

* Rebase on trunk
* Get rid of the trailing spaces

bq.  more specifically, the rates are now tracked together when either a 
response is received or the callback is expired,

I have some concerns with this:

* Do we track the case where we receive a failed response? Specifically, in 
ResponseVerbHandler.doVerb, shouldn't we call updateBackPressureState() also 
when the message is a failure response? 
* If we receive a response after it has timed out, won't we count that request 
twice, incorrectly increasing the rate for that window?

bq. Configuration-wise, we're now left with only the back_pressure_enabled 
boolean and the back_pressure_strategy, and I'd really like to keep the former, 
as it makes way easier to dynamically turn the back-pressure on/off.

It's much better right now and I am not strongly opposed to keeping 
back_pressure_enabled, but I also argue that it is quite easy to comment out 
the strategy and to have an empty strategy in the code that means no 
backpressure.

bq. Talking about the overloaded state and the usage of OverloadedException, I 
agree the latter might be misleading, and I agree some failure conditions could 
lead to requests being wrongly refused, but I'd also like to keep some form of 
"emergency" feedback towards the client: what about throwing OE only if all (or 
a given number depending on the CL?) replicas are overloaded?

I think what we may need is a new companion snitch that sorts the replica by 
backpressure ratio, or an enhanced dynamic snitch that is aware of the 
backpressure strategy. If we sort replicas correctly, then we can throw an 
exception to the client if one of the selected replicas are overloaded. 
However, I think the exception needs to be different. native_protocol_v4.spec 
clearly states:

{code}
0x1001Overloaded: the request cannot be processed because the coordinator 
node is overloaded
{code}

Sorry I totally missed the meaning of this exception in the previous round of 
review. 

bq. Finally, one more wild idea to consider: given this patch greatly reduces 
the number of dropped mutations, and hence the number of inflight hints, what 
do you think about disabling load shedding by the replica side when 
back-pressure is enabled? This way we'd trade "full consistency" for an 
hopefully smaller number of unnecessary hints sent over to "pressured" replicas 
when their callbacks expire by the coordinator side.

By "load shedding by the replica" do we mean dropping mutations that have timed 
out or something else?

Regardless, there is the problem of ensuring that all nodes have backpressure 
enabled, which may not be trivial. I think replicas should have a mechanism of 
protecting themselves, rather than relying on the coordinator but I can comment 
further if we clarify what we mean exactly.

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, 
> limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11950) dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum

2016-07-10 Thread Stefania (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-11950:
-
   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.8
   Status: Resolved  (was: Ready to Commit)

> dtest failure in 
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> -
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.8
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log, 
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log, 
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
>   File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error: 
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11950) dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370116#comment-15370116
 ] 

Stefania commented on CASSANDRA-11950:
--

Committed to 3.8 as 5578d3c7b997146e5320b18e2a8134ee79db6f42, to 3.9 as 
d20c765a7d962b55ce905f393589869dab63d996 and merged into trunk.

Closing this ticket as I am pretty sure the schema race was fixed by 
CASSANDRA-12083.

> dtest failure in 
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> -
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log, 
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log, 
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
>   File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error: 
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[2/4] cassandra git commit: Partial revert of CASSANDRA-11971, cannot recycle buffer in SP.sendMessagesToNonlocalDC

2016-07-10 Thread stefania
Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC

patch by Stefania Alborghetti; reviewed by Jake Luciani for CASSANDRA-11950


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d20c765a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d20c765a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d20c765a

Branch: refs/heads/cassandra-3.9
Commit: d20c765a7d962b55ce905f393589869dab63d996
Parents: 1417a51
Author: Stefania Alborghetti 
Authored: Fri Jul 8 10:26:47 2016 +0800
Committer: Stefania Alborghetti 
Committed: Mon Jul 11 10:57:27 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/service/StorageProxy.java | 7 +--
 2 files changed, 2 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d20c765a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b094b00..9df49f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d20c765a/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java 
b/src/java/org/apache/cassandra/service/StorageProxy.java
index c88c449..3ce8013 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -1284,8 +1284,7 @@ public class StorageProxy implements StorageProxyMBean
 InetAddress target = iter.next();
 
 // Add the other destinations of the same message as a FORWARD_HEADER 
entry
-DataOutputBuffer out = null;
-try (DataOutputBuffer ignored = out = DataOutputBuffer.RECYCLER.get())
+try(DataOutputBuffer out = new DataOutputBuffer())
 {
 out.writeInt(targets.size() - 1);
 while (iter.hasNext())
@@ -1311,10 +1310,6 @@ public class StorageProxy implements StorageProxyMBean
 // DataOutputBuffer is in-memory, doesn't throw IOException
 throw new AssertionError(e);
 }
-finally
-{
-out.recycle();
-}
 }
 
 private static void performLocally(Stage stage, final Runnable runnable)



[4/4] cassandra git commit: Merge branch 'cassandra-3.9' into trunk

2016-07-10 Thread stefania
Merge branch 'cassandra-3.9' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5712e6e2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5712e6e2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5712e6e2

Branch: refs/heads/trunk
Commit: 5712e6e2bc619790cc61699aac8ff1958b2ab903
Parents: 09720f8 d20c765
Author: Stefania Alborghetti 
Authored: Mon Jul 11 10:57:42 2016 +0800
Committer: Stefania Alborghetti 
Committed: Mon Jul 11 10:57:42 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/service/StorageProxy.java | 7 +--
 2 files changed, 2 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5712e6e2/CHANGES.txt
--
diff --cc CHANGES.txt
index 690b1d6,9df49f3..92c6edb
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.10
 + * Reuse DataOutputBuffer from ColumnIndex (CASSANDRA-11970)
 + * Remove DatabaseDescriptor dependency from SegmentedFile (CASSANDRA-11580)
 + * Add supplied username to authentication error messages (CASSANDRA-12076)
 + * Remove pre-startup check for open JMX port (CASSANDRA-12074)
 +
  3.9
+  * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
   * Fix hdr logging for single operation workloads (CASSANDRA-12145)
   * Fix SASI PREFIX search in CONTAINS mode with partial terms 
(CASSANDRA-12073)
   * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5712e6e2/src/java/org/apache/cassandra/service/StorageProxy.java
--



[1/4] cassandra git commit: Partial revert of CASSANDRA-11971, cannot recycle buffer in SP.sendMessagesToNonlocalDC

2016-07-10 Thread stefania
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.8 c31701f60 -> 5578d3c7b
  refs/heads/cassandra-3.9 1417a516c -> d20c765a7
  refs/heads/trunk 09720f81d -> 5712e6e2b


Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC

patch by Stefania Alborghetti; reviewed by Jake Luciani for CASSANDRA-11950


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5578d3c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5578d3c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5578d3c7

Branch: refs/heads/cassandra-3.8
Commit: 5578d3c7b997146e5320b18e2a8134ee79db6f42
Parents: c31701f
Author: Stefania Alborghetti 
Authored: Fri Jul 8 10:26:47 2016 +0800
Committer: Stefania Alborghetti 
Committed: Mon Jul 11 10:53:29 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/service/StorageProxy.java | 7 +--
 2 files changed, 2 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5578d3c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3bd691a..df522ec 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.8
+ * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
  * Upgrade netty to 4.0.37 (CASSANDRA-12032, CASSANDRA-12034)
  * Improve details in compaction log message (CASSANDRA-12080)
  * Allow unset values in CQLSSTableWriter (CASSANDRA-11911)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5578d3c7/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java 
b/src/java/org/apache/cassandra/service/StorageProxy.java
index 90c246e..6aee697 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -1282,8 +1282,7 @@ public class StorageProxy implements StorageProxyMBean
 InetAddress target = iter.next();
 
 // Add the other destinations of the same message as a FORWARD_HEADER 
entry
-DataOutputBuffer out = null;
-try (DataOutputBuffer ignored = out = DataOutputBuffer.RECYCLER.get())
+try(DataOutputBuffer out = new DataOutputBuffer())
 {
 out.writeInt(targets.size() - 1);
 while (iter.hasNext())
@@ -1309,10 +1308,6 @@ public class StorageProxy implements StorageProxyMBean
 // DataOutputBuffer is in-memory, doesn't throw IOException
 throw new AssertionError(e);
 }
-finally
-{
-out.recycle();
-}
 }
 
 private static void performLocally(Stage stage, final Runnable runnable)



[3/4] cassandra git commit: Partial revert of CASSANDRA-11971, cannot recycle buffer in SP.sendMessagesToNonlocalDC

2016-07-10 Thread stefania
Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC

patch by Stefania Alborghetti; reviewed by Jake Luciani for CASSANDRA-11950


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d20c765a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d20c765a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d20c765a

Branch: refs/heads/trunk
Commit: d20c765a7d962b55ce905f393589869dab63d996
Parents: 1417a51
Author: Stefania Alborghetti 
Authored: Fri Jul 8 10:26:47 2016 +0800
Committer: Stefania Alborghetti 
Committed: Mon Jul 11 10:57:27 2016 +0800

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/service/StorageProxy.java | 7 +--
 2 files changed, 2 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d20c765a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b094b00..9df49f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.9
+ * Partial revert of CASSANDRA-11971, cannot recycle buffer in 
SP.sendMessagesToNonlocalDC (CASSANDRA-11950)
  * Fix hdr logging for single operation workloads (CASSANDRA-12145)
  * Fix SASI PREFIX search in CONTAINS mode with partial terms (CASSANDRA-12073)
  * Increase size of flushExecutor thread pool (CASSANDRA-12071)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d20c765a/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java 
b/src/java/org/apache/cassandra/service/StorageProxy.java
index c88c449..3ce8013 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -1284,8 +1284,7 @@ public class StorageProxy implements StorageProxyMBean
 InetAddress target = iter.next();
 
 // Add the other destinations of the same message as a FORWARD_HEADER 
entry
-DataOutputBuffer out = null;
-try (DataOutputBuffer ignored = out = DataOutputBuffer.RECYCLER.get())
+try(DataOutputBuffer out = new DataOutputBuffer())
 {
 out.writeInt(targets.size() - 1);
 while (iter.hasNext())
@@ -1311,10 +1310,6 @@ public class StorageProxy implements StorageProxyMBean
 // DataOutputBuffer is in-memory, doesn't throw IOException
 throw new AssertionError(e);
 }
-finally
-{
-out.recycle();
-}
 }
 
 private static void performLocally(Stage stage, final Runnable runnable)



[jira] [Commented] (CASSANDRA-11950) dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370105#comment-15370105
 ] 

Stefania commented on CASSANDRA-11950:
--

Thanks for the review, added to 3.8 and started CI:

||3.8||3.9||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11950-3.8]|[patch|https://github.com/stef1927/cassandra/commits/11950-3.9]|[patch|https://github.com/stef1927/cassandra/commits/11950]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.8-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.8-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-3.9-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11950-dtest/]|


Will commit as soon as CI results are available.

> dtest failure in 
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> -
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log, 
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log, 
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
>   File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error: 
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12150) cqlsh does not automatically downgrade CQL version

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370094#comment-15370094
 ] 

Stefania commented on CASSANDRA-12150:
--

We should probably mention somewhere that, even though cqlsh now connects to 
older server versions, there may be problems and it may not work in all cases 
since we currently do not test cqlsh against older server versions, 
[~iamaleksey] are we OK with this?

> cqlsh does not automatically downgrade CQL version
> --
>
> Key: CASSANDRA-12150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12150
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Yusuke Takata
>Priority: Minor
>  Labels: cqlsh
> Attachments: patch.txt
>
>
> Cassandra drivers such as the Python driver can automatically connect a 
> supported version, 
> but I found that cqlsh does not automatically downgrade CQL version as the 
> following.
> {code}
> $ cqlsh
> Connection error: ('Unable to connect to any servers', {'127.0.0.1': 
> ProtocolError("cql_version '3.4.2' is not supported by remote (w/ native 
> protocol). Supported versions: [u'3.4.0']",)})
> {code}
> I think that the function is useful for cqlsh too. 
> Could someone review the attached patch?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (CASSANDRA-9467) cqlsh does not connect to older protocol versions

2016-07-10 Thread Stefania (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania resolved CASSANDRA-9467.
-
Resolution: Duplicate

Closing in favor of CASSANDRA-12150

> cqlsh does not connect to older protocol versions
> -
>
> Key: CASSANDRA-9467
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9467
> Project: Cassandra
>  Issue Type: Improvement
> Environment: 2.2rc1 against 2.1 / 2.0
>Reporter: Robert Stupp
>Priority: Minor
>  Labels: cqlsh
> Fix For: 3.x
>
>
> {{cqlsh}} (from curent 2.2 branch) does no longer work against C* 2.1 and 2.0.
> {code}
> bin/cqlsh 
> Connection error: ('Unable to connect to any servers', {'127.0.0.1': 
> ConnectionException('Did not get expected SupportedMessage response; instead, 
> got: Server protocol version (3) does not match the specified driver protocol 
> version (4). Consider setting Cluster.protocol_version to 3.',)})
> {code}
> Related to CASSANDRA-9399 ?
> /cc [~stefania_alborghetti]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12150) cqlsh does not automatically downgrade CQL version

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370089#comment-15370089
 ] 

Stefania commented on CASSANDRA-12150:
--

+1, I've also removed {{DEFAULT_CQL_VERSION}}. 

I'm not sure we need {{--cqlversion}} at all, given that a server will not 
accept an older version, but I've left it for backward compatibility.

Test results are still pending:

|*trunk*||[patch|https://github.com/stef1927/cassandra/commits/12150-cqlsh]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12150-cqlsh-cqlsh-tests/]|


> cqlsh does not automatically downgrade CQL version
> --
>
> Key: CASSANDRA-12150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12150
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Yusuke Takata
>Priority: Minor
>  Labels: cqlsh
> Attachments: patch.txt
>
>
> Cassandra drivers such as the Python driver can automatically connect a 
> supported version, 
> but I found that cqlsh does not automatically downgrade CQL version as the 
> following.
> {code}
> $ cqlsh
> Connection error: ('Unable to connect to any servers', {'127.0.0.1': 
> ProtocolError("cql_version '3.4.2' is not supported by remote (w/ native 
> protocol). Supported versions: [u'3.4.0']",)})
> {code}
> I think that the function is useful for cqlsh too. 
> Could someone review the attached patch?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11950) dtest failure in consistency_test.TestAvailability.test_network_topology_strategy_each_quorum

2016-07-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370042#comment-15370042
 ] 

Stefania commented on CASSANDRA-11950:
--

PR to remove {{UnknownColumnFamilyException}} warnings from the logs is 
[here|https://github.com/riptano/cassandra-dtest/pull/1083], dependent on 
[this|https://github.com/datastax/python-driver/pull/617] driver pull request.

> dtest failure in 
> consistency_test.TestAvailability.test_network_topology_strategy_each_quorum
> -
>
> Key: CASSANDRA-11950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11950
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log, node4.log, node4_debug.log, node5.log, 
> node5_debug.log, node6.log, node6_debug.log, node7.log, node7_debug.log, 
> node8.log, node8_debug.log, node9.log, node9_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/10/testReport/consistency_test/TestAvailability/test_network_topology_strategy_each_quorum
> Failed on CassCI build trunk_large_dtest #10
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 358, in run
> self.tearDown()
>   File "/home/automaton/cassandra-dtest/dtest.py", line 719, in tearDown
> raise AssertionError('Unexpected error in log, see stdout')
> Standard Output
> Unexpected error in node3 log, error: 
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,460 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,462 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-2] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-3] 2016-06-03 14:25:27,464 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-1] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-4] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-5] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-7] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> ERROR [SharedPool-Worker-6] 2016-06-03 14:25:27,465 Keyspace.java:504 - 
> Attempting to mutate non-existant table 03b14ad0-2997-11e6-b8c7-01c3aea11be7 
> (mytestks.users)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements

2016-07-10 Thread Oded Peer (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oded Peer updated CASSANDRA-11424:
--
Fix Version/s: 3.8
   Status: Patch Available  (was: Open)

> Add support to "unset" JSON fields in prepared statements
> -
>
> Key: CASSANDRA-11424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11424
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ralf Steppacher
>Assignee: Oded Peer
>  Labels: client-impacting, cql
> Fix For: 3.8
>
> Attachments: 11424-trunk-V1.txt
>
>
> CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and 
> {{UNSET}} prepared statement parameters.
> When inserting JSON objects it is not possible to profit from this as a 
> prepared statement only has one parameter that is bound to the JSON object as 
> a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for 
> columns omitted from the JSON object.
> Please extend on CASSANDRA-7304 to include JSON support.
> {color:grey}
> (My personal requirement is to be able to insert JSON objects with optional 
> fields without incurring the overhead of creating a tombstone of every column 
> not covered by the JSON object upon initial(!) insert.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements

2016-07-10 Thread Oded Peer (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oded Peer updated CASSANDRA-11424:
--
Attachment: 11424-trunk-V1.txt

> Add support to "unset" JSON fields in prepared statements
> -
>
> Key: CASSANDRA-11424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11424
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ralf Steppacher
>Assignee: Oded Peer
>  Labels: client-impacting, cql
> Attachments: 11424-trunk-V1.txt
>
>
> CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and 
> {{UNSET}} prepared statement parameters.
> When inserting JSON objects it is not possible to profit from this as a 
> prepared statement only has one parameter that is bound to the JSON object as 
> a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for 
> columns omitted from the JSON object.
> Please extend on CASSANDRA-7304 to include JSON support.
> {color:grey}
> (My personal requirement is to be able to insert JSON objects with optional 
> fields without incurring the overhead of creating a tombstone of every column 
> not covered by the JSON object upon initial(!) insert.)
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable

2016-07-10 Thread Jeff Jirsa (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-11715:
---
Status: Patch Available  (was: In Progress)

||Branch||testall||dtest||
| [Trunk|https://github.com/jeffjirsa/cassandra/tree/cassandra-11715] | 
http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-testall/2/testReport/ 
| http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-dtest/lastBuild/ | 
| [3.0|https://github.com/jeffjirsa/cassandra/tree/cassandra-11715-3.0] | 
http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-3.0-testall/lastBuild/testReport/
 | 
http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-3.0-dtest/lastBuild/ | 
| [2.2|https://github.com/jeffjirsa/cassandra/tree/cassandra-11715-2.2] | 
http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-2.2-testall/lastBuild/testReport/
 | 
http://cassci.datastax.com/job/jeffjirsa-cassandra-11715-2.2-dtest/lastBuild/ | 



> Make GCInspector's MIN_LOG_DURATION configurable
> 
>
> Key: CASSANDRA-11715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11715
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Assignee: Jeff Jirsa
>Priority: Minor
>  Labels: lhf
>
> It's common for people to run C* with the G1 collector on appropriately-sized 
> heaps.  Quite often, the target pause time is set to 500ms, but GCI fires on 
> anything over 200ms.  We can already control the warn threshold, but these 
> are acceptable GCs for the configuration and create noise at the INFO log 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)