[jira] [Updated] (CASSANDRA-12876) Negative mean write latency

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12876:
---
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 4.x)
   (was: 3.0.x)
   (was: 2.2.x)
   4.0
   3.11.0
   3.0.11
   2.2.9
Reproduced In: 3.0.9, 2.2.8  (was: 2.2.8, 3.0.9)
   Status: Resolved  (was: Ready to Commit)

Thanks to all three of you for your help on this [~eperott] [~cnlwsu] 
[~alprema] ! Committed to 2.2 as {{7e05f393f39c90ec0447b1cc893ff46901ae3071}} 
and merged up to 3.0, 3.11, and trunk. 

> Negative mean write latency
> ---
>
> Key: CASSANDRA-12876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12876
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Kévin LOVATO
>Assignee: Per Otterström
> Fix For: 2.2.9, 3.0.11, 3.11.0, 4.0
>
> Attachments: 12876-2.2.txt, 12876-2.2-v2.txt, 
> negative_mean_details.PNG, negative_mean_periodicity.PNG, negative_mean.png, 
> negative_mean_read_latency.png
>
>
> The mean write latency returned by JMX turns negative every 30 minutes. As 
> the attached screenshots show, the value turns negative every 30 minutes 
> after the startup of the node.
> We did not experience this behavior in 2.1.16.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jjirsa]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51e3660c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51e3660c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51e3660c

Branch: refs/heads/trunk
Commit: 51e3660c77181403bc841a9eae7a7dd2e8ad3414
Parents: 8ec6090 7e05f39
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:28:47 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:29:11 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/51e3660c/CHANGES.txt
--
diff --cc CHANGES.txt
index 700bde0,214fe97..b720a0e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,47 -1,8 +1,48 @@@
 -2.2.9
 +3.0.11
 + * Hint related logging should include the IP address of the destination in 
addition to 
 +   host ID (CASSANDRA-13205)
 + * Reloading logback.xml does not work (CASSANDRA-13173)
 + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 
(CASSANDRA-13109)
 + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125)
 + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152)
 + * Abort or retry on failed hints delivery (CASSANDRA-13124)
 + * Fix handling of partition with partition-level deletion plus
 +   live rows in sstabledump (CASSANDRA-13177)
 + * Provide user workaround when system_schema.columns does not contain entries
 +   for a table that's in system_schema.tables (CASSANDRA-13180)
 + * Dump threads when unit tests time out (CASSANDRA-13117)
 + * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
 + * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
 + * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
 + * Stress daemon help is incorrect (CASSANDRA-12563)
 + * Remove ALTER TYPE support (CASSANDRA-12443)
 + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
 + * Set javac encoding to utf-8 (CASSANDRA-11077)
 + * Replace empty strings with null values if they cannot be converted 
(CASSANDRA-12794)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620)
 + * Add parent repair session id to anticompaction log message 
(CASSANDRA-12186)
 + * Improve contention handling on failure to acquire MV lock for streaming 
and hints (CASSANDRA-12905)
 + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829)
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Fix negative mean latency metric (CASSANDRA-12876)
   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
   * Fix speculative retry bugs (CASSANDRA-13009)
 - * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
 + * Fix race causing infinite loop if Thrift server is stopped before it 
starts listening (CASSANDRA-12856)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
   * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
   * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
   * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed45d5eb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed45d5eb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed45d5eb

Branch: refs/heads/trunk
Commit: ed45d5eb70f67e6379fe089edf6cbc0b5e23dc50
Parents: 94a8c77 51e3660
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:29:24 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:29:56 2017 -0800

--
 CHANGES.txt |  3 +-
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 35 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed45d5eb/CHANGES.txt
--
diff --cc CHANGES.txt
index b34e5f6,b720a0e..5be6cef
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -17,129 -10,6 +17,130 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
-   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
++ * Fix negative mean latency metric (CASSANDRA-12876)
++ * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo 

[02/10] cassandra git commit: Fix negative mean latency metric

[jjirsa]

Fix negative mean latency metric

Patch by Per Otterstrom; Reviewed by Chris Lohfink for CASSANDRA-12876


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e05f393
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e05f393
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e05f393

Branch: refs/heads/cassandra-3.0
Commit: 7e05f393f39c90ec0447b1cc893ff46901ae3071
Parents: c884b70
Author: Per Otterstrom 
Authored: Wed Jan 18 14:56:01 2017 +0100
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:28:35 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc79796..214fe97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
  * Use only one file pointer when creating commitlog segments (CASSANDRA-12539)
  * Fix speculative retry bugs (CASSANDRA-13009)
  * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
--
diff --git 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
index 2458164..2f8bdf8 100644
--- 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
+++ 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
@@ -352,11 +352,12 @@ public class DecayingEstimatedHistogramReservoir 
implements Reservoir
 public 
EstimatedHistogramReservoirSnapshot(DecayingEstimatedHistogramReservoir 
reservoir)
 {
 final int length = reservoir.decayingBuckets.length();
+final double rescaleFactor = forwardDecayWeight(clock.getTime());
 
 this.decayingBuckets = new long[length];
 
 for (int i = 0; i < length; i++)
-this.decayingBuckets[i] = reservoir.decayingBuckets.get(i);
+this.decayingBuckets[i] = 
Math.round(reservoir.decayingBuckets.get(i) / rescaleFactor);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
index f2d817f..ef1fed3 100644
--- 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
+++ 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
@@ -354,6 +354,32 @@ public class DecayingEstimatedHistogramReservoirTest
 }
 }
 
+@Test
+public void testDecayingMean()
+{
+{
+TestClock clock = new TestClock();
+
+DecayingEstimatedHistogramReservoir histogram = new 
DecayingEstimatedHistogramReservoir(DecayingEstimatedHistogramReservoir.DEFAULT_ZERO_CONSIDERATION,
 DecayingEstimatedHistogramReservoir.DEFAULT_BUCKET_COUNT, clock);
+
+
clock.addMillis(DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS
 - 1_000L);
+
+while (clock.getTime() < 
DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS + 1_000L)
+{
+clock.addMillis(900);
+for (int i = 0; i < 1_000_000; i++)
+{
+histogram.update(1000);
+histogram.update(2000);
+histogram.update(3000);
+histogram.update(4000);
+histogram.update(5000);
+}
+assertEquals(3000D, histogram.getSnapshot().getMean(), 500D);
+}
+}
+}
+
 private void assertEstimatedQuantile(long expectedValue, double 
actualValue)
 {
 assertTrue("Expected at least [" + expectedValue + "] but actual is [" 
+ actualValue + "]", actualValue >= expectedValue);
@@ -363,6 +389,11 @@ public class DecayingEstimatedHistogramReservoirTest
 public class TestClock extends Clock {
 private long tick = 0;
 
+public void 

[03/10] cassandra git commit: Fix negative mean latency metric

[jjirsa]

Fix negative mean latency metric

Patch by Per Otterstrom; Reviewed by Chris Lohfink for CASSANDRA-12876


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e05f393
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e05f393
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e05f393

Branch: refs/heads/cassandra-3.11
Commit: 7e05f393f39c90ec0447b1cc893ff46901ae3071
Parents: c884b70
Author: Per Otterstrom 
Authored: Wed Jan 18 14:56:01 2017 +0100
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:28:35 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc79796..214fe97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
  * Use only one file pointer when creating commitlog segments (CASSANDRA-12539)
  * Fix speculative retry bugs (CASSANDRA-13009)
  * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
--
diff --git 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
index 2458164..2f8bdf8 100644
--- 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
+++ 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
@@ -352,11 +352,12 @@ public class DecayingEstimatedHistogramReservoir 
implements Reservoir
 public 
EstimatedHistogramReservoirSnapshot(DecayingEstimatedHistogramReservoir 
reservoir)
 {
 final int length = reservoir.decayingBuckets.length();
+final double rescaleFactor = forwardDecayWeight(clock.getTime());
 
 this.decayingBuckets = new long[length];
 
 for (int i = 0; i < length; i++)
-this.decayingBuckets[i] = reservoir.decayingBuckets.get(i);
+this.decayingBuckets[i] = 
Math.round(reservoir.decayingBuckets.get(i) / rescaleFactor);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
index f2d817f..ef1fed3 100644
--- 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
+++ 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
@@ -354,6 +354,32 @@ public class DecayingEstimatedHistogramReservoirTest
 }
 }
 
+@Test
+public void testDecayingMean()
+{
+{
+TestClock clock = new TestClock();
+
+DecayingEstimatedHistogramReservoir histogram = new 
DecayingEstimatedHistogramReservoir(DecayingEstimatedHistogramReservoir.DEFAULT_ZERO_CONSIDERATION,
 DecayingEstimatedHistogramReservoir.DEFAULT_BUCKET_COUNT, clock);
+
+
clock.addMillis(DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS
 - 1_000L);
+
+while (clock.getTime() < 
DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS + 1_000L)
+{
+clock.addMillis(900);
+for (int i = 0; i < 1_000_000; i++)
+{
+histogram.update(1000);
+histogram.update(2000);
+histogram.update(3000);
+histogram.update(4000);
+histogram.update(5000);
+}
+assertEquals(3000D, histogram.getSnapshot().getMean(), 500D);
+}
+}
+}
+
 private void assertEstimatedQuantile(long expectedValue, double 
actualValue)
 {
 assertTrue("Expected at least [" + expectedValue + "] but actual is [" 
+ actualValue + "]", actualValue >= expectedValue);
@@ -363,6 +389,11 @@ public class DecayingEstimatedHistogramReservoirTest
 public class TestClock extends Clock {
 private long tick = 0;
 
+public void 

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jjirsa]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51e3660c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51e3660c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51e3660c

Branch: refs/heads/cassandra-3.0
Commit: 51e3660c77181403bc841a9eae7a7dd2e8ad3414
Parents: 8ec6090 7e05f39
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:28:47 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:29:11 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/51e3660c/CHANGES.txt
--
diff --cc CHANGES.txt
index 700bde0,214fe97..b720a0e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,47 -1,8 +1,48 @@@
 -2.2.9
 +3.0.11
 + * Hint related logging should include the IP address of the destination in 
addition to 
 +   host ID (CASSANDRA-13205)
 + * Reloading logback.xml does not work (CASSANDRA-13173)
 + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 
(CASSANDRA-13109)
 + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125)
 + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152)
 + * Abort or retry on failed hints delivery (CASSANDRA-13124)
 + * Fix handling of partition with partition-level deletion plus
 +   live rows in sstabledump (CASSANDRA-13177)
 + * Provide user workaround when system_schema.columns does not contain entries
 +   for a table that's in system_schema.tables (CASSANDRA-13180)
 + * Dump threads when unit tests time out (CASSANDRA-13117)
 + * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
 + * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
 + * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
 + * Stress daemon help is incorrect (CASSANDRA-12563)
 + * Remove ALTER TYPE support (CASSANDRA-12443)
 + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
 + * Set javac encoding to utf-8 (CASSANDRA-11077)
 + * Replace empty strings with null values if they cannot be converted 
(CASSANDRA-12794)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620)
 + * Add parent repair session id to anticompaction log message 
(CASSANDRA-12186)
 + * Improve contention handling on failure to acquire MV lock for streaming 
and hints (CASSANDRA-12905)
 + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829)
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Fix negative mean latency metric (CASSANDRA-12876)
   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
   * Fix speculative retry bugs (CASSANDRA-13009)
 - * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
 + * Fix race causing infinite loop if Thrift server is stopped before it 
starts listening (CASSANDRA-12856)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
   * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
   * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
   * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jjirsa]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1eaef3f9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1eaef3f9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1eaef3f9

Branch: refs/heads/trunk
Commit: 1eaef3f9edb1043319e7a06f6c77afb3c41ecd42
Parents: e1dda71 ed45d5e
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:30:05 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:30:31 2017 -0800

--
 CHANGES.txt |  3 +-
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 35 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1eaef3f9/CHANGES.txt
--
diff --cc CHANGES.txt
index 6c1b9b6,5be6cef..7c38295
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -49,13 -18,12 +49,14 @@@ Merged from 3.0
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
  Merged from 2.2:
-   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
+  * Fix negative mean latency metric (CASSANDRA-12876)
+  * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
  Merged from 2.1:
   * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  
 +
  3.10
   * Fix secondary index queries regression (CASSANDRA-13013)
   * Add duration type to the protocol V5 (CASSANDRA-12850)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1eaef3f9/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
--

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jjirsa]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed45d5eb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed45d5eb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed45d5eb

Branch: refs/heads/cassandra-3.11
Commit: ed45d5eb70f67e6379fe089edf6cbc0b5e23dc50
Parents: 94a8c77 51e3660
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:29:24 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:29:56 2017 -0800

--
 CHANGES.txt |  3 +-
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 35 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed45d5eb/CHANGES.txt
--
diff --cc CHANGES.txt
index b34e5f6,b720a0e..5be6cef
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -17,129 -10,6 +17,130 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
-   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
++ * Fix negative mean latency metric (CASSANDRA-12876)
++ * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
 + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jjirsa]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51e3660c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51e3660c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51e3660c

Branch: refs/heads/cassandra-3.11
Commit: 51e3660c77181403bc841a9eae7a7dd2e8ad3414
Parents: 8ec6090 7e05f39
Author: Jeff Jirsa 
Authored: Sat Feb 11 23:28:47 2017 -0800
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:29:11 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/51e3660c/CHANGES.txt
--
diff --cc CHANGES.txt
index 700bde0,214fe97..b720a0e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,47 -1,8 +1,48 @@@
 -2.2.9
 +3.0.11
 + * Hint related logging should include the IP address of the destination in 
addition to 
 +   host ID (CASSANDRA-13205)
 + * Reloading logback.xml does not work (CASSANDRA-13173)
 + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 
(CASSANDRA-13109)
 + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125)
 + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152)
 + * Abort or retry on failed hints delivery (CASSANDRA-13124)
 + * Fix handling of partition with partition-level deletion plus
 +   live rows in sstabledump (CASSANDRA-13177)
 + * Provide user workaround when system_schema.columns does not contain entries
 +   for a table that's in system_schema.tables (CASSANDRA-13180)
 + * Dump threads when unit tests time out (CASSANDRA-13117)
 + * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
 + * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
 + * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
 + * Stress daemon help is incorrect (CASSANDRA-12563)
 + * Remove ALTER TYPE support (CASSANDRA-12443)
 + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
 + * Set javac encoding to utf-8 (CASSANDRA-11077)
 + * Replace empty strings with null values if they cannot be converted 
(CASSANDRA-12794)
 + * Fixed flacky SSTableRewriterTest: check file counts before calling 
validateCFS (CASSANDRA-12348)
 + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620)
 + * Add parent repair session id to anticompaction log message 
(CASSANDRA-12186)
 + * Improve contention handling on failure to acquire MV lock for streaming 
and hints (CASSANDRA-12905)
 + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829)
 + * Mark MVs as built after successful bootstrap (CASSANDRA-12984)
 + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME 
(CASSANDRA-13040)
 + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021)
 + * Thread local pools never cleaned up (CASSANDRA-13033)
 + * Set RPC_READY to false when draining or if a node is marked as shutdown 
(CASSANDRA-12781)
 + * Make sure sstables only get committed when it's safe to discard commit log 
records (CASSANDRA-12956)
 + * Reject default_time_to_live option when creating or altering MVs 
(CASSANDRA-12868)
 + * Nodetool should use a more sane max heap size (CASSANDRA-12739)
 + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651)
 + * AnticompactionRequestSerializer serializedSize is incorrect 
(CASSANDRA-12934)
 + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535)
 + * Reenable HeapPool (CASSANDRA-12900)
 +Merged from 2.2:
+  * Fix negative mean latency metric (CASSANDRA-12876)
   * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
   * Fix speculative retry bugs (CASSANDRA-13009)
 - * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
 + * Fix race causing infinite loop if Thrift server is stopped before it 
starts listening (CASSANDRA-12856)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
   * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
   * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
   * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)

[04/10] cassandra git commit: Fix negative mean latency metric

[jjirsa]

Fix negative mean latency metric

Patch by Per Otterstrom; Reviewed by Chris Lohfink for CASSANDRA-12876


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e05f393
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e05f393
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e05f393

Branch: refs/heads/trunk
Commit: 7e05f393f39c90ec0447b1cc893ff46901ae3071
Parents: c884b70
Author: Per Otterstrom 
Authored: Wed Jan 18 14:56:01 2017 +0100
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:28:35 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc79796..214fe97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
  * Use only one file pointer when creating commitlog segments (CASSANDRA-12539)
  * Fix speculative retry bugs (CASSANDRA-13009)
  * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
--
diff --git 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
index 2458164..2f8bdf8 100644
--- 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
+++ 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
@@ -352,11 +352,12 @@ public class DecayingEstimatedHistogramReservoir 
implements Reservoir
 public 
EstimatedHistogramReservoirSnapshot(DecayingEstimatedHistogramReservoir 
reservoir)
 {
 final int length = reservoir.decayingBuckets.length();
+final double rescaleFactor = forwardDecayWeight(clock.getTime());
 
 this.decayingBuckets = new long[length];
 
 for (int i = 0; i < length; i++)
-this.decayingBuckets[i] = reservoir.decayingBuckets.get(i);
+this.decayingBuckets[i] = 
Math.round(reservoir.decayingBuckets.get(i) / rescaleFactor);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
index f2d817f..ef1fed3 100644
--- 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
+++ 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
@@ -354,6 +354,32 @@ public class DecayingEstimatedHistogramReservoirTest
 }
 }
 
+@Test
+public void testDecayingMean()
+{
+{
+TestClock clock = new TestClock();
+
+DecayingEstimatedHistogramReservoir histogram = new 
DecayingEstimatedHistogramReservoir(DecayingEstimatedHistogramReservoir.DEFAULT_ZERO_CONSIDERATION,
 DecayingEstimatedHistogramReservoir.DEFAULT_BUCKET_COUNT, clock);
+
+
clock.addMillis(DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS
 - 1_000L);
+
+while (clock.getTime() < 
DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS + 1_000L)
+{
+clock.addMillis(900);
+for (int i = 0; i < 1_000_000; i++)
+{
+histogram.update(1000);
+histogram.update(2000);
+histogram.update(3000);
+histogram.update(4000);
+histogram.update(5000);
+}
+assertEquals(3000D, histogram.getSnapshot().getMean(), 500D);
+}
+}
+}
+
 private void assertEstimatedQuantile(long expectedValue, double 
actualValue)
 {
 assertTrue("Expected at least [" + expectedValue + "] but actual is [" 
+ actualValue + "]", actualValue >= expectedValue);
@@ -363,6 +389,11 @@ public class DecayingEstimatedHistogramReservoirTest
 public class TestClock extends Clock {
 private long tick = 0;
 
+public void 

[01/10] cassandra git commit: Fix negative mean latency metric

[jjirsa]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 c884b701d -> 7e05f393f
  refs/heads/cassandra-3.0 8ec609002 -> 51e3660c7
  refs/heads/cassandra-3.11 94a8c777a -> ed45d5eb7
  refs/heads/trunk e1dda71a4 -> 1eaef3f9e


Fix negative mean latency metric

Patch by Per Otterstrom; Reviewed by Chris Lohfink for CASSANDRA-12876


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7e05f393
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7e05f393
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7e05f393

Branch: refs/heads/cassandra-2.2
Commit: 7e05f393f39c90ec0447b1cc893ff46901ae3071
Parents: c884b70
Author: Per Otterstrom 
Authored: Wed Jan 18 14:56:01 2017 +0100
Committer: Jeff Jirsa 
Committed: Sat Feb 11 23:28:35 2017 -0800

--
 CHANGES.txt |  1 +
 .../DecayingEstimatedHistogramReservoir.java|  3 +-
 ...DecayingEstimatedHistogramReservoirTest.java | 31 
 3 files changed, 34 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc79796..214fe97 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
  * Use only one file pointer when creating commitlog segments (CASSANDRA-12539)
  * Fix speculative retry bugs (CASSANDRA-13009)
  * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
--
diff --git 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
index 2458164..2f8bdf8 100644
--- 
a/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
+++ 
b/src/java/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoir.java
@@ -352,11 +352,12 @@ public class DecayingEstimatedHistogramReservoir 
implements Reservoir
 public 
EstimatedHistogramReservoirSnapshot(DecayingEstimatedHistogramReservoir 
reservoir)
 {
 final int length = reservoir.decayingBuckets.length();
+final double rescaleFactor = forwardDecayWeight(clock.getTime());
 
 this.decayingBuckets = new long[length];
 
 for (int i = 0; i < length; i++)
-this.decayingBuckets[i] = reservoir.decayingBuckets.get(i);
+this.decayingBuckets[i] = 
Math.round(reservoir.decayingBuckets.get(i) / rescaleFactor);
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7e05f393/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
index f2d817f..ef1fed3 100644
--- 
a/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
+++ 
b/test/unit/org/apache/cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java
@@ -354,6 +354,32 @@ public class DecayingEstimatedHistogramReservoirTest
 }
 }
 
+@Test
+public void testDecayingMean()
+{
+{
+TestClock clock = new TestClock();
+
+DecayingEstimatedHistogramReservoir histogram = new 
DecayingEstimatedHistogramReservoir(DecayingEstimatedHistogramReservoir.DEFAULT_ZERO_CONSIDERATION,
 DecayingEstimatedHistogramReservoir.DEFAULT_BUCKET_COUNT, clock);
+
+
clock.addMillis(DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS
 - 1_000L);
+
+while (clock.getTime() < 
DecayingEstimatedHistogramReservoir.LANDMARK_RESET_INTERVAL_IN_MS + 1_000L)
+{
+clock.addMillis(900);
+for (int i = 0; i < 1_000_000; i++)
+{
+histogram.update(1000);
+histogram.update(2000);
+histogram.update(3000);
+histogram.update(4000);
+histogram.update(5000);
+}
+assertEquals(3000D, histogram.getSnapshot().getMean(), 500D);
+}
+}
+}
+
 private void assertEstimatedQuantile(long expectedValue, double 
actualValue)
 {
 assertTrue("Expected at least [" + expectedValue + "] but 

[jira] [Updated] (CASSANDRA-12876) Negative mean write latency

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12876:
---
Status: Ready to Commit  (was: Patch Available)

> Negative mean write latency
> ---
>
> Key: CASSANDRA-12876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12876
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Kévin LOVATO
>Assignee: Per Otterström
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 12876-2.2.txt, 12876-2.2-v2.txt, 
> negative_mean_details.PNG, negative_mean_periodicity.PNG, negative_mean.png, 
> negative_mean_read_latency.png
>
>
> The mean write latency returned by JMX turns negative every 30 minutes. As 
> the attached screenshots show, the value turns negative every 30 minutes 
> after the startup of the node.
> We did not experience this behavior in 2.1.16.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12876) Negative mean write latency

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862660#comment-15862660
 ] 

Chris Lohfink commented on CASSANDRA-12876:
---

+1 from me, could reproduce the negative means before and couldn't afterwards.

> Negative mean write latency
> ---
>
> Key: CASSANDRA-12876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12876
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Kévin LOVATO
>Assignee: Per Otterström
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 12876-2.2.txt, 12876-2.2-v2.txt, 
> negative_mean_details.PNG, negative_mean_periodicity.PNG, negative_mean.png, 
> negative_mean_read_latency.png
>
>
> The mean write latency returned by JMX turns negative every 30 minutes. As 
> the attached screenshots show, the value turns negative every 30 minutes 
> after the startup of the node.
> We did not experience this behavior in 2.1.16.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12876) Negative mean write latency

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862627#comment-15862627
 ] 

Chris Lohfink commented on CASSANDRA-12876:
---

I actually missed that I was reviewer, I'm testing now.

> Negative mean write latency
> ---
>
> Key: CASSANDRA-12876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12876
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability
>Reporter: Kévin LOVATO
>Assignee: Per Otterström
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: 12876-2.2.txt, 12876-2.2-v2.txt, 
> negative_mean_details.PNG, negative_mean_periodicity.PNG, negative_mean.png, 
> negative_mean_read_latency.png
>
>
> The mean write latency returned by JMX turns negative every 30 minutes. As 
> the attached screenshots show, the value turns negative every 30 minutes 
> after the startup of the node.
> We did not experience this behavior in 2.1.16.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13213) compactionstats not available, node eventually OOMs due to pending mutations

[Zachary Girouard (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862589#comment-15862589
 ] 

Zachary Girouard edited comment on CASSANDRA-13213 at 2/12/17 12:16 AM:


Some additional info.

Right now the cluster is an 8 node setup, 32G memory, E5-2530 (10 core, 
2.20Ghz). There are 4 10k RPM drives in use, 1 of them has the commit log and 
the other three are setup as JBOD. The primary use is storing time series data 
with kairosdb. 

The cluster was originally running 2.2 and consisted of three data centers, 
with the other two being roughly the same hardware but only 4 nodes each. I 
consolidated down to the biggest datacenter while trying to debug this issue. I 
saw no problems with 2.2 performance wise. I did have pretty bad disk use 
imbalance, which is why I tried upgrading things.

One thing that may be relevant: the cluster was not being repaired 
consistently, although there are very few deletes occurring, The ones that do 
are usually overwriting old data points with newer values, but that doesn't 
occur frequently at all (I'd be shocked if more than 2% the data points were 
overwritten). Over time the JBOD disk use became very unbalanced, so I'd 
reached a point where there wasn't really enough headroom to do a repair. 






was (Author: zakk4223):
Some additional info.

Right now the cluster is an 8 node setup, 32G memory, E5-2530 (10 core, 
2.20Ghz). There are 4 10k RPM drives in use, 1 of them has the commit log and 
the other three are setup as JBOD. The primary use is storing time series data 
with kairosdb. 

The cluster was originally running 2.2 and consisted of three data centers, 
with the other two being roughly the same hardware but only 4 nodes each. I 
consolidated down to the biggest datacenter while trying to debug this issue. I 
saw no problems with 2.2 performance wise. I did have pretty bad disk use 
imbalance, which is why I tried upgrading things.

One thing that may be relevant: the cluster was not being repaired 
consistently, although there are very view deletes occurring, The ones that do 
are usually overwriting old data points with newer values, but that doesn't 
occur frequently at all (I'd be shocked if more than 2% the data points were 
overwritten). Over time the JBOD disk use became very unbalanced, so I'd 
reached a point where there wasn't really enough headroom to do a repair. 





> compactionstats not available, node eventually OOMs due to pending mutations
> 
>
> Key: CASSANDRA-13213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Local Write-Read Paths
>Reporter: Zachary Girouard
>
> I'm seeing semi-frequent instances of nodetool compactionstats hanging 
> forever. While this is occurring none of the compaction metrics are available 
> via jmx/jconsole.
> Sometimes the node will eventually recover, but I'm seeing a pattern where a 
> node will exhibit this behavior, and then eventually pending mutations start 
> piling up and the node dies due to OOM. Sometimes pending gossip operations 
> starting piling up too, but I think this is due to the impending OOM causing 
> everything to bog down.
> As an experiment I turned auto compaction off on all the nodes and I haven't 
> seen this issue occur since I did that. Additionally, I'm running 
> relocatesstables on some nodes with unthrottled compaction and so far none of 
> them have had any issues handling it.
> I managed to get some stack traces from a dying node:
> All MutationStage threads look similar to this:
> {noformat}
> Name: MutationStage-10
>   
> State: WAITING
> Total blocked: 9  Total waited: 1,959,850
> Stack trace: 
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(Unknown Source)
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:162)
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
> org.apache.cassandra.db.rows.BufferCell.copy(BufferCell.java:122)
> org.apache.cassandra.utils.memory.AbstractAllocator$CloningBTreeRowBuilder.addCell(AbstractAllocator.java:72)
> org.apache.cassandra.db.rows.Rows.copy(Rows.java:51)
> org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:332)
> 

[jira] [Updated] (CASSANDRA-13213) compactionstats not available, node eventually OOMs due to pending mutations

[Zachary Girouard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zachary Girouard updated CASSANDRA-13213:
-
Since Version: 3.9  (was: 3.10)

> compactionstats not available, node eventually OOMs due to pending mutations
> 
>
> Key: CASSANDRA-13213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Local Write-Read Paths
>Reporter: Zachary Girouard
>
> I'm seeing semi-frequent instances of nodetool compactionstats hanging 
> forever. While this is occurring none of the compaction metrics are available 
> via jmx/jconsole.
> Sometimes the node will eventually recover, but I'm seeing a pattern where a 
> node will exhibit this behavior, and then eventually pending mutations start 
> piling up and the node dies due to OOM. Sometimes pending gossip operations 
> starting piling up too, but I think this is due to the impending OOM causing 
> everything to bog down.
> As an experiment I turned auto compaction off on all the nodes and I haven't 
> seen this issue occur since I did that. Additionally, I'm running 
> relocatesstables on some nodes with unthrottled compaction and so far none of 
> them have had any issues handling it.
> I managed to get some stack traces from a dying node:
> All MutationStage threads look similar to this:
> {noformat}
> Name: MutationStage-10
>   
> State: WAITING
> Total blocked: 9  Total waited: 1,959,850
> Stack trace: 
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(Unknown Source)
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:162)
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
> org.apache.cassandra.db.rows.BufferCell.copy(BufferCell.java:122)
> org.apache.cassandra.utils.memory.AbstractAllocator$CloningBTreeRowBuilder.addCell(AbstractAllocator.java:72)
> org.apache.cassandra.db.rows.Rows.copy(Rows.java:51)
> org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:332)
> org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:295)
> org.apache.cassandra.utils.btree.NodeBuilder.addNewKey(NodeBuilder.java:323)
> org.apache.cassandra.utils.btree.NodeBuilder.update(NodeBuilder.java:184)
> org.apache.cassandra.utils.btree.TreeBuilder.update(TreeBuilder.java:95)
> org.apache.cassandra.utils.btree.BTree.update(BTree.java:182)
> org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:156)
> org.apache.cassandra.db.Memtable.put(Memtable.java:284)org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1316)
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> org.apache.cassandra.db.Keyspace.applyFuture(Keyspace.java:425)
> org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:222)
> org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:68)
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> java.lang.Thread.run(Unknown Source)
> {noformat}
> The Compaction threads
> {noformat}
> Name: CompactionExecutor:4
> State: RUNNABLE
> Total blocked: 32,781,277  Total waited: 549
> Stack trace: 
> org.apache.cassandra.io.sstable.format.SSTableReader.getTotalBytes(SSTableReader.java:661)
> org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:669)
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:385)
>- locked org.apache.cassandra.db.compaction.LeveledManifest@55c79600
>
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:119)
>
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:119)
>
> 

[jira] [Commented] (CASSANDRA-13213) compactionstats not available, node eventually OOMs due to pending mutations

[Zachary Girouard (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862589#comment-15862589
 ] 

Zachary Girouard commented on CASSANDRA-13213:
--

Some additional info.

Right now the cluster is an 8 node setup, 32G memory, E5-2530 (10 core, 
2.20Ghz). There are 4 10k RPM drives in use, 1 of them has the commit log and 
the other three are setup as JBOD. The primary use is storing time series data 
with kairosdb. 

The cluster was originally running 2.2 and consisted of three data centers, 
with the other two being roughly the same hardware but only 4 nodes each. I 
consolidated down to the biggest datacenter while trying to debug this issue. I 
saw no problems with 2.2 performance wise. I did have pretty bad disk use 
imbalance, which is why I tried upgrading things.

One thing that may be relevant: the cluster was not being repaired 
consistently, although there are very view deletes occurring, The ones that do 
are usually overwriting old data points with newer values, but that doesn't 
occur frequently at all (I'd be shocked if more than 2% the data points were 
overwritten). Over time the JBOD disk use became very unbalanced, so I'd 
reached a point where there wasn't really enough headroom to do a repair. 





> compactionstats not available, node eventually OOMs due to pending mutations
> 
>
> Key: CASSANDRA-13213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Local Write-Read Paths
>Reporter: Zachary Girouard
>
> I'm seeing semi-frequent instances of nodetool compactionstats hanging 
> forever. While this is occurring none of the compaction metrics are available 
> via jmx/jconsole.
> Sometimes the node will eventually recover, but I'm seeing a pattern where a 
> node will exhibit this behavior, and then eventually pending mutations start 
> piling up and the node dies due to OOM. Sometimes pending gossip operations 
> starting piling up too, but I think this is due to the impending OOM causing 
> everything to bog down.
> As an experiment I turned auto compaction off on all the nodes and I haven't 
> seen this issue occur since I did that. Additionally, I'm running 
> relocatesstables on some nodes with unthrottled compaction and so far none of 
> them have had any issues handling it.
> I managed to get some stack traces from a dying node:
> All MutationStage threads look similar to this:
> {noformat}
> Name: MutationStage-10
>   
> State: WAITING
> Total blocked: 9  Total waited: 1,959,850
> Stack trace: 
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(Unknown Source)
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:162)
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
> org.apache.cassandra.db.rows.BufferCell.copy(BufferCell.java:122)
> org.apache.cassandra.utils.memory.AbstractAllocator$CloningBTreeRowBuilder.addCell(AbstractAllocator.java:72)
> org.apache.cassandra.db.rows.Rows.copy(Rows.java:51)
> org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:332)
> org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:295)
> org.apache.cassandra.utils.btree.NodeBuilder.addNewKey(NodeBuilder.java:323)
> org.apache.cassandra.utils.btree.NodeBuilder.update(NodeBuilder.java:184)
> org.apache.cassandra.utils.btree.TreeBuilder.update(TreeBuilder.java:95)
> org.apache.cassandra.utils.btree.BTree.update(BTree.java:182)
> org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:156)
> org.apache.cassandra.db.Memtable.put(Memtable.java:284)org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1316)
> org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
> org.apache.cassandra.db.Keyspace.applyFuture(Keyspace.java:425)
> org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:222)
> org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:68)
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> 

[jira] [Created] (CASSANDRA-13213) compactionstats not available, node eventually OOMs due to pending mutations

[Zachary Girouard (JIRA)]

Zachary Girouard created CASSANDRA-13213:


 Summary: compactionstats not available, node eventually OOMs due 
to pending mutations
 Key: CASSANDRA-13213
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13213
 Project: Cassandra
  Issue Type: Bug
  Components: Compaction, Local Write-Read Paths
Reporter: Zachary Girouard


I'm seeing semi-frequent instances of nodetool compactionstats hanging forever. 
While this is occurring none of the compaction metrics are available via 
jmx/jconsole.

Sometimes the node will eventually recover, but I'm seeing a pattern where a 
node will exhibit this behavior, and then eventually pending mutations start 
piling up and the node dies due to OOM. Sometimes pending gossip operations 
starting piling up too, but I think this is due to the impending OOM causing 
everything to bog down.

As an experiment I turned auto compaction off on all the nodes and I haven't 
seen this issue occur since I did that. Additionally, I'm running 
relocatesstables on some nodes with unthrottled compaction and so far none of 
them have had any issues handling it.

I managed to get some stack traces from a dying node:

All MutationStage threads look similar to this:
{noformat}
Name: MutationStage-10  
State: WAITING
Total blocked: 9  Total waited: 1,959,850

Stack trace: 
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(Unknown Source)
org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:162)
org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
org.apache.cassandra.db.rows.BufferCell.copy(BufferCell.java:122)
org.apache.cassandra.utils.memory.AbstractAllocator$CloningBTreeRowBuilder.addCell(AbstractAllocator.java:72)
org.apache.cassandra.db.rows.Rows.copy(Rows.java:51)
org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:332)
org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:295)
org.apache.cassandra.utils.btree.NodeBuilder.addNewKey(NodeBuilder.java:323)
org.apache.cassandra.utils.btree.NodeBuilder.update(NodeBuilder.java:184)
org.apache.cassandra.utils.btree.TreeBuilder.update(TreeBuilder.java:95)
org.apache.cassandra.utils.btree.BTree.update(BTree.java:182)
org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:156)
org.apache.cassandra.db.Memtable.put(Memtable.java:284)org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1316)
org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:618)
org.apache.cassandra.db.Keyspace.applyFuture(Keyspace.java:425)
org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:222)
org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:68)
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
java.lang.Thread.run(Unknown Source)
{noformat}

The Compaction threads
{noformat}
Name: CompactionExecutor:4
State: RUNNABLE
Total blocked: 32,781,277  Total waited: 549

Stack trace: 
org.apache.cassandra.io.sstable.format.SSTableReader.getTotalBytes(SSTableReader.java:661)
org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:669)
org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:385)
   - locked org.apache.cassandra.db.compaction.LeveledManifest@55c79600
   
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:119)
   
org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:119)
   
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:261)
   java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
   java.util.concurrent.FutureTask.run(Unknown Source)
   java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
   java.util.concurrent.FutureTask.run(Unknown Source)
   java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   

cassandra git commit: match up logging markers with parameters

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1ffcdb5a6 -> e1dda71a4


match up logging markers with parameters


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e1dda71a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e1dda71a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e1dda71a

Branch: refs/heads/trunk
Commit: e1dda71a4fc43310a0e41dddb4b428fe6ef3b9c6
Parents: 1ffcdb5
Author: Dave Brosius 
Authored: Sat Feb 11 17:47:18 2017 -0500
Committer: Dave Brosius 
Committed: Sat Feb 11 17:47:18 2017 -0500

--
 .../cassandra/streaming/compress/CompressedStreamReader.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e1dda71a/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java
--
diff --git 
a/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java 
b/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java
index d8e329c..da62aa9 100644
--- 
a/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java
+++ 
b/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java
@@ -72,7 +72,7 @@ public class CompressedStreamReader extends StreamReader
 throw new IOException("CF " + tableId + " was dropped during 
streaming");
 }
 
-logger.debug("[Stream #{}] Start receiving file #{} from {}, 
repairedAt = {}, size = {}, ks = '{}', table = '{}'.",
+logger.debug("[Stream #{}] Start receiving file #{} from {}, 
repairedAt = {}, size = {}, ks = '{}', pendingRepair = '{}', table = '{}'.",
  session.planId(), fileSeqNum, session.peer, repairedAt, 
totalSize, cfs.keyspace.getName(), session.getPendingRepair(),
  cfs.getTableName());
 

[jira] [Created] (CASSANDRA-13212) Remove hard dependency on Logback

[Michael Calderero (JIRA)]

Michael Calderero created CASSANDRA-13212:
-

 Summary: Remove hard dependency on Logback
 Key: CASSANDRA-13212
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13212
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Michael Calderero


Hi,

Our application is using Log4J2, SpringBoot and cassandraunit. We just upgraded 
to use cassandra-all version 3.0.10 (to approximately match the DataStax 
version wer're using) and excluded the logback-classic and logback-core 
dependencies from this dependency.

However when I try to run an embedded Cassandra unit test, it always fails with 
the following error:
{quote}
Exception (java.lang.NoClassDefFoundError) encountered during startup: 
ch/qos/logback/core/Context
java.lang.NoClassDefFoundError: ch/qos/logback/core/Context
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:604)
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:558)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:346)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:568)
at 
org.cassandraunit.utils.EmbeddedCassandraServerHelper$1.run(EmbeddedCassandraServerHelper.java:133)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: ch.qos.logback.core.Context
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
{quote}
I looked at the source code of StorageService and it seems like it is invoking 
Logback-specific classes directly, instead of SLF4J ones. This code seems to 
have been introduced by CASSANDRA-5883.

I currently don't see a logback-to-slf4j bridge so seems it seems our options 
are to not use Cassandra (which is not possible) or to try to create a dummy 
bridge to route logback calls to slf4j (which then goes to Log4j2).

Any particular reason why the Logback classes are invoked directly instead 
through SLF4J?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13008) Add vm.max_map_count StartupCheck

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862512#comment-15862512
 ] 

Jay Zhuang commented on CASSANDRA-13008:


Thanks [~Stefania] for the review. You're right, no need to check if it's not 
{{mmap}}: 
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/io/util/CompressedSegmentedFile.java#L46

> Add vm.max_map_count StartupCheck
> -
>
> Key: CASSANDRA-13008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13008
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 13008-3.0.txt
>
>
> It's recommended to set {{vm.max_map_count}} to 1048575 (CASSANDRA-3563)
> When the max_map_count is low, it throws OOM exception, which is hard to link 
> to the real issue of vm.max_map_count.
> The problem happened when we tried to remove one node, all the other nodes in 
> cluster crashed. As each node was trying to load more local SSTable files for 
> streaming.
> I would suggest to add a StartupCheck for max_map_count, at least it could 
> give a warning message to help the debug.
> {code}
> ERROR [STREAM-IN-] JVMStabilityInspector.java:140 - JVM state determined to 
> be unstable.  Exiting forcefully due to:
> java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[na:1.8.0_112]
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937) 
> ~[na:1.8.0_112]
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:152) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:280)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:216)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:173)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:70) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:58) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:96) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile.(CompressedSegmentedFile.java:47)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:132)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:177)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.buildData(SegmentedFile.java:193)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openFinal(BigTableWriter.java:276)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$600(BigTableWriter.java:50)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:313)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.finish(SSTableWriter.java:213)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.finish(SimpleSSTableMultiWriter.java:56)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.streaming.StreamReceiveTask.received(StreamReceiveTask.java:109)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:599) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:482)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:296)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13210) test failure in repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test_not_intersecting_all_ranges

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston reassigned CASSANDRA-13210:
---

Assignee: Blake Eggleston

> test failure in 
> repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test_not_intersecting_all_ranges
> ---
>
> Key: CASSANDRA-13210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13210
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Blake Eggleston
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log, node4_debug.log, node4_gc.log, node4.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_large_dtest/53/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test_not_intersecting_all_ranges
> {noformat}
> Error Message
> 'Repaired at: 0' unexpectedly found in 'SSTable: 
> /tmp/dtest-N7zjo6/test/node1/data0/keyspace1/standard1-a79a0c50efa211e6bf211330662f36ef/md-6-big\nPartitioner:
>  org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance: 
> 0.01\nMinimum timestamp: 148673926323\nMaximum timestamp: 
> 148673926323\nSSTable min local deletion time: 2147483647\nSSTable max 
> local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst 
> token: 296988783704308703 (key=30503337373039503231)\nLast token: 
> 296988783704308703 (key=30503337373039503231)\nEstimated droppable 
> tombstones: 0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair: 
> b099d1f0-efa2-11e6-89ec-d14624f1e47e\nReplay positions covered: 
> {CommitLogPosition(segmentId=1486739234777, 
> position=46796)=CommitLogPosition(segmentId=1486739234777, 
> position=50819)}\ntotalColumnsSet: 5\ntotalRows: 1\nEstimated tombstone drop 
> times:\nCount   Row SizeCell Count\n1 
>  0 0\n2  0 0\n3   
>0 0\n4  0  
>0\n5  0 2\n6   
>0 0\n7  0 
> 0\n8  0 0\n10 
> 0 0\n12 0 0\n14   
>   0 0\n17 0   
>   0\n20 0 0\n24   
>   0 0\n29 0 0\n35 
> 0 0\n42 0 
> 0\n50 0 0\n60 
> 0 0\n72 0 
> 0\n86 0 0\n103
> 0 0\n1240 0\n149  
>   0 0\n1790   
>   0\n2151 0\n258  
>   0 0\n3100 
> 0\n3720 0\n446
> 0 0\n5350 0\n642  
>   0 0\n7700   
>   0\n9240 0\n1109 
>   0 0\n1331   0 
> 0\n1597   0 0\n1916   
> 0 0\n2299   0 0\n2759 
>   0 0\n3311   0   
>   0\n3973   0 0\n4768 
>   0 0\n5722   0 
> 0\n6866   0 0\n8239   
> 0 0\n9887   0 0\n11864
>   0 0\n14237  0   
>   0\n17084  0 0\n20501
>   0 0\n24601  0 
> 0\n29521  0 0\n35425  
> 0 0\n42510  0 0\n51012
>   0 0\n61214 

[jira] [Updated] (CASSANDRA-13204) Thread Leak in OutboundTcpConnection

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13204:

   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 2.2.x)
   (was: 2.1.x)
   4.0
   3.11.0
   2.2.9
   2.1.17
   Status: Resolved  (was: Ready to Commit)

committed as sha {{a6237bf65a95d654b7e702e81fd0d353460d0c89}} to 2.1, 2.2, 3.0, 
3.11, and trunk. Thanks!

> Thread Leak in OutboundTcpConnection
> 
>
> Key: CASSANDRA-13204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13204
> Project: Cassandra
>  Issue Type: Bug
>Reporter: sankalp kohli
>Assignee: Jason Brown
> Fix For: 2.1.17, 2.2.9, 3.0.11, 3.11.0, 4.0
>
>
> We found threads leaking from OutboundTcpConnection to machines which are not 
> part of the cluster and still in Gossip for some reason. There are two issues 
> here, this JIRA will cover the second one which is most important. 
> 1) First issue is that Gossip has information about machines not in the ring 
> which has been replaced out. It causes Cassandra to connect to those machines 
> but due to internode auth, it wont be able to connect to them at the socket 
> level.  
> 2) Second issue is a race between creating a connection and closing a 
> connections which is triggered by the gossip bug explained above. Let me try 
> to explain it using the code
> In OutboundTcpConnection, we are calling closeSocket(true) which will set 
> isStopped=true and also put a close sentinel into the queue to exit the 
> thread. On the ack connection, Gossip tries to send a message which calls 
> connect() which will block for 10 seconds which is RPC timeout. The reason we 
> will block is because Cassandra might not be running there or internode auth 
> will not let it connect. During this 10 seconds, if Gossip calls closeSocket, 
> it will put close sentinel into the queue. When we return from the connect 
> method after 10 seconds, we will clear the backlog queue causing this thread 
> to leak. 
> Proofs from the heap dump of the affected machine which is leaking threads 
> 1. Only ack connection is leaking and not the command connection which is not 
> used by Gossip. 
> 2. We see thread blocked on the backlog queue, isStopped=true and backlog 
> queue is empty. This is happening on the threads which have already leaked. 
> 3. A running thread was blocked on the connect waiting for timeout(10 
> seconds) and we see backlog queue to contain the close sentinel. Once the 
> connect will return false, we will clear the backlog and this thread will 
> have leaked.  
> Interesting bits from j stack 
> 1282 number of threads for "MessagingService-Outgoing-/"
> Thread which is about to leak:
> "MessagingService-Outgoing-/" 
>java.lang.Thread.State: RUNNABLE
>   at sun.nio.ch.Net.connect0(Native Method)
>   at sun.nio.ch.Net.connect(Net.java:454)
>   at sun.nio.ch.Net.connect(Net.java:446)
>   at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
>   - locked <> (a java.lang.Object)
>   - locked <> (a java.lang.Object)
>   - locked <> (a java.lang.Object)
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:137)
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:119)
>   at 
> org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:381)
>   at 
> org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:217)
> Thread already leaked:
> "MessagingService-Outgoing-/"
>java.lang.Thread.State: WAITING (parking)
>   at sun.misc.Unsafe.park(Native Method)
>   - parking to wait for  <> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>   at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>   at 
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
>   at 
> org.apache.cassandra.utils.CoalescingStrategies$DisabledCoalescingStrategy.coalesceInternal(CoalescingStrategies.java:482)
>   at 
> org.apache.cassandra.utils.CoalescingStrategies$CoalescingStrategy.coalesce(CoalescingStrategies.java:213)
>   at 
> org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:190)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[03/15] cassandra git commit: Fix Thread Leak in OutboundTcpConnection

[jasobrown]

Fix Thread Leak in OutboundTcpConnection

patch by jasobrown, reviewed by aweisberg for CASSANDRA-13204


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6237bf6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6237bf6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6237bf6

Branch: refs/heads/trunk
Commit: a6237bf65a95d654b7e702e81fd0d353460d0c89
Parents: f6a7057
Author: Jason Brown 
Authored: Thu Feb 9 14:34:08 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 07:47:29 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5c86687..9ce8d49 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.17
+ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
  * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
  * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java 
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index 1a88220..ff2d929 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -161,8 +161,11 @@ public class OutboundTcpConnection extends Thread
 
 void closeSocket(boolean destroyThread)
 {
-backlog.clear();
 isStopped = destroyThread; // Exit loop to stop the thread
+backlog.clear();
+// in the "destroyThread = true" case, enqueuing the sentinel is 
important mostly to unblock the backlog.take()
+// (via the CoalescingStrategy) in case there's a data race between 
this method enqueuing the sentinel
+// and run() clearing the backlog on connection failure.
 enqueue(CLOSE_SENTINEL, -1);
 }
 
@@ -183,7 +186,7 @@ public class OutboundTcpConnection extends Thread
 final List drainedMessages = new 
ArrayList<>(drainedMessageSize);
 
 outer:
-while (true)
+while (!isStopped)
 {
 try
 {
@@ -199,6 +202,7 @@ public class OutboundTcpConnection extends Thread
 int count = drainedMessages.size();
 //The timestamp of the first message has already been provided to 
the coalescing strategy
 //so skip logging it.
+inner:
 for (QueuedMessage qm : drainedMessages)
 {
 try
@@ -217,8 +221,12 @@ public class OutboundTcpConnection extends Thread
 else if (socket != null || connect())
 writeConnected(qm, count == 1 && backlog.isEmpty());
 else
+{
 // clear out the queue, else gossip messages back up.
+drainedMessages.clear();
 backlog.clear();
+break inner;
+}
 }
 catch (Exception e)
 {

[04/15] cassandra git commit: Fix Thread Leak in OutboundTcpConnection

[jasobrown]

Fix Thread Leak in OutboundTcpConnection

patch by jasobrown, reviewed by aweisberg for CASSANDRA-13204


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6237bf6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6237bf6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6237bf6

Branch: refs/heads/cassandra-3.0
Commit: a6237bf65a95d654b7e702e81fd0d353460d0c89
Parents: f6a7057
Author: Jason Brown 
Authored: Thu Feb 9 14:34:08 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 07:47:29 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5c86687..9ce8d49 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.17
+ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
  * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
  * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java 
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index 1a88220..ff2d929 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -161,8 +161,11 @@ public class OutboundTcpConnection extends Thread
 
 void closeSocket(boolean destroyThread)
 {
-backlog.clear();
 isStopped = destroyThread; // Exit loop to stop the thread
+backlog.clear();
+// in the "destroyThread = true" case, enqueuing the sentinel is 
important mostly to unblock the backlog.take()
+// (via the CoalescingStrategy) in case there's a data race between 
this method enqueuing the sentinel
+// and run() clearing the backlog on connection failure.
 enqueue(CLOSE_SENTINEL, -1);
 }
 
@@ -183,7 +186,7 @@ public class OutboundTcpConnection extends Thread
 final List drainedMessages = new 
ArrayList<>(drainedMessageSize);
 
 outer:
-while (true)
+while (!isStopped)
 {
 try
 {
@@ -199,6 +202,7 @@ public class OutboundTcpConnection extends Thread
 int count = drainedMessages.size();
 //The timestamp of the first message has already been provided to 
the coalescing strategy
 //so skip logging it.
+inner:
 for (QueuedMessage qm : drainedMessages)
 {
 try
@@ -217,8 +221,12 @@ public class OutboundTcpConnection extends Thread
 else if (socket != null || connect())
 writeConnected(qm, count == 1 && backlog.isEmpty());
 else
+{
 // clear out the queue, else gossip messages back up.
+drainedMessages.clear();
 backlog.clear();
+break inner;
+}
 }
 catch (Exception e)
 {

[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ec60900
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ec60900
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ec60900

Branch: refs/heads/trunk
Commit: 8ec60900211c47f3231058de6b09b04f78456eea
Parents: 0cc1c1b c884b70
Author: Jason Brown 
Authored: Sat Feb 11 08:00:56 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:01:34 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/CHANGES.txt
--
diff --cc CHANGES.txt
index 007d06c,fc79796..700bde0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -53,49 -13,6 +53,50 @@@ Merged from 2.2
   * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
   * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
   * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 +Merged from 2.1:
++ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 + * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
 + * cqlsh copy-from: sort user type fields in csv (CASSANDRA-12959)
 +
 +
 +
 +3.0.10
 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Pass root cause to CorruptBlockException when uncompression failed 
(CASSANDRA-12889)
 + * Fix partition count log during compaction (CASSANDRA-12184)
 + * Batch with multiple conditional updates for the same partition causes 
AssertionError (CASSANDRA-12867)
 + * Make AbstractReplicationStrategy extendable from outside its package 
(CASSANDRA-12788)
 + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854)
 + * Don't tell users to turn off consistent rangemovements during rebuild. 
(CASSANDRA-12296)
 + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689)
 + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801)
 + * Include SSTable filename in compacting large row message (CASSANDRA-12384)
 + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330)
 + * Fix ViewTest.testCompaction (CASSANDRA-12789)
 + * Improve avg aggregate functions (CASSANDRA-12417)
 + * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
 + * nodetool stopdaemon errors out (CASSANDRA-12646)
 + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
 + * mx4j does not work in 3.0.8 (CASSANDRA-12274)
 + * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
 + * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
 + * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
 + * Fix exceptions with new vnode allocation (CASSANDRA-12715)
 + * Unify drain and shutdown processes (CASSANDRA-12509)
 + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706)
 + * Fix failure in LogTransactionTest (CASSANDRA-12632)
 + * Fix potentially incomplete non-frozen UDT values when querying with the
 +   full primary key specified (CASSANDRA-12605)
 + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() 
(CASSANDRA-11670)
 + * Establish consistent distinction between non-existing partition and NULL 
value for LWTs on static columns (CASSANDRA-12060)
 + * Extend ColumnIdentifier.internedInstances key to include the type that 
generated the byte buffer (CASSANDRA-12516)
 + * Backport CASSANDRA-10756 (race condition in NativeTransportService 
shutdown) (CASSANDRA-12472)
 + * If CF has no clustering columns, any row cache is full partition cache 
(CASSANDRA-12499)
 + * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
 + * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
 +Merged from 2.2:
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[15/15] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jasobrown]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1ffcdb5a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1ffcdb5a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1ffcdb5a

Branch: refs/heads/trunk
Commit: 1ffcdb5a6fd5d69c8978f3597c4ab8710cdd30ad
Parents: 61b3d34 94a8c77
Author: Jason Brown 
Authored: Sat Feb 11 08:03:00 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:03:31 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1ffcdb5a/CHANGES.txt
--
diff --cc CHANGES.txt
index d2023cf,b34e5f6..6c1b9b6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -51,10 -20,9 +51,11 @@@ Merged from 3.0
  Merged from 2.2:
* Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
  Merged from 2.1:
+  * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  
 +
  3.10
   * Fix secondary index queries regression (CASSANDRA-13013)
   * Add duration type to the protocol V5 (CASSANDRA-12850)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1ffcdb5a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ec60900
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ec60900
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ec60900

Branch: refs/heads/cassandra-3.11
Commit: 8ec60900211c47f3231058de6b09b04f78456eea
Parents: 0cc1c1b c884b70
Author: Jason Brown 
Authored: Sat Feb 11 08:00:56 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:01:34 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/CHANGES.txt
--
diff --cc CHANGES.txt
index 007d06c,fc79796..700bde0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -53,49 -13,6 +53,50 @@@ Merged from 2.2
   * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
   * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
   * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 +Merged from 2.1:
++ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 + * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
 + * cqlsh copy-from: sort user type fields in csv (CASSANDRA-12959)
 +
 +
 +
 +3.0.10
 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Pass root cause to CorruptBlockException when uncompression failed 
(CASSANDRA-12889)
 + * Fix partition count log during compaction (CASSANDRA-12184)
 + * Batch with multiple conditional updates for the same partition causes 
AssertionError (CASSANDRA-12867)
 + * Make AbstractReplicationStrategy extendable from outside its package 
(CASSANDRA-12788)
 + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854)
 + * Don't tell users to turn off consistent rangemovements during rebuild. 
(CASSANDRA-12296)
 + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689)
 + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801)
 + * Include SSTable filename in compacting large row message (CASSANDRA-12384)
 + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330)
 + * Fix ViewTest.testCompaction (CASSANDRA-12789)
 + * Improve avg aggregate functions (CASSANDRA-12417)
 + * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
 + * nodetool stopdaemon errors out (CASSANDRA-12646)
 + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
 + * mx4j does not work in 3.0.8 (CASSANDRA-12274)
 + * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
 + * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
 + * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
 + * Fix exceptions with new vnode allocation (CASSANDRA-12715)
 + * Unify drain and shutdown processes (CASSANDRA-12509)
 + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706)
 + * Fix failure in LogTransactionTest (CASSANDRA-12632)
 + * Fix potentially incomplete non-frozen UDT values when querying with the
 +   full primary key specified (CASSANDRA-12605)
 + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() 
(CASSANDRA-11670)
 + * Establish consistent distinction between non-existing partition and NULL 
value for LWTs on static columns (CASSANDRA-12060)
 + * Extend ColumnIdentifier.internedInstances key to include the type that 
generated the byte buffer (CASSANDRA-12516)
 + * Backport CASSANDRA-10756 (race condition in NativeTransportService 
shutdown) (CASSANDRA-12472)
 + * If CF has no clustering columns, any row cache is full partition cache 
(CASSANDRA-12499)
 + * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
 + * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
 +Merged from 2.2:
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jasobrown]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c884b701
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c884b701
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c884b701

Branch: refs/heads/cassandra-3.0
Commit: c884b701d9cd6d8a6e3efbd01b377ac24d3c8e07
Parents: d24d07e a6237bf
Author: Jason Brown 
Authored: Sat Feb 11 07:59:40 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:00:18 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/CHANGES.txt
--
diff --cc CHANGES.txt
index a109042,9ce8d49..fc79796
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,5 +1,35 @@@
 -2.1.17
 +2.2.9
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
 +Merged from 2.1:
+  * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
   * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/94a8c777
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/94a8c777
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/94a8c777

Branch: refs/heads/trunk
Commit: 94a8c777a176585f04fe2a0e79afe7c2c594c7c3
Parents: 3ae5a4d 8ec6090
Author: Jason Brown 
Authored: Sat Feb 11 08:01:52 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:02:23 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/94a8c777/CHANGES.txt
--
diff --cc CHANGES.txt
index a69cbe3,700bde0..b34e5f6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -17,128 -10,6 +17,129 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
 +  * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
++ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks 

[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/94a8c777
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/94a8c777
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/94a8c777

Branch: refs/heads/cassandra-3.11
Commit: 94a8c777a176585f04fe2a0e79afe7c2c594c7c3
Parents: 3ae5a4d 8ec6090
Author: Jason Brown 
Authored: Sat Feb 11 08:01:52 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:02:23 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/94a8c777/CHANGES.txt
--
diff --cc CHANGES.txt
index a69cbe3,700bde0..b34e5f6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -17,128 -10,6 +17,129 @@@ Merged from 3.0
 live rows in sstabledump (CASSANDRA-13177)
   * Provide user workaround when system_schema.columns does not contain entries
 for a table that's in system_schema.tables (CASSANDRA-13180)
 +Merged from 2.2:
 +  * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 +Merged from 2.1:
++ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 +
 +3.10
 + * Fix secondary index queries regression (CASSANDRA-13013)
 + * Add duration type to the protocol V5 (CASSANDRA-12850)
 + * Fix duration type validation (CASSANDRA-13143)
 + * Fix flaky GcCompactionTest (CASSANDRA-12664)
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks 

[02/15] cassandra git commit: Fix Thread Leak in OutboundTcpConnection

[jasobrown]

Fix Thread Leak in OutboundTcpConnection

patch by jasobrown, reviewed by aweisberg for CASSANDRA-13204


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6237bf6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6237bf6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6237bf6

Branch: refs/heads/cassandra-2.2
Commit: a6237bf65a95d654b7e702e81fd0d353460d0c89
Parents: f6a7057
Author: Jason Brown 
Authored: Thu Feb 9 14:34:08 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 07:47:29 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5c86687..9ce8d49 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.17
+ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
  * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
  * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java 
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index 1a88220..ff2d929 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -161,8 +161,11 @@ public class OutboundTcpConnection extends Thread
 
 void closeSocket(boolean destroyThread)
 {
-backlog.clear();
 isStopped = destroyThread; // Exit loop to stop the thread
+backlog.clear();
+// in the "destroyThread = true" case, enqueuing the sentinel is 
important mostly to unblock the backlog.take()
+// (via the CoalescingStrategy) in case there's a data race between 
this method enqueuing the sentinel
+// and run() clearing the backlog on connection failure.
 enqueue(CLOSE_SENTINEL, -1);
 }
 
@@ -183,7 +186,7 @@ public class OutboundTcpConnection extends Thread
 final List drainedMessages = new 
ArrayList<>(drainedMessageSize);
 
 outer:
-while (true)
+while (!isStopped)
 {
 try
 {
@@ -199,6 +202,7 @@ public class OutboundTcpConnection extends Thread
 int count = drainedMessages.size();
 //The timestamp of the first message has already been provided to 
the coalescing strategy
 //so skip logging it.
+inner:
 for (QueuedMessage qm : drainedMessages)
 {
 try
@@ -217,8 +221,12 @@ public class OutboundTcpConnection extends Thread
 else if (socket != null || connect())
 writeConnected(qm, count == 1 && backlog.isEmpty());
 else
+{
 // clear out the queue, else gossip messages back up.
+drainedMessages.clear();
 backlog.clear();
+break inner;
+}
 }
 catch (Exception e)
 {

[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jasobrown]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c884b701
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c884b701
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c884b701

Branch: refs/heads/cassandra-3.11
Commit: c884b701d9cd6d8a6e3efbd01b377ac24d3c8e07
Parents: d24d07e a6237bf
Author: Jason Brown 
Authored: Sat Feb 11 07:59:40 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:00:18 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/CHANGES.txt
--
diff --cc CHANGES.txt
index a109042,9ce8d49..fc79796
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,5 +1,35 @@@
 -2.1.17
 +2.2.9
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
 +Merged from 2.1:
+  * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
   * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jasobrown]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c884b701
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c884b701
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c884b701

Branch: refs/heads/cassandra-2.2
Commit: c884b701d9cd6d8a6e3efbd01b377ac24d3c8e07
Parents: d24d07e a6237bf
Author: Jason Brown 
Authored: Sat Feb 11 07:59:40 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:00:18 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/CHANGES.txt
--
diff --cc CHANGES.txt
index a109042,9ce8d49..fc79796
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,5 +1,35 @@@
 -2.1.17
 +2.2.9
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
 +Merged from 2.1:
+  * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
   * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jasobrown]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c884b701
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c884b701
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c884b701

Branch: refs/heads/trunk
Commit: c884b701d9cd6d8a6e3efbd01b377ac24d3c8e07
Parents: d24d07e a6237bf
Author: Jason Brown 
Authored: Sat Feb 11 07:59:40 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:00:18 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/CHANGES.txt
--
diff --cc CHANGES.txt
index a109042,9ce8d49..fc79796
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,34 -1,5 +1,35 @@@
 -2.1.17
 +2.2.9
 + * Use only one file pointer when creating commitlog segments 
(CASSANDRA-12539)
 + * Fix speculative retry bugs (CASSANDRA-13009)
 + * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981) 
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fix DynamicEndpointSnitch noop in multi-datacenter situations 
(CASSANDRA-13074)
 + * cqlsh copy-from: encode column names to avoid primary key parsing errors 
(CASSANDRA-12909)
 + * Temporarily fix bug that creates commit log when running offline tools 
(CASSANDRA-8616)
 + * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
 + * Test bind parameters and unset parameters in InsertUpdateIfConditionTest 
(CASSANDRA-12980)
 + * Do not specify local address on outgoing connection when 
listen_on_broadcast_address is set (CASSANDRA-12673)
 + * Use saved tokens when setting local tokens on StorageService.joinRing 
(CASSANDRA-12935)
 + * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
 + * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
 + * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 + * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
 + * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
 + * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)
 + * Fix Util.spinAssertEquals (CASSANDRA-12283)
 + * Fix potential NPE for compactionstats (CASSANDRA-12462)
 + * Prepare legacy authenticate statement if credentials table initialised 
after node startup (CASSANDRA-12813)
 + * Change cassandra.wait_for_tracing_events_timeout_secs default to 0 
(CASSANDRA-12754)
 + * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
 + * Limit colUpdateTimeDelta histogram updates to reasonable deltas 
(CASSANDRA-7)
 + * Fix leak errors and execution rejected exceptions when draining 
(CASSANDRA-12457)
 + * Fix merkle tree depth calculation (CASSANDRA-12580)
 + * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Better handle invalid system roles table (CASSANDRA-12700)
 + * Split consistent range movement flag correction (CASSANDRA-12786)
 + * CompactionTasks now correctly drops sstables out of compaction when not 
enough disk space is available (CASSANDRA-12979)
 +Merged from 2.1:
+  * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
   * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
   * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
   * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c884b701/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[01/15] cassandra git commit: Fix Thread Leak in OutboundTcpConnection

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 f6a705781 -> a6237bf65
  refs/heads/cassandra-2.2 d24d07eb0 -> c884b701d
  refs/heads/cassandra-3.0 0cc1c1b2a -> 8ec609002
  refs/heads/cassandra-3.11 3ae5a4d67 -> 94a8c777a
  refs/heads/trunk 61b3d3490 -> 1ffcdb5a6


Fix Thread Leak in OutboundTcpConnection

patch by jasobrown, reviewed by aweisberg for CASSANDRA-13204


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6237bf6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6237bf6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6237bf6

Branch: refs/heads/cassandra-2.1
Commit: a6237bf65a95d654b7e702e81fd0d353460d0c89
Parents: f6a7057
Author: Jason Brown 
Authored: Thu Feb 9 14:34:08 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 07:47:29 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5c86687..9ce8d49 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.17
+ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
  * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
  * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java 
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index 1a88220..ff2d929 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -161,8 +161,11 @@ public class OutboundTcpConnection extends Thread
 
 void closeSocket(boolean destroyThread)
 {
-backlog.clear();
 isStopped = destroyThread; // Exit loop to stop the thread
+backlog.clear();
+// in the "destroyThread = true" case, enqueuing the sentinel is 
important mostly to unblock the backlog.take()
+// (via the CoalescingStrategy) in case there's a data race between 
this method enqueuing the sentinel
+// and run() clearing the backlog on connection failure.
 enqueue(CLOSE_SENTINEL, -1);
 }
 
@@ -183,7 +186,7 @@ public class OutboundTcpConnection extends Thread
 final List drainedMessages = new 
ArrayList<>(drainedMessageSize);
 
 outer:
-while (true)
+while (!isStopped)
 {
 try
 {
@@ -199,6 +202,7 @@ public class OutboundTcpConnection extends Thread
 int count = drainedMessages.size();
 //The timestamp of the first message has already been provided to 
the coalescing strategy
 //so skip logging it.
+inner:
 for (QueuedMessage qm : drainedMessages)
 {
 try
@@ -217,8 +221,12 @@ public class OutboundTcpConnection extends Thread
 else if (socket != null || connect())
 writeConnected(qm, count == 1 && backlog.isEmpty());
 else
+{
 // clear out the queue, else gossip messages back up.
+drainedMessages.clear();
 backlog.clear();
+break inner;
+}
 }
 catch (Exception e)
 {

[05/15] cassandra git commit: Fix Thread Leak in OutboundTcpConnection

[jasobrown]

Fix Thread Leak in OutboundTcpConnection

patch by jasobrown, reviewed by aweisberg for CASSANDRA-13204


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6237bf6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6237bf6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6237bf6

Branch: refs/heads/cassandra-3.11
Commit: a6237bf65a95d654b7e702e81fd0d353460d0c89
Parents: f6a7057
Author: Jason Brown 
Authored: Thu Feb 9 14:34:08 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 07:47:29 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5c86687..9ce8d49 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.17
+ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
  * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
  * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
  * Fix paging for DISTINCT queries on partition keys and static columns 
(CASSANDRA-13017)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6237bf6/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java 
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index 1a88220..ff2d929 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -161,8 +161,11 @@ public class OutboundTcpConnection extends Thread
 
 void closeSocket(boolean destroyThread)
 {
-backlog.clear();
 isStopped = destroyThread; // Exit loop to stop the thread
+backlog.clear();
+// in the "destroyThread = true" case, enqueuing the sentinel is 
important mostly to unblock the backlog.take()
+// (via the CoalescingStrategy) in case there's a data race between 
this method enqueuing the sentinel
+// and run() clearing the backlog on connection failure.
 enqueue(CLOSE_SENTINEL, -1);
 }
 
@@ -183,7 +186,7 @@ public class OutboundTcpConnection extends Thread
 final List drainedMessages = new 
ArrayList<>(drainedMessageSize);
 
 outer:
-while (true)
+while (!isStopped)
 {
 try
 {
@@ -199,6 +202,7 @@ public class OutboundTcpConnection extends Thread
 int count = drainedMessages.size();
 //The timestamp of the first message has already been provided to 
the coalescing strategy
 //so skip logging it.
+inner:
 for (QueuedMessage qm : drainedMessages)
 {
 try
@@ -217,8 +221,12 @@ public class OutboundTcpConnection extends Thread
 else if (socket != null || connect())
 writeConnected(qm, count == 1 && backlog.isEmpty());
 else
+{
 // clear out the queue, else gossip messages back up.
+drainedMessages.clear();
 backlog.clear();
+break inner;
+}
 }
 catch (Exception e)
 {

[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ec60900
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ec60900
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ec60900

Branch: refs/heads/cassandra-3.0
Commit: 8ec60900211c47f3231058de6b09b04f78456eea
Parents: 0cc1c1b c884b70
Author: Jason Brown 
Authored: Sat Feb 11 08:00:56 2017 -0800
Committer: Jason Brown 
Committed: Sat Feb 11 08:01:34 2017 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/net/OutboundTcpConnection.java | 12 ++--
 2 files changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/CHANGES.txt
--
diff --cc CHANGES.txt
index 007d06c,fc79796..700bde0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -53,49 -13,6 +53,50 @@@ Merged from 2.2
   * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
   * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
   * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
 +Merged from 2.1:
++ * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
 + * Coalescing strategy can enter infinite loop (CASSANDRA-13159)
 + * Upgrade netty version to fix memory leak with client encryption 
(CASSANDRA-13114)
 + * cqlsh copy-from: sort user type fields in csv (CASSANDRA-12959)
 +
 +
 +
 +3.0.10
 + * Disallow offheap_buffers memtable allocation (CASSANDRA-11039)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Pass root cause to CorruptBlockException when uncompression failed 
(CASSANDRA-12889)
 + * Fix partition count log during compaction (CASSANDRA-12184)
 + * Batch with multiple conditional updates for the same partition causes 
AssertionError (CASSANDRA-12867)
 + * Make AbstractReplicationStrategy extendable from outside its package 
(CASSANDRA-12788)
 + * Fix CommitLogTest.testDeleteIfNotDirty (CASSANDRA-12854)
 + * Don't tell users to turn off consistent rangemovements during rebuild. 
(CASSANDRA-12296)
 + * Avoid deadlock due to materialized view lock contention (CASSANDRA-12689)
 + * Fix for KeyCacheCqlTest flakiness (CASSANDRA-12801)
 + * Include SSTable filename in compacting large row message (CASSANDRA-12384)
 + * Fix potential socket leak (CASSANDRA-12329, CASSANDRA-12330)
 + * Fix ViewTest.testCompaction (CASSANDRA-12789)
 + * Improve avg aggregate functions (CASSANDRA-12417)
 + * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
 + * nodetool stopdaemon errors out (CASSANDRA-12646)
 + * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
 + * mx4j does not work in 3.0.8 (CASSANDRA-12274)
 + * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
 + * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
 + * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
 + * Fix exceptions with new vnode allocation (CASSANDRA-12715)
 + * Unify drain and shutdown processes (CASSANDRA-12509)
 + * Fix NPE in ComponentOfSlice.isEQ() (CASSANDRA-12706)
 + * Fix failure in LogTransactionTest (CASSANDRA-12632)
 + * Fix potentially incomplete non-frozen UDT values when querying with the
 +   full primary key specified (CASSANDRA-12605)
 + * Skip writing MV mutations to commitlog on mutation.applyUnsafe() 
(CASSANDRA-11670)
 + * Establish consistent distinction between non-existing partition and NULL 
value for LWTs on static columns (CASSANDRA-12060)
 + * Extend ColumnIdentifier.internedInstances key to include the type that 
generated the byte buffer (CASSANDRA-12516)
 + * Backport CASSANDRA-10756 (race condition in NativeTransportService 
shutdown) (CASSANDRA-12472)
 + * If CF has no clustering columns, any row cache is full partition cache 
(CASSANDRA-12499)
 + * Correct log message for statistics of offheap memtable flush 
(CASSANDRA-12776)
 + * Explicitly set locale for string validation 
(CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
 +Merged from 2.2:
   * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
   * Fail repair if participant dies during sync or anticompaction 
(CASSANDRA-12901)
   * cqlsh COPY: unprotected pk values before converting them if not using 
prepared statements (CASSANDRA-12863)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ec60900/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--

[jira] [Created] (CASSANDRA-13211) cassandra shell script uses bad approach to write "Unable to find java" error to stderr, causing real issue to be masked by a permission error if user has changed us

[Max Bowsher (JIRA)]

Max Bowsher created CASSANDRA-13211:
---

 Summary: cassandra shell script uses bad approach to write "Unable 
to find java" error to stderr, causing real issue to be masked by a permission 
error if user has changed user since logging in
 Key: CASSANDRA-13211
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13211
 Project: Cassandra
  Issue Type: Bug
Reporter: Max Bowsher


The cassandra startup shell script contains this line:

echo Unable to find java executable. Check JAVA_HOME and PATH environment 
variables. > /dev/stderr

The problem here is the construct "> /dev/stderr". If the user invoking 
Cassandra has changed user (for example, by SSHing in as a personal user, and 
then sudo-ing to an application user responsible for executing the Cassandra 
daemon), then the attempt to open /dev/stderr will fail, because it will point 
to a PTY node under /dev/pts/ owned by the original user.

Ultimately this leads to the real problem being masked by the confusing error 
message "bash: /dev/stderr: Permission denied".

The correct technique is to replace "> /dev/stderr" with ">&2" which will write 
to the already open stderr file descriptor, instead of resolving the chain of 
symlinks starting at /dev/stderr, and attempting to reopen the target by name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)