[jira] [Updated] (CASSANDRA-10882) network_topology_test dtest still failing

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10882:
-
Fix Version/s: 3.x
   2.2.x
   2.1.x

> network_topology_test dtest still failing
> -
>
> Key: CASSANDRA-10882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10882
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Philip Thompson
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> It looks like CASSANDRA-8158 may not have been properly resolved:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/176/testReport/replication_test/ReplicationTest/network_topology_test/history/
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/lastCompletedBuild/testReport/replication_test/ReplicationTest/network_topology_test/history/
> http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/lastCompletedBuild/testReport/replication_test/ReplicationTest/network_topology_test/history/
> [~philipthompson] Can you have a look?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-10111:
--
Reproduced In: 3.0.1, 2.2.4, 2.1.12, 2.0.15  (was: 2.0.15, 2.1.12, 2.2.4, 
3.0.1)
Fix Version/s: 3.x

> reconnecting snitch can bypass cluster name check
> -
>
> Key: CASSANDRA-10111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Chris Burroughs
>Assignee: Joel Knighton
>  Labels: gossip
> Fix For: 3.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different 
> listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing 
> node in cluster B (due to an out of data DNS entry).  Cluster B  added all of 
> the nodes from cluster A, somehow bypassing the cluster name mismatch check 
> for this nodes.  The first reference to cluster A nodes in cluster B logs is 
> when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node 
> /8.37.70.168 is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept 
> them out with 'ClusterName mismatch'.  Cluster B however tried to send to 
> send reads/writes to cluster A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since 
> the consequence of crazy merged clusters are really bad (the reason there is 
> the name mismatch check in the first place) I think the hole is reasonable to 
> plug.  I'm not sure exactly what the code path is that skips the check in 
> GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Deleted] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani deleted CASSANDRA-10881:
---


> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10882) network_topology_test dtest still failing

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10882:


 Summary: network_topology_test dtest still failing
 Key: CASSANDRA-10882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10882
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey
Assignee: Philip Thompson


It looks like CASSANDRA-8158 may not have been properly resolved:

http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/176/testReport/replication_test/ReplicationTest/network_topology_test/history/
http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/lastCompletedBuild/testReport/replication_test/ReplicationTest/network_topology_test/history/
http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/lastCompletedBuild/testReport/replication_test/ReplicationTest/network_topology_test/history/

[~philipthompson] Can you have a look?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060800#comment-15060800
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

I have re-created 2.2-All-Comments.patch. I tested this applies locally on a 
2.2 branch pulled in another directory (via git clone 
http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-2.2). Can you 
please confirm this works correctly for you ?

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10883) Unable to create a materialized view for retrieving the data from two different tables

[Sandeep Gopalasetty (JIRA)]

Sandeep Gopalasetty created CASSANDRA-10883:
---

 Summary: Unable to create a materialized view for retrieving the 
data from two different tables
 Key: CASSANDRA-10883
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10883
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: Windows
Reporter: Sandeep Gopalasetty
Priority: Critical
 Fix For: 3.0.0


Unable to create a materialized views for retrieving the data from two 
different tables. Syntactical problems are been arising. Is there any specific 
syntax? Or in which versions we can workout this concept (Materialized views 
from two different tables)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10883) Unable to create a materialized view for retrieving the data from two different tables

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-10883.
--
Resolution: Invalid
  Reviewer:   (was: Carl Yeksigian)

You cannot do this.

> Unable to create a materialized view for retrieving the data from two 
> different tables
> --
>
> Key: CASSANDRA-10883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10883
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
> Fix For: 3.0.0
>
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax? Or in which versions we can workout this concept 
> (Materialized views from two different tables)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10883) Unable to create a materialized view for retrieving the data from two different tables

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060780#comment-15060780
 ] 

T Jake Luciani commented on CASSANDRA-10883:


This is not a feature we support. You can't join tables in Cassandra. Even with 
materialized views.  Please stop opening tickets asking this over and over.

> Unable to create a materialized view for retrieving the data from two 
> different tables
> --
>
> Key: CASSANDRA-10883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10883
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
> Fix For: 3.0.0
>
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax? Or in which versions we can workout this concept 
> (Materialized views from two different tables)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10884) test_refresh_schema_on_timeout_error dtest flapping on CassCI

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10884:


 Summary: test_refresh_schema_on_timeout_error dtest flapping on 
CassCI
 Key: CASSANDRA-10884
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10884
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey


These tests create keyspaces and tables through cqlsh, then runs {{DESCRIBE}} 
to confirm they were successfully created. These tests flap under the novnode 
dtest runs:

http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_refresh_schema_on_timeout_error/history/
http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_refresh_schema_on_timeout_error/history/

I have not reproduced this locally on Linux.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Fix sstableloader not working with upper case keyspace name

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 363e7bd71 -> ed96322a0
  refs/heads/trunk abf7df3bd -> 37986e825


Fix sstableloader not working with upper case keyspace name

patch by Alex Liu; reviewed by yukim for CASSANDRA-10806


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed96322a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed96322a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed96322a

Branch: refs/heads/cassandra-3.0
Commit: ed96322a0dfc106e2270a7d0b9930a5311eadcbf
Parents: 363e7bd
Author: Alex Liu 
Authored: Wed Dec 16 14:34:24 2015 -0600
Committer: Yuki Morishita 
Committed: Wed Dec 16 14:34:24 2015 -0600

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed96322a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 10504c7..7677e38 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.3
+ * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 Merged from 2.2:
  * Add new types to Stress (CASSANDRA-9556)
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed96322a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 840ef26..bb927fc 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -75,7 +75,7 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 
 for (TokenRange tokenRange : tokenRanges)
 {
-Set endpoints = metadata.getReplicas(keyspace, 
tokenRange);
+Set endpoints = 
metadata.getReplicas(Metadata.quote(keyspace), tokenRange);
 Range range = new 
Range<>(tokenFactory.fromString(tokenRange.getStart().getValue().toString()),
  
tokenFactory.fromString(tokenRange.getEnd().getValue().toString()));
 for (Host endpoint : endpoints)

[2/3] cassandra git commit: Fix sstableloader not working with upper case keyspace name

[yukim]

Fix sstableloader not working with upper case keyspace name

patch by Alex Liu; reviewed by yukim for CASSANDRA-10806


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed96322a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed96322a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed96322a

Branch: refs/heads/trunk
Commit: ed96322a0dfc106e2270a7d0b9930a5311eadcbf
Parents: 363e7bd
Author: Alex Liu 
Authored: Wed Dec 16 14:34:24 2015 -0600
Committer: Yuki Morishita 
Committed: Wed Dec 16 14:34:24 2015 -0600

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed96322a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 10504c7..7677e38 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.3
+ * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 Merged from 2.2:
  * Add new types to Stress (CASSANDRA-9556)
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed96322a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 840ef26..bb927fc 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -75,7 +75,7 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 
 for (TokenRange tokenRange : tokenRanges)
 {
-Set endpoints = metadata.getReplicas(keyspace, 
tokenRange);
+Set endpoints = 
metadata.getReplicas(Metadata.quote(keyspace), tokenRange);
 Range range = new 
Range<>(tokenFactory.fromString(tokenRange.getStart().getValue().toString()),
  
tokenFactory.fromString(tokenRange.getEnd().getValue().toString()));
 for (Host endpoint : endpoints)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[yukim]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/37986e82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/37986e82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/37986e82

Branch: refs/heads/trunk
Commit: 37986e825e6706f3ed1c837e0b9523495a0b4e45
Parents: abf7df3 ed96322
Author: Yuki Morishita 
Authored: Wed Dec 16 14:38:19 2015 -0600
Committer: Yuki Morishita 
Committed: Wed Dec 16 14:38:19 2015 -0600

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/37986e82/CHANGES.txt
--
diff --cc CHANGES.txt
index 24b2662,7677e38..4b4a596
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,24 -1,5 +1,26 @@@
 -3.0.3
 +3.2
 + * Establish bootstrap stream sessions sequentially (CASSANDRA-6992)
 + * Sort compactionhistory output by timestamp (CASSANDRA-10464)
 + * More efficient BTree removal (CASSANDRA-9991)
 + * Make tablehistograms accept the same syntax as tablestats (CASSANDRA-10149)
 + * Group pending compactions based on table (CASSANDRA-10718)
 + * Add compressor name in sstablemetadata output (CASSANDRA-9879)
 + * Fix type casting for counter columns (CASSANDRA-10824)
 + * Prevent running Cassandra as root (CASSANDRA-8142)
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
++Merged from 3.0:
+  * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
  Merged from 2.2:
   * Add new types to Stress (CASSANDRA-9556)
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)

[Cassandra Wiki] Trivial Update of "ContributorsGroup" by JonathanEllis

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "ContributorsGroup" page has been changed by JonathanEllis:
https://wiki.apache.org/cassandra/ContributorsGroup?action=diff=53=54

Comment:
alphabetized

  
   * AdminGroup
   * AaronMorton
+  * achilleasa
+  * AdamHolmberg
   * AlekseyYeschenko
   * Alexis Wilke
+  * AlicePorfirio
+  * bhamail
   * Ben McCann
+  * BenedictElliottSmith
   * BenHood
   * BenjaminLerer
+  * CarlYeksigian
   * cassandracomm
+  * Changjiu
   * ChrisBroome
+  * ChrisBurroughs
+  * daniels
   * EricEvans
   * ErnieHershey
   * FlipKromer
@@ -26, +35 @@

   * JeremiahJordan
   * Jim Witschey
   * JoaquinCasares
+  * JoelKnighton
+  * JohnSumsion
+  * JoshuaMcKenzie
+  * KarlLehenbauer
   * LukasGutschmidt
   * LukasWingerberg
   * LyubenTodorov
+  * JonHaddad
   * MakiWatanabe
   * MarcusEriksson
   * MarkWatson
   * MatthewDennis
   * MichaelEdge
   * MichaelKjellman
+  * MichaelShuler
+  * mkjellman
+  * mriou
   * NickBailey
   * Nick Neuberger
+  * ono_matope
+  * OtisGospodnetic
   * PeterSchuller
+  * PauloMotta
   * PavelYaskevich
+  * PhiloYang
   * PierreChalamet
   * PrashantMalik
+  * RobertStupp
+  * RussellHatch
+  * SamTunnicliffe
   * SergeRider
   * StephenBlackheath
   * StephenConnolly
@@ -49, +73 @@

   * thepaul
   * TylerHobbs
   * Victor Lownes
+  * wombat
+  * woolfel
   * yukim
   * zznate
-  * mkjellman
-  * ono_matope
-  * ChrisBurroughs
-  * bhamail
-  * daniels
-  * JonHaddad
-  * PhiloYang
-  * Changjiu
-  * woolfel
-  * MichaelShuler
-  * BenedictElliottSmith
-  * RobertStupp
-  * JoshuaMcKenzie
-  * OtisGospodnetic
-  * CarlYeksigian
-  * JohnSumsion
-  * mriou
-  * achilleasa
-  * RussellHatch
-  * KarlLehenbauer
-  * wombat
-  * AlicePorfirio
-  * SamTunnicliffe
-  * PauloMotta
-  * AdamHolmberg
-  * JoelKnighton
-  * Jim Witschey
  

[jira] [Commented] (CASSANDRA-10838) print 3.0 statistics in sstablemetadata command output

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060735#comment-15060735
 ] 

Yuki Morishita commented on CASSANDRA-10838:


Thanks for the patch!

||branch||testall||dtest||
|[10838|https://github.com/yukim/cassandra/tree/10838]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-10838-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-10838-dtest/lastCompletedBuild/testReport/]|

dtest shows offline_tools_test.py is failing and I see the following exception.

{code}
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at 
org.apache.cassandra.tools.SSTableMetadataViewer.main(SSTableMetadataViewer.java:98)
{code}

Can you add index check to the code and re-submit the patch?

> print 3.0 statistics in sstablemetadata command output
> --
>
> Key: CASSANDRA-10838
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10838
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Shogo Hoshii
>Priority: Minor
> Fix For: 3.x
>
> Attachments: CASSANDRA-10838.txt, sample_result.txt
>
>
> In CASSANDRA-7159, some statistics were added in 2.1.x, and in version 3.0, 
> we can print additional statistics.
> So I would like to print them in sstablemetadata output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of "HowToContribute" by Jim Witschey

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "HowToContribute" page has been changed by Jim Witschey:
https://wiki.apache.org/cassandra/HowToContribute?action=diff=67=68

Comment:
documents use of known_failure decorator

* `git clone https://github.com/riptano/cassandra-dtest.git 
cassandra-dtest`.
   1. Set `$CASSANDRA_DIR` to the location of your cassandra checkout.  For 
example: `export CASSANDRA_DIR=/home/joe/cassandra`.  Make sure you've already 
built Cassandra in this directory. You can build Cassandra by running `ant`.
   1. Run all tests by running `nosetests` from the dtest checkout.  You can 
run a specific module like so: `nosetests cql_tests.py`.  You can run a 
specific test method like this: `nosetests cql_tests.py:TestCQL.counters_test`.
+   * If you encounter any failures, you can confirm whether or not they exist 
in upstream branches by checking to see if the failing tests or test classes 
are tagged with the `known_failure` decorator. This decorator is 
[[https://github.com/riptano/cassandra-dtest/blob/master/tools.py#L329|documented
 inline in the dtests]]. If a test that is known to fail passes, or a test that 
is not known to fail succeeds, you should check the linked JIRA ticket to see 
if you've introduced any detrimental changes to that branch.
  
  === Running the code coverage task ===
   1. Run a basic coverage report of unit tests using `ant codecoverage`.

[jira] [Commented] (CASSANDRA-9813) cqlsh column header can be incorrect when no rows are returned

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060678#comment-15060678
 ] 

Ariel Weisberg commented on CASSANDRA-9813:
---

+1 marked RTC

> cqlsh column header can be incorrect when no rows are returned
> --
>
> Key: CASSANDRA-9813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9813
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Adam Holmberg
>  Labels: cqlsh
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 9813-2.1.txt, Test-for-9813.txt
>
>
> Upon migration, we internally create a pair of surrogate clustering/regular 
> columns for compact static tables. These shouldn't be exposed to the user.
> That is, for the table
> {code}
> CREATE TABLE bar (k int, c int, PRIMARY KEY (k)) WITH COMPACT STORAGE;
> {code}
> {{SELECT * FROM bar}} should not be returning this result set:
> {code}
> cqlsh:test> select * from bar;
>  c | column1 | k | value
> ---+-+---+---
> (0 rows)
> {code}
> Should only contain the defined {{c}} and {{k}} columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9813) cqlsh column header can be incorrect when no rows are returned

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9813:
--
Fix Version/s: 3.0.x

> cqlsh column header can be incorrect when no rows are returned
> --
>
> Key: CASSANDRA-9813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9813
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Adam Holmberg
>  Labels: cqlsh
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 9813-2.1.txt, Test-for-9813.txt
>
>
> Upon migration, we internally create a pair of surrogate clustering/regular 
> columns for compact static tables. These shouldn't be exposed to the user.
> That is, for the table
> {code}
> CREATE TABLE bar (k int, c int, PRIMARY KEY (k)) WITH COMPACT STORAGE;
> {code}
> {{SELECT * FROM bar}} should not be returning this result set:
> {code}
> cqlsh:test> select * from bar;
>  c | column1 | k | value
> ---+-+---+---
> (0 rows)
> {code}
> Should only contain the defined {{c}} and {{k}} columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Sandeep Gopalasetty (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060718#comment-15060718
 ] 

Sandeep Gopalasetty commented on CASSANDRA-10881:
-


Mr. Brandon,

Why u closed it as INVALID.

We were unable to create a materialized views for retrieving the data from two 
different tables. Syntactical problems are been arising. Is there any specific 
syntax?




-- 
Thanks & Regards,
Sandeep Gopalasetty
IIB Developer
Miracle Software Systems,Inc
Email : sgopalase...@miraclesoft.com
Work  : 1-248-233-1889
Mobile: 91-9966991134


__
This email has been scanned by the Symantec Email Security.
__


> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Sandeep Gopalasetty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandeep Gopalasetty reopened CASSANDRA-10881:
-

Hi Carl,
If there is no support for joining data from multiple source table means, in 
real time projects we definitely require this case. Is there any kind of 
alternative carl?
My Schemas:
Table 1. CREATE TABLE courses(code TEXT,description TEXT,days INT,PRIMARY KEY 
(code, description,days));
Values:
INSERT INTO courses(code, description, days) VALUES ('SQL','SQL course',45);
INSERT INTO courses(code, description, days) VALUES ('OAU','Oracle course',45);
INSERT INTO courses(code, description, days) VALUES ('JAV','Java course',90);
INSERT INTO courses(code, description, days) VALUES ('PLS','plsql course',30);
INSERT INTO courses(code, description, days) VALUES ('SAP','SAP course',90);
INSERT INTO courses(code, description, days) VALUES ('DNT','dotnet course',60);
INSERT INTO courses(code, description, days) VALUES ('NWS','Networks 
course',45);
INSERT INTO courses(code, description, days) VALUES ('CLC','Cloud computing 
course',40);
Table 2. CREATE TABLE courseschedule(course TEXT,trainer TEXT,location 
TEXT,PRIMARY KEY (course,trainer));
Values:
INSERT INTO courseschedule(course, trainer, location) VALUES ('SQL','Satish 
Mongam','Mind Space HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('OAU','Vinay 
Raj','Cyber Gateway HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('JAV','Ravi 
Nallani','Cyber Towers HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('PLS','Srikar 
Gondesi','Maitrivanam HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('SAP','Devi 
Prasad Reddy','SAP Labs BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES ('DNT','Rohit 
Kotni','White field BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES ('NWS','Kiran 
Mammuluri','EON IT PUN');
INSERT INTO courseschedule(course, trainer, location) VALUES ('CLC','Viraaj 
Patel','AD Labs HYD');
I want a materialized view as "Job Training". It should retrieve as 
code,description,days from Table "courses" and trainer from "courseschedule" 
WHERE code from courses = course from courseschedule. This view should be 
created, as it is from two different tables. Is there any specific syntax? We 
are unable to create this, could you please help.

> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-10881.
--
Resolution: Invalid

Resolving as invalid.

> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060723#comment-15060723
 ] 

Joshua McKenzie commented on CASSANDRA-10881:
-

As per Carl's comment:

bq. You can only create a view based on a single source table. There is no 
support for joining data from multiple source tables.

JIRA is not a support forum. Please take discussion to the mailing list.

> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Sandeep Gopalasetty (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060724#comment-15060724
 ] 

Sandeep Gopalasetty commented on CASSANDRA-10881:
-


Hi Carl,

we were unable to create a materialized view for retrieving the data
from two different tables.

My Schemas:

Table 1. CREATE TABLE courses(code TEXT,description TEXT,days
INT,PRIMARY KEY (code, description,days));
Values:
INSERT INTO courses(code, description, days) VALUES ('SQL','SQL course',45);
INSERT INTO courses(code, description, days) VALUES ('OAU','Oracle
course',45);
INSERT INTO courses(code, description, days) VALUES ('JAV','Java
course',90);
INSERT INTO courses(code, description, days) VALUES ('PLS','plsql
course',30);
INSERT INTO courses(code, description, days) VALUES ('SAP','SAP course',90);
INSERT INTO courses(code, description, days) VALUES ('DNT','dotnet
course',60);
INSERT INTO courses(code, description, days) VALUES ('NWS','Networks
course',45);
INSERT INTO courses(code, description, days) VALUES ('CLC','Cloud
computing course',40);
Table 2. CREATE TABLE courseschedule(course TEXT,trainer TEXT,location
TEXT,PRIMARY KEY (course,trainer));
Values:
INSERT INTO courseschedule(course, trainer, location) VALUES
('SQL','Satish Mongam','Mind Space HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES
('OAU','Vinay Raj','Cyber Gateway HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES
('JAV','Ravi Nallani','Cyber Towers HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES
('PLS','Srikar Gondesi','Maitrivanam HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES
('SAP','Devi Prasad Reddy','SAP Labs BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES
('DNT','Rohit Kotni','White field BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES
('NWS','Kiran Mammuluri','EON IT PUN');
INSERT INTO courseschedule(course, trainer, location) VALUES
('CLC','Viraaj Patel','AD Labs HYD');

I want a materialized view as "Job Training". It should retrieve as
code,description,days from Table "courses" and trainer from
"courseschedule" WHERE code from courses = course from courseschedule.
This view should be created, as it is from two different tables. Is
there any specific syntax? We are unable to create this, could you
please help.



-- 
Thanks & Regards,
Sandeep Gopalasetty
IIB Developer
Miracle Software Systems,Inc
Email : sgopalase...@miraclesoft.com
Work  : 1-248-233-1889
Mobile: 91-9966991134


__
This email has been scanned by the Symantec Email Security.
__


> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10580:
-
Attachment: (was: 2.2-All-Comments.patch)

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10580:
-
Attachment: 2.2-All-Comments.patch

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10872) Debian Package does not prompt the user to review the config files; it just replaces them causing trouble (since the daemon starts by default)

[Vasilis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060339#comment-15060339
 ] 

Vasilis commented on CASSANDRA-10872:
-

Have you been able to reproduce it [~mshuler]? This morning I installed Ubuntu 
12.04 on two VMs, copied the 2.0.16 debian package from the live cluster, 
installed it using _dpkg --install _, did an update/upgrade in 
order to go to 2.0.17 and I got different behaviour... I wasn't prompted but 
the files were not replaced this time. I cannot understand why yet, but I hope 
this might be useful to you? Have I done something stupid here I followed 
exactly the same path in order to go from _2.0.16 -> 2.0.17_ in both scenarios. 
I also checked that the 2.0.17 package that I got from my _cassandra.list_ 
found in _/var/cache/apt/archives_ is the same as the one in the live cluster.

On a side note, I was under the impression that DC name cannot change once set 
according to the 
[documentation|http://docs.datastax.com/en/cassandra/2.0/cassandra/initialize/initializeMultipleDS.html],
 but then I found CASSANDRA-9474, which in a way I am happy it hasn't been 
fixed for 2.0.x. So, I just changed the _cassandra-rackdc.properties_ and it 
looks like it worked for me.

> Debian Package does not prompt the user to review the config files; it just 
> replaces them causing trouble (since the daemon starts by default)
> --
>
> Key: CASSANDRA-10872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
> Environment: Ubuntu 12.04, Cassandra 2.0.16 -> 2.0.17
>Reporter: Vasilis
>Assignee: Michael Shuler
>Priority: Critical
> Fix For: 2.1.x
>
>
> We run Cassandra 2.0.16 on Ubuntu 12.04 and were trying to upgrade to 2.0.17 
> due to CASSANDRA-9662. The problem is that during the upgrade the we were not 
> prompted how to handle cassandra-env.sh and cassandra-rackdc.properties. 
> The output from the upgrade:
> Setting up cassandra (2.0.17) ...
> Installing new version of config file /etc/cassandra/cassandra-env.sh ...
> Installing new version of config file 
> /etc/cassandra/cassandra-rackdc.properties ...
> This meant that the nodes started automatically after the install with the 
> wrong DC name. 
> I don't think that these config files should have been replaced without the 
> admin being asked; this doesn't appear to comply with standard Debian 
> packages.
> Secondly if CASSANDRA-2356 was implemented the problem would not be as 
> severe; i.e. it would be possible to workaround this issue. Whereas 
> currently, there is no way to prevent the node when upgraded from starting in 
> the wrong DC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060340#comment-15060340
 ] 

Paulo Motta commented on CASSANDRA-10580:
-

bq. can you please confirm if that's what everyone follows ?

You're generating patch correctly. I suspect you're not on the correct branch. 
Is your cassandra-2.2 head b03ce9fd479d3da9c51d118c50480ea96df0d73e ?

If not, you need to synchronize your git repo with 
https://github.com/apache/cassandra.git and checkout the latest cassandra-2.2 
branch, and build your patch on the top of it.

bq. About the tests, I don't think the failures are related to my change. Are 
those failures expected ?

I think they are unrelated too, they're in pair with main branch failures: 
http://cassci.datastax.com/view/trunk/

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie resolved CASSANDRA-10881.
-
Resolution: Not A Problem

Closing this ticket again as "Not a Problem". If you have questions about using 
cassandra, the [cassandra user emailing 
list|http://www.planetcassandra.org/apache-cassandra-mailing-lists/] is the 
right place to go for those answers.

> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of "ContributorsGroup" by JonathanEllis

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "ContributorsGroup" page has been changed by JonathanEllis:
https://wiki.apache.org/cassandra/ContributorsGroup?action=diff=52=53

Comment:
add Jim Witschey

   * JakeLuciani
   * JasonBrown
   * JeremiahJordan
+  * Jim Witschey
   * JoaquinCasares
   * LukasGutschmidt
   * LukasWingerberg

[jira] [Commented] (CASSANDRA-10863) HSHA test_closing_connections test still flaps on 3.0

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060293#comment-15060293
 ] 

Carl Yeksigian commented on CASSANDRA-10863:


It seems like we only ever have 1 socket which is hanging out in 
{{CLOSE_WAIT}}, so it may still be a timing issue.

I'll look into the test failure, but we should keep this test out until we can 
either make it stable or replace it with something else (see also 
CASSANDRA-10879).

> HSHA test_closing_connections test still flaps on 3.0
> -
>
> Key: CASSANDRA-10863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10863
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.0.x
>
>
> The problem reported in CASSANDRA-10570 still seems to be happening on 
> CassCI, as recently as a couple days ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/433/
> [~carlyeks] The fix in #10570 was to increase how long we'd sleep in the 
> tests. Should we just bump it up further, or does this make you suspect a 
> larger problem here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060307#comment-15060307
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

I am not sure what's going on with 2.2. I will look at it again today. I do the 
following to generate patches -- can you please confirm if that's what everyone 
follows ?

1. Change files
2. git status (Confirm that the changes are correct)
3. git add .
4. git commit -m "Foo"
5. git format-patch HEAD~

About the tests, I don't think the failures are related to my change. Are those 
failures expected ?

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10872) Debian Package does not prompt the user to review the config files; it just replaces them causing trouble (since the daemon starts by default)

[Vasilis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vasilis updated CASSANDRA-10872:

Description: 
We run Cassandra 2.0.16 on Ubuntu 12.04 and were trying to upgrade to 2.0.17 
due to CASSANDRA-9662. The problem is that during the upgrade the we were not 
prompted how to handle cassandra-env.sh and cassandra-rackdc.properties. 

The output from the upgrade:
{panel}
...
Setting up cassandra (2.0.17) ...
Installing new version of config file /etc/cassandra/cassandra-env.sh ...
Installing new version of config file 
/etc/cassandra/cassandra-rackdc.properties ...
vm.max_map_count = 1048575
net.ipv4.tcp_keepalive_time = 300
...
{panel}
This meant that the nodes started automatically after the install with the 
wrong DC name. 

I don't think that these config files should have been replaced without the 
admin being asked; this doesn't appear to comply with standard Debian packages.

Secondly if CASSANDRA-2356 was implemented the problem would not be as severe; 
i.e. it would be possible to workaround this issue. Whereas currently, there is 
no way to prevent the node when upgraded from starting in the wrong DC.

  was:
We run Cassandra 2.0.16 on Ubuntu 12.04 and were trying to upgrade to 2.0.17 
due to CASSANDRA-9662. The problem is that during the upgrade the we were not 
prompted how to handle cassandra-env.sh and cassandra-rackdc.properties. 

The output from the upgrade:

Setting up cassandra (2.0.17) ...
Installing new version of config file /etc/cassandra/cassandra-env.sh ...
Installing new version of config file 
/etc/cassandra/cassandra-rackdc.properties ...

This meant that the nodes started automatically after the install with the 
wrong DC name. 

I don't think that these config files should have been replaced without the 
admin being asked; this doesn't appear to comply with standard Debian packages.

Secondly if CASSANDRA-2356 was implemented the problem would not be as severe; 
i.e. it would be possible to workaround this issue. Whereas currently, there is 
no way to prevent the node when upgraded from starting in the wrong DC.


> Debian Package does not prompt the user to review the config files; it just 
> replaces them causing trouble (since the daemon starts by default)
> --
>
> Key: CASSANDRA-10872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
> Environment: Ubuntu 12.04, Cassandra 2.0.16 -> 2.0.17
>Reporter: Vasilis
>Assignee: Michael Shuler
>Priority: Critical
> Fix For: 2.1.x
>
>
> We run Cassandra 2.0.16 on Ubuntu 12.04 and were trying to upgrade to 2.0.17 
> due to CASSANDRA-9662. The problem is that during the upgrade the we were not 
> prompted how to handle cassandra-env.sh and cassandra-rackdc.properties. 
> The output from the upgrade:
> {panel}
> ...
> Setting up cassandra (2.0.17) ...
> Installing new version of config file /etc/cassandra/cassandra-env.sh ...
> Installing new version of config file 
> /etc/cassandra/cassandra-rackdc.properties ...
> vm.max_map_count = 1048575
> net.ipv4.tcp_keepalive_time = 300
> ...
> {panel}
> This meant that the nodes started automatically after the install with the 
> wrong DC name. 
> I don't think that these config files should have been replaced without the 
> admin being asked; this doesn't appear to comply with standard Debian 
> packages.
> Secondly if CASSANDRA-2356 was implemented the problem would not be as 
> severe; i.e. it would be possible to workaround this issue. Whereas 
> currently, there is no way to prevent the node when upgraded from starting in 
> the wrong DC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060376#comment-15060376
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

My HEAD is indeed pointing to different location: 
f1c3df0e848638735c790b4817adf6411a52a064. I pulled down 2.2. via below and then 
made changes.

git clone http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-2.2

I will dig some more on why this did not work correctly.

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9303) Match cassandra-loader options in COPY FROM

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060520#comment-15060520
 ] 

Paulo Motta commented on CASSANDRA-9303:


Looking good, thanks! Some follow-ups below:

bq. CONFIGSECTIONS: this is removed and instead we search the following static 
sections: \[copy\], \[copy-ks-table\], \[copy-ks-table-from\] or 
\[copy-ks-table-to\], in this order.

sounds good! I'd suggest the following \[copy(:ks.table)\] (global and 
per-table copy (to and from) options), \[copy-from(:ks.table)\] (global and 
per-table copy-from options), \[copy-to(:ks.table)\] (global and per-table 
copy-to options) where (:ks.table) is optional. so you can have \[copy\], 
\[copy-to\], \[copy-from\], \[copy-to:ks.table\], \[copy-from:ks.table\].

bq. if no error file is specified I've introduced a default error file called 
import_ks_table.err

nice! maybe we could just add an unique suffix to avoid appending to an 
existing file from a previous execution?

bq. Another thing that follows from the CASSANDRA-9302 review is that the 
INGESTRATE only works if it is much bigger than the CHUNKSIZE. We could address 
it here if you think this is important. 

We can address if it won't take too much time, otherwise we can address it 
separately. Can we maybe improve it by making batchsize adaptive = 
{{min(batchsize, ingest_rate - current_record)}} or something more complicated 
will be needed?

Some minor things I missed before:

* Move {{SKIPCOLS}} to {{COPY_COMMON_OPTIONS}} since it can be used in both 
copy-to and copy-from.
* Regarding the beahvior of {{SKIPCOLS}} with COPY FROM, right now it only 
supports having fewer columns in the CSV. Should we also support actually 
skipping columns in the CSV even if they are present?
** Another related feature to have in the future would be to pick only specific 
columnms from the csv and allowing custom orderings of columns, but we can 
leave that for later if there's a need.

After those are addressed you can probably start making 2.2+ patches.

> Match cassandra-loader options in COPY FROM
> ---
>
> Key: CASSANDRA-9303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> https://github.com/brianmhess/cassandra-loader added a bunch of options to 
> handle real world requirements, we should match those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Sandeep Gopalasetty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandeep Gopalasetty reopened CASSANDRA-10881:
-
Reproduced In: 3.0.0
   Tester: Carl Yeksigian
Since Version: 3.0.0

> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10876) Alter behavior of batch WARN and fail on single partition batches

[Patrick McFadin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick McFadin updated CASSANDRA-10876:

Summary: Alter behavior of batch WARN and fail on single partition batches  
(was: Alter behavior of batch WARN and ERROR on single partition batches)

> Alter behavior of batch WARN and fail on single partition batches
> -
>
> Key: CASSANDRA-10876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10876
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Patrick McFadin
>Priority: Minor
>
> In an attempt to give operator insight into potentially harmful batch usage, 
> Jiras were created to log WARN or fail on certain batch sizes. This ignores 
> the single partition batch, which doesn't create the same issues as a 
> multi-partition batch. 
> The proposal is to ignore size on single partition batch statements. 
> Reference:
> [CASSANDRA-6487|https://issues.apache.org/jira/browse/CASSANDRA-6487]
> [CASSANDRA-8011|https://issues.apache.org/jira/browse/CASSANDRA-8011]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables.

[Sandeep Gopalasetty (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060427#comment-15060427
 ] 

Sandeep Gopalasetty commented on CASSANDRA-10881:
-

Hi Carl,

If there is no support for joining data from multiple source table means, in 
real time projects we definitely require this case. Is there any kind of 
alternative carl?

My Schemas:

Table 1. CREATE TABLE courses(code TEXT,description TEXT,days INT,PRIMARY KEY 
(code, description,days));
  Values:
INSERT INTO courses(code, description, days) VALUES ('SQL','SQL 
course',45);
INSERT INTO courses(code, description, days) VALUES ('OAU','Oracle 
course',45);
INSERT INTO courses(code, description, days) VALUES ('JAV','Java 
course',90);
INSERT INTO courses(code, description, days) VALUES ('PLS','plsql 
course',30);
INSERT INTO courses(code, description, days) VALUES ('SAP','SAP 
course',90);
INSERT INTO courses(code, description, days) VALUES ('DNT','dotnet 
course',60);
INSERT INTO courses(code, description, days) VALUES ('NWS','Networks 
course',45);
INSERT INTO courses(code, description, days) VALUES ('CLC','Cloud 
computing course',40);
Table 2. CREATE TABLE courseschedule(course TEXT,trainer TEXT,location 
TEXT,PRIMARY KEY (course,trainer));
  Values:
INSERT INTO courseschedule(course, trainer, location) VALUES ('SQL','Satish 
Mongam','Mind Space HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('OAU','Vinay 
Raj','Cyber Gateway HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('JAV','Ravi 
Nallani','Cyber Towers HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('PLS','Srikar 
Gondesi','Maitrivanam HYD');
INSERT INTO courseschedule(course, trainer, location) VALUES ('SAP','Devi 
Prasad Reddy','SAP Labs BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES ('DNT','Rohit 
Kotni','White field BLR');
INSERT INTO courseschedule(course, trainer, location) VALUES ('NWS','Kiran 
Mammuluri','EON IT PUN');
INSERT INTO courseschedule(course, trainer, location) VALUES ('CLC','Viraaj 
Patel','AD Labs HYD');

I want a materialized view as "Job Training". It should retrieve as 
code,description,days from Table "courses" and trainer from "courseschedule" 
WHERE code from courses = course from courseschedule. This view should be 
created, as it is from two different tables. Is there any specific syntax? We 
are unable to create this, could you please help.


> Unable to create a materialized view for retrieving the data from two 
> different tables.
> ---
>
> Key: CASSANDRA-10881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Windows
>Reporter: Sandeep Gopalasetty
>Priority: Critical
>
> Unable to create a materialized views for retrieving the data from two 
> different tables. Syntactical problems are been arising. Is there any 
> specific syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10863) HSHA test_closing_connections test still flaps on 3.0

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060462#comment-15060462
 ] 

Jim Witschey commented on CASSANDRA-10863:
--

We've got some new machinery that'll make it easier to filter out known 
failures. I'm documenting that as we speak. I'm ok with keeping this flaky test 
around now that that's in place.

> HSHA test_closing_connections test still flaps on 3.0
> -
>
> Key: CASSANDRA-10863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10863
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.0.x
>
>
> The problem reported in CASSANDRA-10570 still seems to be happening on 
> CassCI, as recently as a couple days ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/433/
> [~carlyeks] The fix in #10570 was to increase how long we'd sleep in the 
> tests. Should we just bump it up further, or does this make you suspect a 
> larger problem here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10872) Debian Package does not prompt the user to review the config files; it just replaces them causing trouble (since the daemon starts by default)

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060486#comment-15060486
 ] 

Michael Shuler commented on CASSANDRA-10872:


I have not tested this out, yet. I'm working on a package build/install job in 
CI, then we can use those packages to automate upgrade testing like this.

> Debian Package does not prompt the user to review the config files; it just 
> replaces them causing trouble (since the daemon starts by default)
> --
>
> Key: CASSANDRA-10872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
> Environment: Ubuntu 12.04, Cassandra 2.0.16 -> 2.0.17
>Reporter: Vasilis
>Assignee: Michael Shuler
>Priority: Critical
> Fix For: 2.1.x
>
>
> We run Cassandra 2.0.16 on Ubuntu 12.04 and were trying to upgrade to 2.0.17 
> due to CASSANDRA-9662. The problem is that during the upgrade the we were not 
> prompted how to handle cassandra-env.sh and cassandra-rackdc.properties. 
> The output from the upgrade:
> {panel}
> ...
> Setting up cassandra (2.0.17) ...
> Installing new version of config file /etc/cassandra/cassandra-env.sh ...
> Installing new version of config file 
> /etc/cassandra/cassandra-rackdc.properties ...
> vm.max_map_count = 1048575
> net.ipv4.tcp_keepalive_time = 300
> ...
> {panel}
> This meant that the nodes started automatically after the install with the 
> wrong DC name. 
> I don't think that these config files should have been replaced without the 
> admin being asked; this doesn't appear to comply with standard Debian 
> packages.
> Secondly if CASSANDRA-2356 was implemented the problem would not be as 
> severe; i.e. it would be possible to workaround this issue. Whereas 
> currently, there is no way to prevent the node when upgraded from starting in 
> the wrong DC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10874) running stress with compaction strategy and replication factor fails on read after write

[Andrew Hust (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Hust updated CASSANDRA-10874:

Description: 
When running a read stress after write stress with a compaction strategy and 
replication factor matching the node count will fail with an exception.  
{code}
Operation x0 on key(s) [38343433384b34364c30]: Data returned was not validated
{code}

Example run:
{code}
ccm create stress -v git:cassandra-3.0 -n 3 -s
ccm node1 stress write n=10M -rate threads=300 -schema replication\(factor=3\) 
compaction\(strategy=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy\)
ccm node1 nodetool flush
ccm node1 nodetool compactionstats # check until quiet
ccm node1 stress read n=10M -rate threads=300
{code}
- This will fail with/out vnodes but will occasionally pass without vnodes. 
- Changing the read phase to be CL=QUORUM will make it pass.  
- Removing the replication factor on write will make it pass.
- Happens on all compaction strategies

So with that in mind I attempted to add a repair after the write phase.  This 
leads to 1 of 2 outcomes.

1: a repair that has a greater than 100% completion, usually stalls after a 
bit, but have seen it get to >400% progress:
{code}
  id   compaction typekeyspace   
table completed totalunit   progress
2d5344c0-9dc8-11e5-9d5f-4fdec8d76c27Validation   keyspace1   
standard1   94722609949   44035292145   bytes215.11%
{code}

2: a repair that has a greatly inflated completed/total value, it will crunch 
for a bit then lockup:
{code}
 id   compaction typekeyspace   
table   completed  totalunit   progress
   8c4cf7f0-a34a-11e5-a321-777be88c58aeValidation   keyspace1   
standard1   0   874811100900   bytes  0.00%

❯ du -sh ~/.ccm/stress/node1/
2.4G  ~/.ccm/stress/node1/
❯ du -sh ~/.ccm/stress
7.1G  ~/.ccm/stress
{code}

This has been reproduced on cassandra-3.0 and cassandra-2.1 both locally and 
using cstar_perf (links below).  
A big twist is that cassandra-2.2 will pass the majority of the time.  It will 
complete successfully without the repair 8 out of 10 runs.  This can be seen in 
the cstar_perf links below.

cstar_perf runs:
http://cstar.datastax.com/tests/id/c8fa27a4-a205-11e5-8fbc-0256e416528f
http://cstar.datastax.com/tests/id/a254c572-a2ce-11e5-a8b9-0256e416528f

  was:
When running a read stress after write stress with a compaction strategy and 
replication factor matching the node count will fail with an exception.  
{code}
Operation x0 on key(s) [38343433384b34364c30]: Data returned was not validated
{code}

Example run:
{code}
ccm create stress -v git:cassandra-3.0 -n 3 -s
ccm node1 stress write n=10M -rate threads=300 -schema replication\(factor=3\) 
compaction\(strategy=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy\)
ccm node1 nodetool flush
ccm node1 nodetool compactionstats # check until quiet
ccm node1 stress read n=10M -rate threads=300
{code}
- This will fail with/out vnodes but will occasionally pass without vnodes. 
- Changing the read phase to be CL=QUORUM will make it pass.  
- Removing the replication factor on write will make it pass.
- Happens on all compaction strategies

So with that in mind I attempted to add a repair after the write phase.  This 
leads to 1 of 2 outcomes.

1: a repair that has a greater than 100% completion, usually stalls after a 
bit, but have seen it get to >400% progress:
{code}
  id   compaction typekeyspace   
table completed totalunit   progress
2d5344c0-9dc8-11e5-9d5f-4fdec8d76c27Validation   keyspace1   
standard1   94722609949   44035292145   bytes215.11%
{code}

2: a repair that has a greatly inflated completed/total value, it will crunch 
for a bit then lockup:
{code}
 id   compaction typekeyspace   
table   completed  totalunit   progress
   8c4cf7f0-a34a-11e5-a321-777be88c58aeValidation   keyspace1   
standard1   0   874811100900   bytes  0.00%

❯ du -sh ~/.ccm/stress/node1/
2.4G  ~/.ccm/stress/node1/
❯ du -sh ~/.ccm/stress
7.1G  ~/.ccm/stress
{code}

This has been reproduced on cassandra-3.0 and cassandra-2.2 both locally and 
using cstar_perf (links below).  
A big twist is that cassandra-2.2 will pass the majority of the time.  It will 
complete successfully without the repair 8 out of 10 runs.  This can be seen in 
the cstar_perf links below.

cstar_perf runs:
http://cstar.datastax.com/tests/id/c8fa27a4-a205-11e5-8fbc-0256e416528f
http://cstar.datastax.com/tests/id/a254c572-a2ce-11e5-a8b9-0256e416528f


> running stress with compaction strategy and replication factor fails on read 
> after write
> 

[jira] [Commented] (CASSANDRA-10880) Paging state between 2.2 and 3.0 are incompatible on protocol v4

[Sandeep Tamhankar (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060536#comment-15060536
 ] 

Sandeep Tamhankar commented on CASSANDRA-10880:
---

Not sure if we want to prefer v3 for Cassandra 2.2 because then we lose UDF/UDA 
support in the driver, don't we? For example, the v3 protocol doc doesn't 
mention how to handle schema change events for FUNCTIONs and AGGREGATEs, while 
the v4 protocol doc does.

> Paging state between 2.2 and 3.0 are incompatible on protocol v4
> 
>
> Key: CASSANDRA-10880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10880
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Priority: Critical
>  Labels: client-impacting
> Fix For: 3.x
>
>
> In CASSANDRA-10254, the paging states generated by 3.0 for the native 
> protocol v4 were made 3.0 specific. This was done because the paging state in 
> pre-3.0 versions contains a serialized cell name, but 3.0 doesn't talk in 
> term of cells internally (at least not the pre-3.0 ones) and so using an 
> old-format cell name when we only have 3.0 nodes is inefficient and inelegant.
> Unfortunately that change was made on the assumption than the protocol v4 was 
> 3.0 only but it's not, it ended up being released with 2.2 and that 
> completely slipped my mind. So in practice, you can't properly have a mixed 
> 2.2/3.0 cluster if your driver is using the protocol v4.
> And unfortunately, I don't think there is an easy way to fix that without 
> breaking something. Concretely, I can see 3 choices:
> # we change 3.0 so that it generates old-format paging states on the v4 
> protocol. The 2 main downsides are that 1) this breaks 3.0 upgrades if the 
> driver is using the v4 protocol, and at least on the java side the only 
> driver versions that support 3.0 will use v4 by default and 2) we're signing 
> off on having sub-optimal paging state until the protocol v5 ships (probably 
> not too soon).
> # we remove the v4 protocol from 2.2. This means 2.2 will have to use v3 
> before upgrade at the risk of breaking upgrade. This is also bad, but I'm not 
> sure the driver version using the v4 protocol are quite ready yet (at least 
> the java driver is not GA yet) so if we work with the drivers teams to make 
> sure the v3 protocol gets prefered by default on 2.2 in the GA versions of 
> these driver, this might be somewhat transparent to users.
> # we don't change anything code-wise, but we document clearly that you can't 
> upgrade from 2.2 to 3.0 if your clients use protocol v4 (so we leave upgrade 
> broken if the v4 protocol is used as it is currently). This is not great, but 
> we can work with the drivers teams here again to make sure drivers prefer the 
> v3 version for 2.2 nodes so most people don't notice in practice.
> I think I'm leaning towards solution 3). It's not great but at least we break 
> no minor upgrades (neither on 2.2, nor on 3.0) which is probably the most 
> important. We'd basically be just adding a new condition on 2.2->3.0 
> upgrades.  We could additionally make 3.0 node completely refuse v4 
> connections if they know a 2.2 nodes is in the cluster for extra safety.
> Ping [~omichallat], [~adutra] and [~aholmber] as you might want to be aware 
> of that ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of "ContributorsGroup" by BrandonWilliams

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "ContributorsGroup" page has been changed by BrandonWilliams:
https://wiki.apache.org/cassandra/ContributorsGroup?action=diff=52=53

   * JakeLuciani
   * JasonBrown
   * JeremiahJordan
+  * Jim Witschey
   * JoaquinCasares
   * LukasGutschmidt
   * LukasWingerberg

[jira] [Created] (CASSANDRA-10885) replication_test.py:ReplicationTest.simple_test dtest flapping on 3.0 no-vnode

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10885:


 Summary: replication_test.py:ReplicationTest.simple_test dtest 
flapping on 3.0 no-vnode
 Key: CASSANDRA-10885
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10885
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey


The test finds one or more replicas missing from the cluster sometimes:

http://cassci.datastax.com/job/cassandra-3.0_novnode_dtest/109/testReport/junit/replication_test/ReplicationTest/simple_test/history/

I have not reproduced it locally on Linux.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10886) test_read_invalid_text dtest fails

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10886:


 Summary: test_read_invalid_text dtest fails 
 Key: CASSANDRA-10886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10886
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Jim Witschey


{{cqlsh_tests/cqlsh_copy_tests.py:CqlshCopyTest.test_read_invalid_text}} seems 
to fail hard on Windows on trunk. These were only recently unskipped when 
CASSANDRA-9302 was closed, so I don't know if they fail on other branches, and 
I won't know until those jobs run.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10797) Bootstrap new node fails with OOM when streaming nodes contains thousands of sstables

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061158#comment-15061158
 ] 

Yuki Morishita commented on CASSANDRA-10797:


I think most of the heap pressure is coming from sstable metadata that is held 
until we finalize (write out to disc), so using SSTableReader will reduce heap 
usage.

I prefer to fix this for 3.x since it has new transactional system so that we 
don't need to deal with lock file by ourselves.


> Bootstrap new node fails with OOM when streaming nodes contains thousands of 
> sstables
> -
>
> Key: CASSANDRA-10797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10797
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Cassandra 2.1.8.621 w/G1GC
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
> Attachments: 10797-nonpatched.png, 10797-patched.png, 
> 10798-nonpatched-500M.png, 10798-patched-500M.png, 112415_system.log, 
> Heapdump_OOM.zip, Screen Shot 2015-12-01 at 7.34.40 PM.png, dtest.tar.gz
>
>
> When adding a new node to an existing DC, it runs OOM after 25-45 minutes
> Upon heapdump revision, it is found the sending nodes are streaming thousands 
> of sstables which in turns blows the bootstrapping node heap 
> {noformat}
> ERROR [RMI Scheduler(0)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [STREAM-IN-/173.36.28.148] 2015-11-24 10:10:44,585 
> StreamSession.java:502 - [Stream #0bb13f50-92cb-11e5-bc8d-f53b7528ffb4] 
> Streaming error occurred
> java.lang.IllegalStateException: Shutdown in progress
> at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) 
> ~[na:1.8.0_65]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239) 
> ~[na:1.8.0_65]
> at 
> org.apache.cassandra.service.StorageService.removeShutdownHook(StorageService.java:747)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector$Killer.killCurrentJVM(JVMStabilityInspector.java:95)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:64)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:66)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:55)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> ERROR [RMI TCP Connection(idle)] 2015-11-24 10:10:44,585 
> JVMStabilityInspector.java:94 - JVM state determined to be unstable.  Exiting 
> forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> ERROR [OptionalTasks:1] 2015-11-24 10:10:44,585 CassandraDaemon.java:223 - 
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Shutdown in progress
> {noformat}
> Attached is the Eclipse MAT report as a zipped web page



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10745) Deprecate PropertyFileSnitch

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061178#comment-15061178
 ] 

Paulo Motta commented on CASSANDRA-10745:
-

Thanks for the input [~sdurity]. We could probably make the 
{{PropertyFileSnitch}} use the {{GossipingPropertyFileSnitch}} internally, 
fetching only the local node's information from 
{{cassandra-topology.properties}} file while ignoring other node's info from 
this file (which will be propagated by gossip as in GPFS).

Alternatively we could still deprecate {{PropertyFileSnitch}} while making 
{{GossipingPropertyFileSnitch}} support the {{cassandra-topology.properties}} 
file to allow having a single global topology file.

> Deprecate PropertyFileSnitch
> 
>
> Key: CASSANDRA-10745
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10745
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Distributed Metadata
>Reporter: Paulo Motta
>Priority: Minor
>
> Opening this ticket to discuss deprecating PropertyFileSnitch, since it's 
> error-prone and more snitch code to maintain (See CASSANDRA-10243). Migration 
> from existing cluster with PropertyFileSnitch to GossipingPropertyFileSnitch 
> is straightforward.
> Is there any useful use case that can be achieved only with 
> PropertyFileSnitch?
> If not objections, we would add deprecation warnings in 2.2.x, 3.0.x, 3.2 and 
> deprecate in 3.4 or 3.6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10874) running stress with compaction strategy and replication factor fails on read after write

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061292#comment-15061292
 ] 

Paulo Motta commented on CASSANDRA-10874:
-

afaik the default stress consistency is ONE for writes and reads, so since 
you're writing with 300 threads, it's expected that some mutations will be 
dropped due to overload and some ONE reads will fail, since dropped mutations 
were not yet hinted (only after 10 minutes). that's why the problem doesn't 
work without replication or with read CL = quorum.

are repairs completing and do stress reads work after it? if so, I suspect 
those might only be reporting/presentation errors.

> running stress with compaction strategy and replication factor fails on read 
> after write
> 
>
> Key: CASSANDRA-10874
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10874
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Andrew Hust
>
> When running a read stress after write stress with a compaction strategy and 
> replication factor matching the node count will fail with an exception.  
> {code}
> Operation x0 on key(s) [38343433384b34364c30]: Data returned was not validated
> {code}
> Example run:
> {code}
> ccm create stress -v git:cassandra-3.0 -n 3 -s
> ccm node1 stress write n=10M -rate threads=300 -schema 
> replication\(factor=3\) 
> compaction\(strategy=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy\)
> ccm node1 nodetool flush
> ccm node1 nodetool compactionstats # check until quiet
> ccm node1 stress read n=10M -rate threads=300
> {code}
> - This will fail with/out vnodes but will occasionally pass without vnodes. 
> - Changing the read phase to be CL=QUORUM will make it pass.  
> - Removing the replication factor on write will make it pass.
> - Happens on all compaction strategies
> So with that in mind I attempted to add a repair after the write phase.  This 
> leads to 1 of 2 outcomes.
> 1: a repair that has a greater than 100% completion, usually stalls after a 
> bit, but have seen it get to >400% progress:
> {code}
>   id   compaction typekeyspace   
> table completed totalunit   progress
> 2d5344c0-9dc8-11e5-9d5f-4fdec8d76c27Validation   keyspace1   
> standard1   94722609949   44035292145   bytes215.11%
> {code}
> 2: a repair that has a greatly inflated completed/total value, it will crunch 
> for a bit then lockup:
> {code}
>  id   compaction typekeyspace   
> table   completed  totalunit   progress
>8c4cf7f0-a34a-11e5-a321-777be88c58aeValidation   keyspace1   
> standard1   0   874811100900   bytes  0.00%
> ❯ du -sh ~/.ccm/stress/node1/
> 2.4G  ~/.ccm/stress/node1/
> ❯ du -sh ~/.ccm/stress
> 7.1G  ~/.ccm/stress
> {code}
> This has been reproduced on cassandra-3.0 and cassandra-2.1 both locally and 
> using cstar_perf (links below).  
> A big twist is that cassandra-2.2 will pass the majority of the time.  It 
> will complete successfully without the repair 8 out of 10 runs.  This can be 
> seen in the cstar_perf links below.
> cstar_perf runs:
> http://cstar.datastax.com/tests/id/c8fa27a4-a205-11e5-8fbc-0256e416528f
> http://cstar.datastax.com/tests/id/a254c572-a2ce-11e5-a8b9-0256e416528f



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10887) Pending range calculator gives wrong pending ranges for moves

[Richard Low (JIRA)]

Richard Low created CASSANDRA-10887:
---

 Summary: Pending range calculator gives wrong pending ranges for 
moves
 Key: CASSANDRA-10887
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10887
 Project: Cassandra
  Issue Type: Bug
  Components: Coordination
Reporter: Richard Low
Priority: Critical


My understanding is the PendingRangeCalculator is meant to calculate who should 
receive extra writes during range movements. However, it adds the wrong ranges 
for moves. An extreme example of this can be seen in the following 
reproduction. Create a 5 node cluster (I did this on 2.0.16 and 2.2.4) and a 
keyspace RF=3 and a simple table. Then start moving a node and immediately kill 
-9 it. Now you see a node as down and moving in the ring. Try a quorum write 
for a partition that is stored on that node - it will fail with a timeout. 
Further, all CAS reads or writes fail immediately with unavailable exception 
because they attempt to include the moving node twice. This is likely to be the 
cause of CASSANDRA-10423.

In my example I had this ring:

127.0.0.1  rack1   Up Normal  170.97 KB   20.00%  
-9223372036854775808
127.0.0.2  rack1   Up Normal  124.06 KB   20.00%  
-5534023222112865485
127.0.0.3  rack1   Down   Moving  108.7 KB40.00%  
1844674407370955160
127.0.0.4  rack1   Up Normal  142.58 KB   0.00%   
1844674407370955161
127.0.0.5  rack1   Up Normal  118.64 KB   20.00%  
5534023222112865484

Node 3 was moving to -1844674407370955160. I added logging to print the pending 
and natural endpoints. For ranges owned by node 3, node 3 appeared in pending 
and natural endpoints. The blockFor is increased to 3 so we’re effectively 
doing CL.ALL operations. This manifests as write timeouts and CAS unavailables 
when the node is down.

The correct pending range for this scenario is node 1 is gaining the range 
(-1844674407370955160, 1844674407370955160). So node 1 should be added as a 
destination for writes and CAS for this range, not node 3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10875) cqlsh fails to decode utf-8 characters for text typed columns.

[Yasuharu Goto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yasuharu Goto updated CASSANDRA-10875:
--
Summary: cqlsh fails to decode utf-8 characters for text typed columns.  
(was: cqlsh decodes text column values as ascii in SELECT statements.)

> cqlsh fails to decode utf-8 characters for text typed columns.
> --
>
> Key: CASSANDRA-10875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10875
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Yasuharu Goto
>Assignee: Yasuharu Goto
>Priority: Minor
> Fix For: 2.1.13, 3.1
>
> Attachments: 10875-2.1.12.txt, 10875-3.1.txt
>
>
> Hi, we've found a bug that cqlsh can't handle unicode text in select 
> conditions even if it were text type.
> {noformat}
> $ ./bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.2-SNAPSHOT | CQL spec 3.3.1 | Native protocol v4]
> Use HELP for help.
> cqlsh> create KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test(txt text primary key);
> cqlsh> insert into test.test (txt) values('日本語');
> cqlsh> select * from test.test where txt='日本語';
> 'ascii' codec can't decode byte 0xe6 in position 35: ordinal not in range(128)
> cqlsh> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061241#comment-15061241
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

That explains it. I will get you the correct patch. Thanks for the explanation.

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061364#comment-15061364
 ] 

Yuki Morishita commented on CASSANDRA-10859:


||branch||testall||dtest||
|[10859|https://github.com/yukim/cassandra/tree/10859]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-10859-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-10859-dtest/lastCompletedBuild/testReport/]|

Patch to correctly create EstimatedHistograms in TableHistograms.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x, 3.x
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9258) Range movement causes CPU & performance impact

[Dikang Gu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dikang Gu updated CASSANDRA-9258:
-
Attachment: Screenshot 2015-12-16 16.11.51.png
Screenshot 2015-12-16 16.11.36.png

[~slebresne], I deploy this to our production environment, and see significant 
improvement of cpu when bootstrapping new node.

I attached the profiling for with and without this patch.

> Range movement causes CPU & performance impact
> --
>
> Key: CASSANDRA-9258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9258
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.4
>Reporter: Rick Branson
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: 0001-pending-ranges-map.patch, Screenshot 2015-12-16 
> 16.11.36.png, Screenshot 2015-12-16 16.11.51.png
>
>
> Observing big CPU & latency regressions when doing range movements on 
> clusters with many tens of thousands of vnodes. See CPU usage increase by 
> ~80% when a single node is being replaced.
> Top methods are:
> 1) Ljava/math/BigInteger;.compareTo in 
> Lorg/apache/cassandra/dht/ComparableObjectToken;.compareTo 
> 2) Lcom/google/common/collect/AbstractMapBasedMultimap;.wrapCollection in 
> Lcom/google/common/collect/AbstractMapBasedMultimap$AsMap$AsMapIterator;.next
> 3) Lorg/apache/cassandra/db/DecoratedKey;.compareTo in 
> Lorg/apache/cassandra/dht/Range;.contains
> Here's a sample stack from a thread dump:
> {code}
> "Thrift:50673" daemon prio=10 tid=0x7f2f20164800 nid=0x3a04af runnable 
> [0x7f2d878d]
>java.lang.Thread.State: RUNNABLE
>   at org.apache.cassandra.dht.Range.isWrapAround(Range.java:260)
>   at org.apache.cassandra.dht.Range.contains(Range.java:51)
>   at org.apache.cassandra.dht.Range.contains(Range.java:110)
>   at 
> org.apache.cassandra.locator.TokenMetadata.pendingEndpointsFor(TokenMetadata.java:916)
>   at 
> org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:775)
>   at 
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:541)
>   at 
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:616)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1101)
>   at 
> org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:1083)
>   at 
> org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:976)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
>   at 
> org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745){code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10877) Unable to read obsolete message version 1; The earliest version supported is 2.0.0

[esala wona (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061282#comment-15061282
 ] 

esala wona commented on CASSANDRA-10877:


I'm not unload  xt_TCPOPTSTRIP, but when I restart cassandra 
service, it go to normal.

> Unable to read obsolete message version 1; The earliest version supported is 
> 2.0.0
> --
>
> Key: CASSANDRA-10877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: esala wona
>
> I used cassandra version 2.1.2, but I get some error about that:
>  error message 
> ERROR [Thread-83674153] 2015-12-15 10:54:42,980 CassandraDaemon.java:153 - 
> Exception in thread Thread[Thread-83674153,5,main]
> java.lang.UnsupportedOperationException: Unable to read obsolete message 
> version 1; The earliest version supported is 2.0.0
> at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
>  ~[apache-cassandra-2.1.2.jar:2.1.2]
> = end 
> == nodetool information 
> ces@ICESSuse3631:/opt/ces/cassandra/bin> ./nodetool gossipinfo
> /192.168.0.1
> generation:1450148624
> heartbeat:299069
> SCHEMA:194168a3-f66e-3ab9-b811-4f7a7c3b89ca
> STATUS:NORMAL,-111061256928956495
> RELEASE_VERSION:2.1.2
> RACK:rack1
> DC:datacenter1
> RPC_ADDRESS:192.144.36.32
> HOST_ID:11f793f0-999b-4ba8-8bdd-0f0c73ae2e23
> NET_VERSION:8
> SEVERITY:0.0
> LOAD:1.3757700946E10
> /192.168.0.2
> generation:1450149068
> heartbeat:297714
> SCHEMA:194168a3-f66e-3ab9-b811-4f7a7c3b89ca
> RELEASE_VERSION:2.1.2
> STATUS:NORMAL,-1108435478195556849
> RACK:rack1
> DC:datacenter1
> RPC_ADDRESS:192.144.36.33
> HOST_ID:0f1a2dab-1d39-4419-bb68-03386c1a79df
> NET_VERSION:8
> SEVERITY:7.611548900604248
> LOAD:8.295301191E9
> end=



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10877) Unable to read obsolete message version 1; The earliest version supported is 2.0.0

[esala wona (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061282#comment-15061282
 ] 

esala wona edited comment on CASSANDRA-10877 at 12/17/15 1:28 AM:
--

I'm not unload  “xt_TCPOPTSTRIP”, but when I restart cassandra service, it go 
to normal.


was (Author: esala):
I'm not unload  xt_TCPOPTSTRIP, but when I restart cassandra 
service, it go to normal.

> Unable to read obsolete message version 1; The earliest version supported is 
> 2.0.0
> --
>
> Key: CASSANDRA-10877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: esala wona
>
> I used cassandra version 2.1.2, but I get some error about that:
>  error message 
> ERROR [Thread-83674153] 2015-12-15 10:54:42,980 CassandraDaemon.java:153 - 
> Exception in thread Thread[Thread-83674153,5,main]
> java.lang.UnsupportedOperationException: Unable to read obsolete message 
> version 1; The earliest version supported is 2.0.0
> at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
>  ~[apache-cassandra-2.1.2.jar:2.1.2]
> = end 
> == nodetool information 
> ces@ICESSuse3631:/opt/ces/cassandra/bin> ./nodetool gossipinfo
> /192.168.0.1
> generation:1450148624
> heartbeat:299069
> SCHEMA:194168a3-f66e-3ab9-b811-4f7a7c3b89ca
> STATUS:NORMAL,-111061256928956495
> RELEASE_VERSION:2.1.2
> RACK:rack1
> DC:datacenter1
> RPC_ADDRESS:192.144.36.32
> HOST_ID:11f793f0-999b-4ba8-8bdd-0f0c73ae2e23
> NET_VERSION:8
> SEVERITY:0.0
> LOAD:1.3757700946E10
> /192.168.0.2
> generation:1450149068
> heartbeat:297714
> SCHEMA:194168a3-f66e-3ab9-b811-4f7a7c3b89ca
> RELEASE_VERSION:2.1.2
> STATUS:NORMAL,-1108435478195556849
> RACK:rack1
> DC:datacenter1
> RPC_ADDRESS:192.144.36.33
> HOST_ID:0f1a2dab-1d39-4419-bb68-03386c1a79df
> NET_VERSION:8
> SEVERITY:7.611548900604248
> LOAD:8.295301191E9
> end=



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10423) Paxos/LWT failures when moving node

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061181#comment-15061181
 ] 

sankalp kohli commented on CASSANDRA-10423:
---

[~slebresne] Why will there be more contention when there are more 
participants? Contention depends upon if you have more clients which are 
updating the same CQL partitions. I don't see how this should be affected. 


> Paxos/LWT failures when moving node
> ---
>
> Key: CASSANDRA-10423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10423
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra version: 2.0.14
> Java-driver version: 2.0.11
>Reporter: Roger Schildmeijer
> Fix For: 2.1.x
>
>
> While moving a node (nodetool move ) we noticed that lwt started 
> failing for some (~50%) requests. The java-driver (version 2.0.11) returned 
> com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout 
> during write query at consistency SERIAL (7 replica were required but only 0 
> acknowledged the write). The cluster was not under heavy load.
> I noticed that the failed lwt requests all took just above 1s. That 
> information and the WriteTimeoutException could indicate that this happens:
> https://github.com/apache/cassandra/blob/cassandra-2.0.14/src/java/org/apache/cassandra/service/StorageProxy.java#L268
> I can't explain why though. Why would there be more cas contention just 
> because a node is moving?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061233#comment-15061233
 ] 

Paulo Motta commented on CASSANDRA-10580:
-

It didn't work. {{git clone 
http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-2.2}} clones the 
trunk branch by default (if you want to clone a specific branch you need to do 
{{git clone -b  
http://git-wip-us.apache.org/repos/asf/cassandra.git}}). You may use {{git 
}} to check which branch you are in and {{git checkout }} to 
switch between branches.

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10580:
-
Attachment: 2.2-All-Comments.patch

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anubhav Kale updated CASSANDRA-10580:
-
Attachment: (was: 2.2-All-Comments.patch)

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10580) Add latency metrics for dropped messages

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061285#comment-15061285
 ] 

Anubhav Kale commented on CASSANDRA-10580:
--

Attached 2.2 patch. Sorry about the goofup.

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10887) Pending range calculator gives wrong pending ranges for moves

[Simon Ashley (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061671#comment-15061671
 ] 

Simon Ashley commented on CASSANDRA-10887:
--

Ran a repro on 2.0.16 based on the description

(1)

create keyspace ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1':3}

(2)

CREATE TABLE ks1.users (
  login varchar,
  email varchar,
  name varchar,
  login_count int,
  PRIMARY KEY (login));

(3)

INSERT INTO ks1.users (login, email, name, login_count) VALUES ('jdoe', 
'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;

(4)

#Run a simple lwt in a loop

cat lwtloop.sh

#!/bin/bash

#loop $1 times 

for (( a=1; a<=$1; a++ ))
do
 cqlsh -e "UPDATE ks1.users SET email = 'jane...@abc.com' WHERE login = 'jdoe' 
IF email = 'j...@abc.com';"
 echo count: $a
done

./lwtloop.sh 1000

This outputs the following with a count

 [applied] | email
---+-
 False | jane...@abc.com

count: 304

(5) Initiate a nodetool move command

 nodetool -h 1.2.3.4 move -- \\-634023222112864484

(6) kill node running move against

sudo kill -9 

(7) lwtloop.sh script reports

:1:Request did not complete within rpc_timeout.
count: 305

followed by

:1:Unable to complete request: one or more nodes were unavailable.
count: 322

(8) Bring node back online

sudo cassandra

lwtloop.sh script now continues

[applied] | email
---+-
 False | jane...@abc.com

count: 587



> Pending range calculator gives wrong pending ranges for moves
> -
>
> Key: CASSANDRA-10887
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10887
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Richard Low
>Priority: Critical
>
> My understanding is the PendingRangeCalculator is meant to calculate who 
> should receive extra writes during range movements. However, it adds the 
> wrong ranges for moves. An extreme example of this can be seen in the 
> following reproduction. Create a 5 node cluster (I did this on 2.0.16 and 
> 2.2.4) and a keyspace RF=3 and a simple table. Then start moving a node and 
> immediately kill -9 it. Now you see a node as down and moving in the ring. 
> Try a quorum write for a partition that is stored on that node - it will fail 
> with a timeout. Further, all CAS reads or writes fail immediately with 
> unavailable exception because they attempt to include the moving node twice. 
> This is likely to be the cause of CASSANDRA-10423.
> In my example I had this ring:
> 127.0.0.1  rack1   Up Normal  170.97 KB   20.00%  
> -9223372036854775808
> 127.0.0.2  rack1   Up Normal  124.06 KB   20.00%  
> -5534023222112865485
> 127.0.0.3  rack1   Down   Moving  108.7 KB40.00%  
> 1844674407370955160
> 127.0.0.4  rack1   Up Normal  142.58 KB   0.00%   
> 1844674407370955161
> 127.0.0.5  rack1   Up Normal  118.64 KB   20.00%  
> 5534023222112865484
> Node 3 was moving to -1844674407370955160. I added logging to print the 
> pending and natural endpoints. For ranges owned by node 3, node 3 appeared in 
> pending and natural endpoints. The blockFor is increased to 3 so we’re 
> effectively doing CL.ALL operations. This manifests as write timeouts and CAS 
> unavailables when the node is down.
> The correct pending range for this scenario is node 1 is gaining the range 
> (-1844674407370955160, 1844674407370955160). So node 1 should be added as a 
> destination for writes and CAS for this range, not node 3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/57d558fc/pylib/cqlshlib/copyutil.py
--
diff --cc pylib/cqlshlib/copyutil.py
index a2fab00,f699e64..a117ec3
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@@ -23,19 -26,25 +26,25 @@@ import sy
  import time
  import traceback
  
- from StringIO import StringIO
+ from calendar import timegm
+ from collections import defaultdict, deque, namedtuple
+ from decimal import Decimal
  from random import randrange
+ from StringIO import StringIO
  from threading import Lock
+ from uuid import UUID
  
  from cassandra.cluster import Cluster
+ from cassandra.cqltypes import ReversedType, UserType
  from cassandra.metadata import protect_name, protect_names
- from cassandra.policies import RetryPolicy, WhiteListRoundRobinPolicy, 
TokenAwarePolicy
- from cassandra.query import tuple_factory
+ from cassandra.policies import RetryPolicy, WhiteListRoundRobinPolicy, 
TokenAwarePolicy, DCAwareRoundRobinPolicy
+ from cassandra.query import BatchStatement, BatchType, SimpleStatement, 
tuple_factory
+ from cassandra.util import Date, Time
  
- 
- import sslhandling
+ from cql3handling import CqlRuleSet
  from displaying import NO_COLOR_MAP
 -from formatting import format_value_default, EMPTY, get_formatter
 +from formatting import format_value_default, DateTimeFormat, EMPTY, 
get_formatter
+ from sslhandling import ssl_settings
  
  
  def parse_options(shell, opts):
@@@ -65,10 -74,13 +74,15 @@@
  # by default the page timeout is 10 seconds per 1000 entries in the page 
size or 10 seconds if pagesize is smaller
  csv_options['pagetimeout'] = int(opts.pop('pagetimeout', max(10, 10 * 
(csv_options['pagesize'] / 1000
  csv_options['maxattempts'] = int(opts.pop('maxattempts', 5))
- csv_options['float_precision'] = shell.display_float_precision
 -csv_options['dtformats'] = opts.pop('timeformat', 
shell.display_time_format)
 +csv_options['dtformats'] = DateTimeFormat(opts.pop('timeformat', 
shell.display_timestamp_format),
 +  shell.display_date_format,
 +  shell.display_nanotime_format)
+ csv_options['float_precision'] = shell.display_float_precision
+ csv_options['chunksize'] = int(opts.pop('chunksize', 1000))
+ csv_options['ingestrate'] = int(opts.pop('ingestrate', 10))
+ csv_options['maxbatchsize'] = int(opts.pop('maxbatchsize', 20))
+ csv_options['minbatchsize'] = int(opts.pop('minbatchsize', 2))
+ csv_options['reportfrequency'] = float(opts.pop('reportfrequency', 0.25))
  
  return csv_options, dialect_options, opts
  
@@@ -371,30 -648,18 +650,18 @@@ class ExportProcess(ChildProcess)
  An child worker process for the export task, ExportTask.
  """
  
- def __init__(self, inmsg, outmsg, ks, cf, columns, dialect_options, 
csv_options,
-  debug, port, cql_version, auth_provider, ssl, 
protocol_version, config_file):
- mp.Process.__init__(self, target=self.run)
- self.inmsg = inmsg
- self.outmsg = outmsg
- self.ks = ks
- self.cf = cf
- self.columns = columns
- self.dialect_options = dialect_options
+ def __init__(self, params):
+ ChildProcess.__init__(self, params=params, target=self.run)
+ self.dialect_options = params['dialect_options']
  self.hosts_to_sessions = dict()
  
- self.debug = debug
- self.port = port
- self.cql_version = cql_version
- self.auth_provider = auth_provider
- self.ssl = ssl
- self.protocol_version = protocol_version
- self.config_file = config_file
- 
+ csv_options = params['csv_options']
  self.encoding = csv_options['encoding']
 -self.time_format = csv_options['dtformats']
 +self.date_time_format = csv_options['dtformats']
  self.float_precision = csv_options['float_precision']
  self.nullval = csv_options['nullval']
- self.maxjobs = csv_options['jobs']
+ self.max_attempts = csv_options['maxattempts']
+ self.max_requests = csv_options['maxrequests']
  self.csv_options = csv_options
  self.formatters = dict()
  
@@@ -600,13 -851,424 +853,424 @@@
  return query
  
  
+ class ImportConversion(object):
+ """
+ A class for converting strings to values when importing from csv, used by 
ImportProcess,
+ the parent.
+ """
+ def __init__(self, parent, table_meta, statement):
+ self.ks = parent.ks
+ self.cf = parent.cf
+ self.columns = parent.columns
+ self.nullval = parent.nullval
+ self.printmsg = parent.printmsg
+ self.table_meta = table_meta
+ self.primary_key_indexes = [self.columns.index(col.name) for col in 
self.table_meta.primary_key]
+ self.partition_key_indexes = 

[3/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/57d558fc/bin/cqlsh.py
--
diff --cc bin/cqlsh.py
index a3fa666,000..42c2923
mode 100644,00..100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@@ -1,2763 -1,0 +1,2486 @@@
 +#!/bin/sh
 +# -*- mode: Python -*-
 +
 +# Licensed to the Apache Software Foundation (ASF) under one
 +# or more contributor license agreements.  See the NOTICE file
 +# distributed with this work for additional information
 +# regarding copyright ownership.  The ASF licenses this file
 +# to you under the Apache License, Version 2.0 (the
 +# "License"); you may not use this file except in compliance
 +# with the License.  You may obtain a copy of the License at
 +#
 +# http://www.apache.org/licenses/LICENSE-2.0
 +#
 +# Unless required by applicable law or agreed to in writing, software
 +# distributed under the License is distributed on an "AS IS" BASIS,
 +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 +# See the License for the specific language governing permissions and
 +# limitations under the License.
 +
 +""":"
 +# bash code here; finds a suitable python interpreter and execs this file.
 +# prefer unqualified "python" if suitable:
 +python -c 'import sys; sys.exit(not (0x020500b0 < sys.hexversion < 
0x0300))' 2>/dev/null \
 +&& exec python "$0" "$@"
 +for pyver in 2.6 2.7 2.5; do
 +which python$pyver > /dev/null 2>&1 && exec python$pyver "$0" "$@"
 +done
 +echo "No appropriate python interpreter found." >&2
 +exit 1
 +":"""
 +
 +from __future__ import with_statement
 +
 +import cmd
 +import codecs
 +import ConfigParser
 +import csv
 +import getpass
 +import locale
- import multiprocessing as mp
 +import optparse
 +import os
 +import platform
 +import sys
 +import time
 +import traceback
 +import warnings
 +import webbrowser
++from StringIO import StringIO
 +from contextlib import contextmanager
- from functools import partial
 +from glob import glob
- from StringIO import StringIO
 +from uuid import UUID
 +
 +if sys.version_info[0] != 2 or sys.version_info[1] != 7:
 +sys.exit("\nCQL Shell supports only Python 2.7\n")
 +
 +description = "CQL Shell for Apache Cassandra"
 +version = "5.0.1"
 +
 +readline = None
 +try:
 +# check if tty first, cause readline doesn't check, and only cares
 +# about $TERM. we don't want the funky escape code stuff to be
 +# output if not a tty.
 +if sys.stdin.isatty():
 +import readline
 +except ImportError:
 +pass
 +
 +CQL_LIB_PREFIX = 'cassandra-driver-internal-only-'
 +
 +CASSANDRA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), 
'..')
 +CASSANDRA_CQL_HTML_FALLBACK = 
'https://cassandra.apache.org/doc/cql3/CQL-2.2.html'
 +
 +if os.path.exists(CASSANDRA_PATH + '/doc/cql3/CQL.html'):
 +# default location of local CQL.html
 +CASSANDRA_CQL_HTML = 'file://' + CASSANDRA_PATH + '/doc/cql3/CQL.html'
 +elif os.path.exists('/usr/share/doc/cassandra/CQL.html'):
 +# fallback to package file
 +CASSANDRA_CQL_HTML = 'file:///usr/share/doc/cassandra/CQL.html'
 +else:
 +# fallback to online version
 +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK
 +
 +# On Linux, the Python webbrowser module uses the 'xdg-open' executable
 +# to open a file/URL. But that only works, if the current session has been
 +# opened from _within_ a desktop environment. I.e. 'xdg-open' will fail,
 +# if the session's been opened via ssh to a remote box.
 +#
 +# Use 'python' to get some information about the detected browsers.
 +# >>> import webbrowser
 +# >>> webbrowser._tryorder
 +# >>> webbrowser._browser
 +#
 +if len(webbrowser._tryorder) == 0:
 +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK
 +elif webbrowser._tryorder[0] == 'xdg-open' and 
os.environ.get('XDG_DATA_DIRS', '') == '':
 +# only on Linux (some OS with xdg-open)
 +webbrowser._tryorder.remove('xdg-open')
 +webbrowser._tryorder.append('xdg-open')
 +
 +# use bundled libs for python-cql and thrift, if available. if there
 +# is a ../lib dir, use bundled libs there preferentially.
 +ZIPLIB_DIRS = [os.path.join(CASSANDRA_PATH, 'lib')]
 +myplatform = platform.system()
 +if myplatform == 'Linux':
 +ZIPLIB_DIRS.append('/usr/share/cassandra/lib')
 +
 +if os.environ.get('CQLSH_NO_BUNDLED', ''):
 +ZIPLIB_DIRS = ()
 +
 +
 +def find_zip(libprefix):
 +for ziplibdir in ZIPLIB_DIRS:
 +zips = glob(os.path.join(ziplibdir, libprefix + '*.zip'))
 +if zips:
 +return max(zips)   # probably the highest version, if multiple
 +
 +cql_zip = find_zip(CQL_LIB_PREFIX)
 +if cql_zip:
 +ver = os.path.splitext(os.path.basename(cql_zip))[0][len(CQL_LIB_PREFIX):]
 +sys.path.insert(0, os.path.join(cql_zip, 'cassandra-driver-' + ver))
 +
 +third_parties = ('futures-', 'six-')
 +
 +for lib in third_parties:
 +lib_zip = find_zip(lib)
 +if lib_zip:
 +sys.path.insert(0, lib_zip)
 +
 

[1/4] cassandra git commit: (cqlsh) further optimise COPY FROM

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 de55c39cd -> 57d558fc1


(cqlsh) further optimise COPY FROM

patch by Stefania Alborghetti; reviewed by Adam Holmberg for
CASSANDRA-9302


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/124f1bd2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/124f1bd2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/124f1bd2

Branch: refs/heads/cassandra-2.2
Commit: 124f1bd2613e400f69f8369ada0ad15c28738530
Parents: 994250c
Author: Stefania Alborghetti 
Authored: Thu Oct 22 17:16:50 2015 +0800
Committer: Aleksey Yeschenko 
Committed: Tue Dec 15 21:03:31 2015 +

--
 CHANGES.txt|   4 +-
 bin/cqlsh  | 285 ++---
 pylib/cqlshlib/copyutil.py | 910 ++--
 pylib/cqlshlib/util.py |  19 +
 4 files changed, 838 insertions(+), 380 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/124f1bd2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8e58703..90f1bca 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,12 +1,10 @@
 2.1.13
-<<< HEAD
+ * (cqlsh) further optimise COPY FROM (CASSANDRA-9302)
  * Allow CREATE TABLE WITH ID (CASSANDRA-9179)
  * Make Stress compiles within eclipse (CASSANDRA-10807)
  * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
  * Allow cancellation of index summary redistribution (CASSANDRA-8805)
-===
  * sstableloader will fail if there are collections in the schema tables 
(CASSANDRA-10700)
->>> 5377183... stableloader will fail if there are collections in the 
schema tables
  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
  * Fix Stress profile parsing on Windows (CASSANDRA-10808)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/124f1bd2/bin/cqlsh
--
diff --git a/bin/cqlsh b/bin/cqlsh
index e72624a..651420d 100755
--- a/bin/cqlsh
+++ b/bin/cqlsh
@@ -37,7 +37,6 @@ import ConfigParser
 import csv
 import getpass
 import locale
-import multiprocessing as mp
 import optparse
 import os
 import platform
@@ -48,7 +47,6 @@ import warnings
 
 from StringIO import StringIO
 from contextlib import contextmanager
-from functools import partial
 from glob import glob
 from uuid import UUID
 
@@ -110,10 +108,10 @@ except ImportError, e:
 
 from cassandra.auth import PlainTextAuthProvider
 from cassandra.cluster import Cluster, PagedResult
-from cassandra.metadata import protect_name, protect_names, protect_value
+from cassandra.metadata import protect_name, protect_names
 from cassandra.policies import WhiteListRoundRobinPolicy
-from cassandra.protocol import QueryMessage, ResultMessage
-from cassandra.query import SimpleStatement, ordered_dict_factory
+from cassandra.protocol import ResultMessage
+from cassandra.query import SimpleStatement, ordered_dict_factory, 
tuple_factory
 
 # cqlsh should run correctly when run out of a Cassandra source tree,
 # out of an unpacked Cassandra tarball, and after a proper package install.
@@ -334,7 +332,7 @@ cqlsh_extra_syntax_rules = r'''
 
  ::= 
   | 
-  | 
+  | 
   ;
 
 # avoiding just "DEBUG" so that this rule doesn't get treated as a terminal
@@ -412,17 +410,20 @@ def complete_copy_column_names(ctxt, cqlsh):
 return set(colnames[1:]) - set(existcols)
 
 
-COPY_OPTIONS = ['DELIMITER', 'QUOTE', 'ESCAPE', 'HEADER', 'NULL', 'ENCODING',
-'TIMEFORMAT', 'JOBS', 'PAGESIZE', 'PAGETIMEOUT', 'MAXATTEMPTS']
+COPY_COMMON_OPTIONS = ['DELIMITER', 'QUOTE', 'ESCAPE', 'HEADER', 'NULL',
+   'MAXATTEMPTS', 'REPORTFREQUENCY']
+COPY_FROM_OPTIONS = ['CHUNKSIZE', 'INGESTRATE', 'MAXBATCHSIZE', 'MINBATCHSIZE']
+COPY_TO_OPTIONS = ['ENCODING', 'TIMEFORMAT', 'PAGESIZE', 'PAGETIMEOUT', 
'MAXREQUESTS']
 
 
 @cqlsh_syntax_completer('copyOption', 'optnames')
 def complete_copy_options(ctxt, cqlsh):
 optnames = map(str.upper, ctxt.get_binding('optnames', ()))
 direction = ctxt.get_binding('dir').upper()
-opts = set(COPY_OPTIONS) - set(optnames)
 if direction == 'FROM':
-opts -= set(['ENCODING', 'TIMEFORMAT', 'JOBS', 'PAGESIZE', 
'PAGETIMEOUT', 'MAXATTEMPTS'])
+opts = set(COPY_COMMON_OPTIONS + COPY_FROM_OPTIONS) - set(optnames)
+elif direction == 'TO':
+opts = set(COPY_COMMON_OPTIONS + COPY_TO_OPTIONS) - set(optnames)
 return opts
 
 
@@ -1520,12 +1521,18 @@ class Shell(cmd.Cmd):
   ESCAPE='\'  - character to appear before the QUOTE char 
when quoted
   HEADER=false- whether to 

[4/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57d558fc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57d558fc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57d558fc

Branch: refs/heads/cassandra-2.2
Commit: 57d558fc1ef8117c41567541b967e2b782d04a50
Parents: de55c39 124f1bd
Author: Aleksey Yeschenko 
Authored: Tue Dec 15 21:37:09 2015 +
Committer: Aleksey Yeschenko 
Committed: Tue Dec 15 21:37:09 2015 +

--
 CHANGES.txt|   4 +
 bin/cqlsh.py   | 329 ++-
 pylib/cqlshlib/copyutil.py | 912 ++--
 pylib/cqlshlib/util.py |  19 +
 4 files changed, 843 insertions(+), 421 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/57d558fc/CHANGES.txt
--
diff --cc CHANGES.txt
index c9074fc,90f1bca..c969a4d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,15 +1,40 @@@
 -2.1.13
 +2.2.5
 + * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 + * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 + * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 + * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 + * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
 +Merged from 2.1:
+  * (cqlsh) further optimise COPY FROM (CASSANDRA-9302)
   * Allow CREATE TABLE WITH ID (CASSANDRA-9179)
   * Make Stress compiles within eclipse (CASSANDRA-10807)
   * Cassandra Daemon should print JVM arguments (CASSANDRA-10764)
   * Allow cancellation of index summary redistribution (CASSANDRA-8805)
+  * sstableloader will fail if there are collections in the schema tables 
(CASSANDRA-10700)
+  * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
   * Fix Stress profile parsing on Windows (CASSANDRA-10808)
  
+ 
 -2.1.12
 +2.2.4
 + * Show CQL help in cqlsh in web browser (CASSANDRA-7225)
 + * Serialize on disk the proper SSTable compression ratio (CASSANDRA-10775)
 + * Reject index queries while the index is building (CASSANDRA-8505)
 + * CQL.textile syntax incorrectly includes optional keyspace for aggregate 
SFUNC and FINALFUNC (CASSANDRA-10747)
 + * Fix JSON update with prepared statements (CASSANDRA-10631)
 + * Don't do anticompaction after subrange repair (CASSANDRA-10422)
 + * Fix SimpleDateType type compatibility (CASSANDRA-10027)
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 + * Fix IllegalArgumentException in DataOutputBuffer.reallocate for large 
buffers (CASSANDRA-10592)
 +Merged from 2.1:
   * Fix incremental repair hang when replica is down (CASSANDRA-10288)
   * Avoid writing range tombstones after END_OF_ROW marker (CASSANDRA-10791)
   * Optimize the way we check if a token is repaired in anticompaction 
(CASSANDRA-10768)

[jira] [Created] (CASSANDRA-10881) Unable to create a materialized view for retrieving the data from two different tables

[Sandeep Gopalasetty (JIRA)]

Sandeep Gopalasetty created CASSANDRA-10881:
---

 Summary: Unable to create a materialized view for retrieving the 
data from two different tables
 Key: CASSANDRA-10881
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10881
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: Windows
Reporter: Sandeep Gopalasetty
Priority: Critical
 Fix For: 3.0.0


Unable to create a materialized views for retrieving the data from two 
different tables. Syntactical problems are been arising. Is there any specific 
syntax?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9494) Need to set TTL with COPY command

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059885#comment-15059885
 ] 

Stefania commented on CASSANDRA-9494:
-

Rebased on the main branches now that CASSANDRA-9302 has been committed. 

CI pending.

> Need to set TTL with COPY command
> -
>
> Key: CASSANDRA-9494
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9494
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Ed Chen
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
>
> I can import a chunk of data into Cassandra table with COPY command like:
> COPY my_table (name, address) FROM my_file.csv WITH option='value' ... ;
> But I am not able to specify a finite TTL in COPY command with "USING TTL 
> 3600", for example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.1.1-tentative [created] 8347d3771

[jira] [Updated] (CASSANDRA-9891) AggregationTest.testAggregateWithWithWriteTimeOrTTL is fragile

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9891:
--
Component/s: CQL

> AggregationTest.testAggregateWithWithWriteTimeOrTTL is fragile
> --
>
> Key: CASSANDRA-9891
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9891
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.1
>
> Attachments: 9891.txt
>
>
> I've seen {{AggregationTest.testAggregateWithWithWriteTimeOrTTL}} fail on 
> cassci on the line
> {noformat}
> assertTrue(row.getInt("ttl(b)") > 4
> {noformat}
> Given that the ttl is set to 5 a few lines above and the CQL {{ttl()}} method 
> returns the actual time-to-live (that is, it decrease with time), it feels 
> safe to assume this test is overly fragile.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9426) Provide a per-table text->blob map for storing extra metadata

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9426:
--
Component/s: Distributed Metadata
 CQL

> Provide a per-table text->blob map for storing extra metadata
> -
>
> Key: CASSANDRA-9426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9426
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Benjamin Lerer
>  Labels: client-impacting
> Fix For: 3.0 beta 1
>
> Attachments: 9426-v2.txt, 9426.txt
>
>
> For some applications that build on Cassandra it's important to be able to 
> attach extra metadata to tables, and have it be distributed via regular 
> Cassandra schema paths.
> I propose a new {{extensions map}} table param for just that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6237) Allow range deletions in CQL

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-6237:
--
Component/s: CQL

> Allow range deletions in CQL
> 
>
> Key: CASSANDRA-6237
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: cql, docs
> Fix For: 3.0 beta 2
>
> Attachments: CASSANDRA-6237.txt
>
>
> We uses RangeTombstones internally in a number of places, but we could expose 
> more directly too. Typically, given a table like:
> {noformat}
> CREATE TABLE events (
> id text,
> created_at timestamp,
> content text,
> PRIMARY KEY (id, created_at)
> )
> {noformat}
> we could allow queries like:
> {noformat}
> DELETE FROM events WHERE id='someEvent' AND created_at < 'Jan 3, 2013';
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10580:

Summary: Add latency metrics for dropped messages  (was: On dropped 
mutations, more details should be logged.)

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10836) Data integrity flappers in upgrade_tests dtests

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10836:
-
Issue Type: Sub-task  (was: Bug)
Parent: CASSANDRA-10664

> Data integrity flappers in upgrade_tests dtests
> ---
>
> Key: CASSANDRA-10836
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10836
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> A number of the {{upgrade_tests}} in dtest are flapping with data integrity 
> bugs. For instance:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/422/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/conditional_delete_test/history/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/421/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/cas_and_list_index_test/history/
> This needs further digging to get good debug information; it could well be, 
> e.g., a race condition in the test. I have not reproduced locally.
> /cc [~philipthompson]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10372) Adds smallint and tinyint to the CQL documentation

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10372:
---
Component/s: Documentation and Website

> Adds smallint and tinyint to the CQL documentation
> --
>
> Key: CASSANDRA-10372
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10372
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation and Website
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.x
>
> Attachments: 10372-2.2.txt
>
>
> CASSANDRA-8951 added {{smallint}} and {{tinyint}} types but the CQL 
> documentation was not updated. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10561) Add a check to cqlsh to require Python-2.7 for version >= 2.2

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10561:
---
Component/s: CQL

> Add a check to cqlsh to require Python-2.7 for version >= 2.2
> -
>
> Key: CASSANDRA-10561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10561
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.4, 3.0.0
>
> Attachments: 10561-2.2-V2.txt, 10561-2.2.txt
>
>
> Not everybody is reading the {{README}} file. Up to now cqlsh was still 
> working with python 2.6 by consequence user complains when we use some 
> python-2.7 features.
> We should still allow python-2.6 with {{2.1}} but enforce the use of python 
> 2.7 in {{2.2+}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch cassandra-3.0 into trunk

[blerer]

Merge branch cassandra-3.0 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/abf7df3b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/abf7df3b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/abf7df3b

Branch: refs/heads/trunk
Commit: abf7df3bd609f999fdcc2dcfb91688be50c46664
Parents: 80250aa 363e7bd
Author: Benjamin Lerer 
Authored: Wed Dec 16 15:20:53 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Dec 16 15:21:07 2015 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/stress/StressProfile.java  |  8 
 .../stress/generate/values/LocalDates.java  | 43 +++
 .../stress/generate/values/SmallInts.java   | 38 +
 .../cassandra/stress/generate/values/Times.java | 37 
 .../stress/generate/values/TinyInts.java| 45 
 .../operations/userdefined/SchemaStatement.java | 16 +--
 .../cassandra/stress/util/JavaDriverClient.java |  1 +
 8 files changed, 185 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/abf7df3b/CHANGES.txt
--
diff --cc CHANGES.txt
index 062ab10,10504c7..24b2662
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,26 -1,11 +1,27 @@@
 -3.0.3
 +3.2
 + * Establish bootstrap stream sessions sequentially (CASSANDRA-6992)
 + * Sort compactionhistory output by timestamp (CASSANDRA-10464)
 + * More efficient BTree removal (CASSANDRA-9991)
 + * Make tablehistograms accept the same syntax as tablestats (CASSANDRA-10149)
 + * Group pending compactions based on table (CASSANDRA-10718)
 + * Add compressor name in sstablemetadata output (CASSANDRA-9879)
 + * Fix type casting for counter columns (CASSANDRA-10824)
 + * Prevent running Cassandra as root (CASSANDRA-8142)
 + * bound maximum in-flight commit log replay mutation bytes to 64 megabytes 
(CASSANDRA-8639)
 + * Normalize all scripts (CASSANDRA-10679)
 + * Make compression ratio much more accurate (CASSANDRA-10225)
 + * Optimize building of Clustering object when only one is created 
(CASSANDRA-10409)
 + * Make index building pluggable (CASSANDRA-10681)
 + * Add sstable flush observer (CASSANDRA-10678)
 + * Improve NTS endpoints calculation (CASSANDRA-10200)
 + * Improve performance of the folderSize function (CASSANDRA-10677)
 + * Add support for type casting in selection clause (CASSANDRA-10310)
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
  Merged from 2.2:
+  * Add new types to Stress (CASSANDRA-9556)
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)
 - * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
 - * Better handling of SSL connection errors inter-node (CASSANDRA-10816)
 - * Disable reloading of GossipingPropertyFileSnitch (CASSANDRA-9474)
 - * Verify tables in pseudo-system keyspaces at startup (CASSANDRA-10761)
  Merged from 2.1:
   * (cqlsh) further optimise COPY FROM (CASSANDRA-9302)
   * Allow CREATE TABLE WITH ID (CASSANDRA-9179)

[1/3] cassandra git commit: Add new types to Stress

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 80250aa01 -> abf7df3bd


Add new types to Stress

patch by ZhaoYang; reviewed by Benjamin Lerer for CASSANDRA-9556


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b03ce9fd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b03ce9fd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b03ce9fd

Branch: refs/heads/trunk
Commit: b03ce9fd479d3da9c51d118c50480ea96df0d73e
Parents: 57d558f
Author: ZhaoYang 
Authored: Wed Dec 16 15:06:06 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Dec 16 15:06:06 2015 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/stress/StressProfile.java  |  8 
 .../stress/generate/values/LocalDates.java  | 43 +++
 .../stress/generate/values/SmallInts.java   | 38 +
 .../cassandra/stress/generate/values/Times.java | 37 
 .../stress/generate/values/TinyInts.java| 45 
 .../operations/userdefined/SchemaStatement.java | 16 +--
 .../cassandra/stress/util/JavaDriverClient.java |  2 +-
 8 files changed, 185 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c969a4d..1480960 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * Add new types to Stress (CASSANDRA-9556)
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)
  * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
  * Better handling of SSL connection errors inter-node (CASSANDRA-10816)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 9c9b921..410f666 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -531,6 +531,14 @@ public class StressProfile implements Serializable
 return new UUIDs(name, config);
 case TIMEUUID:
 return new TimeUUIDs(name, config);
+case TINYINT:
+return new TinyInts(name, config);
+case SMALLINT:
+return new SmallInts(name, config);
+case TIME:
+return new Times(name, config);
+case DATE:
+return new LocalDates(name, config);
 case SET:
 return new Sets(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 case LIST:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
new file mode 100644
index 000..f079d35
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
@@ -0,0 +1,43 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import com.datastax.driver.core.LocalDate;
+import org.apache.cassandra.db.marshal.SimpleDateType;
+import org.joda.time.DateTimeZone;
+import org.joda.time.format.DateTimeFormat;
+import org.joda.time.format.DateTimeFormatter;
+
+public class LocalDates extends Generator
+{
+
+public LocalDates(String name, GeneratorConfig config)
+{
+   

[2/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/363e7bd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/363e7bd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/363e7bd7

Branch: refs/heads/trunk
Commit: 363e7bd716b1c0b798f740b88bcb336c41f33470
Parents: 98178f5 b03ce9f
Author: Benjamin Lerer 
Authored: Wed Dec 16 15:19:20 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Dec 16 15:19:20 2015 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/stress/StressProfile.java  |  8 
 .../stress/generate/values/LocalDates.java  | 43 +++
 .../stress/generate/values/SmallInts.java   | 38 +
 .../cassandra/stress/generate/values/Times.java | 37 
 .../stress/generate/values/TinyInts.java| 45 
 .../operations/userdefined/SchemaStatement.java | 16 +--
 .../cassandra/stress/util/JavaDriverClient.java |  1 +
 8 files changed, 185 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/CHANGES.txt
--
diff --cc CHANGES.txt
index dd1896c,1480960..10504c7
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
 -2.2.5
 +3.0.3
 +Merged from 2.2:
+  * Add new types to Stress (CASSANDRA-9556)
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)
   * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
   * Better handling of SSL connection errors inter-node (CASSANDRA-10816)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
--
diff --cc 
tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
index d0779ed,30d0d4a..7d5f38c
--- a/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
@@@ -114,7 -95,7 +114,8 @@@ public class JavaDriverClien
  .addContactPoint(host)
  .withPort(port)
  
.withPoolingOptions(poolingOpts)
 +.withoutJMXReporting()
+ 
.withProtocolVersion(ProtocolVersion.NEWEST_SUPPORTED)
  .withoutMetrics(); // The 
driver uses metrics 3 with conflict with our version
  if (whitelist != null)
  clusterBuilder.withLoadBalancingPolicy(whitelist);

[jira] [Updated] (CASSANDRA-9606) this query is not supported in new version

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9606:
--
Component/s: CQL

> this query is not supported in new version
> --
>
> Key: CASSANDRA-9606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9606
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: cassandra 2.1.6
> jdk 1.7.0_55
>Reporter: zhaoyan
>Assignee: Benjamin Lerer
> Attachments: 9606-2.0.txt, 9606-2.1.txt, 9606-2.2.txt
>
>
> Background:
> 1、create a table:
> {code}
> CREATE TABLE test (
> a int,
> b int,
> c int,
>   d int,
> PRIMARY KEY (a, b, c)
> );
> {code}
> 2、query by a=1 and b<6
> {code}
> select * from test where a=1 and b<6;
>  a | b | c | d
> ---+---+---+---
>  1 | 3 | 1 | 2
>  1 | 3 | 2 | 2
>  1 | 3 | 4 | 2
>  1 | 3 | 5 | 2
>  1 | 4 | 4 | 2
>  1 | 5 | 5 | 2
> (6 rows)
> {code}
> 3、query by page
> first page：
> {code}
> select * from test where a=1 and b<6 limit 2;
>  a | b | c | d
> ---+---+---+---
>  1 | 3 | 1 | 2
>  1 | 3 | 2 | 2
> (2 rows)
> {code}
> second page：
> {code}
> select * from test where a=1 and b<6 and (b,c) > (3,2) limit 2;
>  a | b | c | d
> ---+---+---+---
>  1 | 3 | 4 | 2
>  1 | 3 | 5 | 2
> (2 rows)
> {code}
> last page：
> {code}
> select * from test where a=1 and b<6 and (b,c) > (3,5) limit 2;
>  a | b | c | d
> ---+---+---+---
>  1 | 4 | 4 | 2
>  1 | 5 | 5 | 2
> (2 rows)
> {code}
> question:
> this query by page is ok when cassandra 2.0.8.
> but is not supported in the latest version 2.1.6
> when execute：
> {code}
> select * from test where a=1 and b<6 and (b,c) > (3,2) limit 2;
> {code}
> get one error message：
> InvalidRequest: code=2200 [Invalid query] message="Column "b" cannot have 
> both tuple-notation inequalities and single-column inequalities: (b, c) > (3, 
> 2)"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10114) Allow count(*) and count(1) to be use as normal aggregation

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10114:
---
Component/s: CQL

> Allow count(*) and count(1) to be use as normal aggregation
> ---
>
> Key: CASSANDRA-10114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10114
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.1, 3.0 beta 1
>
> Attachments: 10114-2.2.txt
>
>
> For the following query: 
> {code}
> SELECT count(*), max(timestamp), min(timestamp) FROM myData WHERE id = ?
> {code}
> Cassandra will throw a {{InvalidSyntaxException}}.
> We should allow {{count(\*)}} and {{count(1)}} to be queried with other 
> aggregations or columns



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10127) Make naming for secondary indexes consistent

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10127:
---
Component/s: Tools
 CQL

> Make naming for secondary indexes consistent
> 
>
> Key: CASSANDRA-10127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10127
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Tools
>Reporter: Sam Tunnicliffe
>Assignee: Benjamin Lerer
> Fix For: 3.0.0 rc1
>
> Attachments: 10127.txt
>
>
> We have a longstanding mismatch between the name of an index as defined in 
> schema and what gets returned from {{SecondaryIndex#getIndexName()}}, which 
> for the builtin index impls is the name of the underlying index CFS, of the 
> form {{.}}.
> This mismatch causes a number of UI inconsistencies:
> {code}nodetool rebuild_index   {code}
> {{}} must be qualified, i.e. include the redundant table name as without 
> it, the rebuild silently fails
> {{system.IndexInfo}} (which is also exposed over JMX) uses the form 
> {{.}}
> {code}cqlsh> describe index [.]{code}
> here, qualifying {{}} with the base table name is an error.
> Generally, anything CQL related uses the index name directly, whereas anthing 
> concerned with building or rebuiling requires the version based on an 
> underlying backing table name. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10743) Failed upgradesstables (upgrade from 2.2.2 to 3.0.0)

[Tomas Ramanauskas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomas Ramanauskas updated CASSANDRA-10743:
--
Attachment: schema.ddl

> Failed upgradesstables (upgrade from 2.2.2 to 3.0.0)
> 
>
> Key: CASSANDRA-10743
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10743
> Project: Cassandra
>  Issue Type: Bug
> Environment: CentOS Linux release 7.1.1503, OpenJDK Runtime 
> Environment (build 1.8.0_65-b17), DSC Cassandra 3.0.0 (tar.gz)
>Reporter: Gábor Auth
> Attachments: faulty-tables.tar.gz, schema.ddl
>
>
> {code}
> [cassandra@dc01-rack01-cass01 ~]$ 
> /home/cassandra/dsc-cassandra-3.0.0/bin/nodetool upgradesstables
> error: null
> -- StackTrace --
> java.lang.UnsupportedOperationException
> at 
> org.apache.cassandra.db.rows.CellPath$EmptyCellPath.get(CellPath.java:143)
> at 
> org.apache.cassandra.db.marshal.CollectionType$CollectionPathSerializer.serializedSize(CollectionType.java:226)
> at 
> org.apache.cassandra.db.rows.BufferCell$Serializer.serializedSize(BufferCell.java:325)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.sizeOfComplexColumn(UnfilteredSerializer.java:297)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:282)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:163)
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:144)
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:112)
> at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:121)
> at 
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:397)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9632) Preserve the Names of Query Parameters in QueryOptions

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9632:
--
Component/s: CQL

> Preserve the Names of Query Parameters in QueryOptions
> --
>
> Key: CASSANDRA-9632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: stephen mallette
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.2, 3.0.0 rc1
>
> Attachments: 9632-2.2-V2.txt, 9632-2.2.txt
>
>
> When writing a custom {{QueryHandler}} that processes named parameters, the 
> {{QueryOptions}} arrive to the various processing methods with the values 
> converted to positional arguments and the names unavailable.  A custom 
> {{QueryHandler}} might want to make use of the names themselves so it would 
> be helpful if they were preserved and exposed with their respective 
> {{ByteBuffer}} values.
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/cql3/QueryOptions.java#L205



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10580) Add latency metrics for dropped messages

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10580:

Component/s: Observability

> Add latency metrics for dropped messages
> 
>
> Key: CASSANDRA-10580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10580
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination, Observability
> Environment: Production
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Fix For: 3.2, 2.2.x
>
> Attachments: 0001-Metrics.patch, 10580-Metrics.patch, 10580.patch, 
> 2.2-All-Comments.patch, CASSANDRA-10580-Head.patch, Trunk-All-Comments.patch, 
> Trunk.patch
>
>
> In our production cluster, we are seeing a large number of dropped mutations. 
> At a minimum, we should print the time the thread took to get scheduled 
> thereby dropping the mutation (We should also print the Message / Mutation so 
> it helps in figuring out which column family was affected). This will help 
> find the right tuning parameter for write_timeout_in_ms. 
> The change is small and is in StorageProxy.java and MessagingTask.java. I 
> will submit a patch shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10870) pushed_notifications_test.py:TestPushedNotifications.restart_node_test flapping on C* 2.1

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10870:
-
Issue Type: Sub-task  (was: Bug)
Parent: CASSANDRA-10664

> pushed_notifications_test.py:TestPushedNotifications.restart_node_test 
> flapping on C* 2.1
> -
>
> Key: CASSANDRA-10870
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10870
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 2.1.x
>
>
> This test flaps on CassCI on 2.1. [~aboudreault] Do I remember correctly that 
> you did some work on these tests in the past few months? If so, could you 
> have a look and see if there's some assumption the test makes that don't hold 
> for 2.1?
> Oddly, it fails frequently under JDK8:
> http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/
> but less frequently on JDK7:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_test/history/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10146) Deprecate v1 and v2 protocol in 2.2, drop support in 3.0

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10146:
---
Component/s: CQL

> Deprecate v1 and v2 protocol in 2.2, drop support in 3.0
> 
>
> Key: CASSANDRA-10146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10146
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Tyler Hobbs
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.0.0 rc1, 2.2.x
>
>
> In 3.0, we would like to use frozen collections in the system keyspaces, and 
> it seems likely that we will eventually want to use tuples or nested 
> collections as well.  Drivers that only support protocol versions 1 and 2 
> will not be able to read these system keyspaces because they cannot decode 
> those types.
> I think this is a good time to start deprecating and dropping support for old 
> protocol versions.  The v3 protocol was introduced in 2.1, so if we remove 
> support for v1 and v2 in 3.0, that gives users two major versions to upgrade 
> their drivers.  Fortunately, upgrading drivers to a version that supports the 
> v3 protocol is generally straightforward.
> The benefits of doing this are:
> * We can use new types in the system keyspaces
> * We can eliminate protocol-version-specific encoding and decoding of 
> collections within Cassandra
> * Driver maintainers can eventually drop support for old protocol versions 
> once all C* versions that support them are EOL
> To avoid a hard drop of v1 and v2 support in 3.0, I propose that we also 
> officially deprecate these in 2.2.  Unfortunately, we don't have 
> protocol-level warnings until v4, so we can't use that to notify users of the 
> deprecation, but the combination of a NEWS.txt entry, warning logs, and 
> (potentially) driver-level warnings should suffice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10473) fix failing dtest for select distinct with static columns on 2.2->3.0 upgrade path

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10473:
---
Component/s: CQL

> fix failing dtest for select distinct with static columns on 2.2->3.0 upgrade 
> path
> --
>
> Key: CASSANDRA-10473
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10473
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Jim Witschey
>Assignee: Benjamin Lerer
> Fix For: 3.0.0 rc2
>
>
> {{upgrade_tests/cql_tests.py:TestCQL.static_columns_with_distinct_test}} 
> fails on the upgrade path from 2.2 to 3.0:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/lastCompletedBuild/testReport/upgrade_tests.cql_tests/TestCQL/static_columns_with_distinct_test/
> Once [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/586] is 
> merged, these tests should also run with this upgrade path on normal 3.0 
> jobs. Until then, you can run it with the following command:
> {code}
> SKIP=false CASSANDRA_VERSION=binary:2.2.0 UPGRADE_TO=git:cassandra-3.0 
> nosetests upgrade_tests/cql_tests.py:TestCQL.static_columns_with_distinct_test
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10863) HSHA test_closing_connections test still flaps on 3.0

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10863:
-
Issue Type: Sub-task  (was: Bug)
Parent: CASSANDRA-10664

> HSHA test_closing_connections test still flaps on 3.0
> -
>
> Key: CASSANDRA-10863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10863
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.0.x
>
>
> The problem reported in CASSANDRA-10570 still seems to be happening on 
> CassCI, as recently as a couple days ago:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/433/
> [~carlyeks] The fix in #10570 was to increase how long we'd sleep in the 
> tests. Should we just bump it up further, or does this make you suspect a 
> larger problem here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10867) thrift_tests.py:TestCQLAccesses.test_range_tombstone_and_static failing on C* 2.1

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10867:
-
Issue Type: Sub-task  (was: Bug)
Parent: CASSANDRA-10664

> thrift_tests.py:TestCQLAccesses.test_range_tombstone_and_static failing on C* 
> 2.1
> -
>
> Key: CASSANDRA-10867
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10867
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>
> http://cassci.datastax.com/job/cassandra-2.1_dtest/376/testReport/thrift_tests/TestCQLAccesses/test_range_tombstone_and_static/history/
> I haven't had enough experience with thrift or the thrift tests to debug 
> this. It passes on 2.2+. I've reproduced this failure locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10381) NullPointerException in cqlsh paging through CF with static columns

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10381:
---
Component/s: CQL

> NullPointerException in cqlsh paging through CF with static columns
> ---
>
> Key: CASSANDRA-10381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10381
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Michael Keeney
>Assignee: Benjamin Lerer
>  Labels: cqlsh, nullpointerexception, range
> Fix For: 2.1.12, 2.2.4, 3.0.0
>
>
> When running select count( * ) from cqlsh with limit, the following NPE 
> occurs:
> select count( * ) from tbl1 limit 5 ; 
> {code}
> ERROR [SharedPool-Worker-4] 2015-09-16 14:49:43,480 QueryMessage.java:132 - 
> Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.containsPreviousLast(RangeSliceQueryPager.java:99)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.fetchPage(RangeSliceQueryPager.java:37)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.pageCountQuery(SelectStatement.java:286)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:230)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:67)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$StatementExecution.execute(DseQueryHandler.java:291)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithTiming(DseQueryHandler.java:223)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithAuditLogging(DseQueryHandler.java:259)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler.process(DseQueryHandler.java:94)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_75]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
> {code}
> Table definition looks something like:
> {code}
> CREATE TABLE tbl1 (
> field1 bigint,
> field2 int,
> field3 timestamp,
> field4 map,
> field5 text static,
> field6 text static,
> field7 text static
> PRIMARY KEY (field1, field2, field3)
> ) WITH CLUSTERING ORDER BY (field2 ASC, field3 ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>...
> {code}
> Following appears in debug log leading up to the error:
> {code}
> DEBUG [SharedPool-Worker-1] 2015-09-17 15:32:06,484  
> AbstractQueryPager.java:95 - Fetched 101 live rows
> DEBUG [SharedPool-Worker-1] 2015-09-17 15:32:06,484  
> AbstractQueryPager.java:133 - 

[jira] [Updated] (CASSANDRA-9556) Add newer data types to cassandra stress

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9556:
--
Summary: Add newer data types to cassandra stress  (was: Add newer data 
types to cassandra stress (e.g. decimal, dates, UDTs))

> Add newer data types to cassandra stress
> 
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: stress
> Fix For: 2.2.x
>
> Attachments: CASSANDRA-9556-2.2.patch
>
>
> Currently you can't define a data model with decimal types and use Cassandra 
> stress with it.  Also, I imagine that holds true with other newer data types 
> such as the new date and time types.  Besides that, now that data models are 
> including user defined types, we should allow users to create those 
> structures with stress as well.  Perhaps we could split out the UDTs into a 
> different ticket if it holds the other types up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Add new types to Stress

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 98178f53a -> 363e7bd71


Add new types to Stress

patch by ZhaoYang; reviewed by Benjamin Lerer for CASSANDRA-9556


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b03ce9fd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b03ce9fd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b03ce9fd

Branch: refs/heads/cassandra-3.0
Commit: b03ce9fd479d3da9c51d118c50480ea96df0d73e
Parents: 57d558f
Author: ZhaoYang 
Authored: Wed Dec 16 15:06:06 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Dec 16 15:06:06 2015 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/stress/StressProfile.java  |  8 
 .../stress/generate/values/LocalDates.java  | 43 +++
 .../stress/generate/values/SmallInts.java   | 38 +
 .../cassandra/stress/generate/values/Times.java | 37 
 .../stress/generate/values/TinyInts.java| 45 
 .../operations/userdefined/SchemaStatement.java | 16 +--
 .../cassandra/stress/util/JavaDriverClient.java |  2 +-
 8 files changed, 185 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c969a4d..1480960 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * Add new types to Stress (CASSANDRA-9556)
  * Add property to allow listening on broadcast interface (CASSANDRA-9748)
  * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
  * Better handling of SSL connection errors inter-node (CASSANDRA-10816)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 9c9b921..410f666 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -531,6 +531,14 @@ public class StressProfile implements Serializable
 return new UUIDs(name, config);
 case TIMEUUID:
 return new TimeUUIDs(name, config);
+case TINYINT:
+return new TinyInts(name, config);
+case SMALLINT:
+return new SmallInts(name, config);
+case TIME:
+return new Times(name, config);
+case DATE:
+return new LocalDates(name, config);
 case SET:
 return new Sets(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 case LIST:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b03ce9fd/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
new file mode 100644
index 000..f079d35
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/LocalDates.java
@@ -0,0 +1,43 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import com.datastax.driver.core.LocalDate;
+import org.apache.cassandra.db.marshal.SimpleDateType;
+import org.joda.time.DateTimeZone;
+import org.joda.time.format.DateTimeFormat;
+import org.joda.time.format.DateTimeFormatter;
+
+public class LocalDates extends Generator
+{
+
+public LocalDates(String name, GeneratorConfig 

[2/2] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/363e7bd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/363e7bd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/363e7bd7

Branch: refs/heads/cassandra-3.0
Commit: 363e7bd716b1c0b798f740b88bcb336c41f33470
Parents: 98178f5 b03ce9f
Author: Benjamin Lerer 
Authored: Wed Dec 16 15:19:20 2015 +0100
Committer: Benjamin Lerer 
Committed: Wed Dec 16 15:19:20 2015 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/stress/StressProfile.java  |  8 
 .../stress/generate/values/LocalDates.java  | 43 +++
 .../stress/generate/values/SmallInts.java   | 38 +
 .../cassandra/stress/generate/values/Times.java | 37 
 .../stress/generate/values/TinyInts.java| 45 
 .../operations/userdefined/SchemaStatement.java | 16 +--
 .../cassandra/stress/util/JavaDriverClient.java |  1 +
 8 files changed, 185 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/CHANGES.txt
--
diff --cc CHANGES.txt
index dd1896c,1480960..10504c7
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
 -2.2.5
 +3.0.3
 +Merged from 2.2:
+  * Add new types to Stress (CASSANDRA-9556)
   * Add property to allow listening on broadcast interface (CASSANDRA-9748)
   * Fix regression in split size on CqlInputFormat (CASSANDRA-10835)
   * Better handling of SSL connection errors inter-node (CASSANDRA-10816)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/363e7bd7/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
--
diff --cc 
tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
index d0779ed,30d0d4a..7d5f38c
--- a/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
+++ b/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java
@@@ -114,7 -95,7 +114,8 @@@ public class JavaDriverClien
  .addContactPoint(host)
  .withPort(port)
  
.withPoolingOptions(poolingOpts)
 +.withoutJMXReporting()
+ 
.withProtocolVersion(ProtocolVersion.NEWEST_SUPPORTED)
  .withoutMetrics(); // The 
driver uses metrics 3 with conflict with our version
  if (whitelist != null)
  clusterBuilder.withLoadBalancingPolicy(whitelist);

[jira] [Commented] (CASSANDRA-10653) Remove dependency on jgrapht for UDT resolution

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060024#comment-15060024
 ] 

Sylvain Lebresne commented on CASSANDRA-10653:
--

+1, but it's probably worth rebasing and re-running the tests before commit.

> Remove dependency on jgrapht for UDT resolution
> ---
>
> Key: CASSANDRA-10653
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10653
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 3.0.x
>
>
> Now that the java-driver no longer pulls it as a dependency, it is silly to 
> pull a whole library for resolving UDTs dependencies.
> Should rewrite the resolution code without jgrapht (maybe reuse whatever code 
> java-driver ended up writing).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9556) Add newer data types to cassandra stress

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060030#comment-15060030
 ] 

Benjamin Lerer edited comment on CASSANDRA-9556 at 12/16/15 2:24 PM:
-

[~jasonstack] Thanks for the patch.


was (Author: blerer):
[~jasonstack] Thanks for all the patch.

> Add newer data types to cassandra stress
> 
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: stress
> Fix For: 2.2.x
>
> Attachments: CASSANDRA-9556-2.2.patch
>
>
> Currently you can't define a data model with decimal types and use Cassandra 
> stress with it.  Also, I imagine that holds true with other newer data types 
> such as the new date and time types.  Besides that, now that data models are 
> including user defined types, we should allow users to create those 
> structures with stress as well.  Perhaps we could split out the UDTs into a 
> different ticket if it holds the other types up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9556) Add newer data types to cassandra stress

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060030#comment-15060030
 ] 

Benjamin Lerer commented on CASSANDRA-9556:
---

[~jasonstack] Thanks for all the patch.

> Add newer data types to cassandra stress
> 
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: stress
> Fix For: 2.2.x
>
> Attachments: CASSANDRA-9556-2.2.patch
>
>
> Currently you can't define a data model with decimal types and use Cassandra 
> stress with it.  Also, I imagine that holds true with other newer data types 
> such as the new date and time types.  Besides that, now that data models are 
> including user defined types, we should allow users to create those 
> structures with stress as well.  Perhaps we could split out the UDTs into a 
> different ticket if it holds the other types up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9767) Allow the selection of columns together with aggregates

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9767:
--
Component/s: CQL

> Allow the selection of columns together with aggregates
> ---
>
> Key: CASSANDRA-9767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9767
> Project: Cassandra
>  Issue Type: Wish
>  Components: CQL
> Environment: Cassandra 2.0.16
> Ubuntu 15.04
>Reporter: Ajay
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.0, 3.0 alpha 1
>
> Attachments: 9767-2.2.txt
>
>
> Lets assume we have a column family as below:
> create table sample ( track_id int, user_id int, country varchar, primary key 
> ((track_id), user_id));
> where track_id is the partition key.
> Now to aggregate the number of rows for a single track_id, we can query using 
> CQL as below:
> select count(*) where track_id = 1 and user_id = 1;
> But that will return only the count. If we need the other columns along with 
> the count, we cannot query as below as it throws error:
>  select count(*), country  from sample where track_id = 1 and user_id = 1;
> Bad Request: line 1:15 mismatched input ',' expecting K_FROM.
> In this case, all rows for a given track_id and user_id will have the same 
> value for country. So we should be able to query as above.  Also in SQL, it 
> is possible to select columns along with aggregate functions.
> Though I know that Cassandra is not analytics (unlike Hadoop and Spark), we 
> need some basic aggregate functions like min, max, avg etcThough 
> performance wise it might not be efficient, but it is better done in the 
> cassandra side (as it uses native protocol) than we getting all rows in the 
> client and doing the basic aggregation.  It cannot used just as a data store 
> (as garbage-in garbage-out). In that context, currently CQL is pretty 
> limited. Just for getting data out of cassandra, we will have to spark though 
> we will not be doing much analytics on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9913) Select * is only returning the first page of data on trunk

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-9913:
--
Component/s: CQL

> Select * is only returning the first page of data on trunk
> --
>
> Key: CASSANDRA-9913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9913
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Philip Thompson
>Assignee: Benjamin Lerer
> Fix For: 3.0 beta 1
>
> Attachments: 9913.txt
>
>
> While doing some testing on the validation harness, I have run into a pretty 
> trivially reproducible problem.
> {code}
> ccm create test -v git:trunk -n 1 -s
> ccm node1 stress write n=2M
> ccm node1 cqlsh
> {code}
> {code}
> Use keyspace1;
> Select * From standard1; (100 rows)
> Select count(*) from standard1; (300 rows)
> {code}
> Despite two million rows being written, I have found that {{select * from 
> standard1}} only returns one page's worth of data. I have used both the java 
> and the python driver to test this. I have also found that {{select count(*) 
> from standard1}} gives a multiple of one page's worth of data, that appears 
> to correspond to page size * RF.
> I have already tried with the patch for CASSANDRA-9775, and that did not 
> resolve this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10043) A NullPointerException is thrown if the column name is unknown for an IN relation

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10043:
---
Component/s: CQL

> A NullPointerException is thrown if the column name is unknown for an IN 
> relation
> -
>
> Key: CASSANDRA-10043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10043
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.2.1, 3.0 beta 2
>
> Attachments: 10043-2.2.txt, 10043-3.0.txt
>
>
> {code}
> cqlsh:test> create table newTable (a int, b int, c int, primary key(a, b));
> cqlsh:test> select * from newTable where d in (1, 2);
> ServerError:  message="java.lang.NullPointerException">
> {code}
> The problem seems to occur only for {{IN}} restrictions 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10880) Paging state between 2.2 and 3.0 are incompatible on protocol v4

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060103#comment-15060103
 ] 

Adam Holmberg commented on CASSANDRA-10880:
---

Just a note on v4 and preferred version: Python driver has supported v4 (and 
defaulted it) since version 2.6.0 ( released in conjunction with Cassandra 
2.2.0 pre-release series in June 2015).

> Paging state between 2.2 and 3.0 are incompatible on protocol v4
> 
>
> Key: CASSANDRA-10880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10880
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Priority: Critical
>  Labels: client-impacting
> Fix For: 3.x
>
>
> In CASSANDRA-10254, the paging states generated by 3.0 for the native 
> protocol v4 were made 3.0 specific. This was done because the paging state in 
> pre-3.0 versions contains a serialized cell name, but 3.0 doesn't talk in 
> term of cells internally (at least not the pre-3.0 ones) and so using an 
> old-format cell name when we only have 3.0 nodes is inefficient and inelegant.
> Unfortunately that change was made on the assumption than the protocol v4 was 
> 3.0 only but it's not, it ended up being released with 2.2 and that 
> completely slipped my mind. So in practice, you can't properly have a mixed 
> 2.2/3.0 cluster if your driver is using the protocol v4.
> And unfortunately, I don't think there is an easy way to fix that without 
> breaking something. Concretely, I can see 3 choices:
> # we change 3.0 so that it generates old-format paging states on the v4 
> protocol. The 2 main downsides are that 1) this breaks 3.0 upgrades if the 
> driver is using the v4 protocol, and at least on the java side the only 
> driver versions that support 3.0 will use v4 by default and 2) we're signing 
> off on having sub-optimal paging state until the protocol v5 ships (probably 
> not too soon).
> # we remove the v4 protocol from 2.2. This means 2.2 will have to use v3 
> before upgrade at the risk of breaking upgrade. This is also bad, but I'm not 
> sure the driver version using the v4 protocol are quite ready yet (at least 
> the java driver is not GA yet) so if we work with the drivers teams to make 
> sure the v3 protocol gets prefered by default on 2.2 in the GA versions of 
> these driver, this might be somewhat transparent to users.
> # we don't change anything code-wise, but we document clearly that you can't 
> upgrade from 2.2 to 3.0 if your clients use protocol v4 (so we leave upgrade 
> broken if the v4 protocol is used as it is currently). This is not great, but 
> we can work with the drivers teams here again to make sure drivers prefer the 
> v3 version for 2.2 nodes so most people don't notice in practice.
> I think I'm leaning towards solution 3). It's not great but at least we break 
> no minor upgrades (neither on 2.2, nor on 3.0) which is probably the most 
> important. We'd basically be just adding a new condition on 2.2->3.0 
> upgrades.  We could additionally make 3.0 node completely refuse v4 
> connections if they know a 2.2 nodes is in the cluster for extra safety.
> Ping [~omichallat], [~adutra] and [~aholmber] as you might want to be aware 
> of that ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10027) ALTER TABLE TYPE check broken

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10027:
---
Component/s: CQL

> ALTER TABLE TYPE check broken
> -
>
> Key: CASSANDRA-10027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10027
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Aaron Ploetz
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.4, 3.0.1, 3.1
>
> Attachments: 10027-2.2.txt
>
>
> I stumbled onto the fact that 2.2.0 will allow you to ALTER TYPE of a 
> {{varint}} to the new {{date}} type.  I thought that was an odd conversion to 
> allow, so I attempted to query it.  I received an error on all subsequent 
> queries, unless I exited or truncated the table.
> After truncating, I could then INSERT and query as normal.  But the new 
> {{varint}} values inserted simply were reflected as an offset of the minimum 
> {{varint}} value.
> I'm not sure why that's happening, but if we could simply prevent type 
> conversion between {{varint}} and {{date}} (and just show the "types are 
> incompatible" message) that should fix this.
> Steps to reproduce:
> {code}
> aploetz@cqlsh:typeconversion> CREATE TABLE varinttest (key int PRIMARY KEY, 
> c1 varint);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
>1 |  1
> (1 rows)
> aploetz@cqlsh:typeconversion> ALTER TABLE varinttest ALTER c1 TYPE date;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1150, in perform_simple_statement
> rows = future.result(self.session.default_timeout)
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 3296, in result
> raise self._final_exception
> error: unpack requires a string argument of length 4
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> NoHostAvailable: ('Unable to complete the operation against any hosts', 
> {: ConnectionShutdown('Connection to 127.0.0.1 is 
> defunct',)})
> aploetz@cqlsh:typeconversion> TRUNCATE varinttest ;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
> (0 rows)
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (2,2);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (3,3);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+-
>1 | -2147483647
>2 | -2147483646
>3 | -2147483645
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10264) Unable to use conditions on static columns for DELETE

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10264:
---
Component/s: CQL

> Unable to use conditions on static columns for DELETE
> -
>
> Key: CASSANDRA-10264
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10264
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 2.2.0
>Reporter: DOAN DuyHai
>Assignee: Benjamin Lerer
> Fix For: 2.1.12, 2.2.4, 3.0.0
>
> Attachments: 10264-2.1-V2.txt, 10264-2.1.txt, 10264-3.0-V2.txt, 
> 10264-3.0.txt
>
>
> {noformat}
> cqlsh:test> create table static_table(id int, stat int static, ord int, val 
> text, primary key(id,ord));
> cqlsh:test> insert into static_table (id,stat,ord,val) VALUES ( 1, 1, 1, '1');
> cqlsh:test> delete from static_table where id=1 and ord=1 if stat != 1;
> Invalid syntax at line 1, char 55
>   delete from static_table where id=1 and ord=1 if stat != 1;
> ^
> {noformat}
> Same error if using =, <, <=, >= or > condition
> According to [~thobbs] the syntax should work. Plus, the error message is 
> wrong



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)