[jira] [Assigned] (CASSANDRA-7622) Implement virtual tables

2016-10-10 Thread Jeff Jirsa (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-7622:
-

Assignee: Jeff Jirsa

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12462) NullPointerException in CompactionInfo.getId(CompactionInfo.java:65)

2016-10-10 Thread Simon Zhou (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564482#comment-15564482
 ] 

Simon Zhou commented on CASSANDRA-12462:


Nice feedback. Thanks [~yukim]. I'll update my patch shortly.

> NullPointerException in CompactionInfo.getId(CompactionInfo.java:65)
> 
>
> Key: CASSANDRA-12462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12462
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonathan DePrizio
> Attachments: 0001-Fix-NPE-when-running-nodetool-compactionstats.patch
>
>
> Note: The same trace is cited in the last comment of 
> https://issues.apache.org/jira/browse/CASSANDRA-11961
> I've noticed that some of my nodes in my 2.1 cluster have fallen way behind 
> on compactions, and have huge numbers (thousands) of uncompacted, tiny 
> SSTables (~30MB or so).
> In diagnosing the issue, I've found that "nodetool compactionstats" returns 
> the exception below.  Restarting cassandra on the node here causes the 
> pending tasks count to jump to ~2000.  Compactions run properly for about an 
> hour, until this exception occurs again.  Once it occurs, I see the pending 
> tasks value rapidly drop towards zero, but without any compactions actually 
> running (the logs show no compactions finishing).  It would seem that this is 
> causing compactions to fail on this node, which is leading to it running out 
> of space, etc.
> [redacted]# nodetool compactionstats
> xss =  -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar 
> -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms12G -Xmx12G 
> -Xmn1000M -Xss255k
> pending tasks: 5
> error: null
> -- StackTrace --
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.db.compaction.CompactionInfo.getId(CompactionInfo.java:65)
>   at 
> org.apache.cassandra.db.compaction.CompactionInfo.asMap(CompactionInfo.java:118)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager.getCompactions(CompactionManager.java:1405)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>   at sun.reflect.misc.Trampoline.invoke(Unknown Source)
>   at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>   at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
>   at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown 
> Source)
>   at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown 
> Source)
>   at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
>   at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
>   at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
>   at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown 
> Source)
>   at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
>   at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown 
> Source)
>   at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown 
> Source)
>   at 
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown 
> Source)
>   at 
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown 
> Source)
>   at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown 
> Source)
>   at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>   at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
>   at sun.rmi.transport.Transport$1.run(Unknown Source)
>   at sun.rmi.transport.Transport$1.run(Unknown Source)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at sun.rmi.transport.Transport.serviceCall(Unknown Source)
>   at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
>   at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown 
> Source)
>   at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown 
> Source)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>   at java.lang.Thread.run(Unknown Source)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12754) Change cassandra.wait_for_tracing_events_timeout_secs default to -1 so C* doesn't wait on trace events to be written before responding to request by default

2016-10-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564406#comment-15564406
 ] 

Stefania commented on CASSANDRA-12754:
--

I've changed the default value to zero, which has the same effect as changing 
it to -1, disabling the wait for pending tracing events. The tests already set 
{{cassandra.wait_for_tracing_events_timeout_secs}} to 15 seconds, so they don't 
need changing.

In order to save CI resources, I've launched the tests only on 2.2 and 3.X 
(which had conflicts):

||2.2||3.X||
|[patch|https://github.com/stef1927/cassandra/commits/12754-2.2]|[patch|https://github.com/stef1927/cassandra/commits/12754-3.X]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12754-2.2-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12754-3.X-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12754-2.2-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12754-3.X-dtest/]|



> Change cassandra.wait_for_tracing_events_timeout_secs default to -1 so C* 
> doesn't wait on trace events to be written before responding to request by 
> default
> 
>
> Key: CASSANDRA-12754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12754
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Stefania
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> [CASSANDRA-11465] introduces a new system property 
> {{cassandra.wait_for_tracing_events_timeout_secs}} that controls whether or 
> not C* waits for events to be written before responding to client.   The 
> current default behavior is to wait up to 1 second and then respond and 
> timeout.  
> If using probabilistic tracing this can cause queries to be randomly delayed 
> up to 1 second.
> Changing the default to -1 (disabled and enabling it explicitly in 
> {{cql_tracing_test.TestCqlTracing.tracing_unknown_impl_test}}.
> Ideally it would be nice to be able to control this behavior on a per request 
> basis (which would require native protocol changes).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12705) Add column definition kind to system schema dropped columns

2016-10-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564372#comment-15564372
 ] 

Stefania commented on CASSANDRA-12705:
--

Thank for the review.

bq. Don't know what it looked like in the original version, but in 
system_schema.columns we store the lowercase values of the enum names. Look at 
kind and clustering_order for examples. Would prefer it to be the same in 
system_schema.dropped_columns, for consistency.

I had noticed this behavior, but I preferred not to do the same because I could 
not understand why convert to lowercase before writing only to convert back to 
uppercase after reading. It seemed like a waste of CPU cycles, but thinking 
about it again, it must be for the benefit of humans reading the tables?

Regardless, for the sake of consistency, I've updated the patch to store 
lowercase values. I've restarted CI and, unless objections or failures, I will 
commit once the results are available.

> Add column definition kind to system schema dropped columns
> ---
>
> Key: CASSANDRA-12705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12705
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 4.0
>
>
> Both regular and static columns can currently be dropped by users, but this 
> information is currently not stored in {{SchemaKeyspace.DroppedColumns}}. As 
> a consequence, {{CFMetadata.getDroppedColumnDefinition}} returns a regular 
> column and this has caused problems such as CASSANDRA-12582.
> We should add the column kind to {{SchemaKeyspace.DroppedColumns}} so that 
> {{CFMetadata.getDroppedColumnDefinition}} can create the correct column 
> definition. However, altering schema tables would cause inter-node 
> communication failures during a rolling upgrade, see CASSANDRA-12236. 
> Therefore we should wait for a full schema migration when upgrading to the 
> next major version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564323#comment-15564323
 ] 

Stefania commented on CASSANDRA-12740:
--

Pull request is [here|https://github.com/riptano/cassandra-dtest/pull/1358].

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 3.0.10, 3.10
>
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Stefania (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12740:
-
   Resolution: Fixed
Fix Version/s: 3.10
   3.0.10
   Status: Resolved  (was: Ready to Commit)

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 3.0.10, 3.10
>
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Stefania (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12740:
-
Labels: cqlsh  (was: )

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 3.0.10, 3.10
>
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Stefania (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12740:
-
Component/s: Tools

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 3.0.10, 3.10
>
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Stefania (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564315#comment-15564315
 ] 

Stefania commented on CASSANDRA-12740:
--

Thanks for the review! I've modified the comment and committed to 3.0 as 
72c9eb2dc6732a1f20e769ae162f02e5766f397f, then merged into 3.X and trunk.

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

2016-10-10 Thread stefania
Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e0a35e58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e0a35e58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e0a35e58

Branch: refs/heads/cassandra-3.X
Commit: e0a35e585230d4072e59b1d6c5caa5a3ee977991
Parents: 3003d21 72c9eb2
Author: Stefania Alborghetti 
Authored: Tue Oct 11 11:26:24 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:26:24 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0a35e58/CHANGES.txt
--
diff --cc CHANGES.txt
index f4dda82,9f7fff8..00ff1eb
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,82 -1,5 +1,83 @@@
 -3.0.10
 +3.10
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Add row offset support to SASI (CASSANDRA-11990)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming (CASSANDRA-11841)
 + * Tracing payload is passed through newSession(..) (CASSANDRA-11706)
 + * avoid deleting non existing sstable files and improve related log messages 
(CASSANDRA-12261)
 + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486)
 + * Retry all internode messages once after a connection is
 +   closed and reopened (CASSANDRA-12192)
 + * Add support to rebuild from targeted replica (CASSANDRA-9875)
 + * Add sequence distribution type to cassandra stress (CASSANDRA-12490)
 + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154)
 + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474)
 + * Extend read/write failure messages with a map of replica addresses
 +   to error codes in the v5 native protocol (CASSANDRA-12311)
 + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
 + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 
12550)
 + * Fix clustering indexes in presence of static columns in SASI 
(CASSANDRA-12378)
 + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
 + * Added slow query log (CASSANDRA-12403)
 + * Count full coordinated request against timeout (CASSANDRA-12256)
 + * Allow TTL with null value on insert and update (CASSANDRA-12216)
 + * Make decommission operation resumable (CASSANDRA-12008)
 + * Add support to one-way targeted repair (CASSANDRA-9876)
 + * Remove clientutil jar (CASSANDRA-11635)
 + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717)
 + * Delay releasing Memtable memory on flush until PostFlush has finished 
running (CASSANDRA-12358)
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 + * Make it possible to compact a given token range (CASSANDRA-10643)
 + * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179)
 + * Collect metrics on queries by consistency level (CASSANDRA-7384)
 + * Add support for GROUP BY to SELECT statement (CASSANDRA-10707)
 + * Deprecate memtable_cleanup_threshold and update default for 
memtable_flush_writers (CASSANDRA-12228)
 + * Upgrade to OHC 0.4.4 (CASSANDRA-12133)
 + * Add version command to 

[3/6] cassandra git commit: Abort cqlsh copy-from in case of no answer after prolonged period of time

2016-10-10 Thread stefania
Abort cqlsh copy-from in case of no answer after prolonged period of time

Patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-12740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/72c9eb2d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/72c9eb2d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/72c9eb2d

Branch: refs/heads/trunk
Commit: 72c9eb2dc6732a1f20e769ae162f02e5766f397f
Parents: 8455286
Author: Stefania Alborghetti 
Authored: Mon Oct 3 08:51:16 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:25:03 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a517995..9f7fff8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
  * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
  * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
  * Fix exceptions with new vnode allocation (CASSANDRA-12715)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 23e739d..d2084d7 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -43,6 +43,7 @@ from select import select
 from uuid import UUID
 from util import profile_on, profile_off
 
+from cassandra import OperationTimedOut
 from cassandra.cluster import Cluster, DefaultConnection
 from cassandra.cqltypes import ReversedType, UserType
 from cassandra.metadata import protect_name, protect_names, protect_value
@@ -362,6 +363,11 @@ class CopyTask(object):
 copy_options['maxinflightmessages'] = 
int(opts.pop('maxinflightmessages', '512'))
 copy_options['maxbackoffattempts'] = 
int(opts.pop('maxbackoffattempts', '12'))
 copy_options['maxpendingchunks'] = int(opts.pop('maxpendingchunks', 
'24'))
+# set requesttimeout to a value high enough so that maxbatchsize rows 
will never timeout if the server
+# responds: here we set it to 1 sec per 10 rows but no less than 60 
seconds
+copy_options['requesttimeout'] = int(opts.pop('requesttimeout', 
max(60, 1 * copy_options['maxbatchsize'] / 10)))
+# set childtimeout higher than requesttimeout so that child processes 
have a chance to report request timeouts
+copy_options['childtimeout'] = int(opts.pop('childtimeout', 
copy_options['requesttimeout'] + 30))
 
 self.check_options(copy_options)
 return CopyOptions(copy=copy_options, dialect=dialect_options, 
unrecognized=opts)
@@ -1186,9 +1192,21 @@ class ImportTask(CopyTask):
 if not self.fname:
 self.send_stdin_rows()
 
+child_timeout = self.options.copy['childtimeout']
+last_recv_num_records = 0
+last_recv_time = time.time()
+
 while self.feeding_result is None or self.receive_meter.total_records 
< self.feeding_result.sent:
 self.receive_results()
 
+if self.feeding_result is not None:
+if self.receive_meter.total_records != last_recv_num_records:
+last_recv_num_records = self.receive_meter.total_records
+last_recv_time = time.time()
+elif (time.time() - last_recv_time) > child_timeout:
+self.shell.printerr("No records inserted in {} seconds, 
aborting".format(child_timeout))
+break
+
 if self.error_handler.max_exceeded() or not 
self.all_processes_running():
 break
 
@@ -2197,6 +2215,7 @@ class ImportProcess(ChildProcess):
 self.use_prepared_statements = options.copy['preparedstatements']
 self.max_inflight_messages = options.copy['maxinflightmessages']
 self.max_backoff_attempts = options.copy['maxbackoffattempts']
+self.request_timeout = options.copy['requesttimeout']
 
 self.dialect_options = options.dialect
 self._session = None
@@ -2223,7 +2242,7 @@ class ImportProcess(ChildProcess):
 connection_class=ConnectionWrapper)
 
 self._session = cluster.connect(self.ks)
-self._session.default_timeout = None
+self._session.default_timeout = self.request_timeout
 

[1/6] cassandra git commit: Abort cqlsh copy-from in case of no answer after prolonged period of time

2016-10-10 Thread stefania
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 84552867b -> 72c9eb2dc
  refs/heads/cassandra-3.X 3003d21a9 -> e0a35e585
  refs/heads/trunk 07bc404ff -> 9f1674926


Abort cqlsh copy-from in case of no answer after prolonged period of time

Patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-12740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/72c9eb2d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/72c9eb2d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/72c9eb2d

Branch: refs/heads/cassandra-3.0
Commit: 72c9eb2dc6732a1f20e769ae162f02e5766f397f
Parents: 8455286
Author: Stefania Alborghetti 
Authored: Mon Oct 3 08:51:16 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:25:03 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a517995..9f7fff8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
  * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
  * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
  * Fix exceptions with new vnode allocation (CASSANDRA-12715)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 23e739d..d2084d7 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -43,6 +43,7 @@ from select import select
 from uuid import UUID
 from util import profile_on, profile_off
 
+from cassandra import OperationTimedOut
 from cassandra.cluster import Cluster, DefaultConnection
 from cassandra.cqltypes import ReversedType, UserType
 from cassandra.metadata import protect_name, protect_names, protect_value
@@ -362,6 +363,11 @@ class CopyTask(object):
 copy_options['maxinflightmessages'] = 
int(opts.pop('maxinflightmessages', '512'))
 copy_options['maxbackoffattempts'] = 
int(opts.pop('maxbackoffattempts', '12'))
 copy_options['maxpendingchunks'] = int(opts.pop('maxpendingchunks', 
'24'))
+# set requesttimeout to a value high enough so that maxbatchsize rows 
will never timeout if the server
+# responds: here we set it to 1 sec per 10 rows but no less than 60 
seconds
+copy_options['requesttimeout'] = int(opts.pop('requesttimeout', 
max(60, 1 * copy_options['maxbatchsize'] / 10)))
+# set childtimeout higher than requesttimeout so that child processes 
have a chance to report request timeouts
+copy_options['childtimeout'] = int(opts.pop('childtimeout', 
copy_options['requesttimeout'] + 30))
 
 self.check_options(copy_options)
 return CopyOptions(copy=copy_options, dialect=dialect_options, 
unrecognized=opts)
@@ -1186,9 +1192,21 @@ class ImportTask(CopyTask):
 if not self.fname:
 self.send_stdin_rows()
 
+child_timeout = self.options.copy['childtimeout']
+last_recv_num_records = 0
+last_recv_time = time.time()
+
 while self.feeding_result is None or self.receive_meter.total_records 
< self.feeding_result.sent:
 self.receive_results()
 
+if self.feeding_result is not None:
+if self.receive_meter.total_records != last_recv_num_records:
+last_recv_num_records = self.receive_meter.total_records
+last_recv_time = time.time()
+elif (time.time() - last_recv_time) > child_timeout:
+self.shell.printerr("No records inserted in {} seconds, 
aborting".format(child_timeout))
+break
+
 if self.error_handler.max_exceeded() or not 
self.all_processes_running():
 break
 
@@ -2197,6 +2215,7 @@ class ImportProcess(ChildProcess):
 self.use_prepared_statements = options.copy['preparedstatements']
 self.max_inflight_messages = options.copy['maxinflightmessages']
 self.max_backoff_attempts = options.copy['maxbackoffattempts']
+self.request_timeout = options.copy['requesttimeout']
 
 self.dialect_options = options.dialect
 self._session = None
@@ -2223,7 +2242,7 @@ class ImportProcess(ChildProcess):
 

[2/6] cassandra git commit: Abort cqlsh copy-from in case of no answer after prolonged period of time

2016-10-10 Thread stefania
Abort cqlsh copy-from in case of no answer after prolonged period of time

Patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-12740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/72c9eb2d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/72c9eb2d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/72c9eb2d

Branch: refs/heads/cassandra-3.X
Commit: 72c9eb2dc6732a1f20e769ae162f02e5766f397f
Parents: 8455286
Author: Stefania Alborghetti 
Authored: Mon Oct 3 08:51:16 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:25:03 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a517995..9f7fff8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Abort cqlsh copy-from in case of no answer after prolonged period of time 
(CASSANDRA-12740)
  * Avoid sstable corrupt exception due to dropped static column 
(CASSANDRA-12582)
  * Make stress use client mode to avoid checking commit log size on startup 
(CASSANDRA-12478)
  * Fix exceptions with new vnode allocation (CASSANDRA-12715)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/72c9eb2d/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 23e739d..d2084d7 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -43,6 +43,7 @@ from select import select
 from uuid import UUID
 from util import profile_on, profile_off
 
+from cassandra import OperationTimedOut
 from cassandra.cluster import Cluster, DefaultConnection
 from cassandra.cqltypes import ReversedType, UserType
 from cassandra.metadata import protect_name, protect_names, protect_value
@@ -362,6 +363,11 @@ class CopyTask(object):
 copy_options['maxinflightmessages'] = 
int(opts.pop('maxinflightmessages', '512'))
 copy_options['maxbackoffattempts'] = 
int(opts.pop('maxbackoffattempts', '12'))
 copy_options['maxpendingchunks'] = int(opts.pop('maxpendingchunks', 
'24'))
+# set requesttimeout to a value high enough so that maxbatchsize rows 
will never timeout if the server
+# responds: here we set it to 1 sec per 10 rows but no less than 60 
seconds
+copy_options['requesttimeout'] = int(opts.pop('requesttimeout', 
max(60, 1 * copy_options['maxbatchsize'] / 10)))
+# set childtimeout higher than requesttimeout so that child processes 
have a chance to report request timeouts
+copy_options['childtimeout'] = int(opts.pop('childtimeout', 
copy_options['requesttimeout'] + 30))
 
 self.check_options(copy_options)
 return CopyOptions(copy=copy_options, dialect=dialect_options, 
unrecognized=opts)
@@ -1186,9 +1192,21 @@ class ImportTask(CopyTask):
 if not self.fname:
 self.send_stdin_rows()
 
+child_timeout = self.options.copy['childtimeout']
+last_recv_num_records = 0
+last_recv_time = time.time()
+
 while self.feeding_result is None or self.receive_meter.total_records 
< self.feeding_result.sent:
 self.receive_results()
 
+if self.feeding_result is not None:
+if self.receive_meter.total_records != last_recv_num_records:
+last_recv_num_records = self.receive_meter.total_records
+last_recv_time = time.time()
+elif (time.time() - last_recv_time) > child_timeout:
+self.shell.printerr("No records inserted in {} seconds, 
aborting".format(child_timeout))
+break
+
 if self.error_handler.max_exceeded() or not 
self.all_processes_running():
 break
 
@@ -2197,6 +2215,7 @@ class ImportProcess(ChildProcess):
 self.use_prepared_statements = options.copy['preparedstatements']
 self.max_inflight_messages = options.copy['maxinflightmessages']
 self.max_backoff_attempts = options.copy['maxbackoffattempts']
+self.request_timeout = options.copy['requesttimeout']
 
 self.dialect_options = options.dialect
 self._session = None
@@ -2223,7 +2242,7 @@ class ImportProcess(ChildProcess):
 connection_class=ConnectionWrapper)
 
 self._session = cluster.connect(self.ks)
-self._session.default_timeout = None
+self._session.default_timeout = 

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

2016-10-10 Thread stefania
Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e0a35e58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e0a35e58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e0a35e58

Branch: refs/heads/trunk
Commit: e0a35e585230d4072e59b1d6c5caa5a3ee977991
Parents: 3003d21 72c9eb2
Author: Stefania Alborghetti 
Authored: Tue Oct 11 11:26:24 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:26:24 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0a35e58/CHANGES.txt
--
diff --cc CHANGES.txt
index f4dda82,9f7fff8..00ff1eb
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,82 -1,5 +1,83 @@@
 -3.0.10
 +3.10
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Add row offset support to SASI (CASSANDRA-11990)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming (CASSANDRA-11841)
 + * Tracing payload is passed through newSession(..) (CASSANDRA-11706)
 + * avoid deleting non existing sstable files and improve related log messages 
(CASSANDRA-12261)
 + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486)
 + * Retry all internode messages once after a connection is
 +   closed and reopened (CASSANDRA-12192)
 + * Add support to rebuild from targeted replica (CASSANDRA-9875)
 + * Add sequence distribution type to cassandra stress (CASSANDRA-12490)
 + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154)
 + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474)
 + * Extend read/write failure messages with a map of replica addresses
 +   to error codes in the v5 native protocol (CASSANDRA-12311)
 + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
 + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 
12550)
 + * Fix clustering indexes in presence of static columns in SASI 
(CASSANDRA-12378)
 + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
 + * Added slow query log (CASSANDRA-12403)
 + * Count full coordinated request against timeout (CASSANDRA-12256)
 + * Allow TTL with null value on insert and update (CASSANDRA-12216)
 + * Make decommission operation resumable (CASSANDRA-12008)
 + * Add support to one-way targeted repair (CASSANDRA-9876)
 + * Remove clientutil jar (CASSANDRA-11635)
 + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717)
 + * Delay releasing Memtable memory on flush until PostFlush has finished 
running (CASSANDRA-12358)
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 + * Make it possible to compact a given token range (CASSANDRA-10643)
 + * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179)
 + * Collect metrics on queries by consistency level (CASSANDRA-7384)
 + * Add support for GROUP BY to SELECT statement (CASSANDRA-10707)
 + * Deprecate memtable_cleanup_threshold and update default for 
memtable_flush_writers (CASSANDRA-12228)
 + * Upgrade to OHC 0.4.4 (CASSANDRA-12133)
 + * Add version command to 

[6/6] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

2016-10-10 Thread stefania
Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9f167492
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9f167492
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9f167492

Branch: refs/heads/trunk
Commit: 9f167492621519c4a991804b271cb6445323a9c1
Parents: 07bc404 e0a35e5
Author: Stefania Alborghetti 
Authored: Tue Oct 11 11:26:53 2016 +0800
Committer: Stefania Alborghetti 
Committed: Tue Oct 11 11:26:53 2016 +0800

--
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 50 +
 2 files changed, 42 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9f167492/CHANGES.txt
--



[jira] [Comment Edited] (CASSANDRA-12767) Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue

2016-10-10 Thread Ye Liang (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564173#comment-15564173
 ] 

Ye Liang edited comment on CASSANDRA-12767 at 10/11/16 2:06 AM:


Thank you very much,I specify a server-side timestamp generator to solve this 
problem.
I notice that when i use a client-side timestamp generator,the cassandra driver 
still attach a client-side timestamp to my insert request(with ifnotexist),i 
want to know when will this timestamp be replaced to a server-side timestamp.I 
think it will be replaced in cassandra server side,if true,can you tell me the 
source code segment about this in Cassandra2.1.15 source code?


was (Author: lyss):
Thank you very much,I specify a server-side timestamp generator to solve this 
problem.
I notice that when i use a client-side timestamp generator,the cassandra driver 
still attach a server-side timestamp to my insert request(with ifnotexist),i 
want to know when will this timestamp be replaced to a server-side timestamp.I 
think it will be replaced in cassandra server side,if true,can you tell me the 
source code segment about this in Cassandra2.1.15 source code?

> Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue
> --
>
> Key: CASSANDRA-12767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12767
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.15
> Cassandra Java Driver 3.1.0
>Reporter: Ye Liang
>Priority: Minor
>
> I use cassandra driver java to do some insert operation.I notice that 
> Cassandra Java Driver use client-side timestamp as default.
> I make my client server one hour ahead than my cassandra server,then i insert 
> some record to an exist table and use sstable2json tool to check my record ' 
> s timestamp.
> i find out that if i insert a record with ifnotexist,the record's timestamp 
> used server-side timestamp,otherwise it use client-side timestamp.
> I think this is a very strange result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12767) Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue

2016-10-10 Thread Ye Liang (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564173#comment-15564173
 ] 

Ye Liang commented on CASSANDRA-12767:
--

Thank you very much,I specify a server-side timestamp generator to solve this 
problem.
I notice that when i use a client-side timestamp generator,the cassandra driver 
still attach a server-side timestamp to my insert request(with ifnotexist),i 
want to know when will this timestamp be replaced to a server-side timestamp.I 
think it will be replaced in cassandra server side,if true,can you tell me the 
source code segment about this in Cassandra2.1.15 source code?

> Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue
> --
>
> Key: CASSANDRA-12767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12767
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.15
> Cassandra Java Driver 3.1.0
>Reporter: Ye Liang
>Priority: Minor
>
> I use cassandra driver java to do some insert operation.I notice that 
> Cassandra Java Driver use client-side timestamp as default.
> I make my client server one hour ahead than my cassandra server,then i insert 
> some record to an exist table and use sstable2json tool to check my record ' 
> s timestamp.
> i find out that if i insert a record with ifnotexist,the record's timestamp 
> used server-side timestamp,otherwise it use client-side timestamp.
> I think this is a very strange result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress

2016-10-10 Thread Ben Slater (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564148#comment-15564148
 ] 

Ben Slater commented on CASSANDRA-12490:


Yes, you're right resetting the counter to zero on setSeed() does result in the 
same row being generated over and over again (which does make me wonder how 
stress is respecting the distribution for the PK value but didn't investigate 
at this point). However, that is pretty easily fixed by having setSeed() set 
the counter to the supplied seed value. I think once we do this SEQ behaves 
very similarly to the other distributions.

I don't think it's correct that stress generates every value if the number of 
unique values it can generate is <= the number of values it is being asked to 
generate for a partition. This would only respect the distribution in the case 
of uniform distribution, however even then I don't think it's guaranteed to be 
completely uniform (and thus generate all values) from n samples of a 1..n 
distribution (you probably need to do many * n to get very close to uniform) - 
it certainly doesn't seem to behave this way in testing. For say normal 
distribution you'd need several * n to cover all the possible values and have 
close to a normal distribution.

I afraid I don't really understand why you think this is abusing the notion of 
distributions when (a) there was already a sequence distribution type in the 
"legacy" distribution sets (presumably for just this purpose) and (b) to me, 
one way of describing this is a uniform distribution with minimal chance of 
collisions (ie it's just another way for selecting values from a range).

Finally, it's not quite correct to say I'm trying to populate all possible 
values for a column, rather trying to generate as many unique values as 
possible (within the specified ranges) for a given sample size (to minimise 
overwriting).

> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds. 
> This ensures generated values don't overlap (unless the sequence wraps) 
> providing more predictable number of inserted records (and generating a base 
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available. 
> It think it would be useful to have this for doing initial load of data for 
> testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (CASSANDRA-12767) Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue

2016-10-10 Thread Ye Liang (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ye Liang resolved CASSANDRA-12767.
--
Resolution: Not A Problem

> Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue
> --
>
> Key: CASSANDRA-12767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12767
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.15
> Cassandra Java Driver 3.1.0
>Reporter: Ye Liang
>Priority: Minor
>
> I use cassandra driver java to do some insert operation.I notice that 
> Cassandra Java Driver use client-side timestamp as default.
> I make my client server one hour ahead than my cassandra server,then i insert 
> some record to an exist table and use sstable2json tool to check my record ' 
> s timestamp.
> i find out that if i insert a record with ifnotexist,the record's timestamp 
> used server-side timestamp,otherwise it use client-side timestamp.
> I think this is a very strange result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12629) All Nodes Replication Strategy

2016-10-10 Thread Jeremiah Jordan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564136#comment-15564136
 ] 

Jeremiah Jordan commented on CASSANDRA-12629:
-

Also I'm fairly sure this patch as is will not behave correctly when used with 
"nodetool repair -pr" to actually split token range across all the nodes right.

> All Nodes Replication Strategy
> --
>
> Key: CASSANDRA-12629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12629
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alwyn Davis
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 12629-trunk.patch
>
>
> When adding a new DC, keyspaces must be manually updated to replicate to the 
> new DC.  This is problematic for system_auth, as it cannot achieve LOCAL_ONE 
> consistency (for a non-cassandra user), until its replication options have 
> been updated on an existing node.
> Ideally, system_auth could be set to an "All Nodes strategy" that will 
> replicate it to all nodes, as they join the cluster.  It also removes the 
> need to update the replication factor for system_auth when adding nodes to 
> the cluster to keep with the recommendation of RF=number of nodes (at least 
> for small clusters).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12629) All Nodes Replication Strategy

2016-10-10 Thread Jeremiah Jordan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15564131#comment-15564131
 ] 

Jeremiah Jordan commented on CASSANDRA-12629:
-

AllStrategy is actually bad to use for this. Auth caches stuff to make the 
queries faster, and if you do login with the default Cassandra user that uses 
QUORUM then you will rarely succeed at logging in if you have a ton of nodes.
There is a reason DataStax never contributed their EverywhereStrategy to OSS, 
it is very easy to screw yourself with it.

> All Nodes Replication Strategy
> --
>
> Key: CASSANDRA-12629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12629
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alwyn Davis
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 12629-trunk.patch
>
>
> When adding a new DC, keyspaces must be manually updated to replicate to the 
> new DC.  This is problematic for system_auth, as it cannot achieve LOCAL_ONE 
> consistency (for a non-cassandra user), until its replication options have 
> been updated on an existing node.
> Ideally, system_auth could be set to an "All Nodes strategy" that will 
> replicate it to all nodes, as they join the cluster.  It also removes the 
> need to update the replication factor for system_auth when adding nodes to 
> the cluster to keep with the recommendation of RF=number of nodes (at least 
> for small clusters).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-12770) Token() CQL Function To Be Used Without Args

2016-10-10 Thread Russell Spitzer (JIRA)
Russell Spitzer created CASSANDRA-12770:
---

 Summary: Token() CQL Function To Be Used Without Args
 Key: CASSANDRA-12770
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12770
 Project: Cassandra
  Issue Type: Improvement
  Components: CQL
Reporter: Russell Spitzer


The Spark Cassandra Connector uses statments like
{code}
SELECT ... WHERE TOKEN(partitionkey) < #
{code}

But I was considering today, why should we pass the partition key names to the 
Token Function?

If we look at the TokenRestriction
https://github.com/apache/cassandra/blob/81f6c784ce967fadb6ed7f58de1328e713eaf53c/src/java/org/apache/cassandra/cql3/TokenRelation.java#L174-L184

The restrictions on the token function basically mean there is only ever 1 
valid invocation of {{Token}}
{code}
Token(partitionkey1, partitionkey2, partitionkey3)
{code}

Now there can be an argument that this is helpful to folks matching {{Tokens}} 
together

{code}
Token(partitionkey1, partitionkey2, partitionkey3) < Token(1,1,3)
{code}

But if we have a single literal on the right hand side
{code}
Token(partitionkey1, partitionkey2, partitionkey3) < 10
{code}
The invocation just seems to have a lot of extra syntax but no benefit.

It would be great if it looked like this instead
{code}
Token() > 0 OR Token() < 3
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11968) More metrics on native protocol requests & responses

2016-10-10 Thread sankalp kohli (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563751#comment-15563751
 ] 

sankalp kohli commented on CASSANDRA-11968:
---

[~snazy] Any updates here.

> More metrics on native protocol requests & responses
> 
>
> Key: CASSANDRA-11968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11968
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> Proposal to add more metrics to the native protocol:
> - number of requests per request-type
> - number of responses by response-type
> - size of request messages in bytes
> - size of response messages in bytes
> - number of in-flight requests (from request arrival to response)
> (Will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[3/5] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84552867
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84552867
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84552867

Branch: refs/heads/trunk
Commit: 84552867b2ad920bbdd648c45423ff4d4d8b75d7
Parents: 2f0e365 c5fdb32
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:44 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:44 2016 -0500

--

--




[2/5] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5fdb32c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5fdb32c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5fdb32c

Branch: refs/heads/trunk
Commit: c5fdb32ceef796514cd0679f425e68c120a3e06e
Parents: be6e6ea 83ae5f3
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:10 2016 -0500

--

--




[1/5] cassandra git commit: Bump version to 2.1.17

2016-10-10 Thread mshuler
Repository: cassandra
Updated Branches:
  refs/heads/trunk be2b8cdee -> 07bc404ff


Bump version to 2.1.17


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83ae5f33
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83ae5f33
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83ae5f33

Branch: refs/heads/trunk
Commit: 83ae5f3398c408e3e28abeb7615f7efb5922ec47
Parents: 87034cd
Author: Michael Shuler 
Authored: Mon Oct 10 17:09:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:09:10 2016 -0500

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/build.xml
--
diff --git a/build.xml b/build.xml
index 8e9f613..f949aec 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index a9a04d2..4bd30ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.17) unstable; urgency=medium
+
+  * New release
+
+ -- Michael Shuler   Mon, 10 Oct 2016 17:07:44 -0500
+
 cassandra (2.1.16) unstable; urgency=medium
 
   * New release



[4/5] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

2016-10-10 Thread mshuler
Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3003d21a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3003d21a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3003d21a

Branch: refs/heads/trunk
Commit: 3003d21a95e9478cb727941e66705f8506c29b37
Parents: cbc7925 8455286
Author: Michael Shuler 
Authored: Mon Oct 10 17:11:15 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:11:15 2016 -0500

--

--




[4/4] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

2016-10-10 Thread mshuler
Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3003d21a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3003d21a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3003d21a

Branch: refs/heads/cassandra-3.X
Commit: 3003d21a95e9478cb727941e66705f8506c29b37
Parents: cbc7925 8455286
Author: Michael Shuler 
Authored: Mon Oct 10 17:11:15 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:11:15 2016 -0500

--

--




[2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5fdb32c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5fdb32c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5fdb32c

Branch: refs/heads/cassandra-3.X
Commit: c5fdb32ceef796514cd0679f425e68c120a3e06e
Parents: be6e6ea 83ae5f3
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:10 2016 -0500

--

--




[3/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84552867
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84552867
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84552867

Branch: refs/heads/cassandra-3.X
Commit: 84552867b2ad920bbdd648c45423ff4d4d8b75d7
Parents: 2f0e365 c5fdb32
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:44 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:44 2016 -0500

--

--




[1/4] cassandra git commit: Bump version to 2.1.17

2016-10-10 Thread mshuler
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.X cbc792531 -> 3003d21a9


Bump version to 2.1.17


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83ae5f33
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83ae5f33
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83ae5f33

Branch: refs/heads/cassandra-3.X
Commit: 83ae5f3398c408e3e28abeb7615f7efb5922ec47
Parents: 87034cd
Author: Michael Shuler 
Authored: Mon Oct 10 17:09:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:09:10 2016 -0500

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/build.xml
--
diff --git a/build.xml b/build.xml
index 8e9f613..f949aec 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index a9a04d2..4bd30ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.17) unstable; urgency=medium
+
+  * New release
+
+ -- Michael Shuler   Mon, 10 Oct 2016 17:07:44 -0500
+
 cassandra (2.1.16) unstable; urgency=medium
 
   * New release



[5/5] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

2016-10-10 Thread mshuler
Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/07bc404f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/07bc404f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/07bc404f

Branch: refs/heads/trunk
Commit: 07bc404ff48a57144c267ac8abf2f6f537dc432c
Parents: be2b8cd 3003d21
Author: Michael Shuler 
Authored: Mon Oct 10 17:11:48 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:11:48 2016 -0500

--

--




[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84552867
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84552867
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84552867

Branch: refs/heads/cassandra-3.0
Commit: 84552867b2ad920bbdd648c45423ff4d4d8b75d7
Parents: 2f0e365 c5fdb32
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:44 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:44 2016 -0500

--

--




[1/3] cassandra git commit: Bump version to 2.1.17

2016-10-10 Thread mshuler
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 2f0e365dc -> 84552867b


Bump version to 2.1.17


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83ae5f33
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83ae5f33
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83ae5f33

Branch: refs/heads/cassandra-3.0
Commit: 83ae5f3398c408e3e28abeb7615f7efb5922ec47
Parents: 87034cd
Author: Michael Shuler 
Authored: Mon Oct 10 17:09:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:09:10 2016 -0500

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/build.xml
--
diff --git a/build.xml b/build.xml
index 8e9f613..f949aec 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index a9a04d2..4bd30ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.17) unstable; urgency=medium
+
+  * New release
+
+ -- Michael Shuler   Mon, 10 Oct 2016 17:07:44 -0500
+
 cassandra (2.1.16) unstable; urgency=medium
 
   * New release



[1/2] cassandra git commit: Bump version to 2.1.17

2016-10-10 Thread mshuler
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 be6e6ea66 -> c5fdb32ce


Bump version to 2.1.17


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83ae5f33
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83ae5f33
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83ae5f33

Branch: refs/heads/cassandra-2.2
Commit: 83ae5f3398c408e3e28abeb7615f7efb5922ec47
Parents: 87034cd
Author: Michael Shuler 
Authored: Mon Oct 10 17:09:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:09:10 2016 -0500

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/build.xml
--
diff --git a/build.xml b/build.xml
index 8e9f613..f949aec 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index a9a04d2..4bd30ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.17) unstable; urgency=medium
+
+  * New release
+
+ -- Michael Shuler   Mon, 10 Oct 2016 17:07:44 -0500
+
 cassandra (2.1.16) unstable; urgency=medium
 
   * New release



[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5fdb32c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5fdb32c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5fdb32c

Branch: refs/heads/cassandra-2.2
Commit: c5fdb32ceef796514cd0679f425e68c120a3e06e
Parents: be6e6ea 83ae5f3
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:10 2016 -0500

--

--




[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2016-10-10 Thread mshuler
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5fdb32c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5fdb32c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5fdb32c

Branch: refs/heads/cassandra-3.0
Commit: c5fdb32ceef796514cd0679f425e68c120a3e06e
Parents: be6e6ea 83ae5f3
Author: Michael Shuler 
Authored: Mon Oct 10 17:10:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:10:10 2016 -0500

--

--




cassandra git commit: Bump version to 2.1.17

2016-10-10 Thread mshuler
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 87034cd05 -> 83ae5f339


Bump version to 2.1.17


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83ae5f33
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83ae5f33
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83ae5f33

Branch: refs/heads/cassandra-2.1
Commit: 83ae5f3398c408e3e28abeb7615f7efb5922ec47
Parents: 87034cd
Author: Michael Shuler 
Authored: Mon Oct 10 17:09:10 2016 -0500
Committer: Michael Shuler 
Committed: Mon Oct 10 17:09:10 2016 -0500

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/build.xml
--
diff --git a/build.xml b/build.xml
index 8e9f613..f949aec 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

http://git-wip-us.apache.org/repos/asf/cassandra/blob/83ae5f33/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index a9a04d2..4bd30ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.17) unstable; urgency=medium
+
+  * New release
+
+ -- Michael Shuler   Mon, 10 Oct 2016 17:07:44 -0500
+
 cassandra (2.1.16) unstable; urgency=medium
 
   * New release



[jira] [Updated] (CASSANDRA-12765) SSTable ignored incorrectly with row level tombstone

2016-10-10 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-12765:
-
Reproduced In: 2.1.15, 2.0.17, 3.x  (was: 2.0.17, 2.1.15, 3.x)
   Status: Patch Available  (was: Open)

> SSTable ignored incorrectly with row level tombstone
> 
>
> Key: CASSANDRA-12765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12765
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Cameron Zemek
> Attachments: 12765.patch
>
>
> {noformat}
> CREATE TABLE test.payload(
>   bucket_id TEXT,
>   name TEXT,
>   data TEXT,
>   PRIMARY KEY (bucket_id, name)
> );
> insert into test.payload (bucket_id, name, data) values 
> ('8772618c9009cf8f5a5e0c18', 'test', 'hello');
> {noformat}
> Flush nodes (nodetool flush)
> {noformat}
> insert into test.payload (bucket_id, name, data) values 
> ('8772618c9009cf8f5a5e0c19', 'test2', 'hello');
> delete from test.payload where bucket_id = '8772618c9009cf8f5a5e0c18';
> {noformat}
> Flush nodes (nodetool flush)
> {noformat}
> select * from test.payload where bucket_id = '8772618c9009cf8f5a5e0c18' and 
> name = 'test';
> {noformat}
> Expected 0 rows but get 1 row back.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error

2016-10-10 Thread Jeff Jirsa (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12014:
---
Reviewer: Jeff Jirsa

> IndexSummary > 2G causes an assertion error
> ---
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444  
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should 
> have a better method of coping with this than throwing the AE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


svn commit: r1764181 - in /cassandra/site: publish/download/index.html src/_data/releases.yaml

2016-10-10 Thread mshuler
Author: mshuler
Date: Mon Oct 10 21:37:45 2016
New Revision: 1764181

URL: http://svn.apache.org/viewvc?rev=1764181=rev
Log:
2.1.16 release

Modified:
cassandra/site/publish/download/index.html
cassandra/site/src/_data/releases.yaml

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1764181=1764180=1764181=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Mon Oct 10 21:37:45 2016
@@ -107,7 +107,7 @@ released against the most recent bug fix
   Apache Cassandra 3.0 is supported until May 2017. The 
latest release is http://www.apache.org/dyn/closer.lua/cassandra/3.0.9/apache-cassandra-3.0.9-bin.tar.gz;>3.0.9
 (http://www.apache.org/dist/cassandra/3.0.9/apache-cassandra-3.0.9-bin.tar.gz.asc;>pgp,
 http://www.apache.org/dist/cassandra/3.0.9/apache-cassandra-3.0.9-bin.tar.gz.md5;>md5
 and http://www.apache.org/dist/cassandra/3.0.9/apache-cassandra-3.0.9-bin.tar.gz.sha1;>sha1),
 released on 2016-09-20.
   Apache Cassandra 2.2 is supported until November 2016. 
The latest release is http://www.apache.org/dyn/closer.lua/cassandra/2.2.8/apache-cassandra-2.2.8-bin.tar.gz;>2.2.8
 (http://www.apache.org/dist/cassandra/2.2.8/apache-cassandra-2.2.8-bin.tar.gz.asc;>pgp,
 http://www.apache.org/dist/cassandra/2.2.8/apache-cassandra-2.2.8-bin.tar.gz.md5;>md5
 and http://www.apache.org/dist/cassandra/2.2.8/apache-cassandra-2.2.8-bin.tar.gz.sha1;>sha1),
 released on 2016-09-28.
   Apache Cassandra 2.1 is supported until November 2016 
with critical fixes only. The latest release is
-http://www.apache.org/dyn/closer.lua/cassandra/2.1.15/apache-cassandra-2.1.15-bin.tar.gz;>2.1.15
 (http://www.apache.org/dist/cassandra/2.1.15/apache-cassandra-2.1.15-bin.tar.gz.asc;>pgp,
 http://www.apache.org/dist/cassandra/2.1.15/apache-cassandra-2.1.15-bin.tar.gz.md5;>md5
 and http://www.apache.org/dist/cassandra/2.1.15/apache-cassandra-2.1.15-bin.tar.gz.sha1;>sha1),
 released on 2016-07-05.
+http://www.apache.org/dyn/closer.lua/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz;>2.1.16
 (http://www.apache.org/dist/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc;>pgp,
 http://www.apache.org/dist/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.md5;>md5
 and http://www.apache.org/dist/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.sha1;>sha1),
 released on 2016-10-10.
 
 
 Older (unsupported) versions of Cassandra are http://archive.apache.org/dist/cassandra/;>archived here.

Modified: cassandra/site/src/_data/releases.yaml
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/_data/releases.yaml?rev=1764181=1764180=1764181=diff
==
--- cassandra/site/src/_data/releases.yaml (original)
+++ cassandra/site/src/_data/releases.yaml Mon Oct 10 21:37:45 2016
@@ -15,5 +15,5 @@ latest:
   date: 2016-09-28
 
 "2.1":
-  name: 2.1.15
-  date: 2016-07-05
+  name: 2.1.16
+  date: 2016-10-10




[jira] [Commented] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Tom van der Woerdt (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563347#comment-15563347
 ] 

Tom van der Woerdt commented on CASSANDRA-12764:


Oh, nice to finally link the IRC name to the Jira name :)

Yes, it was a lot faster. Here's a graph showing what happened the last four 
days: https://i.imgur.com/AdWCCrR.png (graphing inode usage, divide by 8 for 
sstable count)

The red line is the node that started the mess. A botched repair[1] caused a 
nice 100k sstables. This was noticed, and cleaned up.

Sadly it had already synced those 100k sstables to other nodes, which properly 
started compacting the large amounts of files away. But then the regular 
automation jobs started a repair on the node I wiped, streaming all the files 
all over the place :( Sadly I was unaware of this until it was too late, and 
suddenly a lot of nodes on the cluster had 100k sstables :)

The sstable count was slowly going down (very, very slowly) but I figured I'd 
hop on IRC where [~jjirsa] and [~brandon.williams] helped find a workaround 
(the table move). I applied it to the most broken node first. On the graph it's 
the red line, look for the slope at the 10/10 boundary. This morning my script 
broke and it did the final sstables the slow route, but it finished and as you 
can see the scripted version is much faster than just letting compaction run. 
I'm in the progress of applying it to the two most broken nodes now, and will 
let the others just finish.

Anyway, that's the story of how this happened, which was totally my fault :) 
Now I'm just hoping that my mistake can lead to improvements in compaction 
performance.

Tom


[1]: subrange repair (similar to BrianGallew's code) on a LCS table, with 256 
vnodes, and most data not passing validation.

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> 

[jira] [Updated] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Paulo Motta (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12740:

Status: Ready to Commit  (was: Patch Available)

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12740) cqlsh copy tests hang in case of no answer from the server or driver

2016-10-10 Thread Paulo Motta (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563290#comment-15563290
 ] 

Paulo Motta commented on CASSANDRA-12740:
-

Patch and tests look good. Will mark as ready to commit. Before committing, can 
you modify the following comment to mention PYTHON-652:
{noformat}
# occasionally the driver sends false timeouts for rows already processed 
(PYTHON-652)
{noformat}

Thank you!

> cqlsh copy tests hang in case of no answer from the server or driver
> 
>
> Key: CASSANDRA-12740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12740
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
>
> -If we bundle the driver to cqlsh using the 3.6.0 tag or cassandra_test head, 
> some cqlsh copy tests hang, for example {{test_bulk_round_trip_blogposts}}. 
> See CASSANDRA-12736 and CASSANDRA-11534 for some sample failures.-
> If the driver fails to invoke a callback (either error or success), or if the 
> server never answers to the driver, then the copy parent process will wait 
> forever to receive an answer from child processes. We should put a cap to 
> this. We should also use a very high timeout rather than None, so that the 
> driver will notify us if there is no answer from the server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563295#comment-15563295
 ] 

Brandon Williams commented on CASSANDRA-12764:
--

bq. I ended up going for a solution along the lines of "move all data files 
out, then in batches of 500 sstables move them back in, run 'nodetool refresh', 
wait for compaction, then move the next batch".

FWIW, that was my suggestion, and I recall was about 25 times faster.

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at 

[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Brian Hess (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563258#comment-15563258
 ] 

 Brian Hess commented on CASSANDRA-7296:


Consistency Level does feel like the right approach. The THIS_ prefix is in 
line with LOCAL_ in that it would identify the locus of nodes that are 
available for consistency. With LOCAL_ONE, we need just one replica from the 
data center of this coordinator. If no replicas exist (like the RF=0) then you 
get UnavailableException. Namely, you don't reach out to other nodes and proxy 
for another DC, etc. Also note that while the client can certainly see that 
it's talking to a DC with no RF by looking at system tables or driver API 
calls, we still throw the UnavailableException. 

In the THIS_ONE, we are saying that the locus of available nodes for 
consistency level is just the coordonator itself. If that node is not a 
replica, then it should also throw an UnavailableException. It should not 
silently go ask the actual replicas, just like in the LOCAL_ONE case we don't 
ask other DCs. While it is true that the client could know that this node is 
not a replica, it is the same as in LOCAL_ONE and RF. 

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12763) Compaction performance issues when a table has a lot of sstables

2016-10-10 Thread Jeremy Hanna (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563250#comment-15563250
 ] 

Jeremy Hanna commented on CASSANDRA-12763:
--

In terms of the problem being with directory listings, [~brandon.williams] 
mentioned that we could probably even skip the directory listing when doing 
user-defined compactions and just trust the operator.  From there, I was 
curious as to whether a directory cache would cause any other issues if the 
information was ever stale.  Do you have any thoughts on that optimization or 
current problem with the directory listing [~krummas]?

> Compaction performance issues when a table has a lot of sstables
> 
>
> Key: CASSANDRA-12763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12763
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:269" #7536 daemon prio=1 os_prio=4 tid=0x7f4acd40fc00 
> nid=0x14f8 runnable [0x7f4798436000]
>java.lang.Thread.State: RUNNABLE
>   at java.io.UnixFileSystem.list(Native Method)
>   at java.io.File.list(File.java:1122)
>   at java.io.File.listFiles(File.java:1248)
>   at 
> org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles(LogRecord.java:268)
>   at org.apache.cassandra.db.lifecycle.LogRecord.make(LogRecord.java:150)
>   at 
> org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:293)
>   at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:283)
>   at 
> org.apache.cassandra.db.lifecycle.LogTransaction.obsoleted(LogTransaction.java:158)
>   at 
> org.apache.cassandra.db.lifecycle.Helpers.prepareForObsoletion(Helpers.java:134)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doPrepare(LifecycleTransaction.java:193)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:376)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> listFiles is being called over and over, apparently scaling with the number 
> of files in the compaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-12763) Compaction performance issues when a table has a lot of sstables

2016-10-10 Thread Jeremy Hanna (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563250#comment-15563250
 ] 

Jeremy Hanna edited comment on CASSANDRA-12763 at 10/10/16 7:35 PM:


In terms of the problem being with directory listings, [~brandon.williams] 
mentioned that we could probably even skip the directory listing when doing 
user-defined compactions and just trust the operator.  From there (for general 
compactions), I was curious as to whether a directory cache would cause any 
other issues if the information was ever stale.  Do you have any thoughts on 
that optimization or current problem with the directory listing [~krummas]?


was (Author: jeromatron):
In terms of the problem being with directory listings, [~brandon.williams] 
mentioned that we could probably even skip the directory listing when doing 
user-defined compactions and just trust the operator.  From there, I was 
curious as to whether a directory cache would cause any other issues if the 
information was ever stale.  Do you have any thoughts on that optimization or 
current problem with the directory listing [~krummas]?

> Compaction performance issues when a table has a lot of sstables
> 
>
> Key: CASSANDRA-12763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12763
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:269" #7536 daemon prio=1 os_prio=4 tid=0x7f4acd40fc00 
> nid=0x14f8 runnable [0x7f4798436000]
>java.lang.Thread.State: RUNNABLE
>   at java.io.UnixFileSystem.list(Native Method)
>   at java.io.File.list(File.java:1122)
>   at java.io.File.listFiles(File.java:1248)
>   at 
> org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles(LogRecord.java:268)
>   at org.apache.cassandra.db.lifecycle.LogRecord.make(LogRecord.java:150)
>   at 
> org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:293)
>   at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:283)
>   at 
> org.apache.cassandra.db.lifecycle.LogTransaction.obsoleted(LogTransaction.java:158)
>   at 
> org.apache.cassandra.db.lifecycle.Helpers.prepareForObsoletion(Helpers.java:134)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doPrepare(LifecycleTransaction.java:193)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:376)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> listFiles is being called over and over, apparently scaling with the number 
> of files in the compaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12763) Compaction performance issues when a table has a lot of sstables

2016-10-10 Thread Jeremy Hanna (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563219#comment-15563219
 ] 

Jeremy Hanna commented on CASSANDRA-12763:
--

With some other context, it sounds like the smaller sstables may be created as 
a product of using LCS with 256 vnodes and then the repair creates the tiny 
sstables.  In that case it's probably better to use a smaller number of vnodes 
(perhaps 32) in Cassandra 3+ with CASSANDRA-7032 so you don't have as much skew 
and aren't as vulnerable to small sstables with repair.

Keep in mind that if you add a new datacenter, that 7032 relies on getting the 
replication factor to reduce the skew at smaller vnode numbers.  So you'd need 
to create a node in each rack in the new datacenter with 
{{auto_bootstrap=false}}, then you can create an empty keyspace and table with 
replication in that datacenter and add the rest of the nodes with the 7032 
algorithm.

> Compaction performance issues when a table has a lot of sstables
> 
>
> Key: CASSANDRA-12763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12763
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:269" #7536 daemon prio=1 os_prio=4 tid=0x7f4acd40fc00 
> nid=0x14f8 runnable [0x7f4798436000]
>java.lang.Thread.State: RUNNABLE
>   at java.io.UnixFileSystem.list(Native Method)
>   at java.io.File.list(File.java:1122)
>   at java.io.File.listFiles(File.java:1248)
>   at 
> org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles(LogRecord.java:268)
>   at org.apache.cassandra.db.lifecycle.LogRecord.make(LogRecord.java:150)
>   at 
> org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:293)
>   at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:283)
>   at 
> org.apache.cassandra.db.lifecycle.LogTransaction.obsoleted(LogTransaction.java:158)
>   at 
> org.apache.cassandra.db.lifecycle.Helpers.prepareForObsoletion(Helpers.java:134)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doPrepare(LifecycleTransaction.java:193)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:376)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> listFiles is being called over and over, apparently scaling with the number 
> of files in the compaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12042) Decouple messaging protocol versioning from individual message types

2016-10-10 Thread Joshua McKenzie (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12042:

Reviewer: Aleksey Yeschenko

> Decouple messaging protocol versioning from individual message types
> 
>
> Key: CASSANDRA-12042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12042
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Aleksey Yeschenko
>Assignee: Stefania
>Priority: Blocker
> Fix For: 4.0
>
>
> At the moment we have a single constant - 
> {{MessagingService.current_version}} defining serialization format for 
> *everything*, including every possible message type.
> In practice it means that any tiniest change to any message requires bumping 
> the global {{MessagingService}} version.
> This is problematic for several reasons, the primary of which is currently 
> the schema propagation barrier between differently versioned C* nodes. In 
> tick-tock world, it means that any change (say, to a read command message), 
> would require a messaging service bump, putting nodes on split versions of 
> the service, and making schema changes during this now considered minor 
> upgrade, impossible, which is not neat.
> I propose that starting with 4.0 we version all messages individually 
> instead, and separately version messaging service protocol itself - which 
> will basically amount to just framing, once CASSANDRA-8457 is completed.
> In practice, this might be implemented the following way:
> # We use an extra byte with each message to specify the version of that 
> particular message type encoding
> # Instead of relying on messaging service of the sending note (determining 
> which can be racy, especially during upgrades), we use that byte to determine 
> the version of the message during deserialisation
> # On sending side, we can use the gossipped version of Cassandra itself - not 
> the messaging service version - to determine the maximum supported message 
> type version of the destination node
> In the new world, I expect the framing protocol version to change very rarely 
> after 4.0, if ever, and most message types to change extremely rarely as 
> well, with schema, read, and write messages to change version most often.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12042) Decouple messaging protocol versioning from individual message types

2016-10-10 Thread Joshua McKenzie (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12042:

Assignee: Stefania

> Decouple messaging protocol versioning from individual message types
> 
>
> Key: CASSANDRA-12042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12042
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Aleksey Yeschenko
>Assignee: Stefania
>Priority: Blocker
> Fix For: 4.0
>
>
> At the moment we have a single constant - 
> {{MessagingService.current_version}} defining serialization format for 
> *everything*, including every possible message type.
> In practice it means that any tiniest change to any message requires bumping 
> the global {{MessagingService}} version.
> This is problematic for several reasons, the primary of which is currently 
> the schema propagation barrier between differently versioned C* nodes. In 
> tick-tock world, it means that any change (say, to a read command message), 
> would require a messaging service bump, putting nodes on split versions of 
> the service, and making schema changes during this now considered minor 
> upgrade, impossible, which is not neat.
> I propose that starting with 4.0 we version all messages individually 
> instead, and separately version messaging service protocol itself - which 
> will basically amount to just framing, once CASSANDRA-8457 is completed.
> In practice, this might be implemented the following way:
> # We use an extra byte with each message to specify the version of that 
> particular message type encoding
> # Instead of relying on messaging service of the sending note (determining 
> which can be racy, especially during upgrades), we use that byte to determine 
> the version of the message during deserialisation
> # On sending side, we can use the gossipped version of Cassandra itself - not 
> the messaging service version - to determine the maximum supported message 
> type version of the destination node
> In the new world, I expect the framing protocol version to change very rarely 
> after 4.0, if ever, and most message types to change extremely rarely as 
> well, with schema, read, and write messages to change version most often.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jeremiah Jordan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563081#comment-15563081
 ] 

Jeremiah Jordan commented on CASSANDRA-7296:


Just an FYI changes to the CL enum require changes to every driver as well. CL 
is a protocol level option, not part of a query string.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jon Haddad (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563059#comment-15563059
 ] 

Jon Haddad commented on CASSANDRA-7296:
---

It sounds like what you're suggesting is that *every* query setting be moved to 
CQL.  That's a different discussion altogether.  Currently settings that change 
how the protocol behaves go in the protocol, that's how Cassandra works now.  
Trying to change that behavior by starting with a single feature just leaves 
everyone with inconsistencies in how the driver itself behaves.  

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563043#comment-15563043
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 6:22 PM:
--

{quote}
I'm very opposed to the /*disable_snitch=true*/ syntax. We don't use that 
anywhere, and why would we want that to be part of the statement? Making it 
part of the statement removes the ability to disable dynamic snitch at a per 
query level, including it as part of CQL makes it per prepared statement.
It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.
{quote}

Again, this is how most (if not all databases do this). The reason is for RDBMS 
databases the API's are standard (like JDBC) and you can not add new 
functionality in the form of new methods at the driver level.

The win of CQL is it solves everything in the query language. Everything else 
takes something out of the language makes it more like thirft. It is now 
something that EVERY client driver must implement. 

This is why the consistency level makes sense as well because you can fit the 
need without making a new feature that all the clients must implement to get 
the functionality.

Another way to do this is make the options a clear part of the language:

https://msdn.microsoft.com/en-us/library/ms181714.aspx

This is essentially the same thing as /* */ the parser parses it and acts. It 
is only a matter of the syntax.


was (Author: appodictic):
{quote}
I'm very opposed to the /*disable_snitch=true*/ syntax. We don't use that 
anywhere, and why would we want that to be part of the statement? Making it 
part of the statement removes the ability to disable dynamic snitch at a per 
query level, including it as part of CQL makes it per prepared statement.
It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.
{quote}

Again, this is how most (if not all databases do this). The reason is for RDBMS 
databases the API's are standard (like JDBC) and you can not add new 
functionality in the form of new methods.

The win of CQL is it solves everything in the query language. Everything else 
takes something out of the language makes it more like thirft. It is now 
something that EVERY client driver must implement. 

This is why the consistency level makes sense as well because you can fit the 
need without making a new feature that all the clients must implement to get 
the functionality

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563043#comment-15563043
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 6:20 PM:
--

{quote}
I'm very opposed to the /*disable_snitch=true*/ syntax. We don't use that 
anywhere, and why would we want that to be part of the statement? Making it 
part of the statement removes the ability to disable dynamic snitch at a per 
query level, including it as part of CQL makes it per prepared statement.
It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.
{quote}

Again, this is how most (if not all databases do this). The reason is for RDBMS 
databases the API's are standard (like JDBC) and you can not add new 
functionality in the form of new methods.

The win of CQL is it solves everything in the query language. Everything else 
takes something out of the language makes it more like thirft. It is now 
something that EVERY client driver must implement. 

This is why the consistency level makes sense as well because you can fit the 
need without making a new feature that all the clients must implement to get 
the functionality


was (Author: appodictic):
{quote}
I'm very opposed to the /*disable_snitch=true*/ syntax. We don't use that 
anywhere, and why would we want that to be part of the statement? Making it 
part of the statement removes the ability to disable dynamic snitch at a per 
query level, including it as part of CQL makes it per prepared statement.
It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.
{quote}

Again, this is how most (if not all databases do this). The reason is for RDBMS 
databases the API's are standard (like JDBC) and you can not add new 
functionality in the form of new methods.

The point of CQL is it solves everything in the query language, every weird 
switch that takes something out of the language makes it more like thirft. It 
is now something that EVERY client drive must implement.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563043#comment-15563043
 ] 

Edward Capriolo commented on CASSANDRA-7296:


{quote}
I'm very opposed to the /*disable_snitch=true*/ syntax. We don't use that 
anywhere, and why would we want that to be part of the statement? Making it 
part of the statement removes the ability to disable dynamic snitch at a per 
query level, including it as part of CQL makes it per prepared statement.
It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.
{quote}

Again, this is how most (if not all databases do this). The reason is for RDBMS 
databases the API's are standard (like JDBC) and you can not add new 
functionality in the form of new methods.

The point of CQL is it solves everything in the query language, every weird 
switch that takes something out of the language makes it more like thirft. It 
is now something that EVERY client drive must implement.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Wei Deng (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-12764:
-
Labels: lcs  (was: )

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> IntervalTree shows 

[jira] [Updated] (CASSANDRA-12763) Compaction performance issues when a table has a lot of sstables

2016-10-10 Thread Wei Deng (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-12763:
-
Labels: lcs  (was: )

> Compaction performance issues when a table has a lot of sstables
> 
>
> Key: CASSANDRA-12763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12763
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>  Labels: lcs
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:269" #7536 daemon prio=1 os_prio=4 tid=0x7f4acd40fc00 
> nid=0x14f8 runnable [0x7f4798436000]
>java.lang.Thread.State: RUNNABLE
>   at java.io.UnixFileSystem.list(Native Method)
>   at java.io.File.list(File.java:1122)
>   at java.io.File.listFiles(File.java:1248)
>   at 
> org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles(LogRecord.java:268)
>   at org.apache.cassandra.db.lifecycle.LogRecord.make(LogRecord.java:150)
>   at 
> org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:293)
>   at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:283)
>   at 
> org.apache.cassandra.db.lifecycle.LogTransaction.obsoleted(LogTransaction.java:158)
>   at 
> org.apache.cassandra.db.lifecycle.Helpers.prepareForObsoletion(Helpers.java:134)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doPrepare(LifecycleTransaction.java:193)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:376)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> listFiles is being called over and over, apparently scaling with the number 
> of files in the compaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jeremiah Jordan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562992#comment-15562992
 ] 

Jeremiah Jordan commented on CASSANDRA-7296:


Besides what the client API looks like how would people expect this to behave 
if the coordinator is not a replica?  That decision may also affect how the API 
should look from a "least surprises" stand point. If the CL was "THIS_ONE" I 
would expect no data or possibly an UnavailableException (IIRC this is what you 
get from LOCAL_ in a DC with no replicas). If it was a flag called "prefer 
coordinator" of something then I would expect the request to be coordinated to 
replica nodes if the coordinator wasn't one.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Blake Eggleston (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562990#comment-15562990
 ] 

Blake Eggleston commented on CASSANDRA-7296:


bq. are there scenarios where you would query a non-replica and expect it to 
return nothing rather than proxy the request

It's more the guarantee that you're _definitely_ looking at the data on a 
certain node. If we proxy when non-replicas are queried then you can't be sure 
that you're looking at the data on a certain node. If you've made a mistake, 
and queried a non replica, you'll see data from a different node

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jon Haddad (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562951#comment-15562951
 ] 

Jon Haddad commented on CASSANDRA-7296:
---

To Blake's point, are there scenarios where you would query a non-replica and 
expect it to return nothing rather than proxy the request?

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jon Haddad (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562934#comment-15562934
 ] 

Jon Haddad commented on CASSANDRA-7296:
---

I'm very opposed to the /\*disable_snitch=true\*/ syntax.  We don't use that 
anywhere, and why would we want that to be part of the statement?  Making it 
part of the statement removes the ability to disable dynamic snitch at a _per 
query_ level, including it as part of CQL makes it per prepared statement.  

It's not like adding it to the protocol is any different than specifying 
consistency level or a write timestamp.  

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562910#comment-15562910
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 5:30 PM:
--

{quote}
stmt = session.prepare("SELECT * from tab where id = ?", 
consistency_level=ConsistencyLevel.ONE)
stmt.disable_dynamic_snitch()
{quote}

I think it would be better using more standard SQL for optimizations. This is 
the common way query hints are provided.

{quote}
stmt = session.prepare("SELECT /* disable_snitch=true */ * from tab where id = 
?", consistency_level=ConsistencyLevel.ONE)
{quote}

Providing extra methods like this seems thrift like. 
{quote}
stmt.disable_dynamic_snitch()
{quote}
This makes an API not a query language.



was (Author: appodictic):
{quote}
stmt = session.prepare("SELECT * from tab where id = ?", 
consistency_level=ConsistencyLevel.ONE)
stmt.disable_dynamic_snitch()
{quote}

I think it would be better using more standard SQL for optimizations. This is 
the common way query hints are provided.

{quote}
stmt = session.prepare("SELECT /*disable_snitch=true*/ * from tab where id = 
?", consistency_level=ConsistencyLevel.ONE)
{quote}

Providing extra methods like this seems thrift like. 
{quote}
stmt.disable_dynamic_snitch()
{quote}
This makes an API not a query language.


> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562910#comment-15562910
 ] 

Edward Capriolo commented on CASSANDRA-7296:


{quote}
stmt = session.prepare("SELECT * from tab where id = ?", 
consistency_level=ConsistencyLevel.ONE)
stmt.disable_dynamic_snitch()
{quote}

I think it would be better using more standard SQL for optimizations. This is 
the common way query hints are provided.

{quote}
stmt = session.prepare("SELECT /*disable_snitch=true*/ * from tab where id = 
?", consistency_level=ConsistencyLevel.ONE)
{quote}

Providing extra methods like this seems thrift like. 
{quote}
stmt.disable_dynamic_snitch()
{quote}
This makes an API not a query language.


> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Blake Eggleston (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562896#comment-15562896
 ] 

Blake Eggleston commented on CASSANDRA-7296:


Just to clarify, has the goal of the ticket changed to give operators the 
option to always include the coordinator in a read if it's a replica? The goal 
as stated in the ticket description is to give operators the option to perform 
a local only read against the coordinator they’ve connected to, and fail (or 
return nothing) if it's not a replica.

In the context of the original description, combining this option with CLs 
other than ONE doesn’t make much sense.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562870#comment-15562870
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 5:26 PM:
--

I think it makes sense as either but it really makes sense as a consistency 
level as well. 

THIS_ONE might be a better name. Other consistency levels do express WHERE you 
want something to happen:

Aren't we discussing adding consistency levels here?  
https://issues.apache.org/jira/browse/CASSANDRA-8119 

The difference between 8119 and this is that this is implemented in a patch, so 
a rational argument is to do this feature in the least intrusive way. 


was (Author: appodictic):
Is https://issues.apache.org/jira/browse/CASSANDRA-8119 a protocol option as 
well?

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562870#comment-15562870
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 5:27 PM:
--

I think it makes sense as either but it really makes sense as a consistency 
level as well. 

THIS_ONE might be a better name. Other consistency levels do express WHERE you 
want something to happen such as ANY:

Aren't we discussing adding consistency levels here?  
https://issues.apache.org/jira/browse/CASSANDRA-8119 

The difference between 8119 and this is that this is implemented in a patch, so 
a rational argument is to do this feature in the least intrusive way. 


was (Author: appodictic):
I think it makes sense as either but it really makes sense as a consistency 
level as well. 

THIS_ONE might be a better name. Other consistency levels do express WHERE you 
want something to happen:

Aren't we discussing adding consistency levels here?  
https://issues.apache.org/jira/browse/CASSANDRA-8119 

The difference between 8119 and this is that this is implemented in a patch, so 
a rational argument is to do this feature in the least intrusive way. 

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


svn commit: r16458 - in /release/cassandra: 2.1.16/ debian/dists/21x/ debian/dists/21x/main/binary-amd64/ debian/dists/21x/main/binary-i386/ debian/dists/21x/main/source/ debian/pool/main/c/cassandra/

2016-10-10 Thread mshuler
Author: mshuler
Date: Mon Oct 10 17:22:57 2016
New Revision: 16458

Log:
Apache Cassandra 2.1.16 Release

Added:
release/cassandra/2.1.16/
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz   (with props)
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.md5
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.sha1
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.md5
release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.sha1
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz   (with props)
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz.asc
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz.asc.md5
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz.asc.sha1
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz.md5
release/cassandra/2.1.16/apache-cassandra-2.1.16-src.tar.gz.sha1

release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.1.16_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.16.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.16.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.16.orig.tar.gz 
  (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.16.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.16_all.deb   
(with props)
Modified:
release/cassandra/debian/dists/21x/InRelease
release/cassandra/debian/dists/21x/Release
release/cassandra/debian/dists/21x/Release.gpg
release/cassandra/debian/dists/21x/main/binary-amd64/Packages
release/cassandra/debian/dists/21x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/21x/main/binary-amd64/Release
release/cassandra/debian/dists/21x/main/binary-i386/Packages
release/cassandra/debian/dists/21x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/21x/main/binary-i386/Release
release/cassandra/debian/dists/21x/main/source/Release
release/cassandra/debian/dists/21x/main/source/Sources.gz

Added: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc
==
--- release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc (added)
+++ release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc Mon Oct 10 
17:22:57 2016
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAABCAAGBQJX9YUxAAoJEKJ4t4H+SyvaVVUP/RFgMZYw4q7Kj4A4tVc8XSG7
+SxqVRl7VkhyW6wKgyTlxmpAtrYSKhaS6p0w6X5dVYYl+XkvguDh4F3Coo0MTe5G2
+IaA+fBBZaFvvZ+tjgm8154p00ufq6e6ZjZZdEFNQ6wMCY8Z96Py9SNvVeofvm83Y
+8pPL4aRUU3zy2V0L5j1J564BsaKgdnpVNTKZMSbPpgcf9t/53y9r/4+RzztmYkp2
+1KiRNfki7Ww7RbDxp3RRyFp03BIO2UdYMQVizTwk0BpChZWR8cun1/MCsvFq0Cmy
+KMVQ7K1B73KZOp2RZx2GLonDAH4R/bezpzH7d+u945ewpB+vumMt/CM2tgYMOuNb
+dXEagB0C5q7JyLmHUFk9fkAbmvmJ9GKjpjglgsE1wVHU2li4ZUOsGuJfbpUh03q+
+7rppLi+NznG/JTF9P6Ou1Q5I4DCPkdAabtJIbqmWMp8KYID7Rsn6v4xRLQiVPGb+
+h1VW9cTE00DFD5b185i7BhLwnihRXVXCvJpcqNLSE/J7uCsuZsHwayJLMB1TsU+g
+6A35Qwuod8IBREQQ4cVavwxlbIawfQOKLXNJhfXkUO3+GLUFfCBouweSUVlJ3RuK
+j0/ThcIwDTon2Veh9FO9wR6ChH6iOlWpTTLfh6BVcpoK/EcwfBxys+p3kuagbjgG
+P5UgE+R7HPA7vt6utev8
+=FiYC
+-END PGP SIGNATURE-

Added: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.md5
==
--- release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.md5 (added)
+++ release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.md5 Mon Oct 
10 17:22:57 2016
@@ -0,0 +1 @@
+544b8ff6fc155116ff54b5b21d6e5b34
\ No newline at end of file

Added: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.sha1
==
--- release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.sha1 (added)
+++ release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.asc.sha1 Mon 
Oct 10 17:22:57 2016
@@ -0,0 +1 @@
+9d6f73bb0bb0e8133e419b300a20df3940136645
\ No newline at end of file

Added: release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.md5
==
--- release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.md5 (added)
+++ release/cassandra/2.1.16/apache-cassandra-2.1.16-bin.tar.gz.md5 Mon Oct 10 
17:22:57 2016
@@ -0,0 +1 @@
+cc11eadd767e0200d412b7a0bde6a9f5
\ No newline at end of file

Added: 

[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562870#comment-15562870
 ] 

Edward Capriolo commented on CASSANDRA-7296:


Is https://issues.apache.org/jira/browse/CASSANDRA-8119 a protocol option as 
well?

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[cassandra] Git Push Summary

2016-10-10 Thread mshuler
Repository: cassandra
Updated Tags:  refs/tags/2.1.16-tentative [deleted] 87034cd05


[cassandra] Git Push Summary

2016-10-10 Thread mshuler
Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.1.16 [created] 4bb740272


[jira] [Updated] (CASSANDRA-10364) Improve test coverage for schema tables and document 3.0 changes to schema tables

2016-10-10 Thread Aleksey Yeschenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10364:
--
Assignee: (was: Aleksey Yeschenko)

> Improve test coverage for schema tables and document 3.0 changes to schema 
> tables
> -
>
> Key: CASSANDRA-10364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10364
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
> Fix For: 3.0.x
>
>
> In particular,
> {{LegacySchemaMigratorTest.java}}:
> Needed test coverage:
> Legacy schema tables are removed
> New schema tables are written to with the correct timestamp
> Legacy schema tables don't exist in new schema tables
> Migrating tables in general, especially COMPACT ones
> Null values for any optional fields
> Maybe UDTs that refer to other UDTs?
> NTS keyspaces
> Similar coverage is also important to have for {{SchemaKeyspace}}, with no 
> migration involved.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562845#comment-15562845
 ] 

Benedict commented on CASSANDRA-12764:
--

It's been a while since I looked at the major compaction code, but it *should* 
operate on all sstables at once, so that you should only have paid the costly 
finish() call precisely once.  That's kind of the point of it

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> 

[jira] [Comment Edited] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562845#comment-15562845
 ] 

Benedict edited comment on CASSANDRA-12764 at 10/10/16 5:13 PM:


It's been a while since I looked at the major compaction code, but it *should* 
operate on all sstables at once (that's kind of the point of it...), so that 
you should only have paid the costly finish() call precisely once.


was (Author: benedict):
It's been a while since I looked at the major compaction code, but it *should* 
operate on all sstables at once, so that you should only have paid the costly 
finish() call precisely once.  That's kind of the point of it

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> 

[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jon Haddad (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562824#comment-15562824
 ] 

Jon Haddad commented on CASSANDRA-7296:
---

{quote}But short of that, why not directly attack the problem we're trying to 
solve and add a (protocol) option to queries to force the behavior ("always 
pick the coordinator as one replica if it's one"). That sounds less confusing 
to me than have a new CL that will confuse newcomers (as the difference with 
ONE is somewhat subtle for a newcomer). As a bonus, it would also work for CL > 
ONE (since again, it'll just be about forcing the dynamic snitch to pick the 
coordinator if it's a replica).
{quote}

This is a reasonable alternative.  I'm not sure if it's useful outside of 
CL=ONE, but there's probably a use case I'm not thinking of.  

Using the Python driver would look something like this, I'm assuming:

{code}
stmt = session.prepare("SELECT * from tab where id = ?", 
consistency_level=ConsistencyLevel.ONE)
stmt.disable_dynamic_snitch()
session.execute(stmt, [1])
{code}

Plus a bit to direct the driver to a particular replica, which has to happen 
regardless. 


> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Tom van der Woerdt (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562821#comment-15562821
 ] 

Tom van der Woerdt commented on CASSANDRA-12764:


This node was running with a 30GB heap (yes, G1GC).

Stack traces showed that this is what Cassandra was spending a lot of time on 
-- I'd approximate it at 4 days spent doing finish()   (I didn't wait that 
long, of course, but as this grows linearly with number of readers, and it took 
~2 minutes per compaction of 32 sstables to finish, it would take 4 days for 
the full thing to finish)

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> 

[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jeff Jirsa (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562814#comment-15562814
 ] 

Jeff Jirsa commented on CASSANDRA-7296:
---

+1 in favor of protocol option, so users can apply it to other CLs as desired. 



> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12705) Add column definition kind to system schema dropped columns

2016-10-10 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562782#comment-15562782
 ] 

Aleksey Yeschenko commented on CASSANDRA-12705:
---

Don't know what it looked like in the original version, but in 
{{system_schema.columns}} we store the lowercase values of the enum names. Look 
at {{kind}} and {{clustering_order}} for examples. Would prefer it to be the 
same in {{system_schema.dropped_columns}}, for consistency.

Otherwise the patch LGTM.

> Add column definition kind to system schema dropped columns
> ---
>
> Key: CASSANDRA-12705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12705
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 4.0
>
>
> Both regular and static columns can currently be dropped by users, but this 
> information is currently not stored in {{SchemaKeyspace.DroppedColumns}}. As 
> a consequence, {{CFMetadata.getDroppedColumnDefinition}} returns a regular 
> column and this has caused problems such as CASSANDRA-12582.
> We should add the column kind to {{SchemaKeyspace.DroppedColumns}} so that 
> {{CFMetadata.getDroppedColumnDefinition}} can create the correct column 
> definition. However, altering schema tables would cause inter-node 
> communication failures during a rolling upgrade, see CASSANDRA-12236. 
> Therefore we should wait for a full schema migration when upgrading to the 
> next major version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jeremiah Jordan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562736#comment-15562736
 ] 

Jeremiah Jordan commented on CASSANDRA-7296:


+1 for protocol option, not new CL.

Also I think the removal of Severity from DynamicEndpointSnitch 
(CASSANDRA-11738) should reduce the times where it does something very screwy 
in picking replicas.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

2016-10-10 Thread Joel Knighton (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-12761:
--
Status: Awaiting Feedback  (was: Open)

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

2016-10-10 Thread Joel Knighton (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-12761:
--
Status: Open  (was: Patch Available)

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

2016-10-10 Thread Joel Knighton (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562731#comment-15562731
 ] 

Joel Knighton commented on CASSANDRA-12761:
---

+1 to the patch contents - if you could add a CHANGES.txt entry and the "patch 
by XXX; reviewed by YYY for CASSANDRA-ZZZ" line to the commit as described at 
[http://cassandra.apache.org/doc/latest/development/patches.html#creating-a-patch],
 that would be great.

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

2016-10-10 Thread Joel Knighton (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-12761:
--
Fix Version/s: 4.x
   3.x

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

2016-10-10 Thread Joel Knighton (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-12761:
--
Assignee: Guy Bolton King

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12767) Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue

2016-10-10 Thread Jeremy Hanna (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562712#comment-15562712
 ] 

Jeremy Hanna commented on CASSANDRA-12767:
--

{{ifnotexists}} uses a lightweight transaction within the cluster to provide 
some guarantees.  You can't do this with a client-side timestamp.  So for 
anything that doesn't require a server-side timestamp, it uses a client-side 
timestamp.

Generally though, it's best to do clock syncing (e.g. ntp) within the cluster 
and on client machines, so in practice it shouldn't matter very much either way.

> Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue
> --
>
> Key: CASSANDRA-12767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12767
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.15
> Cassandra Java Driver 3.1.0
>Reporter: Ye Liang
>Priority: Minor
>
> I use cassandra driver java to do some insert operation.I notice that 
> Cassandra Java Driver use client-side timestamp as default.
> I make my client server one hour ahead than my cassandra server,then i insert 
> some record to an exist table and use sstable2json tool to check my record ' 
> s timestamp.
> i find out that if i insert a record with ifnotexist,the record's timestamp 
> used server-side timestamp,otherwise it use client-side timestamp.
> I think this is a very strange result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562698#comment-15562698
 ] 

Edward Capriolo edited comment on CASSANDRA-7296 at 10/10/16 4:10 PM:
--

{quote}
Basically, despite this being arguably confusing to most, I'm not sure we have 
really quantified the advantage this brings us, which is a shame 
{quote}

It brings one key thing. The clients do logic to control where to route 
request, they do this because they want the lowest latency. We want the server 
to respect the brain power of the client and carry out the operation where it 
decided, not forward the request elsewhere like it (sometimes) does now 
incurring more latency on some requests and making them hard to debug.


was (Author: appodictic):
{quote}
Basically, despite this being arguably confusing to most, I'm not sure we have 
really quantified the advantage this brings us, which is a shame 
{quote}

It brings one key thing. The clients do logic to control where to route 
request, they do this because they want the lowest latency. We want the server 
to respect the brain power of the client and carry out the operation where it 
decided, not forward the request elsewhere like it (sometimes) does now.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562698#comment-15562698
 ] 

Edward Capriolo commented on CASSANDRA-7296:


{quote}
Basically, despite this being arguably confusing to most, I'm not sure we have 
really quantified the advantage this brings us, which is a shame 
{quote}

It brings one key thing. The clients do logic to control where to route 
request, they do this because they want the lowest latency. We want the server 
to respect the brain power of the client and carry out the operation where it 
decided, not forward the request elsewhere like it (sometimes) does now.

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562659#comment-15562659
 ] 

Sylvain Lebresne commented on CASSANDRA-7296:
-

bq. I'm concerned it will prove to be a step backwards in real clusters, where 
coordinator disk latencies may truly jump up up significantly

Right, by we also have rapid read protection now that might limit that problem 
reasonably well. But anyway, I'm not making any strong claim here, that's why I 
started my sentence by "I'm not even entirely sure". Basically, despite this 
being arguably confusing to most, I'm not sure we have really quantified the 
advantage this brings us, which is a shame (but it's not like I'm volunteering 
for experimenting here so ).

To clarify, my main point was that I dislike the idea of providing this through 
a new CL, and I'd rather have that being a protocol level query option (we have 
to change the protocol _anyway_).

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Jeff Jirsa (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562632#comment-15562632
 ] 

Jeff Jirsa commented on CASSANDRA-7296:
---

{quote}
First, I'm not even entirely sure than letting the dynamic snitch bypass the 
coordinator if it's a replica is a good idea in the first place. Everyone more 
or less agree that doing token-aware routing is a good thing nowadays, and it's 
certainly confusing that the dynamic snitch may screw that up. If the dynamic 
snitch was a perfect and instantaneous view of latencies, then that could make 
sense, but it's not. Anyway, I think it's worth at least evaluating making even 
the dynamic snitch always pick the local node if it's a replica, as I'm not 
sure the benefit of not doing so outweigh the confusion it creates.
{quote}

Emotionally, I want this to be the right answer (principle of least 
astonishment), but I don't think it is. I'm concerned it will prove to be a 
step backwards in real clusters, where coordinator disk latencies may truly 
jump up up significantly (imagine all compaction threads running scrub/cleanup, 
where not only is the disk likely completely utilized, but the # of sstables on 
disk grows because all compaction threads are in use, so reads are more 
expensive than normal - in this case, dsnitch DOES save us, and implementing 
this type of change would be very hard to work around in production with most 
drivers).



> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12768) CQL often queries static columns unnecessarily

2016-10-10 Thread Sylvain Lebresne (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-12768:
-
Status: Patch Available  (was: Open)

Patch attached below:
| [12768-3.0|https://github.com/pcmanus/cassandra/commits/12768-3.0] | 
[utests|http://cassci.datastax.com/job/pcmanus-12768-3.0-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-12768-3.0-dtest] |
| [12768-3.X|https://github.com/pcmanus/cassandra/commits/12768-3.X] | 
[utests|http://cassci.datastax.com/job/pcmanus-12768-3.X-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-12768-3.X-dtest] |

The 3.X patch is a tad bigger because 1) {{ColumnFilter}} has been refactored a 
bit since 3.0 and 2) I've tried to clean up naming a bit more than in 3.0 where 
I optimized more for less changes.


> CQL often queries static columns unnecessarily
> --
>
> Key: CASSANDRA-12768
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12768
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.x
>
>
> While looking at CASSANDRA-12694 (which isn't directly related, but some of 
> the results in this ticket are explained by this), I realized that CQL was 
> always querying static columns even in cases where this is unnecessary.
> More precisely, for reasons long described elsewhere, we have to query all 
> the columns for a row (we have optimizations, see CASSANDRA-10657, but they 
> don't change that general fact) to be able to distinguish between the case 
> where a row doesn't exist from when it exists but has no values for the 
> columns selected by the query. *However*, this really only extend to 
> "regular" columns (static columns play no role in deciding whether a 
> particular row exists or not) but the implementation in 3.x, which is in 
> {{ColumnFilter}}, still always query all static columns.
> We shouldn't do that and it's arguably a performance regression from 2.x. 
> Which is why I'm tentatively marking this a bug and for the 3.0 line. It's a 
> tiny bit scary for 3.0 though so really more asking for other opinions and 
> I'd be happy to stick to 3.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12754) Change cassandra.wait_for_tracing_events_timeout_secs default to -1 so C* doesn't wait on trace events to be written before responding to request by default

2016-10-10 Thread Paulo Motta (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12754:

Reproduced In: 3.0.9, 2.2.8  (was: 2.2.8, 3.0.9)
 Reviewer: Paulo Motta

> Change cassandra.wait_for_tracing_events_timeout_secs default to -1 so C* 
> doesn't wait on trace events to be written before responding to request by 
> default
> 
>
> Key: CASSANDRA-12754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12754
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Stefania
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> [CASSANDRA-11465] introduces a new system property 
> {{cassandra.wait_for_tracing_events_timeout_secs}} that controls whether or 
> not C* waits for events to be written before responding to client.   The 
> current default behavior is to wait up to 1 second and then respond and 
> timeout.  
> If using probabilistic tracing this can cause queries to be randomly delayed 
> up to 1 second.
> Changing the default to -1 (disabled and enabling it explicitly in 
> {{cql_tracing_test.TestCqlTracing.tracing_unknown_impl_test}}.
> Ideally it would be nice to be able to control this behavior on a per request 
> basis (which would require native protocol changes).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12268) Make MV Index creation robust for wide referent rows

2016-10-10 Thread T Jake Luciani (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562439#comment-15562439
 ] 

T Jake Luciani commented on CASSANDRA-12268:


Overall +1, I see you removed the iterator approach for live mutations and only 
use it for building MVs. Good idea since that would open a window for partial 
updates to MVs which would be bad.  Is there a follow up ticket you can open 
for the live mutations?  It should be rare to OOM live but things like range 
tombstones on a large partition would still OOM.  We need some way to make the 
batchlog updates transactional (at least locally)

The branches need a rebase and re-run the tests but if those are clean please 
commit! Thx.

> Make MV Index creation robust for wide referent rows
> 
>
> Key: CASSANDRA-12268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12268
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jonathan Shook
>Assignee: Carl Yeksigian
> Fix For: 3.0.x, 3.x
>
> Attachments: 12268.py
>
>
> When creating an index for a materialized view for extant data, heap pressure 
> is very dependent on the cardinality of of rows associated with each index 
> value. With the way that per-index value rows are created within the index, 
> this can cause unbounded heap pressure, which can cause OOM. This appears to 
> be a side-effect of how each index row is applied atomically as with batches.
> The commit logs can accumulate enough during the process to prevent the node 
> from being restarted. Given that this occurs during global index creation, 
> this can happen on multiple nodes, making stable recovery of a node set 
> difficult, as co-replicas become unavailable to assist in back-filling data 
> from commitlogs.
> While it is understandable that you want to avoid having relatively wide rows 
>  even in materialized views, this represents a particularly difficult 
> scenario for triage.
> The basic recommendation for improving this is to sub-group the index 
> creation into smaller chunks internally, providing a maximal bound against 
> the heap pressure when it is needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12694) PAXOS Update Corrupted empty row exception

2016-10-10 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562438#comment-15562438
 ] 

Sylvain Lebresne commented on CASSANDRA-12694:
--

Had a closer look at this. But first, I don't really agree that the new outputs 
from test are better. The case in question deletes a row if it exists. As the 
condition is entirely based on the row existence, I don't think making the 
output change based on partition-related information is necessary meaningful. 
In fact, we shouldn't *have* to read static content to even decide if a row 
exists or not.

In all fairness, we never carefully defined what the CAS result set output 
contained, so what's better or not is debatable, but I think we should 
preserver the output for 2 main reasons: 1) to not break backward compatibility 
(we shouldn't unless we think a behavior was really a bug and I don't think 
this can be said) and 2) forcing to query static content when the existence is 
on a row is inefficient and I'd rather not back that inefficient in.

Anyway, the reason we get this output difference is actually the genuine 
bug/inefficiency from CASSANDRA-12768: we have a condition on a row, so no 
reason to read any static content.


> PAXOS Update Corrupted empty row exception
> --
>
> Key: CASSANDRA-12694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12694
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: 3 node cluster using RF=3 running on cassandra 3.7
>Reporter: Cameron Zemek
>Assignee: Alex Petrov
>
> {noformat}
> cqlsh> create table test.test (test_id TEXT, last_updated TIMESTAMP, 
> message_id TEXT, PRIMARY KEY(test_id));
> update test.test set last_updated = 1474494363669 where test_id = 'test1' if 
> message_id = null;
> {noformat}
> Then nodetool flush on the all 3 nodes.
> {noformat}
> cqlsh> update test.test set last_updated = 1474494363669 where test_id = 
> 'test1' if message_id = null;
> ServerError: 
> {noformat}
> From cassandra log
> {noformat}
> ERROR [SharedPool-Worker-1] 2016-09-23 12:09:13,179 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0x7a22599e, 
> L:/127.0.0.1:9042 - R:/127.0.0.1:58297]
> java.io.IOError: java.io.IOException: Corrupt empty row found in unfiltered 
> partition
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:224)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:212)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:125)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:249)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:87) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.ReadResponse$DataResponse.digest(ReadResponse.java:192)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:80) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:139) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:145)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.awaitResultsAndRetryOnDigestMismatch(StorageProxy.java:1714)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1663) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1604) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1523) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.readOne(StorageProxy.java:1497) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.readOne(StorageProxy.java:1491) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:249) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:441)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:416)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:208)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:239) 
> ~[main/:na]

[jira] [Updated] (CASSANDRA-12769) An invalid insert request crashes the C* node

2016-10-10 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jérémi Kurzanski updated CASSANDRA-12769:
-
  Environment: osx, linux, windows, cassandra-driver-core-3.1.0  (was: osx, 
linux, windows)
Reproduced In: 3.9, 3.7  (was: 3.7, 3.9)

> An invalid insert request crashes the C* node
> -
>
> Key: CASSANDRA-12769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12769
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: osx, linux, windows, cassandra-driver-core-3.1.0
>Reporter: Jérémi Kurzanski
> Attachments: CrashNode.java
>
>
> The node crash with a OOM error when it receives an invalid insert including 
> a plain string instead of a List. 
> Here is an extract of the log : 
> {code}
> ERROR [Native-Transport-Requests-1] 2016-10-10 15:51:57,096 
> JVMStabilityInspector.java:141 - JVM state determined to be unstable.  
> Exiting forcefully due to:
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.(ArrayList.java:152) ~[na:1.8.0_60]
> at 
> org.apache.cassandra.serializers.ListSerializer.deserializeForNativeProtocol(ListSerializer.java:87)
>  ~[apache-cassandra-3.9.jar:3.9]
> at 
> org.apache.cassandra.cql3.Lists$Value.fromSerialized(Lists.java:150) 
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.Lists$Marker.bind(Lists.java:255) 
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.Lists$Setter.execute(Lists.java:308) 
> ~[apache-cassandra-3.9.jar:3.9]
> {code}
> Please find enclosed a unitTest to reproduce the problem. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-12769) An invalid insert request crashes the C* node

2016-10-10 Thread JIRA
Jérémi Kurzanski created CASSANDRA-12769:


 Summary: An invalid insert request crashes the C* node
 Key: CASSANDRA-12769
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12769
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: osx, linux, windows
Reporter: Jérémi Kurzanski
 Attachments: CrashNode.java

The node crash with a OOM error when it receives an invalid insert including a 
plain string instead of a List. 

Here is an extract of the log : 
{code}
ERROR [Native-Transport-Requests-1] 2016-10-10 15:51:57,096 
JVMStabilityInspector.java:141 - JVM state determined to be unstable.  Exiting 
forcefully due to:
java.lang.OutOfMemoryError: Java heap space
at java.util.ArrayList.(ArrayList.java:152) ~[na:1.8.0_60]
at 
org.apache.cassandra.serializers.ListSerializer.deserializeForNativeProtocol(ListSerializer.java:87)
 ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cql3.Lists$Value.fromSerialized(Lists.java:150) 
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cql3.Lists$Marker.bind(Lists.java:255) 
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.cql3.Lists$Setter.execute(Lists.java:308) 
~[apache-cassandra-3.9.jar:3.9]
{code}

Please find enclosed a unitTest to reproduce the problem. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2016-10-10 Thread Brian Hess (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562407#comment-15562407
 ] 

 Brian Hess commented on CASSANDRA-7296:


Just my 2 cents, but having this be a per-table option is not really a great 
solution for the debugging issue.  I'd be okay if we had that as a default, but 
we'd certainly need to support having this behavior even if that table option 
isn't set (it would be unfortunate to have to ALTER the table to get this 
behavior).

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12767) Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue

2016-10-10 Thread Ye Liang (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ye Liang updated CASSANDRA-12767:
-
Description: 
I use cassandra driver java to do some insert operation.I notice that Cassandra 
Java Driver use client-side timestamp as default.

I make my client server one hour ahead than my cassandra server,then i insert 
some record to an exist table and use sstable2json tool to check my record ' s 
timestamp.

i find out that if i insert a record with ifnotexist,the record's timestamp 
used server-side timestamp,otherwise it use client-side timestamp.

I think this is a very strange result.

  was:
When I use Cassandra Java Driver to connect a C* cluster with a Protocol 
Version 3,such as :
Builder builder = Cluster.builder().withProtocolVersion(ProtocolVersion.V3);

I insert some record to an exist table using ifnotexist,for example:
QueryBuilder.insertInto(xxx).ifNotExists();

Then,i will delete the record normally.

i do the two step over and over again

I find something strange to me :
insert and delete operation are always success(no exception and the response 
looks ok).but before i delete the record,i use select statement to query my 
record.when i insert the record for the first time  i can always  query my 
record.but after that i seldom query my record successfully between insert and 
delete.

I just use a single node cassandra cluster to exclude the effect of data 
consistency.And when i use a Protocol Version 2,it always works well when i 
query my record between insert and delete


> Cassandra Java Driver 3.1.0 Client-Side timestamp generation issue
> --
>
> Key: CASSANDRA-12767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12767
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.15
> Cassandra Java Driver 3.1.0
>Reporter: Ye Liang
>Priority: Minor
>
> I use cassandra driver java to do some insert operation.I notice that 
> Cassandra Java Driver use client-side timestamp as default.
> I make my client server one hour ahead than my cassandra server,then i insert 
> some record to an exist table and use sstable2json tool to check my record ' 
> s timestamp.
> i find out that if i insert a record with ifnotexist,the record's timestamp 
> used server-side timestamp,otherwise it use client-side timestamp.
> I think this is a very strange result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12744) Randomness of stress distributions is not good

2016-10-10 Thread Paulo Motta (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562403#comment-15562403
 ] 

Paulo Motta commented on CASSANDRA-12744:
-

some 
[cqlsh_copy_tests|http://cassci.datastax.com/job/tjake-stress-random-dtest/lastCompletedBuild/testReport/cqlsh_tests.cqlsh_copy_tests/]
 are outputting the following error:
{noformat}
cassandra-stress did not import enough records
{noformat}

do you think this could be related to the distribution change? if not, can you 
maybe rebase and resubmit those?

> Randomness of stress distributions is not good
> --
>
> Key: CASSANDRA-12744
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12744
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
>Priority: Minor
>  Labels: stress
> Fix For: 3.0.10
>
>
> The randomness of our distributions is pretty bad.  We are using the 
> JDKRandomGenerator() but in testing of uniform(1..3) we see for 100 
> iterations it's only outputting 3.  If you bump it to 10k it hits all 3 
> values. 
> I made a change to just use the default commons math random generator and now 
> see all 3 values for n=10



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-12764) Compaction performance issues with many sstables, during transaction commit phase

2016-10-10 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562395#comment-15562395
 ] 

Benedict commented on CASSANDRA-12764:
--

bq. Sadly, that would've taken 20 days

Well, that sounds like a more pressing concern.  These tickets shouldn't have 
materially affected the ETA of a major compaction.  It's possible it entered a 
GC death spiral, as probably >6Gb of heap would have been needed for the reader 
buffers alone (though it's been a while since I've checked our behaviours here; 
it's possible they're lazily initialised and with tiny sstables they might not 
all have been needed to be materialised at once).  

Anyway, it sounds potentially worth filing a ticket to investigate that, as it 
should be the get-out-of-jail free card for the problems you had.

Perhaps it's worth filing a ticket to explicitly make major compaction robust 
to literally arbitrary numbers of files also.  i.e. by performing multiple 
rounds of "major compaction" if the file count is too high to do at once.

> Compaction performance issues with many sstables, during transaction commit 
> phase
> -
>
> Key: CASSANDRA-12764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12764
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:308" #7541 daemon prio=1 os_prio=4 tid=0x7fa22af35400 
> nid=0x41eb runnable [0x7fdbea48d000]
>java.lang.Thread.State: RUNNABLE
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)
>   at java.util.TimSort.sort(TimSort.java:220)
>   at java.util.Arrays.sort(Arrays.java:1438)
>   at com.google.common.collect.Ordering.sortedCopy(Ordering.java:817)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:209)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:211)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at 
> org.apache.cassandra.utils.IntervalTree$IntervalNode.(IntervalTree.java:210)
>   at org.apache.cassandra.utils.IntervalTree.(IntervalTree.java:50)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.(SSTableIntervalTree.java:40)
>   at 
> org.apache.cassandra.db.lifecycle.SSTableIntervalTree.build(SSTableIntervalTree.java:50)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:288)
>   at org.apache.cassandra.db.lifecycle.View$4.apply(View.java:283)
>   at 
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:216)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:128)
>   at org.apache.cassandra.db.lifecycle.Tracker.apply(Tracker.java:101)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:307)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkpoint(LifecycleTransaction.java:288)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:368)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> 

[jira] [Updated] (CASSANDRA-12759) cassandra-stress shows the incorrect JMX port in settings output

2016-10-10 Thread T Jake Luciani (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-12759:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Thanks committed in {{cbc792531fd7282178b82fd0f333b4540409b117}}

> cassandra-stress shows the incorrect JMX port in settings output
> 
>
> Key: CASSANDRA-12759
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12759
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.10
>
> Attachments: 
> 0001-Show-the-correct-value-for-JMX-port-in-cassandra-str.patch
>
>
> CASSANDRA-11914 introduces settings output for cassandra-stress; in that 
> output, the JMX port is incorrectly reported.  The attached patch fixes this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


  1   2   >