[jira] [Commented] (CASSANDRA-7576) DateType columns not properly converted to TimestampType when in ReversedType columns.

[Ben Hood (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068328#comment-14068328
 ] 

Ben Hood commented on CASSANDRA-7576:
-

FWIW, you could verify this patch by removing this workaround 
(https://github.com/gocql/gocql/pull/154) that we put into place to deal with 
this issue in gocql.

 DateType columns not properly converted to TimestampType when in ReversedType 
 columns.
 --

 Key: CASSANDRA-7576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7576
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Karl Rieb
 Attachments: DataType_CASSANDRA_7576.patch

   Original Estimate: 0.25h
  Remaining Estimate: 0.25h

 The {{org.apache.cassandra.transport.DataType.fromType(AbstractType)}} method 
 has a bug that prevents sending the correct Protocol ID for reversed 
 {{DateType}} columns.   This results in clients receiving Protocol ID {{0}}, 
 which maps to a {{CUSTOM}} type, for timestamp columns that are clustered in 
 reverse order.  
 Some clients can handle this properly since they recognize the 
 {{org.apache.cassandra.db.marshal.DateType}} marshaling type, however the 
 native Datastax java-driver does not.  It will produce errors like the one 
 below when trying to prepare queries against such tables:
 {noformat}
 com.datastax.driver.core.exceptions.InvalidTypeException: Invalid type 
 for value 2 of CQL type 'org.apache.cassandra.db.marshal.DateType', expecting 
 class java.nio.ByteBuffer but class java.util.Date provided
   at com.datastax.driver.core.BoundStatement.bind(BoundStatement.java:190)
   at 
 com.datastax.driver.core.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:103)
 {noformat}
 On the Cassandra side, there is a check for {{DateType}} columns that is 
 supposed to convert these columns to TimestampType.  However, the check is 
 skipped when the column is also reversed.  Specifically:
 {code:title=DataType.java|borderStyle=solid}
 public static PairDataType, Object fromType(AbstractType type)
 {
 // For CQL3 clients, ReversedType is an implementation detail and they
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
 // For compatibility sake, we still return DateType as the timestamp type 
 in resultSet metadata (#5723)
 else if (type instanceof DateType)
 type = TimestampType.instance;
 // ...
 {code}
 The *else if* should be changed to just an *if*, like so:
 {code:title=DataType.java|borderStyle=solid}
 public static PairDataType, Object fromType(AbstractType type)
 {
 // For CQL3 clients, ReversedType is an implementation detail and they
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
 // For compatibility sake, we still return DateType as the timestamp type 
 in resultSet metadata (#5723)
 if (type instanceof DateType)
 type = TimestampType.instance;
 // ...
 {code}
 This bug is preventing us from upgrading our 1.2.11 cluster to 2.0.9 because 
 our clients keep throwing exceptions trying to read or write data to tables 
 with reversed timestamp columns. This issue can be reproduced by creating a 
 CQL table in Cassandra 1.2.11 that clusters on a timestamp in reverse, then 
 upgrading the node to 2.0.9.  When querying the metadata for the table, the 
 node will return Protocol ID 0 (CUSTOM) instead of Protocol ID 11 (TIMESTAMP).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7454) NPE When Prepared Statement ID is not Found

[Ben Hood (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068366#comment-14068366
 ] 

Ben Hood commented on CASSANDRA-7454:
-

I can confirm that this fix (as applied to 2.1-rc4) has resolved part of the an 
issue reported in https://issues.apache.org/jira/browse/CASSANDRA-7566.

When I run the the TestReprepareStatement test case from the gocql integration 
suite against the 2.1-rc3 tarball, it fails:

{code}
bool:gocql 0x6e6562$ go test . -v -test.run=TestReprepareStatement
=== RUN TestReprepareStatement
--- FAIL: TestReprepareStatement (0.31 seconds)
cassandra_test.go:588: Failed to execute query for reprepare statement: 
java.lang.NullPointerException
FAIL
exit status 1
{code}

But when I run it against the 2.1-rc4 tarball, it passes:

{code}
bool:gocql 0x6e6562$ go test . -v -test.run=TestReprepareStatement
=== RUN TestReprepareStatement
--- PASS: TestReprepareStatement (0.25 seconds)
PASS
ok  github.com/relops/gocql 0.267s
{code}

 NPE When Prepared Statement ID is not Found
 ---

 Key: CASSANDRA-7454
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7454
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Fix For: 2.1.0

 Attachments: 7475.txt


 CASSANDRA-6855 introduced a NullPointerException when an unknown prepared 
 statement ID is used.
 You'll see a stack trace like this:
 {noformat}
 ERROR [SharedPool-Worker-4] 2014-06-26 15:02:04,911 ErrorMessage.java:218 - 
 Unexpected exception during request
 java.lang.NullPointerException: null
 at 
 org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:105)
  ~[main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:412)
  [main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:309)
  [main/:na]
 at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
  [netty-all-4.0.20.Final.jar:4.0.20.Final]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_40]
 at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:162)
  [main/:na]
 at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) 
 [main/:na]
 at java.lang.Thread.run(Thread.java:724) [na:1.7.0_40]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068388#comment-14068388
 ] 

Benedict commented on CASSANDRA-7507:
-

Unfortunately it looks like we have a *lot* of places where we catch 
IOException and do something specific. I'd say the safest thing to do is create 
a special method for handling special exceptions that we call any time we hit 
an IOException, which will ordinarily do nothing, but in case of 'too many open 
files' (or anything else we later decide to include) performs a sys exit

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068435#comment-14068435
 ] 

Marcus Eriksson commented on CASSANDRA-6851:


in general this looks good to me, pushed a branch with a few updates: 
https://github.com/krummas/cassandra/commits/russellspitzer/6851

it fixes a few nits (style mostly) and tries to pick better sstables by sorting 
them by first key in the sstable for standard grouping

we should probably target trunk for this as well

 Improve anticompaction after incremental repair
 ---

 Key: CASSANDRA-6851
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Marcus Eriksson
Assignee: Russell Alexander Spitzer
Priority: Minor
  Labels: compaction, lhf
 Fix For: 2.1.1


 After an incremental repair we iterate over all sstables and split them in 
 two parts, one containing the repaired data and one the unrepaired. We could 
 in theory double the number of sstables on a node.
 To avoid this we could make anticompaction also do a compaction, for example, 
 if we are to anticompact 10 sstables, we could anticompact those to 2.
 Note that we need to avoid creating too big sstables though, if we 
 anticompact all sstables on a node it would essentially be a major compaction.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6851) Improve anticompaction after incremental repair

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-6851:
---

Fix Version/s: (was: 2.1.1)
   3.0

 Improve anticompaction after incremental repair
 ---

 Key: CASSANDRA-6851
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Marcus Eriksson
Assignee: Russell Alexander Spitzer
Priority: Minor
  Labels: compaction, lhf
 Fix For: 3.0


 After an incremental repair we iterate over all sstables and split them in 
 two parts, one containing the repaired data and one the unrepaired. We could 
 in theory double the number of sstables on a node.
 To avoid this we could make anticompaction also do a compaction, for example, 
 if we are to anticompact 10 sstables, we could anticompact those to 2.
 Note that we need to avoid creating too big sstables though, if we 
 anticompact all sstables on a node it would essentially be a major compaction.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068437#comment-14068437
 ] 

Marcus Eriksson commented on CASSANDRA-6434:


Blocks on CASSANDRA-5839 - once we have the repair history we can figure out 
actual repair times for keys

 Repair-aware gc grace period 
 -

 Key: CASSANDRA-6434
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: sankalp kohli
Assignee: Marcus Eriksson
 Fix For: 3.0


 Since the reason for gcgs is to ensure that we don't purge tombstones until 
 every replica has been notified, it's redundant in a world where we're 
 tracking repair times per sstable (and repairing frequentily), i.e., a world 
 where we default to incremental repair a la CASSANDRA-5351.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-5839) Save repair data to system table

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-5839:
--

Assignee: Marcus Eriksson  (was: Jimmy Mårdell)

 Save repair data to system table
 

 Key: CASSANDRA-5839
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5839
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.0.10

 Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch


 As noted in CASSANDRA-2405, it would be useful to store repair results, 
 particularly with sub-range repair available (CASSANDRA-5280).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7443) SSTable Pluggability v2

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068458#comment-14068458
 ] 

Benedict commented on CASSANDRA-7443:
-

First impression is pretty positive - this is looking a lot cleaner than I 
might have hoped.

A few initial comments, obviously bearing in mind it's still in progress:

* I'm not super keen on using Class.getConstructor() for instantiating a 
reader/writer. Would prefer a factory method / class; using reflection for long 
type parameter lists worries me.
* Would prefer DatabaseDescriptor.getDefaultSSTableFormat, and permit overrides 
per Table
* LazilyCompactedRow is still tightly coupled with Big format (relates to 
AbstractCompactedRow as you mentioned)
* Streaming _may_ need a bit more work/thought - seems that compression format 
of sstables is coupled quite tightly with compression stream writer, and also 
assumes we can stream a single file range. Might not want to impose that 
requirement (we probably don't for 3.0 format - a set of ranges is more likely, 
but I would prefer to abstract the concept of a stream Chunk to be format 
specific anyway, to remove the coupling)

 SSTable Pluggability v2
 ---

 Key: CASSANDRA-7443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7443
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani
Assignee: T Jake Luciani
 Fix For: 3.0


 As part of a wider effort to improve the performance of our storage engine we 
 will need to support basic pluggability of the SSTable reader/writer. We 
 primarily need this to support the current SSTable format and new SSTable 
 format in the same version.  This will also let us encapsulate the changes in 
 a single layer vs forcing the whole engine to change at once.
 We previously discussed how to accomplish this in CASSANDRA-3067
   



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-6751) Setting -Dcassandra.fd_initial_value_ms Results in NPE

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6751:


Fix Version/s: 2.0.8

 Setting -Dcassandra.fd_initial_value_ms Results in NPE
 --

 Key: CASSANDRA-6751
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6751
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Dave Brosius
Priority: Minor
 Fix For: 1.2.17, 2.0.8

 Attachments: 6751.txt


 Start Cassandra with {{-Dcassandra.fd_initial_value_ms=1000}} and you'll get 
 the following stacktrace:
 {noformat}
  INFO [main] 2014-02-21 14:45:57,731 StorageService.java (line 617) Starting 
 up server gossip
 ERROR [main] 2014-02-21 14:45:57,736 CassandraDaemon.java (line 464) 
 Exception encountered during startup
 java.lang.ExceptionInInitializerError
 at org.apache.cassandra.gms.Gossiper.init(Gossiper.java:178)
 at org.apache.cassandra.gms.Gossiper.clinit(Gossiper.java:71)
 at 
 org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:618)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:583)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:480)
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.cassandra.gms.FailureDetector.getInitialValue(FailureDetector.java:81)
 at 
 org.apache.cassandra.gms.FailureDetector.clinit(FailureDetector.java:48)
 ... 8 more
 ERROR [StorageServiceShutdownHook] 2014-02-21 14:45:57,754 
 CassandraDaemon.java (line 191) Exception in thread 
 Thread[StorageServiceShutdownHook,5,main]
 java.lang.NoClassDefFoundError: Could not initialize class 
 org.apache.cassandra.gms.Gossiper
 at 
 org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:550)
 at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
 at java.lang.Thread.run(Thread.java:724)
 {noformat}
 Glancing at the code, this is because the FailureDetector logger isn't 
 initialized when the static initialization of {{INITIAL_VALUE}} happens.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[06/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

[brandonwilliams]

Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095

Branch: refs/heads/cassandra-2.1
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:49:13 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
 
 java_ver_output=`${JAVA:-java} -version 21`
 
-jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
+jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
{print $2}'`
 JVM_VERSION=${jvmver%_*}
 JVM_PATCH_VERSION=${jvmver#*_}
 
@@ -100,7 +100,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 fi
 
 
-jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
+jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
$1}'`
 case $jvm in
 OpenJDK)
 JVM_VENDOR=OpenJDK

[11/12] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

[brandonwilliams]

Merge branch 'cassandra-2.1.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b14f6a5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b14f6a5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b14f6a5

Branch: refs/heads/cassandra-2.1
Commit: 3b14f6a5dbc91b3e8d2efa95840a61604dd6fbcb
Parents: 82fa5d5 df7bd7f
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:51:10 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:51:10 2014 -0500

--
 CHANGES.txt   | 2 +-
 build.xml | 2 +-
 conf/cassandra-env.sh | 4 ++--
 debian/changelog  | 6 ++
 4 files changed, 10 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/CHANGES.txt
--
diff --cc CHANGES.txt
index 8fa96f6,5ba30cc..2aa2efa
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,4 +1,15 @@@
 +2.1.1
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Fail to start if commit log replay detects a problem (CASSANDRA-7125)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 + * Fix native protocol drop user type notification (CASSANDRA-7571)
 +
 +
- 2.1.0-final
+ 2.1.0-rc4
   * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
 (CASSANDRA-7551)
   * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/build.xml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/conf/cassandra-env.sh
--

[09/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0

[brandonwilliams]

Merge branch 'cassandra-2.0' into cassandra-2.1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7

Branch: refs/heads/trunk
Commit: df7bd7f725f6df5102730de89647418dd9f9473d
Parents: d872e2c 8754809
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:50:18 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:50:18 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh
--

[07/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0

[brandonwilliams]

Merge branch 'cassandra-2.0' into cassandra-2.1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7

Branch: refs/heads/cassandra-2.1
Commit: df7bd7f725f6df5102730de89647418dd9f9473d
Parents: d872e2c 8754809
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:50:18 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:50:18 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh
--

[02/12] git commit: update versioning for 2.1.0-rc4 release

[brandonwilliams]

update versioning for 2.1.0-rc4 release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d872e2c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d872e2c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d872e2c7

Branch: refs/heads/trunk
Commit: d872e2c74685c0c8c019b2ef46aefa443efa95b9
Parents: 8fd8832
Author: Eric Evans eev...@apache.org
Authored: Fri Jul 18 13:42:11 2014 -0500
Committer: Eric Evans eev...@apache.org
Committed: Fri Jul 18 13:42:11 2014 -0500

--
 CHANGES.txt  | 2 +-
 build.xml| 2 +-
 debian/changelog | 6 ++
 3 files changed, 8 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ff417b1..5ba30cc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,4 @@
-2.1.0-final
+2.1.0-rc4
  * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
(CASSANDRA-7551)
  * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/build.xml
--
diff --git a/build.xml b/build.xml
index fd0c72b..ca973cc 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.1.0-rc3/
+property name=base.version value=2.1.0-rc4/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 6f8e3ec..7a38d2d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.0~rc4) unstable; urgency=medium
+
+  * New RC release
+
+ -- Eric Evans eev...@apache.org  Fri, 18 Jul 2014 13:40:48 -0500
+
 cassandra (2.1.0~rc3) unstable; urgency=medium
 
   * New RC release

[03/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

[brandonwilliams]

Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095

Branch: refs/heads/cassandra-2.1.0
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:49:13 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
 
 java_ver_output=`${JAVA:-java} -version 21`
 
-jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
+jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
{print $2}'`
 JVM_VERSION=${jvmver%_*}
 JVM_PATCH_VERSION=${jvmver#*_}
 
@@ -100,7 +100,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 fi
 
 
-jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
+jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
$1}'`
 case $jvm in
 OpenJDK)
 JVM_VENDOR=OpenJDK

[01/12] git commit: update versioning for 2.1.0-rc4 release

[brandonwilliams]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 337e4a829 - 875480958
  refs/heads/cassandra-2.1 82fa5d507 - 3b14f6a5d
  refs/heads/cassandra-2.1.0 d872e2c74 - df7bd7f72
  refs/heads/trunk 8d8fed522 - 2093270c7


update versioning for 2.1.0-rc4 release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d872e2c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d872e2c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d872e2c7

Branch: refs/heads/cassandra-2.1
Commit: d872e2c74685c0c8c019b2ef46aefa443efa95b9
Parents: 8fd8832
Author: Eric Evans eev...@apache.org
Authored: Fri Jul 18 13:42:11 2014 -0500
Committer: Eric Evans eev...@apache.org
Committed: Fri Jul 18 13:42:11 2014 -0500

--
 CHANGES.txt  | 2 +-
 build.xml| 2 +-
 debian/changelog | 6 ++
 3 files changed, 8 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ff417b1..5ba30cc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,4 @@
-2.1.0-final
+2.1.0-rc4
  * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
(CASSANDRA-7551)
  * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/build.xml
--
diff --git a/build.xml b/build.xml
index fd0c72b..ca973cc 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.1.0-rc3/
+property name=base.version value=2.1.0-rc4/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 6f8e3ec..7a38d2d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.1.0~rc4) unstable; urgency=medium
+
+  * New RC release
+
+ -- Eric Evans eev...@apache.org  Fri, 18 Jul 2014 13:40:48 -0500
+
 cassandra (2.1.0~rc3) unstable; urgency=medium
 
   * New RC release

[10/12] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

[brandonwilliams]

Merge branch 'cassandra-2.1.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b14f6a5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b14f6a5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b14f6a5

Branch: refs/heads/trunk
Commit: 3b14f6a5dbc91b3e8d2efa95840a61604dd6fbcb
Parents: 82fa5d5 df7bd7f
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:51:10 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:51:10 2014 -0500

--
 CHANGES.txt   | 2 +-
 build.xml | 2 +-
 conf/cassandra-env.sh | 4 ++--
 debian/changelog  | 6 ++
 4 files changed, 10 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/CHANGES.txt
--
diff --cc CHANGES.txt
index 8fa96f6,5ba30cc..2aa2efa
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,4 +1,15 @@@
 +2.1.1
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Fail to start if commit log replay detects a problem (CASSANDRA-7125)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 + * Fix native protocol drop user type notification (CASSANDRA-7571)
 +
 +
- 2.1.0-final
+ 2.1.0-rc4
   * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
 (CASSANDRA-7551)
   * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/build.xml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/conf/cassandra-env.sh
--

[08/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0

[brandonwilliams]

Merge branch 'cassandra-2.0' into cassandra-2.1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7

Branch: refs/heads/cassandra-2.1.0
Commit: df7bd7f725f6df5102730de89647418dd9f9473d
Parents: d872e2c 8754809
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:50:18 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:50:18 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh
--

[05/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

[brandonwilliams]

Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095

Branch: refs/heads/cassandra-2.0
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:49:13 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
 
 java_ver_output=`${JAVA:-java} -version 21`
 
-jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
+jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
{print $2}'`
 JVM_VERSION=${jvmver%_*}
 JVM_PATCH_VERSION=${jvmver#*_}
 
@@ -100,7 +100,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 fi
 
 
-jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
+jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
$1}'`
 case $jvm in
 OpenJDK)
 JVM_VENDOR=OpenJDK

[04/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

[brandonwilliams]

Fix jvm version detection when JAVA_TOOL_OPTIONS in environment

Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095

Branch: refs/heads/trunk
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:49:13 2014 -0500

--
 conf/cassandra-env.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
 
 java_ver_output=`${JAVA:-java} -version 21`
 
-jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
+jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
{print $2}'`
 JVM_VERSION=${jvmver%_*}
 JVM_PATCH_VERSION=${jvmver#*_}
 
@@ -100,7 +100,7 @@ if [ $JVM_VERSION \ 1.7 ] ; then
 fi
 
 
-jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
+jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
$1}'`
 case $jvm in
 OpenJDK)
 JVM_VENDOR=OpenJDK

[12/12] git commit: Merge branch 'cassandra-2.1' into trunk

[brandonwilliams]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2093270c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2093270c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2093270c

Branch: refs/heads/trunk
Commit: 2093270c7c51700996577fd679f644bcfb17239e
Parents: 8d8fed5 3b14f6a
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jul 21 07:51:21 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jul 21 07:51:21 2014 -0500

--
 CHANGES.txt   | 2 +-
 build.xml | 2 +-
 conf/cassandra-env.sh | 4 ++--
 debian/changelog  | 6 ++
 4 files changed, 10 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2093270c/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2093270c/build.xml
--

[jira] [Updated] (CASSANDRA-5839) Save repair data to system table

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-5839:
---

Fix Version/s: (was: 2.0.10)
   3.0

 Save repair data to system table
 

 Key: CASSANDRA-5839
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5839
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 3.0

 Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch


 As noted in CASSANDRA-2405, it would be useful to store repair results, 
 particularly with sub-range repair available (CASSANDRA-5280).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (CASSANDRA-7572) cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in environment

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-7572.
-

   Resolution: Fixed
Fix Version/s: 2.1.0
   2.0.10
 Reviewer: Brandon Williams

bq. Parsing this is not guaranteed to work as there is no contract for this 
output. The recommended way of dealing with this is via a Java application 
(which will have access to each of these pieces).

While I totally agree with you on this point, we've been using this method for 
years without any problem until now, so I committed the change with grep.

 cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in 
 environment
 ---

 Key: CASSANDRA-7572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7572
 Project: Cassandra
  Issue Type: Bug
  Components: Config
 Environment: Linux/darwin/etc
Reporter: Gregory Ramsperger
 Fix For: 2.0.10, 2.1.0


 Parsing of Java version is done in cassandra-env.sh via
 {code}
 java_ver_output=`${JAVA:-java} -version 21`
 jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
 {code}
 This fails if the first line of output from `java -version` does not contain 
 the version. When JAVA_TOOL_OPTIONS is set as the output is
 {quote}
 Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
 java version 1.7.0_40
 Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
 Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
 {quote}
 Parsing this is not guaranteed to work as there is no contract for this 
 output. The recommended way of dealing with this is via a Java application 
 (which will have access to each of these pieces).
 If you want to keep with this model of doing things, you can apply a very 
 small but hacky patch:
 {code}
 -jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
 +jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
 {print $2}'`
  JVM_VERSION=${jvmver%_*}
  JVM_PATCH_VERSION=${jvmver#*_}
 @@ -100,7 +100,7 @@
  fi
 -jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
 +jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
 $1}'`
  case $jvm in
  OpenJDK)
 {code}
 Otherwise, you need to write a java app, compiled for older versions, that 
 returns this data via the java.specification.version and a combination of 
 java.vm.name and java.vm.vendor.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-7572) cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in environment

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-7572:
---

Assignee: Gregory Ramsperger

 cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in 
 environment
 ---

 Key: CASSANDRA-7572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7572
 Project: Cassandra
  Issue Type: Bug
  Components: Config
 Environment: Linux/darwin/etc
Reporter: Gregory Ramsperger
Assignee: Gregory Ramsperger
 Fix For: 2.0.10, 2.1.0


 Parsing of Java version is done in cassandra-env.sh via
 {code}
 java_ver_output=`${JAVA:-java} -version 21`
 jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
 {code}
 This fails if the first line of output from `java -version` does not contain 
 the version. When JAVA_TOOL_OPTIONS is set as the output is
 {quote}
 Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
 java version 1.7.0_40
 Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
 Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
 {quote}
 Parsing this is not guaranteed to work as there is no contract for this 
 output. The recommended way of dealing with this is via a Java application 
 (which will have access to each of these pieces).
 If you want to keep with this model of doing things, you can apply a very 
 small but hacky patch:
 {code}
 -jvmver=`echo $java_ver_output | awk -F'' 'NR==1 {print $2}'`
 +jvmver=`echo $java_ver_output | grep 'java version' | awk -F'' 'NR==1 
 {print $2}'`
  JVM_VERSION=${jvmver%_*}
  JVM_PATCH_VERSION=${jvmver#*_}
 @@ -100,7 +100,7 @@
  fi
 -jvm=`echo $java_ver_output | awk 'NR==2 {print $1}'`
 +jvm=`echo $java_ver_output | grep -A 1 'java version' | awk 'NR==2 {print 
 $1}'`
  case $jvm in
  OpenJDK)
 {code}
 Otherwise, you need to write a java app, compiled for older versions, that 
 returns this data via the java.specification.version and a combination of 
 java.vm.name and java.vm.vendor.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes

[Mike Adamson (JIRA)]

Mike Adamson created CASSANDRA-7578:
---

 Summary: Non-superusers can't login to tools because of no access 
to system.schema_usertypes
 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Blocker
 Fix For: 2.1.0


When I try to login to cqlsh as a non-superuser I get the following error:
{noformat}
ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
permission on table system.schema_usertypes or any of its parents
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

buildbot success in ASF Buildbot on cassandra-trunk

[buildbot]

The Buildbot has detected a restored build on builder cassandra-trunk while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/478

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] 2093270c7c51700996577fd679f644bcfb17239e
Blamelist: Brandon Williams brandonwilli...@apache.org,Eric Evans 
eev...@apache.org

Build succeeded!

sincerely,
 -The Buildbot

[jira] [Updated] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes

[Mike Adamson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Adamson updated CASSANDRA-7578:


Attachment: 7578.patch

 Non-superusers can't login to tools because of no access to 
 system.schema_usertypes
 ---

 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7578.patch


 When I try to login to cqlsh as a non-superuser I get the following error:
 {noformat}
 ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
 permission on table system.schema_usertypes or any of its parents
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

buildbot failure in ASF Buildbot on cassandra-trunk

[buildbot]

The Buildbot has detected a new failure on builder cassandra-trunk while 
building libcloud.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/479

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] 075383509e507f88b2c105b4f5fde5b4d4af5e68
Blamelist: Tomaz Muraus to...@apache.org

BUILD FAILED: failed

sincerely,
 -The Buildbot

[jira] [Updated] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7578:


Reviewer: Aleksey Yeschenko

 Non-superusers can't login to tools because of no access to 
 system.schema_usertypes
 ---

 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7578.patch


 When I try to login to cqlsh as a non-superuser I get the following error:
 {noformat}
 ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
 permission on table system.schema_usertypes or any of its parents
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7546:


Attachment: 7546.20.txt

Attaching my suggested alternative approach, which I think is a little simpler 
to reason about: if we loop for any reason we immediately take the monitor, if 
unsafe is available to us; if it isn't we don't benefit from the optimisation. 
We don't change the logic for updating, as taking the monitor doesn't guarantee 
we have exclusive access, it only guarantees we have exclusive access versus 
those writes that have themselves spun once, thereby capping the amount of 
competition.

For 2.1 I suggest sticking with the same approach, then in 3.0/.1 we'll take a 
look at the more complex approach of making writes lazy under competition.

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068588#comment-14068588
 ] 

Jonathan Ellis commented on CASSANDRA-7507:
---

Let's keep the scope of this ticket to dealing with OOM.  Open files can be 
another ticket.

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068592#comment-14068592
 ] 

Jonathan Ellis commented on CASSANDRA-6434:
---

We could approximate it as max(repairtime) for any sstable that may contain the 
key.  (And we already have the logic that says, checking bloom filters is 
expensive so avoid it until the last minute.)

 Repair-aware gc grace period 
 -

 Key: CASSANDRA-6434
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: sankalp kohli
Assignee: Marcus Eriksson
 Fix For: 3.0


 Since the reason for gcgs is to ensure that we don't purge tombstones until 
 every replica has been notified, it's redundant in a world where we're 
 tracking repair times per sstable (and repairing frequentily), i.e., a world 
 where we default to incremental repair a la CASSANDRA-5351.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068616#comment-14068616
 ] 

Joshua McKenzie commented on CASSANDRA-7507:


Was planning on creating another ticket for that effort and doing a little 
tidying up on this patch to make it more general rather than oom-specific on 
naming.  I should be able to get to that today.

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068730#comment-14068730
 ] 

Benedict commented on CASSANDRA-7507:
-

It's worth noting then that this applies also to catch (Throwable), although it 
looks a little less widespread.

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

graham sanderson updated CASSANDRA-7546:


Attachment: 7546.20_alt.txt

Thanks for taking a look Benedict - I am in the process of actually measuring 
the effect via cassandra stress in the real world for code paths that might hit 
this (actual user initiated contended write on a partition, hint based 
contented write, and low cardinality secondary indexes).

I had initially shied away from the non obviously paired monitor usage because 
a) it requires unsafe, and b) whilst monitorenter and monitorexit do exist as 
separate byte codes, I seem to recall that there is code in the monitor fast 
path (especially w.r.t. biased locking which we probably wan't to avoid) that 
expects them to be trivially paired... c) on the negative side this forces two 
loops under contention, but that is better than unbounded obviously.

I'm not sure if b) is an actual issue (or what effect it has - does it matter 
if the monitor is always inflated for example).

As for c) I guess I'll have to measure and see.

Note I attached the current version of the 2.0 patch I was testing with, which 
pushed the loop body into a class similar to ColumnUpdater in 2.1 (which I 
would update accordingly)... 

again any attempt to further simplify the three separate loops, didn't really 
make things any simpler looking.

I'm not sure my code (whilst not blazingly pretty) is insanely hard to reason 
about... clearly someone always makes progress (since any change in state is 
dependent on a CAS success or failure due to someone else succeeding); the 
un-contended and highly contended cases are pretty simple, and any 
flip-flopping between the count and raw state again always means we made 
progress AND must (still) be under low contention anyway.

In any case, I'll defer to your judgement as to what you'd like to see in the 
code base... as I say some actual numbers outside of a synthetic test should 
help

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_alt.txt, suggestion1.txt, 
 suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7579) File descriptor exhaustion can lead to unreliable state in exception condition

[Joshua McKenzie (JIRA)]

Joshua McKenzie created CASSANDRA-7579:
--

 Summary: File descriptor exhaustion can lead to unreliable state 
in exception condition
 Key: CASSANDRA-7579
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7579
 Project: Cassandra
  Issue Type: New Feature
Reporter: Joshua McKenzie
Priority: Minor


If the JVM runs out of file descriptors we can get into an unreliable state 
(similar to CASSANDRA-7507 on OOM) where we cannot trust our shutdown hook to 
run successfully to completion.  We need to check IOExceptions and appropriate 
Throwable to see if we have a FileNotFoundException w/message Too many files 
open and forcefully shutdown the Daemon in these cases.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7507:
---

Attachment: 7507_v2.txt

Attaching a v2 w/naming that's not OOM-centric and exposes an external method 
for others to call to cause a hardExit (i.e. CASSANDRA-7579)

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt, 7507_v2.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068803#comment-14068803
 ] 

Benedict commented on CASSANDRA-7546:
-

bq. I'm not sure my code (whilst not blazingly pretty) is insanely hard to 
reason about... 

I'm not suggesting it is by any means abhorrent, only that we can achieve the 
desired goal with fewer changes, so unless there's a lot of evidence that the 
extra complexity is worth it, we should stick with the simpler approach (this 
also means less pollution of the instruction cache in a very hot part of the 
codebase, which is a good thing). I would suggest running a stress workload 
with a fixed number of threads, with increasing numbers of partitions (from 1 
up to  number of threads) and see how the curve changes, if you want to 
benchmark this closely.

As to (b): since we only ever acquire the lock when we are contending, it must 
always be inflated anyway, so this shouldn't be an issue.

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_alt.txt, suggestion1.txt, 
 suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068856#comment-14068856
 ] 

Benedict commented on CASSANDRA-7507:
-

Starting a thread is sometimes impossible after an OOM (since OOM can be thrown 
for having exhausted our permitted thread count), so we should probably execute 
our shutdown directly in the thread that caught the exception. Also, we need to 
(IMO) deal with all the catch (Throwable) instances that do not propagate the 
exception - esp. any worker threads in SEP, DebuggableExecutor etc, but there 
are other places we could get bitten as well.

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt, 7507_v2.txt


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh

[Mike Adamson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Adamson updated CASSANDRA-7580:


Description: 
On a system configured with:

 * PasswordAuthenticator
 * CassandraAuthorizer

If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}

  was:
On a system configured with:
 * PasswordAuthenticator
 * CassandraAuthorizer
If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}


 Super user cannot list permissions with cqlsh
 -

 Key: CASSANDRA-7580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Critical
 Fix For: 2.1.0


 On a system configured with:
  * PasswordAuthenticator
  * CassandraAuthorizer
 If you login as a super user and issue:
 {noformat}
 list all permissions;
 {noformat}
 where permissions have been granted to a user you get back:
 {noformat}
 Keyspace None not found.
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7580) Super user cannot list permissions with cqlsh

[Mike Adamson (JIRA)]

Mike Adamson created CASSANDRA-7580:
---

 Summary: Super user cannot list permissions with cqlsh
 Key: CASSANDRA-7580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Critical
 Fix For: 2.1.0


On a system configured with:
 * PasswordAuthenticator
 * CassandraAuthorizer
If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7580:


Attachment: 7580.txt

Patch against 2.1.0

 Super user cannot list permissions with cqlsh
 -

 Key: CASSANDRA-7580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Critical
 Fix For: 2.1.0

 Attachments: 7580.txt


 On a system configured with:
  * PasswordAuthenticator
  * CassandraAuthorizer
 If you login as a super user and issue:
 {noformat}
 list all permissions;
 {noformat}
 where permissions have been granted to a user you get back:
 {noformat}
 Keyspace None not found.
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7580:


Reviewer: Tyler Hobbs
Priority: Minor  (was: Critical)

 Super user cannot list permissions with cqlsh
 -

 Key: CASSANDRA-7580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.1.0

 Attachments: 7580.txt


 On a system configured with:
  * PasswordAuthenticator
  * CassandraAuthorizer
 If you login as a super user and issue:
 {noformat}
 list all permissions;
 {noformat}
 where permissions have been granted to a user you get back:
 {noformat}
 Keyspace None not found.
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (CASSANDRA-7580) Super user cannot list permissions with cqlsh

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-7580:
---

Assignee: Brandon Williams

 Super user cannot list permissions with cqlsh
 -

 Key: CASSANDRA-7580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Assignee: Brandon Williams
Priority: Critical
 Fix For: 2.1.0

 Attachments: 7580.txt


 On a system configured with:
  * PasswordAuthenticator
  * CassandraAuthorizer
 If you login as a super user and issue:
 {noformat}
 list all permissions;
 {noformat}
 where permissions have been granted to a user you get back:
 {noformat}
 Keyspace None not found.
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

graham sanderson updated CASSANDRA-7546:


Attachment: 7546.20_2.txt

Yes, I agree with you; simpler is better - I didn't like the fact that it was 
hard to disentangle my code.

So I have attached a different patch 7546.20_2.txt, which is about the same as 
yours except that it tracks a very minimal state heuristic to avoid the first 
unlocked loop when it thinks it is contended... this helps under load where it 
attempts about 1.11 loop per mutation in the heavily contented case vs 1.93 
with 7546.20.txt (or about 18 without any patch at all).

It turns out to be rather hard to measure this with cassandra-stress in 
isolation - basically the looping causes very high memory allocation rates 
which may or may not be a problem based on the current state of memory in the 
JVM (e.g. currently resident recent memtables) and other activity. Also, I may 
open a separate ticket to allow batch (thrift and maybe CQL) inserts in 
cassandra-stress to bump up the load a little more.

That said, clearly this race is undesirable, and we have certainly observed its 
effects in practice in the hinting case (which is doubly unfortunate since you 
are already now doing more work per node than you thought, and you are going 
down a contention path you weren't expecting based on your own partitioning). 
Note when it goes wrong it goes really wrong!

So, I think settling for a patch with minimal code or uncontended path impact, 
that caps us nicely is the right way to go

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069059#comment-14069059
 ] 

graham sanderson commented on CASSANDRA-7546:
-

FYI here are the same synthetic test results for 7546.20_2.txt

{code}
[junit] --
[junit] 1 THREAD; ELEMENT SIZE 64
[junit] 
[junit] Threads = 1 elements = 10 (of size 64) partitions = 1
[junit]  original code:
[junit]   Duration = 993ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 34 ms for 3 collections
[junit]   Approx allocation = 553MB vs 8MB; ratio to raw data size = 
69.13799428571429
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 761ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 34 ms for 3 collections
[junit]   Approx allocation = 579MB vs 8MB; ratio to raw data size = 
72.31675047619048
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit] 
[junit] Threads = 1 elements = 10 (of size 64) partitions = 16
[junit]  original code:
[junit]   Duration = 780ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 25 ms for 2 collections
[junit]   Approx allocation = 436MB vs 8MB; ratio to raw data size = 
54.48992095238095
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 671ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 24 ms for 2 collections
[junit]   Approx allocation = 477MB vs 8MB; ratio to raw data size = 
59.545997142857146
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit] 
[junit] Threads = 1 elements = 10 (of size 64) partitions = 256
[junit]  original code:
[junit]   Duration = 452ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 11 ms for 1 collections
[junit]   Approx allocation = 321MB vs 8MB; ratio to raw data size = 
40.14510761904762
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 460ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 10 ms for 1 collections
[junit]   Approx allocation = 341MB vs 8MB; ratio to raw data size = 
42.63770857142857
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit] 
[junit] Threads = 1 elements = 10 (of size 64) partitions = 1024
[junit]  original code:
[junit]   Duration = 462ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 14 ms for 1 collections
[junit]   Approx allocation = 264MB vs 8MB; ratio to raw data size = 
32.99879142857143
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 543ms maxConcurrency = 1
[junit]   GC for PS Scavenge: 14 ms for 1 collections
[junit]   Approx allocation = 272MB vs 8MB; ratio to raw data size = 
34.047360952380956
[junit]   loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0 
sync 0/0 up 0 down 0
[junit] 
[junit] 
[junit] --
[junit] 100 THREADS; ELEMENT SIZE 64
[junit] 
[junit] Threads = 100 elements = 10 (of size 64) partitions = 1
[junit]  original code:
[junit]   Duration = 2318ms maxConcurrency = 100
[junit]   GC for PS Scavenge: 119 ms for 32 collections
[junit]   Approx allocation = 10547MB vs 8MB; ratio to raw data size = 
1316.62704
[junit]   loopRatio (closest to 1 best) 18.35448 raw 10/1835448 counted 
0/0 sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 1315ms maxConcurrency = 100
[junit]   GC for PS Scavenge: 14 ms for 1 collections
[junit]   Approx allocation = 629MB vs 8MB; ratio to raw data size = 
78.62949142857143
[junit]   loopRatio (closest to 1 best) 1.11563 raw 13653/13653 counted 0/0 
sync 88223/97910 up 0 down 0
[junit] 
[junit] 
[junit] Threads = 100 elements = 10 (of size 64) partitions = 16
[junit]  original code:
[junit]   Duration = 215ms maxConcurrency = 100
[junit]   GC for PS Scavenge: 23 ms for 2 collections
[junit]   Approx allocation = 776MB vs 8MB; ratio to raw data size = 
96.92138285714286
[junit]   loopRatio (closest to 1 best) 1.95927 raw 10/195927 counted 
0/0 sync 0/0 up 0 down 0
[junit] 
[junit]  modified code: 
[junit]   Duration = 201ms maxConcurrency = 99
[junit]   GC for PS Scavenge: 9 ms for 1 collections

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069065#comment-14069065
 ] 

Benedict commented on CASSANDRA-7546:
-

I'll take a look at your patch shortly, but in the meantime it's worth pointing 
out cassandra-stress does now support fairly complex CQL inserts including 
various sizes of batch updates, with fine grained control over how large a 
partition to generate, and what percentage of that total partition to update at 
any point. Take a look at the sample stress profiles under the tools hierarchy 
on latest 2.1

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7507) OOM creates unreliable state - die instantly better

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7507:
---

Attachment: exceptionHandlingResults.txt
findSwallowedExceptions.py

Some quick and dirty parsing of the code base gives me:
{code:title=Parse_Results}
Total Exceptions caught and rethrown as something other than RuntimeException: 
69  
Total Exceptions caught and rethrown as RuntimeException: 76
   
Total Exceptions Swallowed: 89  
  
Total Exceptions analyzed: 234  
  
{code}

While incredibly rough and not handling a bunch of edge-cases in the exception 
handling blocks, as a general metric of how pervasive these blanket catches are 
within the code-base this data should be helpful to get a general feel for how 
much work we're proposing here.

Attaching findSwallowedExceptions.py and exceptionHandlingResults.txt which is 
output from the script.

While I agree that this is something we need to look into, I'm not sure 2.1.1 
and this ticket is the right place to make changes that are this extensive.  
3.0 and another ticket to specifically go through the code-base and clean up 
blanket exception catch blocks to not swallow and be very specific on the logic 
surrounding rethrows would work for me.  WDYT [~jbellis]?

 OOM creates unreliable state - die instantly better
 ---

 Key: CASSANDRA-7507
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
 Project: Cassandra
  Issue Type: New Feature
Reporter: Karl Mueller
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7507_v1.txt, 7507_v2.txt, exceptionHandlingResults.txt, 
 findSwallowedExceptions.py


 I had a cassandra node run OOM. My heap had enough headroom, there was just 
 something which either was a bug or some unfortunate amount of short-term 
 memory utilization. This resulted in the following error:
  WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java 
 (line 1713) Some hints were not written before shutdown.  This is not 
 supposed to happen.  You should (a) run repair, and (b) file a bug report
 There are no other messages of relevance besides the OOM error about 90 
 minutes earlier.
 My (limited) understanding of the JVM and Cassandra says that when it goes 
 OOM, it will attempt to signal cassandra to shut down cleanly. The problem, 
 in my view, is that with an OOM situation, nothing is guaranteed anymore. I 
 believe it's impossible to reliably cleanly shut down at this point, and 
 therefore it's wrong to even try. 
 Yes, ideally things could be written out, flushed to disk, memory messages 
 written, other nodes notified, etc. but why is there any reason to believe 
 any of those steps could happen? Would happen? Couldn't bad data be written 
 at this point to disk rather than good data? Some network messages delivered, 
 but not others?
 I think Cassandra should have the option to (and possibly default) to kill 
 itself immediately upon the OOM condition happening in a hard way, and not 
 rely on the java-based clean shutdown process. Cassandra already handles 
 recovery from unclean shutdown, and it's not a big deal. My node, for 
 example, kept in a sort-of alive state for 90 minutes where who knows what it 
 was doing or not doing.
 I don't know enough about the JVM and options for it to know the best exact 
 implementation of die instantly on OOM, but it should be something that's 
 possible either with some flags or a C library (which doesn't rely on java 
 memory to do something which it may not be able to get!)
 Short version: a kill -9 of all C* processes in that instance without needing 
 more java memory, when OOM is raised



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069173#comment-14069173
 ] 

graham sanderson commented on CASSANDRA-7546:
-

Excellent - I will take a look in the 2.1 branch - I was wondering if there 
were some sample profiles.

The main problem we have in 2.0.x is that if we are under relatively heavy 
sustained write load, so we are allocating memtable slabs along with all the 
small short lived objects in the commit log and write path... you add to that 
hinting which means more memtable slabs, and now because of single partition 
for hints, much larger snap trees (whose somewhat contentious 
lazy-copy-on-write may or may not make things worse, I don't know)... under 
that allocation rate we spill huge numbers of small (possibly snap tree nodes) 
objects into the tenured gen along with the slabs, which tends to lead to 
promotion failure and need for compaction.

I'll have to play around, but I don't think it is easy to capture the effect of 
excessive (intended to be) temporary object allocation in a stress test as 
opposed to excessive CPU because the GC can cope really well until it doesn't.

Note my belief is your new tree in 2.1 probably mitigates the problem quite a 
bit (no contention in the tree, wider nodes, less rebalancing etc), though I 
suggest we still fix the CAS loop allocation there too.


 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069218#comment-14069218
 ] 

Aleksey Yeschenko commented on CASSANDRA-7578:
--

Not available to anyone without access to table.schema_usertypes, only when 
both authn and authz are enabled. Not blocker-worthy.

 Non-superusers can't login to tools because of no access to 
 system.schema_usertypes
 ---

 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Priority: Blocker
 Fix For: 2.1.0

 Attachments: 7578.patch


 When I try to login to cqlsh as a non-superuser I get the following error:
 {noformat}
 ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
 permission on table system.schema_usertypes or any of its parents
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7578) Add system.schema_usertypes to the readable-by-default whitelist

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7578:
-

 Priority: Minor  (was: Blocker)
Fix Version/s: (was: 2.1.0)
   2.1.1
 Assignee: Mike Adamson
  Summary: Add system.schema_usertypes to the readable-by-default 
whitelist  (was: Non-superusers can't login to tools because of no access to 
system.schema_usertypes)

 Add system.schema_usertypes to the readable-by-default whitelist
 

 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7578.patch


 When I try to login to cqlsh as a non-superuser I get the following error:
 {noformat}
 ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
 permission on table system.schema_usertypes or any of its parents
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7468) Add time-based execution to cassandra-stress

[Matt Kennedy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Kennedy updated CASSANDRA-7468:


Attachment: trunk-7468-rebase.patch

 Add time-based execution to cassandra-stress
 

 Key: CASSANDRA-7468
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7468
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Matt Kennedy
Assignee: Matt Kennedy
Priority: Minor
 Fix For: 2.1.1

 Attachments: trunk-7468-rebase.patch, trunk-7468.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7468) Add time-based execution to cassandra-stress

[Matt Kennedy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069268#comment-14069268
 ] 

Matt Kennedy commented on CASSANDRA-7468:
-

Rebased to trunk. Changed '-d' parameter to '-duration'. Note, running this 
without the latest DataStax Java driver (2.1-beta2) results in some seemingly 
extraneous stack traces, but they don't seem to affect functionality.

 Add time-based execution to cassandra-stress
 

 Key: CASSANDRA-7468
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7468
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Matt Kennedy
Assignee: Matt Kennedy
Priority: Minor
 Fix For: 2.1.1

 Attachments: trunk-7468-rebase.patch, trunk-7468.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069309#comment-14069309
 ] 

sankalp kohli commented on CASSANDRA-2434:
--

This will be very nice to have in 2.0
cc [~brandon.williams]

 range movements can violate consistency
 ---

 Key: CASSANDRA-2434
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Assignee: T Jake Luciani
 Fix For: 2.1 beta2

 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt


 My reading (a while ago) of the code indicates that there is no logic 
 involved during bootstrapping that avoids consistency level violations. If I 
 recall correctly it just grabs neighbors that are currently up.
 There are at least two issues I have with this behavior:
 * If I have a cluster where I have applications relying on QUORUM with RF=3, 
 and bootstrapping complete based on only one node, I have just violated the 
 supposedly guaranteed consistency semantics of the cluster.
 * Nodes can flap up and down at any time, so even if a human takes care to 
 look at which nodes are up and things about it carefully before 
 bootstrapping, there's no guarantee.
 A complication is that not only does it depend on use-case where this is an 
 issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster 
 which is otherwise used for QUORUM operations you may wish to accept 
 less-than-quorum nodes during bootstrap in various emergency situations.
 A potential easy fix is to have bootstrap take an argument which is the 
 number of hosts to bootstrap from, or to assume QUORUM if none is given.
 (A related concern is bootstrapping across data centers. You may *want* to 
 bootstrap to a local node and then do a repair to avoid sending loads of data 
 across DC:s while still achieving consistency. Or even if you don't care 
 about the consistency issues, I don't think there is currently a way to 
 bootstrap from local nodes only.)
 Thoughts?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7468) Add time-based execution to cassandra-stress

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7468:


Attachment: 7468v2.txt

Attached a slightly tweaked version that uses 
Uninterruptibles.sleepUninterruptibly instead of Thread.sleep(), and also asks 
that the duration units be specified, for the hell of it (supporting s, m and 
h)... duration seemed a bit strange without it :-)

For the record, and posterity, the switch is actually duration=, not -duration

If you review the changes I'll commit to trunk + 2.1

 Add time-based execution to cassandra-stress
 

 Key: CASSANDRA-7468
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7468
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Matt Kennedy
Assignee: Matt Kennedy
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7468v2.txt, trunk-7468-rebase.patch, trunk-7468.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7581) Stop read requests for partitions which don't belong to a node

[sankalp kohli (JIRA)]

sankalp kohli created CASSANDRA-7581:


 Summary: Stop read requests for partitions which don't belong to a 
node
 Key: CASSANDRA-7581
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7581
 Project: Cassandra
  Issue Type: Bug
Reporter: sankalp kohli
Priority: Minor


A C* instance will serve CF level reads with null values or old values to a 
co-ordinator even if the instance is not responsible for the data. 
This is bad if co-ordinator has a wrong view of the ring and will return wrong 
results back to the client. 
A solution to this problem could be to not respond or return error to such read 
requests. 
Tricky part here is to respond to read requests when the instance has just lost 
ownership for a few ring delay. 
 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair

[Russell Alexander Spitzer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069324#comment-14069324
 ] 

Russell Alexander Spitzer commented on CASSANDRA-6851:
--

The only thing I worry about is changing the signature on,
bikeshed
{code}
public CollectionCollectionSSTableReader 
groupSSTablesForAntiCompaction(CollectionSSTableReader sstablesToGroup, int 
groupSize)
{code}

I was wondering if we might want multiple implementations, some of which may 
not have a set group size. I know this is bikesheding, but maybe we want to 
change the signature to maxGroupSize? 
/bikeshed

I'm +1 with everything else

 Improve anticompaction after incremental repair
 ---

 Key: CASSANDRA-6851
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Marcus Eriksson
Assignee: Russell Alexander Spitzer
Priority: Minor
  Labels: compaction, lhf
 Fix For: 3.0


 After an incremental repair we iterate over all sstables and split them in 
 two parts, one containing the repaired data and one the unrepaired. We could 
 in theory double the number of sstables on a node.
 To avoid this we could make anticompaction also do a compaction, for example, 
 if we are to anticompact 10 sstables, we could anticompact those to 2.
 Note that we need to avoid creating too big sstables though, if we 
 anticompact all sstables on a node it would essentially be a major compaction.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6572) Workload recording / playback

[Lyuben Todorov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069328#comment-14069328
 ] 

Lyuben Todorov commented on CASSANDRA-6572:
---

variablesClone was used for debugging the values being stored for replay so it 
was indeed redundant, pushing 
[6228df6|https://github.com/lyubent/cassandra/commit/6228df6e5f047b29476744b1012802de37edda37]
 that makes replay of prepared statements functional, now string statements and 
prepared statements can be mixed and replayed. The commit also addressed the 
above issues. 

Next up is logging of thread  session ids.

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: no need to enforce case, when using equalsIgnoreCase

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2093270c7 - 453292f3c


no need to enforce case, when using equalsIgnoreCase


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/453292f3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/453292f3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/453292f3

Branch: refs/heads/trunk
Commit: 453292f3c6a443aed72a4b53203f1a46b26d9aaf
Parents: 2093270
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Jul 21 17:41:43 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Jul 21 17:41:43 2014 -0400

--
 tools/stress/src/org/apache/cassandra/stress/StressProfile.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/453292f3/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 13f26a2..8037ba5 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -492,7 +492,7 @@ public class StressProfile implements Serializable
 while (iter.hasNext())
 {
 Map.EntryString, V e = iter.next();
-if (!e.getKey().toLowerCase().equalsIgnoreCase(e.getKey()))
+if (!e.getKey().equalsIgnoreCase(e.getKey()))
 {
 reinsert.add(e);
 iter.remove();

[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069351#comment-14069351
 ] 

Brandon Williams commented on CASSANDRA-2434:
-

Does seem like mostly new code we could just add with the flag defaulting to 
off to give 2.0 the option.

 range movements can violate consistency
 ---

 Key: CASSANDRA-2434
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Assignee: T Jake Luciani
 Fix For: 2.1 beta2

 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt


 My reading (a while ago) of the code indicates that there is no logic 
 involved during bootstrapping that avoids consistency level violations. If I 
 recall correctly it just grabs neighbors that are currently up.
 There are at least two issues I have with this behavior:
 * If I have a cluster where I have applications relying on QUORUM with RF=3, 
 and bootstrapping complete based on only one node, I have just violated the 
 supposedly guaranteed consistency semantics of the cluster.
 * Nodes can flap up and down at any time, so even if a human takes care to 
 look at which nodes are up and things about it carefully before 
 bootstrapping, there's no guarantee.
 A complication is that not only does it depend on use-case where this is an 
 issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster 
 which is otherwise used for QUORUM operations you may wish to accept 
 less-than-quorum nodes during bootstrap in various emergency situations.
 A potential easy fix is to have bootstrap take an argument which is the 
 number of hosts to bootstrap from, or to assume QUORUM if none is given.
 (A related concern is bootstrapping across data centers. You may *want* to 
 bootstrap to a local node and then do a repair to avoid sending loads of data 
 across DC:s while still achieving consistency. Or even if you don't care 
 about the consistency issues, I don't think there is currently a way to 
 bootstrap from local nodes only.)
 Thoughts?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: remove unbound method template parm

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 453292f3c - 19b7d56fe


remove unbound method template parm


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19b7d56f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19b7d56f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19b7d56f

Branch: refs/heads/trunk
Commit: 19b7d56fece687b108e1b6ee024e0f09592a529e
Parents: 453292f
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Jul 21 17:46:45 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Jul 21 17:46:45 2014 -0400

--
 src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19b7d56f/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java 
b/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
index 92fabf1..9469a9c 100644
--- a/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
@@ -223,7 +223,7 @@ public class KeyspaceMetrics
  * @param MetricValue 
  * @return Gaugegt;Long that computes sum of MetricValue.getValue()
  */
-private T extends Number GaugeLong createKeyspaceGauge(String name, 
final MetricValue extractor)
+private GaugeLong createKeyspaceGauge(String name, final MetricValue 
extractor)
 {
 allMetrics.add(name);
 return Metrics.newGauge(factory.createMetricName(name), new 
GaugeLong()

buildbot success in ASF Buildbot on cassandra-trunk

[buildbot]

The Buildbot has detected a restored build on builder cassandra-trunk while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/480

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] 453292f3c6a443aed72a4b53203f1a46b26d9aaf
Blamelist: Dave Brosius dbros...@mebigfatguy.com

Build succeeded!

sincerely,
 -The Buildbot

[jira] [Commented] (CASSANDRA-7468) Add time-based execution to cassandra-stress

[Matt Kennedy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069387#comment-14069387
 ] 

Matt Kennedy commented on CASSANDRA-7468:
-

Thanks for the review, units are a welcome addition. I'm also relieved you got 
rid of the countInSeconds boolean to do it. I felt cheap doing it that way :-)

Everything else looks good to me.

 Add time-based execution to cassandra-stress
 

 Key: CASSANDRA-7468
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7468
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Matt Kennedy
Assignee: Matt Kennedy
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7468v2.txt, trunk-7468-rebase.patch, trunk-7468.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069396#comment-14069396
 ] 

Benedict commented on CASSANDRA-7546:
-

My concern with the approach you've outlined is that we're barely a hair's 
breadth from a lock: as soon as we hit _any_ contention, we inflate to locking 
behaviour. This is good for large partitions, and most likely bad for small 
ones, and more to the point seems barely worth the complexity of not just 
making it a lock in the first place. On further consideration, I think I would 
perhaps prefer to run this lock-inflation behaviour based on the size of the 
aborted changes, so if the amount of work we've wasted exceeds some threshold 
we decide it's high time all threads were stopped to let us finish. We could in 
this scenario flip a switch that requires all modifications to acquire the 
monitor once we hit this threshold once; I would be fine with this behaviour, 
and it would be simple. 

I do wonder how much of a problem this is in 2.1, though. I wonder if the 
largest problem with these racy modifications isn't actually the massive 
amounts of memtable arena allocations they incur in 2.0 with all their 
transformation.apply() calls (which reallocate the mutation on the arena), 
which is most likely what causes the promotion failures, as they cannot be 
collected. I wonder if we shouldn't simply backport the logic to only allocate 
these once, or at most twice (the first time we race). It seems much more 
likely to me that this is where the pain is being felt.

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7559) Switch Stress from using math3.pair because it is unserializable

[Russell Alexander Spitzer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069424#comment-14069424
 ] 

Russell Alexander Spitzer commented on CASSANDRA-7559:
--

[~tjake], This is the patch we were talking about a few days ago for fixing 
StressD. I would be grateful if you could review and make sure I haven't done 
anything destructive.

 Switch Stress from using math3.pair because it is unserializable 
 -

 Key: CASSANDRA-7559
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7559
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Russell Alexander Spitzer
Assignee: Russell Alexander Spitzer
 Attachments: CASSANDRA-7559-2.1.patch


 Stress uses org.apache.commons.math3.util.Pair to hold information in 
 settings because eventually it is used in commons.math3.distributions. This 
 makes the settings unserializable so we can't run with StressDemon.
 {code}
 /bin/cassandra-stress user no_warmup ops(insert=1) n=1 
 profile=cqlstress-example.yaml -sendto 127.0.0.1
 Exception in thread main java.io.NotSerializableException: 
 org.apache.commons.math3.util.Pair
   at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
   at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
 at java.util.ArrayList.writeObject(ArrayList.java:742)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
 java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
   at 
 java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
   at 
 java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
   at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
   at 
 java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
   at 
 java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
   at 
 java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
   at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
   at 
 java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
   at 
 java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
   at 
 java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
   at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
   at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
   at org.apache.cassandra.stress.Stress.main(Stress.java:78)
 Control-C caught. Canceling running action and shutting down...
 {code}
 To fix this we can pass around serializable pairs and convert to 
 commons.math3 pairs before we actually pass the objects to the distribution 
 code. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Give read access to system.schema_usertypes to all authenticated users

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 3b14f6a5d - 3d2da7987


Give read access to system.schema_usertypes to all authenticated users

patch by Mike Adamson; reviewed by Aleksey Yeschenko for CASSANDRA-7578


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d2da798
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d2da798
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d2da798

Branch: refs/heads/cassandra-2.1
Commit: 3d2da7987e3a6d0f1e088af9c661068335cc9f67
Parents: 3b14f6a
Author: Mike Adamson madam...@datastax.com
Authored: Mon Jul 21 14:36:01 2014 +0100
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:27:22 2014 +0300

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/ClientState.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2aa2efa..f05c98e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -7,6 +7,8 @@
  * Configurable client timeout for cqlsh (CASSANDRA-7516)
  * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
  * Fix native protocol drop user type notification (CASSANDRA-7571)
+ * Give read access to system.schema_usertypes to all authenticated users
+   (CASSANDRA-7578)
 
 
 2.1.0-rc4

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/src/java/org/apache/cassandra/service/ClientState.java
--
diff --git a/src/java/org/apache/cassandra/service/ClientState.java 
b/src/java/org/apache/cassandra/service/ClientState.java
index be3b895..c0396cb 100644
--- a/src/java/org/apache/cassandra/service/ClientState.java
+++ b/src/java/org/apache/cassandra/service/ClientState.java
@@ -67,7 +67,8 @@ public class ClientState
SystemKeyspace.PEERS_CF,
SystemKeyspace.SCHEMA_KEYSPACES_CF,
SystemKeyspace.SCHEMA_COLUMNFAMILIES_CF,
-   SystemKeyspace.SCHEMA_COLUMNS_CF };
+   SystemKeyspace.SCHEMA_COLUMNS_CF,
+   SystemKeyspace.SCHEMA_USER_TYPES_CF};
 for (String cf : cfs)
 
READABLE_SYSTEM_RESOURCES.add(DataResource.columnFamily(Keyspace.SYSTEM_KS, 
cf));
 

[1/2] git commit: Give read access to system.schema_usertypes to all authenticated users

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 19b7d56fe - 9c9fa1498


Give read access to system.schema_usertypes to all authenticated users

patch by Mike Adamson; reviewed by Aleksey Yeschenko for CASSANDRA-7578


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d2da798
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d2da798
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d2da798

Branch: refs/heads/trunk
Commit: 3d2da7987e3a6d0f1e088af9c661068335cc9f67
Parents: 3b14f6a
Author: Mike Adamson madam...@datastax.com
Authored: Mon Jul 21 14:36:01 2014 +0100
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:27:22 2014 +0300

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/ClientState.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2aa2efa..f05c98e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -7,6 +7,8 @@
  * Configurable client timeout for cqlsh (CASSANDRA-7516)
  * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
  * Fix native protocol drop user type notification (CASSANDRA-7571)
+ * Give read access to system.schema_usertypes to all authenticated users
+   (CASSANDRA-7578)
 
 
 2.1.0-rc4

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/src/java/org/apache/cassandra/service/ClientState.java
--
diff --git a/src/java/org/apache/cassandra/service/ClientState.java 
b/src/java/org/apache/cassandra/service/ClientState.java
index be3b895..c0396cb 100644
--- a/src/java/org/apache/cassandra/service/ClientState.java
+++ b/src/java/org/apache/cassandra/service/ClientState.java
@@ -67,7 +67,8 @@ public class ClientState
SystemKeyspace.PEERS_CF,
SystemKeyspace.SCHEMA_KEYSPACES_CF,
SystemKeyspace.SCHEMA_COLUMNFAMILIES_CF,
-   SystemKeyspace.SCHEMA_COLUMNS_CF };
+   SystemKeyspace.SCHEMA_COLUMNS_CF,
+   SystemKeyspace.SCHEMA_USER_TYPES_CF};
 for (String cf : cfs)
 
READABLE_SYSTEM_RESOURCES.add(DataResource.columnFamily(Keyspace.SYSTEM_KS, 
cf));
 

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

[aleksey]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9c9fa149
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9c9fa149
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9c9fa149

Branch: refs/heads/trunk
Commit: 9c9fa1498096c51db7692a453f4af77f9276adfa
Parents: 19b7d56 3d2da79
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:30:10 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:30:10 2014 +0300

--
 CHANGES.txt| 2 ++
 src/java/org/apache/cassandra/service/ClientState.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9c9fa149/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9c9fa149/src/java/org/apache/cassandra/service/ClientState.java
--

[jira] [Updated] (CASSANDRA-7578) Give read access to system.schema_usertypes to all authenticated users

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-7578:
-

Summary: Give read access to system.schema_usertypes to all authenticated 
users  (was: Add system.schema_usertypes to the readable-by-default whitelist)

 Give read access to system.schema_usertypes to all authenticated users
 --

 Key: CASSANDRA-7578
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7578.patch


 When I try to login to cqlsh as a non-superuser I get the following error:
 {noformat}
 ErrorMessage code=2100 [Unauthorized] message=User mike has no SELECT 
 permission on table system.schema_usertypes or any of its parents
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat

[Alex Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069450#comment-14069450
 ] 

Alex Liu commented on CASSANDRA-6454:
-

cassandra_pig.yaml uses Murmur3Partitioner instead of ByteOrderedPartitioner. 
ByteOrderedPartitioner is used for other unit tests.

 Pig support for hadoop CqlInputFormat
 -

 Key: CASSANDRA-6454
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6454
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
 Fix For: 2.0.10

 Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, 
 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt


 CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it



--
This message was sent by Atlassian JIRA
(v6.2#6252)

git commit: Fix ReversedType(DateType) mapping to native protocol

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 875480958 - ff211a4c4


Fix ReversedType(DateType) mapping to native protocol

patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c

Branch: refs/heads/cassandra-2.0
Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2
Parents: 8754809
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:42:24 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:42:24 2014 +0300

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d339309..9909760 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.10
+ * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576)
  * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541)
  * Fix range merging when DES scores are zero (CASSANDRA-7535)
  * Warn when SSL certificates have expired (CASSANDRA-7528)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java
--
diff --git a/src/java/org/apache/cassandra/transport/DataType.java 
b/src/java/org/apache/cassandra/transport/DataType.java
index 1656d24..8cd7194 100644
--- a/src/java/org/apache/cassandra/transport/DataType.java
+++ b/src/java/org/apache/cassandra/transport/DataType.java
@@ -146,8 +146,9 @@ public enum DataType implements 
OptionCodec.CodecableDataType
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
+
 // For compatibility sake, we still return DateType as the timestamp 
type in resultSet metadata (#5723)
-else if (type instanceof DateType)
+if (type instanceof DateType)
 type = TimestampType.instance;
 
 DataType dt = dataTypeMap.get(type);

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

[aleksey]

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a7242ec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a7242ec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a7242ec

Branch: refs/heads/trunk
Commit: 3a7242eca51e703c33ed8a4ca7d61a66018f7a81
Parents: 9c9fa14 f9af6ab
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:46:32 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:46:32 2014 +0300

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a7242ec/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069459#comment-14069459
 ] 

Brandon Williams commented on CASSANDRA-6454:
-

Is there a reason we need to switch to it for this? (not that I disagree with 
it at all)

 Pig support for hadoop CqlInputFormat
 -

 Key: CASSANDRA-6454
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6454
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
 Fix For: 2.0.10

 Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, 
 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt


 CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[1/2] git commit: Fix ReversedType(DateType) mapping to native protocol

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 3d2da7987 - f9af6ab58


Fix ReversedType(DateType) mapping to native protocol

patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c

Branch: refs/heads/cassandra-2.1
Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2
Parents: 8754809
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:42:24 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:42:24 2014 +0300

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d339309..9909760 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.10
+ * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576)
  * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541)
  * Fix range merging when DES scores are zero (CASSANDRA-7535)
  * Warn when SSL certificates have expired (CASSANDRA-7528)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java
--
diff --git a/src/java/org/apache/cassandra/transport/DataType.java 
b/src/java/org/apache/cassandra/transport/DataType.java
index 1656d24..8cd7194 100644
--- a/src/java/org/apache/cassandra/transport/DataType.java
+++ b/src/java/org/apache/cassandra/transport/DataType.java
@@ -146,8 +146,9 @@ public enum DataType implements 
OptionCodec.CodecableDataType
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
+
 // For compatibility sake, we still return DateType as the timestamp 
type in resultSet metadata (#5723)
-else if (type instanceof DateType)
+if (type instanceof DateType)
 type = TimestampType.instance;
 
 DataType dt = dataTypeMap.get(type);

[1/3] git commit: Fix ReversedType(DateType) mapping to native protocol

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9c9fa1498 - 3a7242eca


Fix ReversedType(DateType) mapping to native protocol

patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c

Branch: refs/heads/trunk
Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2
Parents: 8754809
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:42:24 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:42:24 2014 +0300

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d339309..9909760 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.10
+ * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576)
  * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541)
  * Fix range merging when DES scores are zero (CASSANDRA-7535)
  * Warn when SSL certificates have expired (CASSANDRA-7528)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java
--
diff --git a/src/java/org/apache/cassandra/transport/DataType.java 
b/src/java/org/apache/cassandra/transport/DataType.java
index 1656d24..8cd7194 100644
--- a/src/java/org/apache/cassandra/transport/DataType.java
+++ b/src/java/org/apache/cassandra/transport/DataType.java
@@ -146,8 +146,9 @@ public enum DataType implements 
OptionCodec.CodecableDataType
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
+
 // For compatibility sake, we still return DateType as the timestamp 
type in resultSet metadata (#5723)
-else if (type instanceof DateType)
+if (type instanceof DateType)
 type = TimestampType.instance;
 
 DataType dt = dataTypeMap.get(type);

[jira] [Commented] (CASSANDRA-7570) CqlPagingRecordReader is broken

[Alex Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069460#comment-14069460
 ] 

Alex Liu commented on CASSANDRA-7570:
-

It can be fixed at CPRR by changing  to = ( to =), then filter the first 
row if it's equal to the last row fetched. 

Since we have new CRR, we should depreciate CPRR. Should we fix this issue or 
just warn user that compact table is broken, should use the new CRR instead.

 CqlPagingRecordReader is broken
 ---

 Key: CASSANDRA-7570
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7570
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Brandon Williams
Assignee: Alex Liu
 Fix For: 2.0.10


 As mentioned on CASSANDRA-7059, it broke CPRR.  It's not quite as simple as 
 changing the greater than to a greater than equal, either, since that makes 
 the task run forever.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[aleksey]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9af6ab5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9af6ab5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9af6ab5

Branch: refs/heads/cassandra-2.1
Commit: f9af6ab5874f4d33e32af1af8934bd7c0bed1724
Parents: 3d2da79 ff211a4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:46:11 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:46:11 2014 +0300

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/CHANGES.txt
--
diff --cc CHANGES.txt
index f05c98e,9909760..028128d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,29 -1,5 +1,31 @@@
 -2.0.10
 +2.1.1
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Fail to start if commit log replay detects a problem (CASSANDRA-7125)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 + * Fix native protocol drop user type notification (CASSANDRA-7571)
 + * Give read access to system.schema_usertypes to all authenticated users
 +   (CASSANDRA-7578)
++Merged from 2.0:
+  * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576)
 +
 +
 +2.1.0-rc4
 + * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
 +   (CASSANDRA-7551)
 + * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)
 + * Anti-compaction proceeds if any part of the repair failed (CASANDRA-7521)
 + * Add missing table name to DROP INDEX responses and notifications 
(CASSANDRA-7539)
 + * Bump CQL version to 3.2.0 and update CQL documentation (CASSANDRA-7527)
 + * Fix configuration error message when running nodetool ring (CASSANDRA-7508)
 + * Support conditional updates, tuple type, and the v3 protocol in cqlsh 
(CASSANDRA-7509)
 + * Handle queries on multiple secondary index types (CASSANDRA-7525)
 + * Fix cqlsh authentication with v3 native protocol (CASSANDRA-7564)
 + * Fix NPE when unknown prepared statement ID is used (CASSANDRA-7454)
 +Merged from 2.0:
   * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541)
   * Fix range merging when DES scores are zero (CASSANDRA-7535)
   * Warn when SSL certificates have expired (CASSANDRA-7528)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/src/java/org/apache/cassandra/transport/DataType.java
--

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

[aleksey]

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9af6ab5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9af6ab5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9af6ab5

Branch: refs/heads/trunk
Commit: f9af6ab5874f4d33e32af1af8934bd7c0bed1724
Parents: 3d2da79 ff211a4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jul 22 01:46:11 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jul 22 01:46:11 2014 +0300

--
 CHANGES.txt   | 2 ++
 src/java/org/apache/cassandra/transport/DataType.java | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/CHANGES.txt
--
diff --cc CHANGES.txt
index f05c98e,9909760..028128d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,29 -1,5 +1,31 @@@
 -2.0.10
 +2.1.1
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Fail to start if commit log replay detects a problem (CASSANDRA-7125)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 + * Fix native protocol drop user type notification (CASSANDRA-7571)
 + * Give read access to system.schema_usertypes to all authenticated users
 +   (CASSANDRA-7578)
++Merged from 2.0:
+  * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576)
 +
 +
 +2.1.0-rc4
 + * Updated memtable_cleanup_threshold and memtable_flush_writers defaults 
 +   (CASSANDRA-7551)
 + * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505)
 + * Anti-compaction proceeds if any part of the repair failed (CASANDRA-7521)
 + * Add missing table name to DROP INDEX responses and notifications 
(CASSANDRA-7539)
 + * Bump CQL version to 3.2.0 and update CQL documentation (CASSANDRA-7527)
 + * Fix configuration error message when running nodetool ring (CASSANDRA-7508)
 + * Support conditional updates, tuple type, and the v3 protocol in cqlsh 
(CASSANDRA-7509)
 + * Handle queries on multiple secondary index types (CASSANDRA-7525)
 + * Fix cqlsh authentication with v3 native protocol (CASSANDRA-7564)
 + * Fix NPE when unknown prepared statement ID is used (CASSANDRA-7454)
 +Merged from 2.0:
   * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541)
   * Fix range merging when DES scores are zero (CASSANDRA-7535)
   * Warn when SSL certificates have expired (CASSANDRA-7528)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/src/java/org/apache/cassandra/transport/DataType.java
--

[jira] [Commented] (CASSANDRA-7570) CqlPagingRecordReader is broken

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069463#comment-14069463
 ] 

Brandon Williams commented on CASSANDRA-7570:
-

We probably at least need to fix it in 2.0, since that's where we broke it, but 
it's also a bit late to deprecate in 2.1.

 CqlPagingRecordReader is broken
 ---

 Key: CASSANDRA-7570
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7570
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Brandon Williams
Assignee: Alex Liu
 Fix For: 2.0.10


 As mentioned on CASSANDRA-7059, it broke CPRR.  It's not quite as simple as 
 changing the greater than to a greater than equal, either, since that makes 
 the task run forever.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069504#comment-14069504
 ] 

graham sanderson commented on CASSANDRA-7546:
-

{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the 
largest problem with these racy modifications isn't actually the massive 
amounts of memtable arena allocations they incur in 2.0 with all their 
transformation.apply() calls (which reallocate the mutation on the arena), 
which is most likely what causes the promotion failures, as they cannot be 
collected. I wonder if we shouldn't simply backport the logic to only allocate 
these once, or at most twice (the first time we race). It seems much more 
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the 
at most twice refers to looping once then locking. Certainly avoiding any 
repeated cloning of the cells is good, however I'm still pretty sure based on 
PrintFLSStatistics that the slabs themselves are not the biggest problem (I 
suspect SnapTreeMap nodes, combined with high rebalancing cost of huge trees in 
the hint case since the keys are almost entirely sorted).

Are you suggesting a one way switch per Atomic*Columns instance that flips 
after a number waster operations? That sounds reasonable... I'd expect that a 
partition for a table is either likely to have high contention or not based on 
the schema design/use case. I have no idea how long these instances hang around 
in practice (presumably not insanely long)

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069504#comment-14069504
 ] 

graham sanderson edited comment on CASSANDRA-7546 at 7/21/14 11:28 PM:
---

{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the 
largest problem with these racy modifications isn't actually the massive 
amounts of memtable arena allocations they incur in 2.0 with all their 
transformation.apply() calls (which reallocate the mutation on the arena), 
which is most likely what causes the promotion failures, as they cannot be 
collected. I wonder if we shouldn't simply backport the logic to only allocate 
these once, or at most twice (the first time we race). It seems much more 
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the 
at most twice refers to looping once then locking, and what is reasonable to 
modify in 2.0.x now. Certainly avoiding any repeated cloning of the cells is 
good, however I'm still pretty sure based on PrintFLSStatistics that the slabs 
themselves are not the biggest problem (I suspect SnapTreeMap nodes, combined 
with high rebalancing cost of huge trees in the hint case since the keys are 
almost entirely sorted).

Are you suggesting a one way switch per Atomic*Columns instance that flips 
after a number waster operations? That sounds reasonable... I'd expect that a 
partition for a table is either likely to have high contention or not based on 
the schema design/use case. I have no idea how long these instances hang around 
in practice (presumably not insanely long) at least if they are being actively 
used since I assume they get flushed eventually in that case, and if they 
aren't it doesn't really matter anyway


was (Author: graham sanderson):
{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the 
largest problem with these racy modifications isn't actually the massive 
amounts of memtable arena allocations they incur in 2.0 with all their 
transformation.apply() calls (which reallocate the mutation on the arena), 
which is most likely what causes the promotion failures, as they cannot be 
collected. I wonder if we shouldn't simply backport the logic to only allocate 
these once, or at most twice (the first time we race). It seems much more 
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the 
at most twice refers to looping once then locking. Certainly avoiding any 
repeated cloning of the cells is good, however I'm still pretty sure based on 
PrintFLSStatistics that the slabs themselves are not the biggest problem (I 
suspect SnapTreeMap nodes, combined with high rebalancing cost of huge trees in 
the hint case since the keys are almost entirely sorted).

Are you suggesting a one way switch per Atomic*Columns instance that flips 
after a number waster operations? That sounds reasonable... I'd expect that a 
partition for a table is either likely to have high contention or not based on 
the schema design/use case. I have no idea how long these instances hang around 
in practice (presumably not insanely long)

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7576) DateType columns not properly converted to TimestampType when in ReversedType columns.

[Karl Rieb (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069557#comment-14069557
 ] 

Karl Rieb commented on CASSANDRA-7576:
--

Thanks [~iamaleksey]!  No problems with the 'patch by', it's fine by me.  

Do you know approximately how long before *2.0.10* is released?  I'm guessing 
some time around August?

 DateType columns not properly converted to TimestampType when in ReversedType 
 columns.
 --

 Key: CASSANDRA-7576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7576
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Karl Rieb
Assignee: Karl Rieb
 Fix For: 2.0.10, 2.1.1

 Attachments: DataType_CASSANDRA_7576.patch

   Original Estimate: 0.25h
  Remaining Estimate: 0.25h

 The {{org.apache.cassandra.transport.DataType.fromType(AbstractType)}} method 
 has a bug that prevents sending the correct Protocol ID for reversed 
 {{DateType}} columns.   This results in clients receiving Protocol ID {{0}}, 
 which maps to a {{CUSTOM}} type, for timestamp columns that are clustered in 
 reverse order.  
 Some clients can handle this properly since they recognize the 
 {{org.apache.cassandra.db.marshal.DateType}} marshaling type, however the 
 native Datastax java-driver does not.  It will produce errors like the one 
 below when trying to prepare queries against such tables:
 {noformat}
 com.datastax.driver.core.exceptions.InvalidTypeException: Invalid type 
 for value 2 of CQL type 'org.apache.cassandra.db.marshal.DateType', expecting 
 class java.nio.ByteBuffer but class java.util.Date provided
   at com.datastax.driver.core.BoundStatement.bind(BoundStatement.java:190)
   at 
 com.datastax.driver.core.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:103)
 {noformat}
 On the Cassandra side, there is a check for {{DateType}} columns that is 
 supposed to convert these columns to TimestampType.  However, the check is 
 skipped when the column is also reversed.  Specifically:
 {code:title=DataType.java|borderStyle=solid}
 public static PairDataType, Object fromType(AbstractType type)
 {
 // For CQL3 clients, ReversedType is an implementation detail and they
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
 // For compatibility sake, we still return DateType as the timestamp type 
 in resultSet metadata (#5723)
 else if (type instanceof DateType)
 type = TimestampType.instance;
 // ...
 {code}
 The *else if* should be changed to just an *if*, like so:
 {code:title=DataType.java|borderStyle=solid}
 public static PairDataType, Object fromType(AbstractType type)
 {
 // For CQL3 clients, ReversedType is an implementation detail and they
 // shouldn't have to care about it.
 if (type instanceof ReversedType)
 type = ((ReversedType)type).baseType;
 // For compatibility sake, we still return DateType as the timestamp type 
 in resultSet metadata (#5723)
 if (type instanceof DateType)
 type = TimestampType.instance;
 // ...
 {code}
 This bug is preventing us from upgrading our 1.2.11 cluster to 2.0.9 because 
 our clients keep throwing exceptions trying to read or write data to tables 
 with reversed timestamp columns. This issue can be reproduced by creating a 
 CQL table in Cassandra 1.2.11 that clusters on a timestamp in reverse, then 
 upgrading the node to 2.0.9.  When querying the metadata for the table, the 
 node will return Protocol ID 0 (CUSTOM) instead of Protocol ID 11 (TIMESTAMP).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat

[Alex Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069620#comment-14069620
 ] 

Alex Liu commented on CASSANDRA-6454:
-

Most of Cassandra deployment uses Murmur3Partitioner, testing on 
Murmur3Partitioner covers the general case. Some unit tests still use old 
ByteOrderedPartitioner, so just update the cassanra.yaml to Murmur3Partitioner 
breaks other unit tests. That's the reason I create a new yaml file.

 Pig support for hadoop CqlInputFormat
 -

 Key: CASSANDRA-6454
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6454
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
 Fix For: 2.0.10

 Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, 
 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt


 CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory

[graham sanderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069666#comment-14069666
 ] 

graham sanderson commented on CASSANDRA-7546:
-

Alternatively if you are saying, let each thread keep working while they still 
believe they can win, or while they have something to do that can be reused if 
they lose, then maybe give them one last shot to try again if they lose and 
haven't done anything reusable, then make them block... I'm okay with that. (of 
course on 2.0.x. today, that pretty much boils down to your patch!)

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, 
 suggestion1.txt, suggestion1_21.txt


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors

[Ryan McGuire (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-7582:


Assignee: Russ Hatch

 2.1 multi-dc upgrade errors
 ---

 Key: CASSANDRA-7582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Russ Hatch

 Multi-dc upgrade [was working from 2.0 - 2.1 fairly 
 recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
  but is currently failing.
 Running 
 upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
  I get the following errors when starting 2.1 upgraded from 2.0:
 {code}
 ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay 
 failed due to replaying a mutation for a missing table. This error can be 
 ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on 
 the command line
 ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception 
 encountered during startup
 java.lang.RuntimeException: 
 org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
 cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
 [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
  [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546) 
 [main/:na]
 Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't 
 find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
  ~[main/:na]
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115) 
 ~[main/:na]
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (CASSANDRA-7582) 2.1 multi-dc upgrade errors

[Ryan McGuire (JIRA)]

Ryan McGuire created CASSANDRA-7582:
---

 Summary: 2.1 multi-dc upgrade errors
 Key: CASSANDRA-7582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire


Multi-dc upgrade [was working from 2.0 - 2.1 fairly 
recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
 but is currently failing.

Running 
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
 I get the following errors when starting 2.1 upgraded from 2.0:

{code}
ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay 
failed due to replaying a mutation for a missing table. This error can be 
ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on the 
command line
ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception 
encountered during startup
java.lang.RuntimeException: 
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546) 
[main/:na]
Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
at 
org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
 ~[main/:na]
at 
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
 ~[main/:na]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
 ~[main/:na]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
 ~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
 ~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
 ~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137) 
~[main/:na]
at 
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115) 
~[main/:na]
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7582) 2.1 multi-dc upgrade errors

[Ryan McGuire (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069788#comment-14069788
 ] 

Ryan McGuire commented on CASSANDRA-7582:
-

[~rhatch] can you repro and bisect this? It was passing last week.

 2.1 multi-dc upgrade errors
 ---

 Key: CASSANDRA-7582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Russ Hatch

 Multi-dc upgrade [was working from 2.0 - 2.1 fairly 
 recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
  but is currently failing.
 Running 
 upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
  I get the following errors when starting 2.1 upgraded from 2.0:
 {code}
 ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay 
 failed due to replaying a mutation for a missing table. This error can be 
 ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on 
 the command line
 ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception 
 encountered during startup
 java.lang.RuntimeException: 
 org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
 cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
 [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
  [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546) 
 [main/:na]
 Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't 
 find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
  ~[main/:na]
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115) 
 ~[main/:na]
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors

[Ryan McGuire (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-7582:


Since Version: 2.1 rc3

 2.1 multi-dc upgrade errors
 ---

 Key: CASSANDRA-7582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Russ Hatch

 Multi-dc upgrade [was working from 2.0 - 2.1 fairly 
 recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
  but is currently failing.
 Running 
 upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
  I get the following errors when starting 2.1 upgraded from 2.0:
 {code}
 ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay 
 failed due to replaying a mutation for a missing table. This error can be 
 ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on 
 the command line
 ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception 
 encountered during startup
 java.lang.RuntimeException: 
 org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
 cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
 [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
  [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546) 
 [main/:na]
 Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't 
 find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
  ~[main/:na]
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115) 
 ~[main/:na]
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7370) Create a new system table node_config to load cassandra.yaml config data.

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7370:


Attachment: (was: 7370-v1-prototype.diff)

 Create a new system table node_config to load cassandra.yaml config data.
 ---

 Key: CASSANDRA-7370
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7370
 Project: Cassandra
  Issue Type: Wish
  Components: Config
Reporter: Hayato Shimizu
Assignee: Robert Stupp
Priority: Minor
  Labels: ponies
 Attachments: 7370-v3.txt


 Currently the node configuration information specified in cassandra.yaml can 
 only be viewed via JMX or by looking at the file on individual machines.
 As an administrator, it would be extremely useful to be able to execute 
 queries like the following example;
 select concurrent_reads from system.node_config;
 which will list all the concurrent_reads value from all of the nodes in a 
 cluster.
 This will require a new table in the system keyspace and the data to be 
 loaded (if required) during the bootstrap, and updated when MBeans attribute 
 value updates are performed. The data from other nodes in the cluster is also 
 required in the table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7370) Create a new system table node_config to load cassandra.yaml config data.

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7370:


Attachment: 7370-v3.txt

merge trunk

 Create a new system table node_config to load cassandra.yaml config data.
 ---

 Key: CASSANDRA-7370
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7370
 Project: Cassandra
  Issue Type: Wish
  Components: Config
Reporter: Hayato Shimizu
Assignee: Robert Stupp
Priority: Minor
  Labels: ponies
 Attachments: 7370-v3.txt


 Currently the node configuration information specified in cassandra.yaml can 
 only be viewed via JMX or by looking at the file on individual machines.
 As an administrator, it would be extremely useful to be able to execute 
 queries like the following example;
 select concurrent_reads from system.node_config;
 which will list all the concurrent_reads value from all of the nodes in a 
 cluster.
 This will require a new table in the system keyspace and the data to be 
 loaded (if required) during the bootstrap, and updated when MBeans attribute 
 value updates are performed. The data from other nodes in the cluster is also 
 required in the table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7370) Create a new system table node_config to load cassandra.yaml config data.

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7370:


Attachment: (was: 7370-v2-with-jmx.diff)

 Create a new system table node_config to load cassandra.yaml config data.
 ---

 Key: CASSANDRA-7370
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7370
 Project: Cassandra
  Issue Type: Wish
  Components: Config
Reporter: Hayato Shimizu
Assignee: Robert Stupp
Priority: Minor
  Labels: ponies
 Attachments: 7370-v3.txt


 Currently the node configuration information specified in cassandra.yaml can 
 only be viewed via JMX or by looking at the file on individual machines.
 As an administrator, it would be extremely useful to be able to execute 
 queries like the following example;
 select concurrent_reads from system.node_config;
 which will list all the concurrent_reads value from all of the nodes in a 
 cluster.
 This will require a new table in the system keyspace and the data to be 
 loaded (if required) during the bootstrap, and updated when MBeans attribute 
 value updates are performed. The data from other nodes in the cluster is also 
 required in the table.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-7582:
--

  Component/s: Core
 Priority: Critical  (was: Major)
Fix Version/s: 2.1.0

 2.1 multi-dc upgrade errors
 ---

 Key: CASSANDRA-7582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ryan McGuire
Assignee: Russ Hatch
Priority: Critical
 Fix For: 2.1.0


 Multi-dc upgrade [was working from 2.0 - 2.1 fairly 
 recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
  but is currently failing.
 Running 
 upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
  I get the following errors when starting 2.1 upgraded from 2.0:
 {code}
 ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay 
 failed due to replaying a mutation for a missing table. This error can be 
 ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on 
 the command line
 ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception 
 encountered during startup
 java.lang.RuntimeException: 
 org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
 cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300) 
 [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
  [main/:na]
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546) 
 [main/:na]
 Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't 
 find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
  ~[main/:na]
 at 
 org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
  ~[main/:na]
 at 
 org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
  ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137) 
 ~[main/:na]
 at 
 org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115) 
 ~[main/:na]
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[Vijay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069862#comment-14069862
 ] 

Vijay commented on CASSANDRA-7438:
--

Attached patch makes the off heap/Serialization Cache configurable. (the 
default is still SerializationCache).

Regarding performance, the performance of the new cache is obviously better 
when the JNI overhead is less than the GC overhead, but for the smaller sized 
caches which can fit in memory the performance is little lower which is 
understandable (but both of them out perform pagecache performance by a large 
number). Here are the numbers.

OffheapCacheProvider

Running READ with 1200 threads  for 1000 iterations
ops   ,op/s,   key/s,mean, med, .95, .99,.999, 
max,   time,   stderr
2030355   , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,  
3014.5,1.0,  0.0
2395480   ,  202845,  202845, 5.8, 5.4, 5.8,20.2,   522.4,   
545.9,2.8,  0.0
2638600   ,  221368,  221368, 5.4, 5.3, 5.8,16.3,78.8,   
131.5,3.9,  0.57860
2891705   ,  221976,  221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0,  0.60478
3147747   ,  222527,  222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2,  0.58659
3394999   ,  221527,  221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3,  0.55884
3663559   ,  226114,  226114, 5.3, 5.2, 5.6,15.0,84.4,   
110.7,8.5,  0.52924
3911154   ,  223831,  223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6,  0.50018
4152946   ,  223246,  223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8,   10.7,  0.47323
4403162   ,  228532,  228532, 5.2, 5.2, 5.6,23.2,   107.4,   
121.4,   11.8,  0.44856
4641021   ,  225196,  225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4,   12.8,  0.42557
4889523   ,  222826,  222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0,   13.9,  0.40476
5124891   ,  223203,  223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8,   15.0,  0.38602
5375262   ,  221222,  221222, 5.4, 5.2, 5.6,18.4,94.2,   
115.1,   16.1,  0.36899
5616470   ,  224022,  224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8,   17.2,  0.35349
5866825   ,  223000,  223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2,   18.3,  0.33882
6125601   ,  225757,  225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0,   19.5,  0.32535
6348030   ,  192703,  192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5,   20.6,  0.31282
6483574   ,  128520,  128520, 9.3, 8.4,10.9,19.5,88.7,
99.0,   21.7,  0.30329
6626176   ,  137199,  137199, 8.7, 8.4,10.6,14.0,32.7,
40.9,   22.7,  0.29771
6768401   ,  136860,  136860, 8.8, 8.4,10.3,14.1,35.1,
40.8,   23.8,  0.29213
6911785   ,  138204,  138204, 8.7, 8.3,10.2,13.7,34.1,
37.8,   24.8,  0.28669
7055951   ,  138633,  138633, 8.7, 8.3,10.5,32.0,40.5,
46.9,   25.8,  0.28130
7199084   ,  137731,  137731, 8.7, 8.4,10.2,14.0,33.4,
40.9,   26.9,  0.27623
7338032   ,  133201,  133201, 9.0, 8.4,10.9,34.0,39.4,
43.8,   27.9,  0.27116
7480439   ,  137059,  137059, 8.8, 8.4,10.2,13.9,35.9,
39.5,   29.0,  0.26663
7647810   ,  161209,  161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9,   30.0,  0.26185
7898882   ,  226498,  226498, 5.3, 5.2, 5.6,19.7,   108.5,   
119.3,   31.1,  0.25629
8136305   ,  223840,  223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2,   32.2,  0.24838
8372076   ,  223790,  223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0,   33.2,  0.24095
8633758   ,  232914,  232914, 5.1, 5.2, 5.6,17.5,   138.4,   
182.0,   34.4,  0.23397
8869214   ,  43,  43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9,   35.4,  0.22717
9121652   ,  223037,  223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8,   36.5,  0.22105
9360286   ,  225070,  225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1,   37.6,  0.21524
9609676   ,  224089,  224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3,   38.7,  0.20967
9848551   ,  222123,  222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2,   39.8,  0.20440
1000  ,  229511,  229511, 5.0, 5.2, 5.8,60.0,74.3,   
132.0,   40.5,  0.19935


Results:
real op rate  : 247211
adjusted op rate stderr   : 0
key rate  : 247211
latency mean  : 5.4
latency median: 3.5
latency 95th percentile   : 5.5
latency 99th percentile   : 6.1
latency 99.9th percentile : 83.4
latency max   : 3014.5
Total operation 

[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[Vijay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069862#comment-14069862
 ] 

Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:49 AM:
---

Attached patch makes the off heap/Serialization Cache configurable. (the 
default is still SerializationCache).

Regarding performance, the performance of the new cache is obviously better 
when the JNI overhead is less than the GC overhead, but for the smaller sized 
caches which can fit in memory the performance is little lower which is 
understandable (but both of them out perform pagecache performance by a large 
number). Here are the numbers.

*OffheapCacheProvider*
{panel}
Running READ with 1200 threads  for 1000 iterations
ops   ,op/s,   key/s,mean, med, .95, .99,.999, 
max,   time,   stderr
2030355   , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,  
3014.5,1.0,  0.0
2395480   ,  202845,  202845, 5.8, 5.4, 5.8,20.2,   522.4,   
545.9,2.8,  0.0
2638600   ,  221368,  221368, 5.4, 5.3, 5.8,16.3,78.8,   
131.5,3.9,  0.57860
2891705   ,  221976,  221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0,  0.60478
3147747   ,  222527,  222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2,  0.58659
3394999   ,  221527,  221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3,  0.55884
3663559   ,  226114,  226114, 5.3, 5.2, 5.6,15.0,84.4,   
110.7,8.5,  0.52924
3911154   ,  223831,  223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6,  0.50018
4152946   ,  223246,  223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8,   10.7,  0.47323
4403162   ,  228532,  228532, 5.2, 5.2, 5.6,23.2,   107.4,   
121.4,   11.8,  0.44856
4641021   ,  225196,  225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4,   12.8,  0.42557
4889523   ,  222826,  222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0,   13.9,  0.40476
5124891   ,  223203,  223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8,   15.0,  0.38602
5375262   ,  221222,  221222, 5.4, 5.2, 5.6,18.4,94.2,   
115.1,   16.1,  0.36899
5616470   ,  224022,  224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8,   17.2,  0.35349
5866825   ,  223000,  223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2,   18.3,  0.33882
6125601   ,  225757,  225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0,   19.5,  0.32535
6348030   ,  192703,  192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5,   20.6,  0.31282
6483574   ,  128520,  128520, 9.3, 8.4,10.9,19.5,88.7,
99.0,   21.7,  0.30329
6626176   ,  137199,  137199, 8.7, 8.4,10.6,14.0,32.7,
40.9,   22.7,  0.29771
6768401   ,  136860,  136860, 8.8, 8.4,10.3,14.1,35.1,
40.8,   23.8,  0.29213
6911785   ,  138204,  138204, 8.7, 8.3,10.2,13.7,34.1,
37.8,   24.8,  0.28669
7055951   ,  138633,  138633, 8.7, 8.3,10.5,32.0,40.5,
46.9,   25.8,  0.28130
7199084   ,  137731,  137731, 8.7, 8.4,10.2,14.0,33.4,
40.9,   26.9,  0.27623
7338032   ,  133201,  133201, 9.0, 8.4,10.9,34.0,39.4,
43.8,   27.9,  0.27116
7480439   ,  137059,  137059, 8.8, 8.4,10.2,13.9,35.9,
39.5,   29.0,  0.26663
7647810   ,  161209,  161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9,   30.0,  0.26185
7898882   ,  226498,  226498, 5.3, 5.2, 5.6,19.7,   108.5,   
119.3,   31.1,  0.25629
8136305   ,  223840,  223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2,   32.2,  0.24838
8372076   ,  223790,  223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0,   33.2,  0.24095
8633758   ,  232914,  232914, 5.1, 5.2, 5.6,17.5,   138.4,   
182.0,   34.4,  0.23397
8869214   ,  43,  43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9,   35.4,  0.22717
9121652   ,  223037,  223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8,   36.5,  0.22105
9360286   ,  225070,  225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1,   37.6,  0.21524
9609676   ,  224089,  224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3,   38.7,  0.20967
9848551   ,  222123,  222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2,   39.8,  0.20440
1000  ,  229511,  229511, 5.0, 5.2, 5.8,60.0,74.3,   
132.0,   40.5,  0.19935


Results:
real op rate  : 247211
adjusted op rate stderr   : 0
key rate  : 247211
latency mean  : 5.4
latency median: 3.5
latency 95th percentile   : 5.5
latency 99th percentile   : 6.1
latency 99.9th percentile : 83.4
latency max  

[jira] [Updated] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[Vijay (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-7438:
-

Attachment: 0001-CASSANDRA-7438.patch

 Serializing Row cache alternative (Fully off heap)
 --

 Key: CASSANDRA-7438
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux
Reporter: Vijay
Assignee: Vijay
  Labels: performance
 Fix For: 3.0

 Attachments: 0001-CASSANDRA-7438.patch


 Currently SerializingCache is partially off heap, keys are still stored in 
 JVM heap as BB, 
 * There is a higher GC costs for a reasonably big cache.
 * Some users have used the row cache efficiently in production for better 
 results, but this requires careful tunning.
 * Overhead in Memory for the cache entries are relatively high.
 So the proposal for this ticket is to move the LRU cache logic completely off 
 heap and use JNI to interact with cache. We might want to ensure that the new 
 implementation match the existing API's (ICache), and the implementation 
 needs to have safe memory access, low overhead in memory and less memcpy's 
 (As much as possible).
 We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[Vijay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069862#comment-14069862
 ] 

Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:51 AM:
---

Attached patch makes the off heap/Serialization Cache configurable. (the 
default is still SerializationCache).

Regarding performance, the performance of the new cache is obviously better 
when the JNI overhead is less than the GC overhead, but for the smaller sized 
caches which can fit in memory the performance is little lower which is 
understandable (but both of them out perform pagecache performance by a large 
number). Here are the numbers.

*OffheapCacheProvider*
{panel}
Running READ with 1200 threads  for 1000 iterations
ops   ,op/s,   key/s,mean, med, .95, .99,.999, 
max,   time,   stderr
2030355   , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,  
3014.5,1.0,  0.0
2395480   ,  202845,  202845, 5.8, 5.4, 5.8,20.2,   522.4,   
545.9,2.8,  0.0
2638600   ,  221368,  221368, 5.4, 5.3, 5.8,16.3,78.8,   
131.5,3.9,  0.57860
2891705   ,  221976,  221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0,  0.60478
3147747   ,  222527,  222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2,  0.58659
3394999   ,  221527,  221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3,  0.55884
3663559   ,  226114,  226114, 5.3, 5.2, 5.6,15.0,84.4,   
110.7,8.5,  0.52924
3911154   ,  223831,  223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6,  0.50018
4152946   ,  223246,  223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8,   10.7,  0.47323
4403162   ,  228532,  228532, 5.2, 5.2, 5.6,23.2,   107.4,   
121.4,   11.8,  0.44856
4641021   ,  225196,  225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4,   12.8,  0.42557
4889523   ,  222826,  222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0,   13.9,  0.40476
5124891   ,  223203,  223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8,   15.0,  0.38602
5375262   ,  221222,  221222, 5.4, 5.2, 5.6,18.4,94.2,   
115.1,   16.1,  0.36899
5616470   ,  224022,  224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8,   17.2,  0.35349
5866825   ,  223000,  223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2,   18.3,  0.33882
6125601   ,  225757,  225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0,   19.5,  0.32535
6348030   ,  192703,  192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5,   20.6,  0.31282
6483574   ,  128520,  128520, 9.3, 8.4,10.9,19.5,88.7,
99.0,   21.7,  0.30329
6626176   ,  137199,  137199, 8.7, 8.4,10.6,14.0,32.7,
40.9,   22.7,  0.29771
6768401   ,  136860,  136860, 8.8, 8.4,10.3,14.1,35.1,
40.8,   23.8,  0.29213
6911785   ,  138204,  138204, 8.7, 8.3,10.2,13.7,34.1,
37.8,   24.8,  0.28669
7055951   ,  138633,  138633, 8.7, 8.3,10.5,32.0,40.5,
46.9,   25.8,  0.28130
7199084   ,  137731,  137731, 8.7, 8.4,10.2,14.0,33.4,
40.9,   26.9,  0.27623
7338032   ,  133201,  133201, 9.0, 8.4,10.9,34.0,39.4,
43.8,   27.9,  0.27116
7480439   ,  137059,  137059, 8.8, 8.4,10.2,13.9,35.9,
39.5,   29.0,  0.26663
7647810   ,  161209,  161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9,   30.0,  0.26185
7898882   ,  226498,  226498, 5.3, 5.2, 5.6,19.7,   108.5,   
119.3,   31.1,  0.25629
8136305   ,  223840,  223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2,   32.2,  0.24838
8372076   ,  223790,  223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0,   33.2,  0.24095
8633758   ,  232914,  232914, 5.1, 5.2, 5.6,17.5,   138.4,   
182.0,   34.4,  0.23397
8869214   ,  43,  43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9,   35.4,  0.22717
9121652   ,  223037,  223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8,   36.5,  0.22105
9360286   ,  225070,  225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1,   37.6,  0.21524
9609676   ,  224089,  224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3,   38.7,  0.20967
9848551   ,  222123,  222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2,   39.8,  0.20440
1000  ,  229511,  229511, 5.0, 5.2, 5.8,60.0,74.3,   
132.0,   40.5,  0.19935


Results:
real op rate  : 247211
adjusted op rate stderr   : 0
key rate  : 247211
latency mean  : 5.4
latency median: 3.5
latency 95th percentile   : 5.5
latency 99th percentile   : 6.1
latency 99.9th percentile : 83.4
latency max  

[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)

[Vijay (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069862#comment-14069862
 ] 

Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:54 AM:
---

Attached patch makes the off heap/Serialization Cache configurable. (the 
default is still SerializationCache).

Regarding performance, the performance of the new cache is obviously better 
when the JNI overhead is less than the GC overhead. For smaller size cache that 
can fit in the JVM heap, the performance is a little lower understandablely 
(but both of them out perform pagecache performance by a large number). Here 
are the numbers.

*OffheapCacheProvider*
{panel}
Running READ with 1200 threads  for 1000 iterations
ops   ,op/s,   key/s,mean, med, .95, .99,.999, 
max,   time,   stderr
2030355   , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,  
3014.5,1.0,  0.0
2395480   ,  202845,  202845, 5.8, 5.4, 5.8,20.2,   522.4,   
545.9,2.8,  0.0
2638600   ,  221368,  221368, 5.4, 5.3, 5.8,16.3,78.8,   
131.5,3.9,  0.57860
2891705   ,  221976,  221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0,  0.60478
3147747   ,  222527,  222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2,  0.58659
3394999   ,  221527,  221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3,  0.55884
3663559   ,  226114,  226114, 5.3, 5.2, 5.6,15.0,84.4,   
110.7,8.5,  0.52924
3911154   ,  223831,  223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6,  0.50018
4152946   ,  223246,  223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8,   10.7,  0.47323
4403162   ,  228532,  228532, 5.2, 5.2, 5.6,23.2,   107.4,   
121.4,   11.8,  0.44856
4641021   ,  225196,  225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4,   12.8,  0.42557
4889523   ,  222826,  222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0,   13.9,  0.40476
5124891   ,  223203,  223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8,   15.0,  0.38602
5375262   ,  221222,  221222, 5.4, 5.2, 5.6,18.4,94.2,   
115.1,   16.1,  0.36899
5616470   ,  224022,  224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8,   17.2,  0.35349
5866825   ,  223000,  223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2,   18.3,  0.33882
6125601   ,  225757,  225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0,   19.5,  0.32535
6348030   ,  192703,  192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5,   20.6,  0.31282
6483574   ,  128520,  128520, 9.3, 8.4,10.9,19.5,88.7,
99.0,   21.7,  0.30329
6626176   ,  137199,  137199, 8.7, 8.4,10.6,14.0,32.7,
40.9,   22.7,  0.29771
6768401   ,  136860,  136860, 8.8, 8.4,10.3,14.1,35.1,
40.8,   23.8,  0.29213
6911785   ,  138204,  138204, 8.7, 8.3,10.2,13.7,34.1,
37.8,   24.8,  0.28669
7055951   ,  138633,  138633, 8.7, 8.3,10.5,32.0,40.5,
46.9,   25.8,  0.28130
7199084   ,  137731,  137731, 8.7, 8.4,10.2,14.0,33.4,
40.9,   26.9,  0.27623
7338032   ,  133201,  133201, 9.0, 8.4,10.9,34.0,39.4,
43.8,   27.9,  0.27116
7480439   ,  137059,  137059, 8.8, 8.4,10.2,13.9,35.9,
39.5,   29.0,  0.26663
7647810   ,  161209,  161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9,   30.0,  0.26185
7898882   ,  226498,  226498, 5.3, 5.2, 5.6,19.7,   108.5,   
119.3,   31.1,  0.25629
8136305   ,  223840,  223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2,   32.2,  0.24838
8372076   ,  223790,  223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0,   33.2,  0.24095
8633758   ,  232914,  232914, 5.1, 5.2, 5.6,17.5,   138.4,   
182.0,   34.4,  0.23397
8869214   ,  43,  43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9,   35.4,  0.22717
9121652   ,  223037,  223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8,   36.5,  0.22105
9360286   ,  225070,  225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1,   37.6,  0.21524
9609676   ,  224089,  224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3,   38.7,  0.20967
9848551   ,  222123,  222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2,   39.8,  0.20440
1000  ,  229511,  229511, 5.0, 5.2, 5.8,60.0,74.3,   
132.0,   40.5,  0.19935


Results:
real op rate  : 247211
adjusted op rate stderr   : 0
key rate  : 247211
latency mean  : 5.4
latency median: 3.5
latency 95th percentile   : 5.5
latency 99th percentile   : 6.1
latency 99.9th percentile : 83.4
latency max