[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Peter Schuller (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127292#comment-13127292
 ] 

Peter Schuller commented on CASSANDRA-2466:
---

Good point. What's your feeling on the approach of modifying the bitset to use 
a number of small byte arrays? (Probably at some fixed size to help 
fragmentation.)

It does mean an additional level of indirection in an array of arrays, and it's 
not clear (to me at least) that it is expected to usually reside in CPU cache.


 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor

 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2348) Allow periods in secondary index column names

[David Semeria (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127295#comment-13127295
 ] 

David Semeria commented on CASSANDRA-2348:
--

Because underscore is a valid character in a JSON property name.

We must be sure the column name separator cannot appear in the property names 
themselves, otherwise when we read-in and parse the column names we would risk 
not splitting on correct boundaries.

Not to worry though, we switched to : which appears to be legal in v. 0.8+ 
secondary index references.


 Allow periods in secondary index column names
 -

 Key: CASSANDRA-2348
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2348
 Project: Cassandra
  Issue Type: Wish
  Components: API
Reporter: David Semeria
Priority: Minor
  Labels: index, json

 Premise: it is natural to map JSON objects to column names using periods to 
 denote the object hierarchy.
 For example {food:{fruit;{a:apples,b:bananas,c:cherries}}}
 Would map to the following CF names: food.fruit.a, food.fruit.b, food.fruit.c
 However, secondary index names cannot contain periods which means workarounds 
 (ie denormalization) must be used to index such columns.
 It would be nice if this restriction could be removed.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Anastasyev updated CASSANDRA-2466:
---

Attachment: OpenBitSet.java

I made it already for our cassandra deployment (because we have huge bloom 
filters). Attaching it - may be you'll find it useful.

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Issue Comment Edited) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127313#comment-13127313
 ] 

Oleg Anastasyev edited comment on CASSANDRA-2466 at 10/14/11 6:53 AM:
--

I made it already for our cassandra deployment (because we have huge bloom 
filters). Attaching it as is (will rebase it to head if you'll find it useful).

  was (Author: m0nstermind):
I made it already for our cassandra deployment (because we have huge bloom 
filters). Attaching it - may be you'll find it useful.
  
 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2749) fine-grained control over data directories

[Marcus Eriksson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127329#comment-13127329
 ] 

Marcus Eriksson commented on CASSANDRA-2749:


Biggest part left is figuring out how to estimate disk space to know where to 
write an sstable, ill work on that this weekend

Other than that small cleanups (codestyle), unit tests and adding the config 
parameter

 fine-grained control over data directories
 --

 Key: CASSANDRA-2749
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2749
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor
 Fix For: 1.1

 Attachments: 
 0001-Make-it-possible-to-put-column-families-in-subdirect.patch, 
 2749_backwards_compatible_v1.patch, 2749_backwards_compatible_v2.patch


 Currently Cassandra supports multiple data directories but no way to control 
 what sstables are placed where. Particularly for systems with mixed SSDs and 
 rotational disks, it would be nice to pin frequently accessed columnfamilies 
 to the SSDs.
 Postgresql does this with tablespaces 
 (http://www.postgresql.org/docs/9.0/static/manage-ag-tablespaces.html) but we 
 should probably avoid using that name because of confusing similarity to 
 keyspaces.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3331) Apache Daemon missing from the binary tarball

[Sylvain Lebresne (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127335#comment-13127335
 ] 

Sylvain Lebresne commented on CASSANDRA-3331:
-

I'm also fine with reverting this, but I'd add a few remarks:
  # if we do revert, it would be nice to fix the .bat file so that it don't 
look for it in our specific repository that we would not ship (i.e, do a little 
bit more than reverting because the .bat before this patch was clearly 
suggesting we intended to ship it but did not). Ideally we would just expect it 
to be installed in whatever is supposed to be standard for windows and offer an 
helpful message if it is not found (since we can't do what we do for the debian 
package, i.e make it a dependency of the package)
  # it's been committed to 1.0.0 and I don't really want to re-roll the vote 
for that. That would mean we ship procrun for 1.0.0 but then never after that. 
A little weird imho.

 Apache Daemon missing from the binary tarball
 -

 Key: CASSANDRA-3331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3331
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Benjamin Coverston
Assignee: Benjamin Coverston
 Fix For: 1.0.0

 Attachments: 0001-add-files-to-src-and-bin-add-uninstall.patch, 
 0002-adding-daemon-directory.patch


 Apparently the tools used to run the binary release are missing from the 
 binary tarball.
 I will verify that they are in the 1.0 branch, then update the ticket so we 
 can ensure that they are included.
 Ben

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3127) Message (inter-node) compression

[Sylvain Lebresne (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127341#comment-13127341
 ] 

Sylvain Lebresne commented on CASSANDRA-3127:
-

bq. Let's go with this: intranode_message_compression_threshold  0 means 
compress messages larger than it. = 0 means off

Agreed with the idea. Though to nitpick, I would find it more natural to have 
== 0 means 'compress all messages'. And maybe  0 means off.

bq. Let's leave off the cross-dc complexity for now.

Totally agree.

 Message (inter-node) compression
 

 Key: CASSANDRA-3127
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3127
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Sylvain Lebresne
Priority: Minor

 CASSANDRA-3015 adds compression of streams. But it could be useful to also 
 compress some messages.
 Compressing messages is easy, but what may be little bit trickier is when and 
 what messages to compress to get the best performances.
 The simple solution would be to just have it either always on or always off. 
 But for very small messages (gossip?) that may be counter-productive. On the 
 other side of the spectrum, this is likely always a good choice to compress 
 for say the exchange of merkle trees across data-centers. We could maybe 
 define a size of messages after which we start to compress. Maybe the option 
 to only compress for cross data-center messages would be useful too (but I 
 may also just be getting carried away). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3272) READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)

[Sylvain Lebresne (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127361#comment-13127361
 ] 

Sylvain Lebresne commented on CASSANDRA-3272:
-

Shouldn't we move the validation up. We already have a validateConsistencyLevel 
in ThriftValidation for instance, we could just add a flag to if it's a write 
or read operation and throw the right exception there. If only for consistency, 
but I think it's nice to keep validating the queries upfront so we don't have 
to later anyway.

Which btw make me see that this validateConsistencyLevel is not called by CQL 
(which thus don't do the right check this function already does I suppose, or 
duplicate it). It would be worth checking if that's the only place where CQL is 
more loose than thrift in validating the query (and maybe we could refactor a 
bit the validation code so it's easier to apply it to both thrift and CQL).

 READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
 ---

 Key: CASSANDRA-3272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3272
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Jackson Chung
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.1

 Attachments: 3272.txt


 READ EACH_QUORUM:Returns the record with the most recent timestamp once 
 a quorum of replicas in each data center of the cluster has responded.
 In other words, if a DC is down and the QUORUM could not be reached on that 
 DC, read should fail.
 test case:
 - Cassandra version 0.8.6:
 INFO [main] 2011-09-28 22:26:24,297 StorageService.java (line 371) Cassandra 
 version: 0.8.6
 - 6-node cluster with 2 DC and 3 node each. RF=3 in each DC:
 [default@Keyspace3] describe keyspace;
 Keyspace: Keyspace3:
 Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
 Durable Writes: true
 Options: [DC2:3, DC1:3]
 Column Families:
 ColumnFamily: test
 Key Validation Class: org.apache.cassandra.db.marshal.BytesType
 Default column value validator: org.apache.cassandra.db.marshal.BytesType
 Columns sorted by: org.apache.cassandra.db.marshal.BytesType
 Row cache size / save period in seconds: 0.0/0
 Key cache size / save period in seconds: 20.0/14400
 Memtable thresholds: 1.0875/1440/232 (millions of ops/minutes/MB)
 GC grace seconds: 864000
 Compaction min/max thresholds: 4/32
 Read repair chance: 1.0
 Replicate on write: true
 Built indexes: []
 all nodes are up, insert a row:
 $ nodetool -h localhost ring
 Address DC Rack Status State Load Owns Token
 141784319550391026443072753096570088106
 10.34.79.179 DC1 RAC1 Up Normal 11.13 KB 16.67% 0
 10.34.70.163 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 28356863910078205288614550619314017621
 10.35.81.147 DC1 RAC1 Up Normal 11.14 KB 16.67% 
 56713727820156410577229101238628035242
 10.84.233.170 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 85070591730234615865843651857942052864
 10.195.201.236 DC1 RAC1 Up Normal 11.14 KB 16.67% 
 113427455640312821154458202477256070485
 10.118.147.73 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 141784319550391026443072753096570088106
 - insert a value 
 [default@Keyspace3] set 
 test[utf8('test-key-1')][utf8('test-col')]=utf8('test-value');
 Value inserted.
 sanity check (cli connects to a node in DC1) :
 [default@Keyspace3] consistencylevel as EACH_QUORUM;  
 
 Consistency level is set to 'EACH_QUORUM'.
 [default@Keyspace3] get test[utf8('test-key-1')];   
 = (column=746573742d636f6c, value=test-value, timestamp=1317249361722000)
 Returned 1 results
 shut down DC2:
 $ nodetool -h localhost ring
 Address DC  RackStatus State   LoadOwns   
  Token   
   
  141784319550391026443072753096570088106 
 10.34.79.179DC1 RAC1Up Normal  51.86 KB16.67% 
  0   
 10.34.70.163DC2 RAC1Down   Normal  51.88 KB16.67% 
  28356863910078205288614550619314017621  
 10.35.81.147DC1 RAC1Up Normal  47.5 KB 16.67% 
  56713727820156410577229101238628035242  
 10.84.233.170   DC2 RAC1Down   Normal  51.88 KB16.67% 
  85070591730234615865843651857942052864  
 10.195.201.236  DC1 RAC1Up Normal  47.5 KB 16.67% 
  113427455640312821154458202477256070485 
 10.118.147.73   DC2 RAC1Down   Normal  51.88 KB16.67% 
  141784319550391026443072753096570088106  
 [default@Keyspace3] get test[utf8('test-key-1')];   
 = (column=746573742d636f6c, value=746573742d76616c7565, 
 

[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127366#comment-13127366
 ] 

David Allsopp commented on CASSANDRA-2854:
--

Will try to take a look. Probably needs to detect the OS in Ant to modify the 
regex passed to split(), e.g.:

condition property=path.sep value=;
   os family=windows/
/condition
condition property=path.sep value=:
   os family=unix/
/condition

fail unless=foo.pathNo path.sep set for this OS!/fail

Or (possibly) use unix-style paths in the first place if Ant handles them OK on 
Windows (but not sure where they originate from; this may not be possible)...

 Java Build Path is broken in Eclipse after running generate-eclipse-files Ant 
 target
 

 Key: CASSANDRA-2854
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2854
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
Affects Versions: 0.8.1
 Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse
Reporter: David Allsopp
Priority: Minor

 Following the instructions in 
 http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out 
 v0.8.1:
 After running the 'generate-eclipse-files' Ant target, the build path is set 
 up to include all the libraries in build/lib/jars, but each library has ;C 
 (semicolon, letter C) appended to it, so Eclipse can't find the libraries - 
 there are about 55 errors like:
 Project 'cassandra-0.8.1' is missing required library: 
 '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183241 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/db/compaction/CompactionManager.java

[slebresne]

Author: slebresne
Date: Fri Oct 14 08:50:27 2011
New Revision: 1183241

URL: http://svn.apache.org/viewvc?rev=1183241view=rev
Log:
Only count compaction as active (for throttling) once the compaction lock has 
been acquired.
patch by Fabien Rousseau and slebresne; reviewed by jbellis for CASSANDRA-3344

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183241r1=1183240r2=1183241view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 08:50:27 2011
@@ -8,6 +8,8 @@
  * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
  * fix ColumnIndexer to use long offsets (CASSANDRA-3358)
  * Display CLI version string on startup (CASSANDRA-3196)
+ * Only count compaction as active (for throttling) when they have
+   successfully acquired the compaction lock (CASSANDRA-3344)
 
 
 0.8.7

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1183241r1=1183240r2=1183241view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
 Fri Oct 14 08:50:27 2011
@@ -1216,7 +1216,7 @@ public class CompactionManager implement
 
 public int getActiveCompactions()
 {
-return executor.getActiveCount() + validationExecutor.getActiveCount();
+return CompactionExecutor.compactions.size();
 }
 
 private static class CompactionExecutor extends 
DebuggableThreadPoolExecutor

svn commit: r1183243 - in /cassandra/branches/cassandra-1.0.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/

[slebresne]

Author: slebresne
Date: Fri Oct 14 08:52:05 2011
New Revision: 1183243

URL: http://svn.apache.org/viewvc?rev=1183243view=rev
Log:
merge from 0.8

Modified:
cassandra/branches/cassandra-1.0.0/   (props changed)
cassandra/branches/cassandra-1.0.0/CHANGES.txt
cassandra/branches/cassandra-1.0.0/contrib/   (props changed)

cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java

Propchange: cassandra/branches/cassandra-1.0.0/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:52:05 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
 /cassandra/branches/cassandra-0.7:1026516-1170333,1172024
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950,1183241
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689

Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1183243r1=1183242r2=1183243view=diff
==
--- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Fri Oct 14 08:52:05 2011
@@ -1,3 +1,9 @@
+1.0.1
+ * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
+ * Only count compaction as active (for throttling) when they have
+   successfully acquired the compaction lock (CASSANDRA-3344)
+
+
 1.0.0-final
  * close scrubbed sstable fd before deleting it (CASSANDRA-3318)
  * fix bug preventing obsolete commitlog segments from being removed
@@ -30,7 +36,6 @@ Fixes merged from 0.8 below:
  * Nodetool no longer leaks threads and closes JMX connections (CASSANDRA-3309)
  * fix truncate allowing data to be replayed post-restart (CASSANDRA-3297)
  * Move SimpleAuthority and SimpleAuthenticator to examples (CASSANDRA-2922)
- * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
 
 
 1.0.0-rc2

Propchange: cassandra/branches/cassandra-1.0.0/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:52:05 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
 /cassandra/branches/cassandra-0.7/contrib:1026516-1170333,1172024
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950,1183241
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689

Propchange: 
cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:52:05 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1170333,1172024
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950

svn commit: r1183245 - in /cassandra/branches/cassandra-1.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/

[slebresne]

Author: slebresne
Date: Fri Oct 14 08:54:00 2011
New Revision: 1183245

URL: http://svn.apache.org/viewvc?rev=1183245view=rev
Log:
merge from 1.0.0

Modified:
cassandra/branches/cassandra-1.0/   (props changed)
cassandra/branches/cassandra-1.0/CHANGES.txt
cassandra/branches/cassandra-1.0/contrib/   (props changed)

cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java

Propchange: cassandra/branches/cassandra-1.0/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:54:00 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
 /cassandra/branches/cassandra-0.7:1026516-1183000
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
 /cassandra/branches/cassandra-1.0:1167106,1167185
-/cassandra/branches/cassandra-1.0.0:1167104-1181093,1181741,1182951
+/cassandra/branches/cassandra-1.0.0:1167104-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020
 /cassandra/trunk:1167085-1167102,1169870

Modified: cassandra/branches/cassandra-1.0/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1183245r1=1183244r2=1183245view=diff
==
--- cassandra/branches/cassandra-1.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0/CHANGES.txt Fri Oct 14 08:54:00 2011
@@ -12,6 +12,9 @@
  * fix ColumnIndexer to use long offsets (CASSANDRA-3358)
  * limit nodetool to 32MB of heap (CASSANDRA-3124)
  * (CQL) update parser to accept timestamp instead of date (CASSANDRA-3149)
+ * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
+ * Only count compaction as active (for throttling) when they have
+   successfully acquired the compaction lock (CASSANDRA-3344)
 
 
 1.0.0-final
@@ -49,7 +52,6 @@ Fixes merged from 0.8 below:
  * Force flush of system table after updating/removing a token (CASSANDRA-3243)
  * Make Pig storage handle implements LoadMetadata (CASSANDRA-2777)
  * Improved CLI exceptions (CASSANDRA-3312)
- * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
 
 
 1.0.0-rc2

Propchange: cassandra/branches/cassandra-1.0/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:54:00 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
 /cassandra/branches/cassandra-0.7/contrib:1026516-1183000
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
 /cassandra/branches/cassandra-1.0/contrib:1167106,1167185
-/cassandra/branches/cassandra-1.0.0/contrib:1167104-1181093,1181741,1182951
+/cassandra/branches/cassandra-1.0.0/contrib:1167104-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020
 /cassandra/trunk/contrib:1167085-1167102,1169870

Propchange: 
cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:54:00 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1183000
 

svn commit: r1183247 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/

[slebresne]

Author: slebresne
Date: Fri Oct 14 08:55:08 2011
New Revision: 1183247

URL: http://svn.apache.org/viewvc?rev=1183247view=rev
Log:
merge from 1.0

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:55:08 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
 /cassandra/branches/cassandra-0.7:1026516-1183000
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
-/cassandra/branches/cassandra-1.0:1167085-1183003
-/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951
+/cassandra/branches/cassandra-1.0:1167085-1183003,1183245
+/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1183247r1=1183246r2=1183247view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Fri Oct 14 08:55:08 2011
@@ -14,6 +14,9 @@
  * (Hadoop) skip empty rows when slicing the entire row (CASSANDRA-2855)
  * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
  * fix ColumnIndexer to use long offsets (CASSANDRA-3358)
+ * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
+ * Only count compaction as active (for throttling) when they have
+   successfully acquired the compaction lock (CASSANDRA-3344)
 
 
 1.0.0-final
@@ -51,7 +54,6 @@ Fixes merged from 0.8 below:
  * Force flush of system table after updating/removing a token (CASSANDRA-3243)
  * Make Pig storage handle implements LoadMetadata (CASSANDRA-2777)
  * Improved CLI exceptions (CASSANDRA-3312)
- * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
 
 
 1.0.0-rc2

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:55:08 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
 /cassandra/branches/cassandra-0.7/contrib:1026516-1183000
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
-/cassandra/branches/cassandra-1.0/contrib:1167085-1183003
-/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951
+/cassandra/branches/cassandra-1.0/contrib:1167085-1183003,1183245
+/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020
 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 08:55:08 2011
@@ -1,11 +1,11 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1183000
 

[jira] [Commented] (CASSANDRA-3344) Compaction throttling can be too slow

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127388#comment-13127388
 ] 

Hudson commented on CASSANDRA-3344:
---

Integrated in Cassandra-0.8 #372 (See 
[https://builds.apache.org/job/Cassandra-0.8/372/])
Only count compaction as active (for throttling) once the compaction lock 
has been acquired.
patch by Fabien Rousseau and slebresne; reviewed by jbellis for CASSANDRA-3344

slebresne : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1183241
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java


 Compaction throttling can be too slow
 -

 Key: CASSANDRA-3344
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3344
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.0
Reporter: Fabien Rousseau
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8.8, 1.0.1

 Attachments: 001-CASSANDRA-3344.patch, 3344.patch, 3344_v2.patch


 Compaction throttling needs to know how many active compactions are running 
 (to divide bandwith for each active compaction).
 The way active compaction is counted can be broken because it counts the 
 number of active threads in the executor BUT the thread starts by acquiring a 
 lock.
 If the lock can't be acquired immediately : the thread is seen as active 
 but does not participate in IO operations.
 The case can happen when major compaction are triggered (major compaction 
 acquire a write lock, while minor compactions acquire a read lock).
 Having compaction througput to 16Mb/s, we observed is the following  (two 
 times) :
  - only 1 active compaction (a long one for a few hours) starting at 16Mb/s, 
 then after some time running at 2Mb/s, thus taking a very long time to 
 complete
  - many pending compactions
 Using JMX and monitoring the stack trace of the compaction threads showed 
 that :
  - 1 thread was effectively compacting
  - 1 thread was waiting to acquire the write lock (due to a major compaction)
  - 6 threads were waiting to acquire the read lock (probably due to the 
 thread above trying to acquire the write lock)
 Attached is a proposed patch (very simple, not yet tested) which counts only 
 active compactions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2524) Use SSTableBoundedScanner for cleanup

[Sylvain Lebresne (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127403#comment-13127403
 ] 

Sylvain Lebresne commented on CASSANDRA-2524:
-

This looks good to me, though one thing that goes away with that is that for cf 
that use the BoundedScanner, we don't cleanup the row cache. This is not a 
really big deal, but it's perfectly possible that following the cleanup, rows 
that are still on the node get evicted from cache before rows that are not in 
the ranges of the node, so it's a slight regression. Maybe we could simply add 
a cache cleanup that iterate through the cache and only keeps keys in range? 
Though one may argue it's on the fringe of this ticket, I'd prefer we add this 
here so we commit something that is an improvement, rather than an improvement 
with a slight regression to be fixed later.

 Use SSTableBoundedScanner for cleanup
 -

 Key: CASSANDRA-2524
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2524
 Project: Cassandra
  Issue Type: Improvement
Reporter: Stu Hood
Assignee: Stu Hood
Priority: Minor
  Labels: lhf
 Attachments: 
 0001-Use-a-SSTableBoundedScanner-for-cleanup-and-improve-cl.txt, 
 0002-Oops.-When-indexes-or-counters-are-in-use-must-continu.txt


 SSTableBoundedScanner seeks rather than scanning through rows, so it would be 
 significantly more efficient than the existing per-key filtering that cleanup 
 does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127481#comment-13127481
 ] 

Jonathan Ellis commented on CASSANDRA-2466:
---

What effect did you observe when deploying this, Oleg?

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2348) Allow periods in secondary index column names

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2348.
---

Resolution: Not A Problem

WFM.

 Allow periods in secondary index column names
 -

 Key: CASSANDRA-2348
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2348
 Project: Cassandra
  Issue Type: Wish
  Components: API
Reporter: David Semeria
Priority: Minor
  Labels: index, json

 Premise: it is natural to map JSON objects to column names using periods to 
 denote the object hierarchy.
 For example {food:{fruit;{a:apples,b:bananas,c:cherries}}}
 Would map to the following CF names: food.fruit.a, food.fruit.b, food.fruit.c
 However, secondary index names cannot contain periods which means workarounds 
 (ie denormalization) must be used to index such columns.
 It would be nice if this restriction could be removed.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3331) Apache Daemon missing from the binary tarball

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127487#comment-13127487
 ] 

Jonathan Ellis commented on CASSANDRA-3331:
---

bq. Ideally we would just expect it to be installed in whatever is supposed to 
be standard for windows 

Is there a standard?  It just comes as a .zip, not a proper installer.

bq. That would mean we ship procrun for 1.0.0 but then never after that

The alternative is to keep it for all of 1.0 which will set the expectation 
that we'll keep doing it...  I'd rather bite the bullet early.  Everyone waits 
for 1.0.1 anyway right? :)

 Apache Daemon missing from the binary tarball
 -

 Key: CASSANDRA-3331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3331
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Benjamin Coverston
Assignee: Benjamin Coverston
 Fix For: 1.0.0

 Attachments: 0001-add-files-to-src-and-bin-add-uninstall.patch, 
 0002-adding-daemon-directory.patch


 Apparently the tools used to run the binary release are missing from the 
 binary tarball.
 I will verify that they are in the 1.0 branch, then update the ticket so we 
 can ensure that they are included.
 Ben

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127586#comment-13127586
 ] 

Oleg Anastasyev commented on CASSANDRA-2466:


Promotion failures caused by bloom filters gone away completely.
 
I did not measured performance overhead specifically on bloom filters, so all I 
can tell that it was not introduced measureable performance degradation on 
production cluster operation from client perspective. 

(currently we have ~3300 thrift single column read reqs/sec per node with ~1ms 
avg call duration measured from client; 98% reads are bounced by bloom filters; 
size of single bloom filter is up to 250Mb)


 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Trivial Update of HadoopSupport by jeremyhanna

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HadoopSupport page has been changed by jeremyhanna:
http://wiki.apache.org/cassandra/HadoopSupport?action=diffrev1=44rev2=45

Comment:
Fixed typo.

   * '''cassandra.range.batch.size''' - the default is 4096, but you may need 
to lower this depending on your data.  This is either specified in your hadoop 
configuration or using 
`org.apache.cassandra.hadoop.ConfigHelper.setRangeBatchSize`.
   * '''rpc_timeout_in_ms''' - this is set in your `cassandra.yaml` (in 0.6 
it's `RpcTimeoutInMillis` in `storage-conf.xml`).  The rpc timeout is not for 
timing out from the client but between nodes.  This can be increased to reduce 
chances of timing out.
  
- If you still see timeout exceptions with resultant failed jobs and/or 
blacklisted tasktrackers, there are settings that can give Cassandra more 
latitude before failing the jobs.  An example of usage (in either the job 
configuration or taskracker mapred-site.xml):
+ If you still see timeout exceptions with resultant failed jobs and/or 
blacklisted tasktrackers, there are settings that can give Cassandra more 
latitude before failing the jobs.  An example of usage (in either the job 
configuration or tasktracker mapred-site.xml):
  
  {{{
  property

[jira] [Commented] (CASSANDRA-3067) Simple SSTable Pluggability

[Sylvain Lebresne (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127618#comment-13127618
 ] 

Sylvain Lebresne commented on CASSANDRA-3067:
-


Had an initial look at the patches. I haven't try applying the patches yet 
(which likely need rebase anyway), so following comments are fairly high level.

For the abstract interfaces to SSTable{Reader, Writer, *Iterator}, I would 
prefer calling them AbstractSSTable* and keep the concrete implementation with 
their current name. It should make the diffs much more digestible and much more 
easy to check. Same thing for future merges. And we kind of use Abstract* for 
abstract place around the place so it would rather more consistent than less.

For patch 2, I'm not too sure about what this refactoring buys us, in 
particular as long as files are versioned by one character. I understand that 
to support other kind of sstables, we would need to somehow distinguish between 
kind of sstables, but this doesn't seem to fully solve that problem.  I think 
we already had kind of the same problem with compression, where we went with 
using the presence of a sstable component to decide if it's compressed or not. 
I'm happy to have a discussion on how to best evolve the sstable versioning but 
not sure about the solution here. Follows the same holds for patch 6.

Small nitpick: In patch1, I'm not too fond of introducing a compareTo for 
SSTableIdentityIterator (given a SSTableIdentityIterator is not really equal to 
it's key), is there an underlying reason ?


 Simple SSTable Pluggability
 ---

 Key: CASSANDRA-3067
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3067
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Stu Hood
Assignee: Stu Hood
 Fix For: 1.1

 Attachments: 
 0001-CASSANDRA-3067-Create-an-ABC-for-SSTableIdentityIterat.txt, 
 0002-CASSANDRA-3067-Move-from-linear-SSTable-versions-to-fe.txt, 
 0003-CASSANDRA-3067-Create-an-ABC-for-SSTableWriter.txt, 
 0004-CASSANDRA-3067-Rename-SSTable-Names-Slice-Iterator-to-.txt, 
 0005-CASSANDRA-3067-Create-ABCs-for-SSTableReader-and-KeyIt.txt, 
 0006-CASSANDRA-3067-Allow-overriding-the-current-sstable-ve.txt


 CASSANDRA-2995 proposes full storage engine pluggability, which is probably 
 unavoidable in the long run. For now though, I'd like to propose an 
 incremental alternative that preserves the sstable model, but allows it to 
 evolve non-linearly.
 The sstable version field could allow for simple switching between writable 
 sstable types, without moving all the way to differentiating between engines 
 as CASSANDRA-2995 requires. This can be accomplished by moving towards a 
 feature flags model (with a mapping between versions and feature sets), 
 rather than a linear versions model (where versions can be strictly ordered 
 and all versions above X have a feature).
 There are restrictions on this approach:
 * It's sufficient for an alternate SSTable(Writer|Reader|*) set to require a 
 patch to enable (rather than a JAR)
 * Filenames/descriptors/components must conform to the existing conventions

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2

[Gopalakrishnan Rajagopal (Created) (JIRA)]

Read data inconsistancy in Cassandra 1.0.0-rc2
--

 Key: CASSANDRA-3360
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3360
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.0.0
Reporter: Gopalakrishnan Rajagopal


When qsuper column for a particular key is being queried
using hector-core-0.8.0-2,
the data retrieved is inconsistent. I mean, for the key that I use to fetch 
data, there are 7 sub columns actually. But the query returns 1 or 3 sub 
columns depending on which nodes respond to it. (I tested by bringing down 
each one of the three nodes in turn).  

When I tried to fetch the data for the same key using cassandra-cli tool, I 
get all the 7 sub columns for both the consistancy levels ONE and QUORUM. 

Below is the code that I used to fetch data

superColumnQuery = HFactory.createSuperColumnQuery
(keyspaceOperator, 
stringSerializer, 
stringSerializer, stringSerializer, stringSerializer);
superColumnQuery.setColumnFamily(cfName).setKey
(key).setSuperName(scName);
result=superColumnQuery.execute();
superColumn=result.get();
columnList=superColumn.getColumns();



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183417 - in /cassandra/site: publish/download/index.html src/content/download/index.html

[slebresne]

Author: slebresne
Date: Fri Oct 14 16:45:18 2011
New Revision: 1183417

URL: http://svn.apache.org/viewvc?rev=1183417view=rev
Log:
Update link to java CQL driver on website

Modified:
cassandra/site/publish/download/index.html
cassandra/site/src/content/download/index.html

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1183417r1=1183416r2=1183417view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Fri Oct 14 16:45:18 2011
@@ -131,11 +131,11 @@
   /p
   ul
 li
-  a href=http://www.apache.org/dist/cassandra/drivers/java;
+  a href=http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/;
   Java (JDBC)/a
 /li
 li
-  a 
href=http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/downloads/list;
+  a href=http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/;
   Python (DBAPI2)/a
 /li
   /ul

Modified: cassandra/site/src/content/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/content/download/index.html?rev=1183417r1=1183416r2=1183417view=diff
==
--- cassandra/site/src/content/download/index.html (original)
+++ cassandra/site/src/content/download/index.html Fri Oct 14 16:45:18 2011
@@ -90,11 +90,11 @@
   /p
   ul
 li
-  a href=http://www.apache.org/dist/cassandra/drivers/java;
+  a href=http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/;
   Java (JDBC)/a
 /li
 li
-  a 
href=http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/downloads/list;
+  a href=http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/;
   Python (DBAPI2)/a
 /li
   /ul

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127720#comment-13127720
 ] 

Jonathan Ellis commented on CASSANDRA-2466:
---

Is serialization the same, or at least backwards-compatible w/ existing OBS BFs?

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127721#comment-13127721
 ] 

Jonathan Ellis commented on CASSANDRA-3360:
---

If the CLI is behaving correctly, it sounds like a Hector bug?

 Read data inconsistancy in Cassandra 1.0.0-rc2
 --

 Key: CASSANDRA-3360
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3360
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.0.0
Reporter: Gopalakrishnan Rajagopal

 When qsuper column for a particular key is being queried
 using hector-core-0.8.0-2,
 the data retrieved is inconsistent. I mean, for the key that I use to fetch 
 data, there are 7 sub columns actually. But the query returns 1 or 3 sub 
 columns depending on which nodes respond to it. (I tested by bringing down 
 each one of the three nodes in turn).  
 When I tried to fetch the data for the same key using cassandra-cli tool, I 
 get all the 7 sub columns for both the consistancy levels ONE and QUORUM. 
 Below is the code that I used to fetch data
 superColumnQuery = HFactory.createSuperColumnQuery
 (keyspaceOperator, 
 stringSerializer, 
 stringSerializer, stringSerializer, stringSerializer);
 superColumnQuery.setColumnFamily(cfName).setKey
 (key).setSuperName(scName);
 result=superColumnQuery.execute();
 superColumn=result.get();
 columnList=superColumn.getColumns();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Anastasyev updated CASSANDRA-2466:
---

Attachment: BloomFilterSerializer.java

You mean on-disk format ? Yes. Specifics are handled by BloomFilterSerializer. 
Attached serializer as implemented for 0.6.


 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: BloomFilterSerializer.java, OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Issue Comment Edited) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127728#comment-13127728
 ] 

Oleg Anastasyev edited comment on CASSANDRA-2466 at 10/14/11 5:46 PM:
--

You mean on-disk format ? Yes. Specifics are handled by BloomFilterSerializer. 
Attached serializer as implemented for 0.6.
(as far as i see no changes are required in BloomFilter itself)

  was (Author: m0nstermind):
You mean on-disk format ? Yes. Specifics are handled by 
BloomFilterSerializer. Attached serializer as implemented for 0.6.

  
 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: BloomFilterSerializer.java, OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127735#comment-13127735
 ] 

Jonathan Ellis commented on CASSANDRA-2466:
---

Are there any other moving parts to this? Could we ask you to submit a patch 
against trunk?

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: BloomFilterSerializer.java, OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2

[Gopalakrishnan Rajagopal (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127743#comment-13127743
 ] 

Gopalakrishnan Rajagopal commented on CASSANDRA-3360:
-

I am not too sure... the same hector version and code works with 1.0.0-beta1 
version though. So this just is a compatibilty issue with 1.0.0-rc2?

 Read data inconsistancy in Cassandra 1.0.0-rc2
 --

 Key: CASSANDRA-3360
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3360
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.0.0
Reporter: Gopalakrishnan Rajagopal

 When qsuper column for a particular key is being queried
 using hector-core-0.8.0-2,
 the data retrieved is inconsistent. I mean, for the key that I use to fetch 
 data, there are 7 sub columns actually. But the query returns 1 or 3 sub 
 columns depending on which nodes respond to it. (I tested by bringing down 
 each one of the three nodes in turn).  
 When I tried to fetch the data for the same key using cassandra-cli tool, I 
 get all the 7 sub columns for both the consistancy levels ONE and QUORUM. 
 Below is the code that I used to fetch data
 superColumnQuery = HFactory.createSuperColumnQuery
 (keyspaceOperator, 
 stringSerializer, 
 stringSerializer, stringSerializer, stringSerializer);
 superColumnQuery.setColumnFamily(cfName).setKey
 (key).setSuperName(scName);
 result=superColumnQuery.execute();
 superColumn=result.get();
 columnList=superColumn.getColumns();

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Oleg Anastasyev (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127749#comment-13127749
 ] 

Oleg Anastasyev commented on CASSANDRA-2466:


Of course. I'll try to prepare it till monday.

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: BloomFilterSerializer.java, OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL

[Jackson Chung (Created) (JIRA)]

add logging to StorageProxy for mutations and CL


 Key: CASSANDRA-3361
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jackson Chung
Priority: Minor
 Attachments: 3361.patch

Through 
https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
 the read command and CL is logged (under DEBUG) and it is very helpful. But I 
don't think I have seen the same for mutation...

It'd be nice if we could also log this for the mutation (and others?), please 
see the attached patch (just done on mutation) . Note also this prints the 
IMutation which I did wonder if it is a good idea (could the list be so large 
that it floods the log instead of being useful?)

sample output:
{noformat}
DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171) 
Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70', 
modifications=[ColumnFamily(testindx [testcol:false:1@1318619350892000,])])]/ONE
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL

[Jackson Chung (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jackson Chung updated CASSANDRA-3361:
-

Attachment: 3361.patch

 add logging to StorageProxy for mutations and CL
 

 Key: CASSANDRA-3361
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jackson Chung
Priority: Minor
 Attachments: 3361.patch


 Through 
 https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
  the read command and CL is logged (under DEBUG) and it is very helpful. But 
 I don't think I have seen the same for mutation...
 It'd be nice if we could also log this for the mutation (and others?), please 
 see the attached patch (just done on mutation) . Note also this prints the 
 IMutation which I did wonder if it is a good idea (could the list be so large 
 that it floods the log instead of being useful?)
 sample output:
 {noformat}
 DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171) 
 Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70', 
 modifications=[ColumnFamily(testindx 
 [testcol:false:1@1318619350892000,])])]/ONE
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183489 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java

[brandonwilliams]

Author: brandonwilliams
Date: Fri Oct 14 20:27:43 2011
New Revision: 1183489

URL: http://svn.apache.org/viewvc?rev=1183489view=rev
Log:
Make CFIF use rpc_endpoint prior to trying endpoint.
Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for
CASSANDRA-3214

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183489r1=1183488r2=1183489view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 20:27:43 2011
@@ -10,6 +10,8 @@
  * Display CLI version string on startup (CASSANDRA-3196)
  * Only count compaction as active (for throttling) when they have
successfully acquired the compaction lock (CASSANDRA-3344)
+ * (Hadoop) make CFIF try rpc_address or fallback to listen_address
+   (CASSANDRA-3214)
 
 
 0.8.7

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java?rev=1183489r1=1183488r2=1183489view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
 Fri Oct 14 20:27:43 2011
@@ -188,13 +188,17 @@ public class ColumnFamilyInputFormat ext
 {
 ArrayListInputSplit splits = new ArrayListInputSplit();
 ListString tokens = getSubSplits(keyspace, cfName, range, conf);
-
+assert range.rpc_endpoints.size() == range.endpoints.size() : 
rpc_endpoints size must match endpoints size;
 // turn the sub-ranges into InputSplits
 String[] endpoints = range.endpoints.toArray(new 
String[range.endpoints.size()]);
 // hadoop needs hostname, not ip
-for (int i = 0; i  endpoints.length; i++)
+int endpointIndex = 0;
+for (String endpoint: range.rpc_endpoints)
 {
-endpoints[i] = 
InetAddress.getByName(endpoints[i]).getHostName();
+String endpoint_address = endpoint;
+   if(endpoint_address == null || endpoint_address == 
0.0.0.0)
+   endpoint_address = 
range.endpoints.get(endpointIndex);
+   endpoints[endpointIndex++] = 
InetAddress.getByName(endpoint_address).getHostName();
 }
 
 for (int i = 1; i  tokens.size(); i++)
@@ -210,7 +214,7 @@ public class ColumnFamilyInputFormat ext
 private ListString getSubSplits(String keyspace, String cfName, 
TokenRange range, Configuration conf) throws IOException
 {
 int splitsize = ConfigHelper.getInputSplitSize(conf);
-for (String host : range.endpoints)
+for (String host : range.rpc_endpoints)
 {
 try
 {

[jira] [Resolved] (CASSANDRA-3214) Make CFIF use rpc_endpoint prior to trying endpoint

[Brandon Williams (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-3214.
-

Resolution: Fixed

Committed with formatting fixes.  Thanks!

 Make CFIF use rpc_endpoint prior to trying endpoint
 ---

 Key: CASSANDRA-3214
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3214
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.8.6
 Environment: Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 / 
Reporter: Eldon Stegall
Assignee: Eldon Stegall
Priority: Minor
 Fix For: 0.8.8

 Attachments: CASSANDRA-3214-make-cfif-use-rpc-endpoints-v1.txt, 
 CASSANDRA-3214.patch


 Following up on CASSANDRA-3187 , we probably need to attempt to use the 
 rpc_endpoint address first, and then fall back to the gossip endpoint if we 
 don't get what we want.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3214) Make CFIF use rpc_endpoint prior to trying endpoint

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127868#comment-13127868
 ] 

Hudson commented on CASSANDRA-3214:
---

Integrated in Cassandra-0.8 #373 (See 
[https://builds.apache.org/job/Cassandra-0.8/373/])
Make CFIF use rpc_endpoint prior to trying endpoint.
Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for
CASSANDRA-3214

brandonwilliams : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1183489
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java


 Make CFIF use rpc_endpoint prior to trying endpoint
 ---

 Key: CASSANDRA-3214
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3214
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.8.6
 Environment: Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 / 
Reporter: Eldon Stegall
Assignee: Eldon Stegall
Priority: Minor
 Fix For: 0.8.8

 Attachments: CASSANDRA-3214-make-cfif-use-rpc-endpoints-v1.txt, 
 CASSANDRA-3214.patch


 Following up on CASSANDRA-3187 , we probably need to attempt to use the 
 rpc_endpoint address first, and then fall back to the gossip endpoint if we 
 don't get what we want.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183506 - in /cassandra/branches/cassandra-0.8: ./ contrib/pig/src/java/org/apache/cassandra/hadoop/pig/ src/java/org/apache/cassandra/client/ src/java/org/apache/cassandra/hadoop/ test/d

[brandonwilliams]

Author: brandonwilliams
Date: Fri Oct 14 21:22:14 2011
New Revision: 1183506

URL: http://svn.apache.org/viewvc?rev=1183506view=rev
Log:
Unify hadoop support for accept CDL for initial thrift address
Patch by Eldon Stegall, reviewed by brandonwilliams for CASSANDRA-3185

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java

cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java

cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/client/TestRingCache.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183506r1=1183505r2=1183506view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:22:14 2011
@@ -12,6 +12,8 @@
successfully acquired the compaction lock (CASSANDRA-3344)
  * (Hadoop) make CFIF try rpc_address or fallback to listen_address
(CASSANDRA-3214)
+ * (Hadoop) accept comma delimited lists of initial thrift connections
+   (CASSANDRA-3185)
 
 
 0.8.7

Modified: 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1183506r1=1183505r2=1183506view=diff
==
--- 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 Fri Oct 14 21:22:14 2011
@@ -545,7 +545,7 @@ public class CassandraStorage extends Lo
 Cassandra.Client client = null;
 try
 {
-client = 
createConnection(ConfigHelper.getInitialAddress(conf), 
ConfigHelper.getRpcPort(conf), true);
+client = ConfigHelper.getClientFromAddressList(conf);
 CfDef cfDef = null;
 client.set_keyspace(keyspace);
 KsDef ksDef = client.describe_keyspace(keyspace);
@@ -579,21 +579,6 @@ public class CassandraStorage extends Lo
 }
 }
 
-private static Cassandra.Client createConnection(String host, Integer 
port, boolean framed) throws IOException
-{
-TSocket socket = new TSocket(host, port);
-TTransport trans = framed ? new TFramedTransport(socket) : socket;
-try
-{
-trans.open();
-}
-catch (TTransportException e)
-{
-throw new IOException(unable to connect to server, e);
-}
-return new Cassandra.Client(new TBinaryProtocol(trans));
-}
-
 private static String cfdefToString(CfDef cfDef)
 {
 assert cfDef != null;

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java?rev=1183506r1=1183505r2=1183506view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
 Fri Oct 14 21:22:14 2011
@@ -21,25 +21,22 @@ import java.io.IOException;
 import java.net.InetAddress;
 import java.net.UnknownHostException;
 import java.nio.ByteBuffer;
-import java.util.HashSet;
 import java.util.List;
-import java.util.Set;
-
-import com.google.common.collect.ArrayListMultimap;
-import com.google.common.collect.Multimap;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.dht.IPartitioner;
 import org.apache.cassandra.dht.Range;
 import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.hadoop.ConfigHelper;
 import org.apache.cassandra.thrift.Cassandra;
 import org.apache.cassandra.thrift.InvalidRequestException;
 import org.apache.cassandra.thrift.TokenRange;
+import org.apache.hadoop.conf.Configuration;
 import org.apache.thrift.TException;
-import org.apache.cassandra.thrift.TBinaryProtocol;
-import org.apache.thrift.transport.TFramedTransport;
-import 

[jira] [Updated] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1

[Pavel Yaskevich (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-3342:
---

Attachment: CASSANDRA-3342.patch

 cassandra-cli allows setting min_compaction_threshold to 1
 --

 Key: CASSANDRA-3342
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20 
 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
Reporter: Jason Baker
Assignee: Pavel Yaskevich
 Fix For: 0.8.8, 1.0.1

 Attachments: CASSANDRA-3342.patch


 {{
 [root@Apture] update column family MagicLinks with min_compaction_threshold=1 
 and max_compaction_threshold=20;
 b98e3b80-f3a3-11e0--76abb4a6dbbf
 Waiting for schema agreement...
 ... schemas agree across the cluster
 }}
 I'm told that a min_compaction_threshold of 1 is nonsensical.  I had a spell 
 where my servers stopped doing compactions.  Once I upped the 
 min_compaction_threshold, they started compacting again.  I'm unable to 
 confirm for sure that this was the case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1

[Brandon Williams (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127870#comment-13127870
 ] 

Brandon Williams commented on CASSANDRA-3342:
-

+1 with minor typo fixes: s/then/than/ and s/greather/greater/

 cassandra-cli allows setting min_compaction_threshold to 1
 --

 Key: CASSANDRA-3342
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20 
 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
Reporter: Jason Baker
Assignee: Pavel Yaskevich
 Fix For: 0.8.8, 1.0.1

 Attachments: CASSANDRA-3342.patch


 {{
 [root@Apture] update column family MagicLinks with min_compaction_threshold=1 
 and max_compaction_threshold=20;
 b98e3b80-f3a3-11e0--76abb4a6dbbf
 Waiting for schema agreement...
 ... schemas agree across the cluster
 }}
 I'm told that a min_compaction_threshold of 1 is nonsensical.  I had a spell 
 where my servers stopped doing compactions.  Once I upped the 
 min_compaction_threshold, they started compacting again.  I'm unable to 
 confirm for sure that this was the case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127871#comment-13127871
 ] 

Jonathan Ellis commented on CASSANDRA-2466:
---

Great!

 bloom filters should avoid huge array allocations to avoid fragmentation 
 concerns
 -

 Key: CASSANDRA-2466
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2466
 Project: Cassandra
  Issue Type: Bug
Reporter: Peter Schuller
Priority: Minor
 Attachments: BloomFilterSerializer.java, OpenBitSet.java


 The fact that bloom filters are backed by single large arrays of longs is 
 expected to interact badly with promotion of objects into old gen with CMS, 
 due to fragmentation concerns (as discussed in CASSANDRA-2463).
 It should be less of an issue than CASSANDRA-2463 in the sense that you need 
 to have a lot of rows before the array sizes become truly huge. For 
 comparison, the ~ 143 million row key limit implied by the use of 'int' in 
 BitSet prior to the switch to OpenBitSet translates roughly to 238 MB 
 (assuming the limitation factor there was the addressability of the bits with 
 a 32 bit int, which is my understanding).
 Having a preliminary look at OpenBitSet with an eye towards replacing the 
 single long[] with multiple arrays, it seems that if we're willing to drop 
 some of the functionality that is not used for bloom filter purposes, the 
 bits[i] indexing should be pretty easy to augment with modulo to address an 
 appropriate smaller array. Locality is not an issue since the bloom filter 
 case is the worst possible case for locality anyway, and it doesn't matter 
 whether it's one huge array or a number of ~ 64k arrays.
 Callers may be affected like BloomFilterSerializer which cares about the 
 underlying bit array.
 If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) 
 some additional acrobatics would be necessary and presumably at a noticable 
 performance cost if such operations were to be used in performance critical 
 places.
 An argument against touching OpenBitSet is that it seems to be pretty 
 carefully written and tested and has some non-trivial details and people have 
 seemingly benchmarked it quite carefully. On the other hand, the improvement 
 would then apply to other things as well, such as the bitsets used to keep 
 track of in-core pages (off the cuff for scale, a 64 gig sstable should imply 
 a 2 mb bit set, with one bit per 4k page).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183509 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java

[jbellis]

Author: jbellis
Date: Fri Oct 14 21:35:42 2011
New Revision: 1183509

URL: http://svn.apache.org/viewvc?rev=1183509view=rev
Log:
add debug log of mutations + consistencylevel in SP.mutate
patch by Jackson Chung; reviewed by jbellis for CASSANDRA-3361

Modified:

cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java

Modified: 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1183509r1=1183508r2=1183509view=diff
==
--- 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
 (original)
+++ 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
 Fri Oct 14 21:35:42 2011
@@ -169,6 +169,7 @@ public class StorageProxy implements Sto
  */
 public static void mutate(List? extends IMutation mutations, 
ConsistencyLevel consistency_level) throws UnavailableException, 
TimeoutException
 {
+logger.debug(Mutations/ConsistencyLevel are {}/{}, mutations, 
consistency_level);
 final String localDataCenter = 
DatabaseDescriptor.getEndpointSnitch().getDatacenter(FBUtilities.getBroadcastAddress());
 
 long startTime = System.nanoTime();

svn commit: r1183510 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/thrift/ThriftValidation.java

[xedin]

Author: xedin
Date: Fri Oct 14 21:35:54 2011
New Revision: 1183510

URL: http://svn.apache.org/viewvc?rev=1183510view=rev
Log:
ColumnFamily min_compaction_threshold should be = 2
patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3342

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183510r1=1183509r2=1183510view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:35:54 2011
@@ -14,7 +14,7 @@
(CASSANDRA-3214)
  * (Hadoop) accept comma delimited lists of initial thrift connections
(CASSANDRA-3185)
-
+ * ColumnFamily min_compaction_threshold should be = 2 (CASSANDRA-3342)
 
 0.8.7
  * Kill server on wrapped OOME such as from FileChannel.map (CASSANDRA-3201)

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1183510r1=1183509r2=1183510view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
 Fri Oct 14 21:35:54 2011
@@ -679,19 +679,11 @@ public class ThriftValidation
 {
 if (cf_def.isSetMin_compaction_threshold()  
cf_def.isSetMax_compaction_threshold())
 {
-if ((cf_def.min_compaction_threshold  
cf_def.max_compaction_threshold)
- cf_def.max_compaction_threshold != 0)
-{
-throw new ConfigurationException(min_compaction_threshold 
cannot be greater than max_compaction_threshold);
-}
+validateMinCompactionThreshold(cf_def.min_compaction_threshold, 
cf_def.max_compaction_threshold);
 }
 else if (cf_def.isSetMin_compaction_threshold())
 {
-if (cf_def.min_compaction_threshold  
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD)
-{
-throw new 
ConfigurationException(String.format(min_compaction_threshold cannot be 
greather than max_compaction_threshold (default %d),
-   
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD));
-}
+validateMinCompactionThreshold(cf_def.min_compaction_threshold, 
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD);
 }
 else if (cf_def.isSetMax_compaction_threshold())
 {
@@ -706,6 +698,16 @@ public class ThriftValidation
 }
 }
 
+public static void validateMinCompactionThreshold(int 
min_compaction_threshold, int max_compaction_threshold) throws 
ConfigurationException
+{
+if (min_compaction_threshold = 1)
+throw new ConfigurationException(min_compaction_threshold cannot 
be less than 2.);
+
+if (min_compaction_threshold  max_compaction_threshold  
max_compaction_threshold != 0)
+throw new 
ConfigurationException(String.format(min_compaction_threshold cannot be 
greater than max_compaction_threshold %d,
+
max_compaction_threshold));
+}
+
 public static void 
validateMemtableSettings(org.apache.cassandra.thrift.CfDef cf_def) throws 
ConfigurationException
 {
 if (cf_def.isSetMemtable_flush_after_mins())

[jira] [Resolved] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3361.
---

   Resolution: Fixed
Fix Version/s: 1.0.1
 Reviewer: jbellis
 Assignee: Jackson Chung

committed, thanks!

 add logging to StorageProxy for mutations and CL
 

 Key: CASSANDRA-3361
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 1.0.1

 Attachments: 3361.patch


 Through 
 https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
  the read command and CL is logged (under DEBUG) and it is very helpful. But 
 I don't think I have seen the same for mutation...
 It'd be nice if we could also log this for the mutation (and others?), please 
 see the attached patch (just done on mutation) . Note also this prints the 
 IMutation which I did wonder if it is a good idea (could the list be so large 
 that it floods the log instead of being useful?)
 sample output:
 {noformat}
 DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171) 
 Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70', 
 modifications=[ColumnFamily(testindx 
 [testcol:false:1@1318619350892000,])])]/ONE
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput

[Pavel Yaskevich (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127890#comment-13127890
 ] 

Pavel Yaskevich commented on CASSANDRA-2999:


I think that this should be moved to 0.8.8 instead 1.0.1 because 
memtable_throughput_in_mb was removed in 1.0.0. Also CLI does not accept float 
because that parameter is int, do you think that this is a good idea to change 
it to float even if it would be used only for testing proposes?

 cassandra-cli doesn't accept floats for memtable_thoughput
 --

 Key: CASSANDRA-2999
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2999
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
 Fix For: 1.0.1


 The cassandra-cli does not accept floats for the value of memtable_throughput 
 (memtable_throughput_in_mb).  This means that you can't specify a value with 
 sub-MB resolution, such as 0.5.
 This probably doesn't matter much for production, but it's useful for testing 
 really low memtable thresholds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput

[Brandon Williams (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127893#comment-13127893
 ] 

Brandon Williams commented on CASSANDRA-2999:
-

I'm -1 on changing from an int to a float in what will be an oldstable release 
very shortly.

 cassandra-cli doesn't accept floats for memtable_thoughput
 --

 Key: CASSANDRA-2999
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2999
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
 Fix For: 1.0.1


 The cassandra-cli does not accept floats for the value of memtable_throughput 
 (memtable_throughput_in_mb).  This means that you can't specify a value with 
 sub-MB resolution, such as 0.5.
 This probably doesn't matter much for production, but it's useful for testing 
 really low memtable thresholds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput

[Pavel Yaskevich (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich resolved CASSANDRA-2999.


Resolution: Won't Fix

 cassandra-cli doesn't accept floats for memtable_thoughput
 --

 Key: CASSANDRA-2999
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2999
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
Priority: Trivial
 Fix For: 1.0.1


 The cassandra-cli does not accept floats for the value of memtable_throughput 
 (memtable_throughput_in_mb).  This means that you can't specify a value with 
 sub-MB resolution, such as 0.5.
 This probably doesn't matter much for production, but it's useful for testing 
 really low memtable thresholds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1183516 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java

[jbellis]

Author: jbellis
Date: Fri Oct 14 21:54:12 2011
New Revision: 1183516

URL: http://svn.apache.org/viewvc?rev=1183516view=rev
Log:
r/m redundant validateColumnFamily calls
patch by jbellis

Modified:

cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java

Modified: 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java?rev=1183516r1=1183515r2=1183516view=diff
==
--- 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
 (original)
+++ 
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
 Fri Oct 14 21:54:12 2011
@@ -73,11 +73,10 @@ public class QueryProcessor
 
 public static final String DEFAULT_KEY_NAME = 
bufferToString(CFMetaData.DEFAULT_KEY_NAME);
 
-private static Listorg.apache.cassandra.db.Row getSlice(String keyspace, 
SelectStatement select)
+private static Listorg.apache.cassandra.db.Row getSlice(CFMetaData 
metadata, SelectStatement select)
 throws InvalidRequestException, TimedOutException, UnavailableException
 {
 QueryPath queryPath = new QueryPath(select.getColumnFamily());
-CFMetaData metadata = validateColumnFamily(keyspace, 
select.getColumnFamily());
 ListReadCommand commands = new ArrayListReadCommand();
 
 // ...of a list of column names
@@ -91,13 +90,13 @@ public class QueryProcessor
 ByteBuffer key = 
rawKey.getByteBuffer(metadata.getKeyValidator());
 
 validateKey(key);
-commands.add(new SliceByNamesReadCommand(keyspace, key, 
queryPath, columnNames));
+commands.add(new SliceByNamesReadCommand(metadata.ksName, key, 
queryPath, columnNames));
 }
 }
 // ...a range (slice) of column names
 else
 {
-AbstractType? comparator = select.getComparator(keyspace);
+AbstractType? comparator = select.getComparator(metadata.ksName);
 ByteBuffer start = 
select.getColumnStart().getByteBuffer(comparator);
 ByteBuffer finish = 
select.getColumnFinish().getByteBuffer(comparator);
 
@@ -107,7 +106,7 @@ public class QueryProcessor
 
 validateKey(key);
 validateSliceRange(metadata, start, finish, 
select.isColumnsReversed());
-commands.add(new SliceFromReadCommand(keyspace,
+commands.add(new SliceFromReadCommand(metadata.ksName,
   key,
   queryPath,
   start,
@@ -144,13 +143,13 @@ public class QueryProcessor
 return columnNames;
 }
 
-private static Listorg.apache.cassandra.db.Row multiRangeSlice(String 
keyspace, SelectStatement select)
+private static Listorg.apache.cassandra.db.Row 
multiRangeSlice(CFMetaData metadata, SelectStatement select)
 throws TimedOutException, UnavailableException, InvalidRequestException
 {
 Listorg.apache.cassandra.db.Row rows;
 IPartitioner? p = StorageService.getPartitioner();
 
-AbstractType? keyType = Schema.instance.getCFMetaData(keyspace, 
select.getColumnFamily()).getKeyValidator();
+AbstractType? keyType = 
Schema.instance.getCFMetaData(metadata.ksName, 
select.getColumnFamily()).getKeyValidator();
 
 ByteBuffer startKey = (select.getKeyStart() != null)
? select.getKeyStart().getByteBuffer(keyType)
@@ -170,7 +169,6 @@ public class QueryProcessor
 }
 AbstractBounds bounds = new Bounds(startToken, finishToken);
 
-CFMetaData metadata = validateColumnFamily(keyspace, 
select.getColumnFamily());
 // XXX: Our use of Thrift structs internally makes me Sad. :(
 SlicePredicate thriftSlicePredicate = slicePredicateFromSelect(select, 
metadata);
 validateSlicePredicate(metadata, thriftSlicePredicate);
@@ -181,7 +179,7 @@ public class QueryProcessor
 
 try
 {
-rows = StorageProxy.getRangeSlice(new RangeSliceCommand(keyspace,
+rows = StorageProxy.getRangeSlice(new 
RangeSliceCommand(metadata.ksName,
 
select.getColumnFamily(),
 null,
 
thriftSlicePredicate,
@@ -220,10 +218,9 @@ public class QueryProcessor
 return rows.subList(0, select.getNumRecords()  rows.size() ? 
select.getNumRecords() : rows.size());
 }
 
-private static Listorg.apache.cassandra.db.Row getIndexedSlices(String 
keyspace, SelectStatement 

svn commit: r1183517 - in /cassandra/trunk: ./ bin/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cql/

[jbellis]

Author: jbellis
Date: Fri Oct 14 21:56:38 2011
New Revision: 1183517

URL: http://svn.apache.org/viewvc?rev=1183517view=rev
Log:
merge from 1.0

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/bin/nodetool
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 21:56:38 2011
@@ -4,7 +4,7 @@
 /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
-/cassandra/branches/cassandra-1.0:1167085-1183003,1183245
+/cassandra/branches/cassandra-1.0:1167085-1183245
 
/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1183517r1=1183516r2=1183517view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Fri Oct 14 21:56:38 2011
@@ -12,11 +12,12 @@
  * (CQL) allow numeric keyspace names in USE statement (CASSANDRA-3350)
  * reduce network copies (CASSANDRA-)
  * (Hadoop) skip empty rows when slicing the entire row (CASSANDRA-2855)
- * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
  * fix ColumnIndexer to use long offsets (CASSANDRA-3358)
  * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
  * Only count compaction as active (for throttling) when they have
successfully acquired the compaction lock (CASSANDRA-3344)
+ * limit nodetool to 32MB of heap (CASSANDRA-3124)
+ * (CQL) update parser to accept timestamp instead of date (CASSANDRA-3149)
 
 
 1.0.0-final

Modified: cassandra/trunk/bin/nodetool
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/bin/nodetool?rev=1183517r1=1183516r2=1183517view=diff
==
--- cassandra/trunk/bin/nodetool (original)
+++ cassandra/trunk/bin/nodetool Fri Oct 14 21:56:38 2011
@@ -55,8 +55,10 @@ case `uname` in
 ;;
 esac
 
-$JAVA -cp $CLASSPATH -Dstorage-config=$CASSANDRA_CONF \
--Dlog4j.configuration=log4j-tools.properties \
-org.apache.cassandra.tools.NodeCmd $@
+$JAVA -cp $CLASSPATH \
+  -Xmx32m \
+  -Dlog4j.configuration=log4j-tools.properties \
+  -Dstorage-config=$CASSANDRA_CONF \
+  org.apache.cassandra.tools.NodeCmd $@
 
 # vi:ai sw=4 ts=4 tw=0 et

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 21:56:38 2011
@@ -4,7 +4,7 @@
 
/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
-/cassandra/branches/cassandra-1.0/contrib:1167085-1183003,1183245
+/cassandra/branches/cassandra-1.0/contrib:1167085-1183245
 
/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951,1183243
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Oct 14 21:56:38 2011
@@ -4,7 +4,7 @@
 
/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1183002,1183241
 
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
 
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018
-/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1183003,1183245

svn commit: r1183518 - in /cassandra/branches/cassandra-0.8: CHANGES.txt contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

[brandonwilliams]

Author: brandonwilliams
Date: Fri Oct 14 21:57:29 2011
New Revision: 1183518

URL: http://svn.apache.org/viewvc?rev=1183518view=rev
Log:
Add 0.8+ types and key validation type to pig schema.
Patch by Steeve Morin, reviewed by brandonwilliams for CASSANDRA-3280

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183518r1=1183517r2=1183518view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:57:29 2011
@@ -15,6 +15,8 @@
  * (Hadoop) accept comma delimited lists of initial thrift connections
(CASSANDRA-3185)
  * ColumnFamily min_compaction_threshold should be = 2 (CASSANDRA-3342)
+ * (Pig) add 0.8+ types and key validation type in schema (CASSANDRA-3280)
+
 
 0.8.7
  * Kill server on wrapped OOME such as from FileChannel.map (CASSANDRA-3201)

Modified: 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1183518r1=1183517r2=1183518view=diff
==
--- 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 Fri Oct 14 21:57:29 2011
@@ -107,7 +107,7 @@ public class CassandraStorage extends Lo
 return limit;
 }
 
-   @Override
+@Override
 public Tuple getNext() throws IOException
 {
 try
@@ -122,7 +122,7 @@ public class CassandraStorage extends Lo
 assert key != null  cf != null;
 
 // and wrap it in a tuple
-   Tuple tuple = TupleFactory.getInstance().newTuple(2);
+Tuple tuple = TupleFactory.getInstance().newTuple(2);
 ArrayListTuple columns = new ArrayListTuple();
 tuple.set(0, new DataByteArray(key.array(), 
key.position()+key.arrayOffset(), key.limit()+key.arrayOffset()));
 for (Map.EntryByteBuffer, IColumn entry : cf.entrySet())
@@ -187,10 +187,12 @@ public class CassandraStorage extends Lo
 ArrayListAbstractType marshallers = new ArrayListAbstractType();
 AbstractType comparator = null;
 AbstractType default_validator = null;
+AbstractType key_validator = null;
 try
 {
-comparator = TypeParser.parse(cfDef.comparator_type);
-default_validator = 
TypeParser.parse(cfDef.default_validation_class);
+comparator = TypeParser.parse(cfDef.getComparator_type());
+default_validator = 
TypeParser.parse(cfDef.getDefault_validation_class());
+key_validator = TypeParser.parse(cfDef.getKey_validation_class());
 }
 catch (ConfigurationException e)
 {
@@ -199,13 +201,14 @@ public class CassandraStorage extends Lo
 
 marshallers.add(comparator);
 marshallers.add(default_validator);
+marshallers.add(key_validator);
 return marshallers;
 }
 
-private MapByteBuffer,AbstractType getValidatorMap(CfDef cfDef) throws  
IOException
+private MapByteBuffer, AbstractType getValidatorMap(CfDef cfDef) throws 
IOException
 {
 MapByteBuffer, AbstractType validators = new HashMapByteBuffer, 
AbstractType();
-for (ColumnDef cd : cfDef.column_metadata)
+for (ColumnDef cd : cfDef.getColumn_metadata())
 {
 if (cd.getValidation_class() != null  
!cd.getValidation_class().isEmpty())
 {
@@ -236,6 +239,18 @@ public class CassandraStorage extends Lo
 this.reader = reader;
 }
 
+public static MapString, String getQueryMap(String query)
+{
+String[] params = query.split();
+MapString, String map = new HashMapString, String();
+for (String param : params)
+{
+String[] keyValue = param.split(=);
+map.put(keyValue[0], keyValue[1]);
+}
+return map;
+}
+
 private void setLocationFromUri(String location) throws IOException
 {
 // parse uri into keyspace and columnfamily
@@ -247,18 +262,18 @@ public class CassandraStorage extends Lo
 String[] urlParts = location.split(\\?);
 if (urlParts.length  1)
 {
-for (String param : urlParts[1].split())
-{
-String[] pair = param.split(=);
-if (pair[0].equals(slice_start))
-

[jira] [Commented] (CASSANDRA-3280) Pig Storage Handler: Add =0.8.1 types, Guess right type for Key in Schema

[Brandon Williams (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127904#comment-13127904
 ] 

Brandon Williams commented on CASSANDRA-3280:
-

Committed, thanks!

 Pig Storage Handler: Add =0.8.1 types, Guess right type for Key in Schema
 --

 Key: CASSANDRA-3280
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3280
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.8.1
Reporter: Steeve Morin
  Labels: patch
 Fix For: 0.8.8

 Attachments: new_types_and_key_type.txt, new_types_and_key_type_v2.txt


 Various patches in the Pig Storage Handler:
 - correctly guess the right Pig type for the Row Key
 - add support for FloatType, DoubleType, -UUIDType (as String) and DateType 
 (as time epoch)- (removed in v2 because it would break when storing data back 
 in Cassandra)
 - add support to specify correct type comparator in SlicePredicate
 *SlicePredicate comparator:*
 For instance:
 {quote}
 {{raw =  LOAD 
 'cassandra://ks/cf?slice_start=2.5slice_end=5.3comparator=DoubleType' USING 
 CassandraStorage() AS ();}}
 {quote}
 It's an optional parameter. If it's not present, it will default to 
 BytesType. Which mean you must use hex strings.
 Hence {{slice_start=00slice_end=FF}} isn't the same as 
 {{slice_start=00slice_end=FFcomparator=AsciiType}} !

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3272) READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3272:
--

Attachment: 3272-v2.txt

v2 attached

 READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
 ---

 Key: CASSANDRA-3272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3272
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Jackson Chung
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.1

 Attachments: 3272-v2.txt, 3272.txt


 READ EACH_QUORUM:Returns the record with the most recent timestamp once 
 a quorum of replicas in each data center of the cluster has responded.
 In other words, if a DC is down and the QUORUM could not be reached on that 
 DC, read should fail.
 test case:
 - Cassandra version 0.8.6:
 INFO [main] 2011-09-28 22:26:24,297 StorageService.java (line 371) Cassandra 
 version: 0.8.6
 - 6-node cluster with 2 DC and 3 node each. RF=3 in each DC:
 [default@Keyspace3] describe keyspace;
 Keyspace: Keyspace3:
 Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
 Durable Writes: true
 Options: [DC2:3, DC1:3]
 Column Families:
 ColumnFamily: test
 Key Validation Class: org.apache.cassandra.db.marshal.BytesType
 Default column value validator: org.apache.cassandra.db.marshal.BytesType
 Columns sorted by: org.apache.cassandra.db.marshal.BytesType
 Row cache size / save period in seconds: 0.0/0
 Key cache size / save period in seconds: 20.0/14400
 Memtable thresholds: 1.0875/1440/232 (millions of ops/minutes/MB)
 GC grace seconds: 864000
 Compaction min/max thresholds: 4/32
 Read repair chance: 1.0
 Replicate on write: true
 Built indexes: []
 all nodes are up, insert a row:
 $ nodetool -h localhost ring
 Address DC Rack Status State Load Owns Token
 141784319550391026443072753096570088106
 10.34.79.179 DC1 RAC1 Up Normal 11.13 KB 16.67% 0
 10.34.70.163 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 28356863910078205288614550619314017621
 10.35.81.147 DC1 RAC1 Up Normal 11.14 KB 16.67% 
 56713727820156410577229101238628035242
 10.84.233.170 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 85070591730234615865843651857942052864
 10.195.201.236 DC1 RAC1 Up Normal 11.14 KB 16.67% 
 113427455640312821154458202477256070485
 10.118.147.73 DC2 RAC1 Up Normal 11.14 KB 16.67% 
 141784319550391026443072753096570088106
 - insert a value 
 [default@Keyspace3] set 
 test[utf8('test-key-1')][utf8('test-col')]=utf8('test-value');
 Value inserted.
 sanity check (cli connects to a node in DC1) :
 [default@Keyspace3] consistencylevel as EACH_QUORUM;  
 
 Consistency level is set to 'EACH_QUORUM'.
 [default@Keyspace3] get test[utf8('test-key-1')];   
 = (column=746573742d636f6c, value=test-value, timestamp=1317249361722000)
 Returned 1 results
 shut down DC2:
 $ nodetool -h localhost ring
 Address DC  RackStatus State   LoadOwns   
  Token   
   
  141784319550391026443072753096570088106 
 10.34.79.179DC1 RAC1Up Normal  51.86 KB16.67% 
  0   
 10.34.70.163DC2 RAC1Down   Normal  51.88 KB16.67% 
  28356863910078205288614550619314017621  
 10.35.81.147DC1 RAC1Up Normal  47.5 KB 16.67% 
  56713727820156410577229101238628035242  
 10.84.233.170   DC2 RAC1Down   Normal  51.88 KB16.67% 
  85070591730234615865843651857942052864  
 10.195.201.236  DC1 RAC1Up Normal  47.5 KB 16.67% 
  113427455640312821154458202477256070485 
 10.118.147.73   DC2 RAC1Down   Normal  51.88 KB16.67% 
  141784319550391026443072753096570088106  
 [default@Keyspace3] get test[utf8('test-key-1')];   
 = (column=746573742d636f6c, value=746573742d76616c7565, 
 timestamp=1317249361722000)
 Returned 1 results.
 tried with pycassaShell:
  col_fam.get('test-key-1',read_consistency_level=pycassa.ConsistencyLevel.EACH_QUORUM)
 OrderedDict([('test-col', 'test-value')])

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127914#comment-13127914
 ] 

Hudson commented on CASSANDRA-3342:
---

Integrated in Cassandra-0.8 #374 (See 
[https://builds.apache.org/job/Cassandra-0.8/374/])
ColumnFamily min_compaction_threshold should be = 2
patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3342

xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1183510
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java


 cassandra-cli allows setting min_compaction_threshold to 1
 --

 Key: CASSANDRA-3342
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20 
 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
Reporter: Jason Baker
Assignee: Pavel Yaskevich
 Fix For: 0.8.8, 1.0.1

 Attachments: CASSANDRA-3342.patch


 {{
 [root@Apture] update column family MagicLinks with min_compaction_threshold=1 
 and max_compaction_threshold=20;
 b98e3b80-f3a3-11e0--76abb4a6dbbf
 Waiting for schema agreement...
 ... schemas agree across the cluster
 }}
 I'm told that a min_compaction_threshold of 1 is nonsensical.  I had a spell 
 where my servers stopped doing compactions.  Once I upped the 
 min_compaction_threshold, they started compacting again.  I'm unable to 
 confirm for sure that this was the case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3185) Unify support across different map/reduce related classes for comma seperated list of hosts for initial thrift port connection.

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127916#comment-13127916
 ] 

Hudson commented on CASSANDRA-3185:
---

Integrated in Cassandra-0.8 #374 (See 
[https://builds.apache.org/job/Cassandra-0.8/374/])
Unify hadoop support for accept CDL for initial thrift address
Patch by Eldon Stegall, reviewed by brandonwilliams for CASSANDRA-3185

brandonwilliams : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1183506
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
* 
/cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java
* 
/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/client/TestRingCache.java


 Unify support across different map/reduce related classes for comma seperated 
 list of hosts for initial thrift port connection.
 ---

 Key: CASSANDRA-3185
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3185
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.8.0
 Environment: Hadoop-0.20+pig-0.8
Reporter: Eldon Stegall
Assignee: Eldon Stegall
Priority: Minor
 Fix For: 0.8.8

 Attachments: 
 unify-comma-seperated-hadoop-initial-thrift-connection-v4.patch


 Unify support across different map/reduce related classes for comma seperated 
 list of hosts for initial thrift port connection. Column family input format 
 already had support for this since CASSANDRA-2807, and RingCache did it the 
 same way, but it was coded seperately. This JIRA should add pig support, and 
 abstract a common method to iterate over these seeds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3280) Pig Storage Handler: Add =0.8.1 types, Guess right type for Key in Schema

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127915#comment-13127915
 ] 

Hudson commented on CASSANDRA-3280:
---

Integrated in Cassandra-0.8 #374 (See 
[https://builds.apache.org/job/Cassandra-0.8/374/])
Add 0.8+ types and key validation type to pig schema.
Patch by Steeve Morin, reviewed by brandonwilliams for CASSANDRA-3280

brandonwilliams : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1183518
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java


 Pig Storage Handler: Add =0.8.1 types, Guess right type for Key in Schema
 --

 Key: CASSANDRA-3280
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3280
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.8.1
Reporter: Steeve Morin
  Labels: patch
 Fix For: 0.8.8

 Attachments: new_types_and_key_type.txt, new_types_and_key_type_v2.txt


 Various patches in the Pig Storage Handler:
 - correctly guess the right Pig type for the Row Key
 - add support for FloatType, DoubleType, -UUIDType (as String) and DateType 
 (as time epoch)- (removed in v2 because it would break when storing data back 
 in Cassandra)
 - add support to specify correct type comparator in SlicePredicate
 *SlicePredicate comparator:*
 For instance:
 {quote}
 {{raw =  LOAD 
 'cassandra://ks/cf?slice_start=2.5slice_end=5.3comparator=DoubleType' USING 
 CassandraStorage() AS ();}}
 {quote}
 It's an optional parameter. If it's not present, it will default to 
 BytesType. Which mean you must use hex strings.
 Hence {{slice_start=00slice_end=FF}} isn't the same as 
 {{slice_start=00slice_end=FFcomparator=AsciiType}} !

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3335) ThreadPoolExecutor creates threads as non-daemon and will block on shutdown by default

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3335:
--

Attachment: 3335-v2.txt

v2 adds a log line before running any scheduled task. Maybe that will help 
pinpoint the culprit.

 ThreadPoolExecutor creates threads as non-daemon and will block on shutdown 
 by default
 --

 Key: CASSANDRA-3335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3335
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.0.1

 Attachments: 3335-v2.txt, 3335.txt


 This is most obviously visible in OptionalTasks which should not block 
 shutdown, but often does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3335) ThreadPoolExecutor creates threads as non-daemon and will block on shutdown by default

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127926#comment-13127926
 ] 

Jonathan Ellis commented on CASSANDRA-3335:
---

(all the task logging is done w/ the StorageService logger so it can be enabled 
separately from the reset of the package involved)

 ThreadPoolExecutor creates threads as non-daemon and will block on shutdown 
 by default
 --

 Key: CASSANDRA-3335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3335
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.0.1

 Attachments: 3335-v2.txt, 3335.txt


 This is most obviously visible in OptionalTasks which should not block 
 shutdown, but often does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3362) allow sub-row repair

[Jonathan Ellis (Created) (JIRA)]

allow sub-row repair


 Key: CASSANDRA-3362
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3362
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
Priority: Minor


With large rows, it would be nice to not have to send an entire row if a small 
part is out of sync.  Could we use the row index blocks as repair atoms instead 
of the full row?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3327) Support TimeUUID in CassandraStorage

[Brandon Williams (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-3327:


Attachment: 3327.txt

 Support TimeUUID in CassandraStorage
 

 Key: CASSANDRA-3327
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3327
 Project: Cassandra
  Issue Type: Bug
  Components: Contrib
Affects Versions: 0.8.7
 Environment: Cassandra 0.8.6 Build #348 (CASSANDRA-2777 + 
 CASSANDRA-2810)
Reporter: Manuel Kreutz
Assignee: Brandon Williams
  Labels: pig
 Fix For: 0.8.8

 Attachments: 3327.txt


 Cassandra CLI:
 {code}
 grunt raw = LOAD 'cassandra://TEST/CF'
  USING CassandraStorage()
  AS (
  key:chararray,
  columns:bag {
  column:tuple(
  name,
  value
  )
  });
 grunt describe raw;
 raw: {key: chararray,columns: {(name: bytearray,value: bytearray)}}
 log_test =
 FOREACH raw
 GENERATE
 (CHARARRAY) key,
 flatten(columns);
 grunt DUMP log_test;
 {code}
 Returns:
 {code}
 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
 open iterator for alias log_test. Backend error : Unexpected data type 
 java.util.UUID found in stream. Note only standard Pig type is supported when 
 you output from UDF/LoadFunc
 at org.apache.pig.PigServer.openIterator(PigServer.java:890)
 at 
 org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655)
 at 
 org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
 at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
 at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
 at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
 at org.apache.pig.Main.run(Main.java:487)
 at org.apache.pig.Main.main(Main.java:108)
 Caused by: java.lang.RuntimeException: Unexpected data type java.util.UUID 
 found in stream. Note only standard Pig type is supported when you output 
 from UDF/LoadFunc
 at 
 org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
 at 
 org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:542)
 at 
 org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
 at 
 org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
 at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
 at 
 org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:498)
 at 
 org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:263)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
 at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
 {code}
 According to driftx on IRC the setTupleValue function in CassandraStorage 
 needs to handle the uuid case and cast it to a DataByteArray.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3363) Allow one leveled compaction task to kick off another

[Jonathan Ellis (Created) (JIRA)]

Allow one leveled compaction task to kick off another
-

 Key: CASSANDRA-3363
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.0.1


Leveled compaction wants to prevent multiple tasks from running at once, but 
this check also defeats the kick off another compaction if there is more work 
to do code in CompactionTask.  So currently LCS relies completely on the 
every-five-minutes compaction check, which is not enough to keep up with heavy 
insert load.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3363:
--

Reviewer: brandon.williams  (was: slebresne)

 Allow one leveled compaction task to kick off another
 -

 Key: CASSANDRA-3363
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.0.1


 Leveled compaction wants to prevent multiple tasks from running at once, but 
 this check also defeats the kick off another compaction if there is more 
 work to do code in CompactionTask.  So currently LCS relies completely on 
 the every-five-minutes compaction check, which is not enough to keep up with 
 heavy insert load.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3363:
--

Reviewer: slebresne  (was: brandon.williams)

 Allow one leveled compaction task to kick off another
 -

 Key: CASSANDRA-3363
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.0.1

 Attachments: 3363.txt


 Leveled compaction wants to prevent multiple tasks from running at once, but 
 this check also defeats the kick off another compaction if there is more 
 work to do code in CompactionTask.  So currently LCS relies completely on 
 the every-five-minutes compaction check, which is not enough to keep up with 
 heavy insert load.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3363:
--

Attachment: 3363.txt

Patch attached that encapsulates CT.submitNextTask and 
CM.getBackgroundSubmitter so LCT can override and wrap them.

Kind of ugly, open to better ideas.

 Allow one leveled compaction task to kick off another
 -

 Key: CASSANDRA-3363
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.0.1

 Attachments: 3363.txt


 Leveled compaction wants to prevent multiple tasks from running at once, but 
 this check also defeats the kick off another compaction if there is more 
 work to do code in CompactionTask.  So currently LCS relies completely on 
 the every-five-minutes compaction check, which is not enough to keep up with 
 heavy insert load.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3334) dropping index causes some inflight mutations to fail

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3334:
--

Attachment: 0002-fix.patch
0001-cleanup.patch

Patch to fix attached, as well as patch to clean up the SecondaryIndexManager a 
bit.

 dropping index causes some inflight mutations to fail
 -

 Key: CASSANDRA-3334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3334
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0.0
 Environment: windows
Reporter: Radim Kolar
Priority: Minor
  Labels: indexing
 Fix For: 1.0.1

 Attachments: 0001-cleanup.patch, 0002-fix.patch


 dropping index causes some inflight mutations to fail. hector on client side 
 didnt throw any exception
  INFO [MigrationStage:1] 2011-10-07 23:11:53,742 Migration.java (line 119) 
 Applying migration fb1a8540-f128-11e0--23b38323f4da Update column family 
 to 
 org.apache.cassandra.config.CFMetaData@786669[cfId=1000,ksName=test,cfName=sipdb,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=null,comment=phone
  calls routing 
 information,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=0.0,replicateOnWrite=false,gcGraceSeconds=0,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@8bb33c,mergeShardsChance=0.1,keyAlias=java.nio.HeapByteBuffer[pos=461
  lim=464 cap=466],column_metadata={java.nio.HeapByteBuffer[pos=0 lim=3 
 cap=3]=ColumnDefinition{name=6b616d, 
 validator=org.apache.cassandra.db.marshal.AsciiType, index_type=null, 
 index_name='null'}},compactionStrategyClass=class 
 org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
  INFO [MigrationStage:1] 2011-10-07 23:11:53,805 ColumnFamilyStore.java (line 
 664) Enqueuing flush of Memtable-Migrations@27537043(7860/9825 
 serialized/live bytes, 1 ops)
  INFO [MigrationStage:1] 2011-10-07 23:11:53,820 ColumnFamilyStore.java (line 
 664) Enqueuing flush of Memtable-Schema@8340427(3320/4150 serialized/live 
 bytes, 3 ops)
  INFO [FlushWriter:3] 2011-10-07 23:11:53,820 Memtable.java (line 237) 
 Writing Memtable-Migrations@27537043(7860/9825 serialized/live bytes, 1 ops)
  INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 273) 
 Completed flushing \var\lib\cassandra\data\system\Migrations-h-14-Data.db 
 (7924 bytes)
  INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 237) 
 Writing Memtable-Schema@8340427(3320/4150 serialized/live bytes, 3 ops)
  INFO [CompactionExecutor:4] 2011-10-07 23:11:55,008 CompactionTask.java 
 (line 119) Compacting 
 [SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-13-Data.db'),
  
 SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-14-Data.db'), 
 SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-11-Data.db'), 
 SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-12-Data.db')]
  INFO [FlushWriter:3] 2011-10-07 23:11:56,430 Memtable.java (line 273) 
 Completed flushing \var\lib\cassandra\data\system\Schema-h-14-Data.db (3470 
 bytes)
  INFO [CompactionExecutor:3] 2011-10-07 23:11:56,446 CompactionTask.java 
 (line 119) Compacting 
 [SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-13-Data.db'), 
 SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-14-Data.db'), 
 SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-12-Data.db'), 
 SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-11-Data.db')]
 ERROR [MutationStage:56] 2011-10-07 23:11:56,508 AbstractCassandraDaemon.java 
 (line 133) Fatal exception in thread Thread[MutationStage:56,5,main]
 java.lang.AssertionError
   at 
 org.apache.cassandra.db.index.SecondaryIndexManager.applyIndexUpdates(SecondaryIndexManager.java:369)
   at org.apache.cassandra.db.Table.apply(Table.java:457)
   at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:253)
   at 
 org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:436)
   at 
 org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1263)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)
 ERROR [MutationStage:51] 2011-10-07 23:11:56,539 AbstractCassandraDaemon.java 
 (line 

[jira] [Commented] (CASSANDRA-3069) multiget support in CQL (UNION / OR)

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128048#comment-13128048
 ] 

Jonathan Ellis commented on CASSANDRA-3069:
---

Pavel, do you remember which ticket it was where we decided that full OR 
support was too messy?

 multiget support in CQL (UNION / OR)
 

 Key: CASSANDRA-3069
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3069
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.0.1

 Attachments: CASSANDRA-3069.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3143:
--

Reviewer: slebresne  (was: jbellis)

 Global caches (key/row)
 ---

 Key: CASSANDRA-3143
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3143
 Project: Cassandra
  Issue Type: Improvement
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: Core
 Fix For: 1.1

 Attachments: 0001-global-key-cache.patch, 
 0002-global-row-cache-and-ASC.readSaved-changed-to-abstra.patch, 
 0003-CacheServiceMBean-and-correct-key-cache-loading.patch, 
 0004-key-row-cache-tests-and-tweaks.patch, 
 0005-cleanup-of-the-CFMetaData-and-thrift-avro-CfDef-and-.patch


 Caches are difficult to configure well as ColumnFamilies are added, similar 
 to how memtables were difficult pre-CASSANDRA-2006.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3317) SStable corruption possible caused by Leveldb compaction

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128050#comment-13128050
 ] 

Jonathan Ellis commented on CASSANDRA-3317:
---

I can't reproduce.  Are you using the Cassandra stress tool or something else?

 SStable corruption possible caused by Leveldb compaction
 

 Key: CASSANDRA-3317
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3317
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0.0
Reporter: Radim Kolar

 During stress testing of leveldb compaction i found that sometimes it create 
 corrupted ssfile and
 cassandra starts with error. I am not sure what caused that error, but i cant 
 reproduce it if i switch to standard
 compaction strategy.
 ERROR [SSTableBatchOpen:1] 2011-10-05 15:31:31,781 
 AbstractCassandraDaemon.java
 (line 133) Fatal exception in thread Thread[SSTableBatchOpen:1,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java
 :129)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.jav
 a:196)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44
 1)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
 utor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
 .java:908)
 at java.lang.Thread.run(Thread.java:662)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-3267) Failed to change replication factor on 3 node cluster

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3267.
---

Resolution: Not A Problem

See wiki.apache.org/cassandra/FAQ#schema_disagreement for how to solve schema 
disagreement problems.  (And CASSANDRA-1391 for how we plan to fix that 
permanently.)

 Failed to change replication factor on 3 node cluster
 -

 Key: CASSANDRA-3267
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3267
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.8.5
 Environment: Linux, openjdk16, cassandra 0.8.5
Reporter: Timur Tulyaganov

 3-node cluster config all up and running failed to change replication factor
 /usr/local/cassandra/bin/nodetool -h 10.0.1.3 ring Address Status State Load 
 Owns Token
 Address DC  RackStatus State   LoadOwns   
  Token   
   
  57715317233367107622067286720208938865  
 10.0.1.1  datacenter1 rack1   Up Normal  1.49 GB 78.15%  
 20543402371996174596346065790779111550  
 10.0.1.2  datacenter1 rack1   Up Normal  1011.54 MB  7.59%   
 33454860067234500516210522518260948578  
 10.0.1.3  datacenter1 rack1   Up Normal  232.25 MB   14.26%  
 57715317233367107622067286720208938865 
 [default@API] describe keyspace API;
 Keyspace: API:
   Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
   Durable Writes: true
 Options: [replication_factor:1]
   Column Families:
 ColumnFamily: AdCampaign
   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
   Default column value validator: 
 org.apache.cassandra.db.marshal.BytesType
 [default@API] update keyspace API with 
 strategy_options=[{replication_factor:2}];
 null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3250) fsync the directory after new sstable or commit log segment are created

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3250:
--

 Reviewer: jbellis
  Component/s: Core
 Priority: Minor  (was: Major)
Fix Version/s: 1.0.1
 Assignee: Pavel Yaskevich

 fsync the directory after new sstable or commit log segment are created
 ---

 Key: CASSANDRA-3250
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3250
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Zhu Han
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.0.1


 The mannual of fsync said:
 bq.   Calling  fsync()  does  not  necessarily  ensure  that  the entry in 
 the directory containing the file has also reached disk.  For that an 
 explicit fsync() on a file descriptor for the directory is also needed.
 At least on ext4, syncing the directory is a must to have step, as described 
 by [1]. Otherwise, the new sstables or commit logs could be missed after 
 crash even if itself is synced. 
 Unfortunately, JVM does not provide an approach to sync the directory...
 [1] 
 http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/don%E2%80%99t-fear-fsync

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3170) Schema versions output should be on separate lines for separate versions

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3170:
-

Assignee: Pavel Yaskevich

 Schema versions output should be on separate lines for separate versions
 

 Key: CASSANDRA-3170
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3170
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: cli, lhf

 On the CLI if you do a 'describe cluster;' it would be really nice to have 
 different versions on different lines or some way to call out multiple 
 versions more.  Right now it's a UUID with a list of nodes for one, but with 
 multiple versions, it's on the same line - another UUID and another list of 
 nodes.  That's hard to distinguish between one version and multiple versions 
 at a glance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3217) javax.management.InstanceAlreadyExistsException: during startup

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3217:
-

Assignee: Pavel Yaskevich

The second report here is confusing to me, I'm not sure how it could fail once 
but then succeed with no schema changes in between.

 javax.management.InstanceAlreadyExistsException: during startup
 ---

 Key: CASSANDRA-3217
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3217
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.8.5
Reporter: Thibaut
Assignee: Pavel Yaskevich
Priority: Minor

 Hi,
 I got this error on one machines after restarting (the node could have been 
 killed due to an outofmemory error). It would crash on each restart on this 
 error.
 I copied over the system table from another machine and deleted the 
 locationinfo files and everything worked fine.
 I also had to delete 2 db files which coulnd't be loaded due to UTF-8 
 encoding errors. These rows were from the same table_feedfetch. The node 
 might have memory errors, I'm going to check this. So it might be related to 
 that.
 ERROR [main] 2011-09-16 16:13:18,468 AbstractCassandraDaemon.java (line 358) 
 Exception encountered during startup.
 java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException: 
 org.apache.cassandra.db:type=ColumnFamilies,keyspace=table_feedfetch,columnfamily=table_feedfetch
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:303)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:465)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:435)
 at org.apache.cassandra.db.Table.initCf(Table.java:369)
 at org.apache.cassandra.db.Table.init(Table.java:306)
 at org.apache.cassandra.db.Table.open(Table.java:111)
 at 
 org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:187)
 at 
 org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:341)
 at 
 org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:97)
 Caused by: javax.management.InstanceAlreadyExistsException: 
 org.apache.cassandra.db:type=ColumnFamilies,keyspace=table_feedfetch,columnfamily=table_feedfetch
 at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
 at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
 at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:299)
 ... 8 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3189) redhat init script can report success (exit 0) for startup failures

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3189:
-

Assignee: paul cannon

 redhat init script can report success (exit 0) for startup failures
 ---

 Key: CASSANDRA-3189
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3189
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
Affects Versions: 0.8.5
Reporter: Eric Evans
Assignee: paul cannon
Priority: Minor
 Fix For: 1.0.1


 The Redhat init script's start target merely returns the status code of the 
 JVM.  In a perfect world that would be adequate, but the JVM does not always 
 return a non-zero status when it should.
 The most obvious fix is to test that the process is live before exiting. See 
 the Debian init script for an example of this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3130) CQL queries should alow talbe names to be qualified by keyspace

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3130:
-

Assignee: Pavel Yaskevich

 CQL queries should alow talbe names to be qualified by keyspace
 ---

 Key: CASSANDRA-3130
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3130
 Project: Cassandra
  Issue Type: New Feature
Reporter: Edward Capriolo
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.0.1


 While the 0.6.X api was ugly in terms of method signatures, it did allow 
 you to use the same client to query multiple keyspaces without having to call 
 set_keyspace(String). I totally dislike set_keyspace but I know the thrift 
 API is definitely not changing.
 The following command sequence is three RPC operations.
 {noformat}
 select * from cf;
 use otherkeyspace;
 select * from othercf;
 {noformat}
 CQL should allow us to do:
 {noformat}
 select * from keyspace1.cf;
 select * from keyspace2.cf;
 {noformat}
 This will make the connection pool management on the client much easier.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3105) MarshallException no longer showing for incompatible key_validation_class

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3105:
-

Assignee: Pavel Yaskevich

 MarshallException no longer showing for incompatible key_validation_class
 -

 Key: CASSANDRA-3105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3105
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Affects Versions: 0.8.4
 Environment: Mac OS X 10.6.8
 Cassandra-cli 0.8.4
 Cassandra 0.8.1
Reporter: Anthony Ikeda
Assignee: Pavel Yaskevich
Priority: Minor

 Recently I tried querying a column family which had the default bytes 
 key_validation_class using a string. Rather than the usual MarshallException 
 error displayed, it just displays 'null'
 [default@unknown] connect localhost/9160;
 Connected to: Test Cluster on localhost/9160
 [default@unknown] use RegistryFoundation; 
 Authenticated to keyspace: RegistryFoundation
 [default@RegistryFoundation] list Registry limit 1;   
 
 ---
 RowKey: REG:e38e47af-da8d-4cea-917b-44b93198643d
 = (column=c-createInfo, 
 value=7b2274696d657374616d70223a22323031312d30382d31315431333a33343a32362e3136312d30373a3030222c2270726f66696c654964223a2265636f6d6d3a70742f7072663a61696b65646140777367632e636f6d227d,
  timestamp=1313094866191002)
 = (column=c-externalId, value=15cf9a223f8ad92768499dbdd497c524, 
 timestamp=1313094866191001)
 = (column=c-name, value=416e74686f6e79277320776973686c697374, 
 timestamp=1313094866191000)
 = (column=l-pt-bfbb2342-c85e-43c9-bb69-b3bb54c723fb, value=7074, 
 timestamp=1313094866388000)
 = (column=p-ecomm:pt/prf:aik...@wsgc.com, value=4f574e4552, 
 timestamp=1313094866191003)
 1 Row Returned.
 [default@RegistryFoundation] assume Registry keys as bytes;   
 
 Assumption for column family 'Registry' added successfully.
 [default@RegistryFoundation] get 
 Registry['REG:e38e47af-da8d-4cea-917b-44b93198643d'];
 null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3065) Major file corruption after running nodetool cleanup

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3065:
-

Assignee: Sylvain Lebresne

 Major file corruption after running nodetool cleanup
 

 Key: CASSANDRA-3065
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3065
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.3
Reporter: Benjamin Schrauwen
Assignee: Sylvain Lebresne

 After running nodetool cleanup on two of the nodes in my 4 node cluster, 
 almost all SSTables on those those machine got corrupted. I am not able to 
 read them anymore with sstable2json, and the cassandra daemon is repetitively 
 throwing:
 ERROR [ReadStage:11] 2011-08-20 04:44:46,846 AbstractCassandraDaemon.java 
 (line 139) Fatal exception in thread Thread[ReadStage:11,5,main]
 java.lang.RuntimeException: java.lang.IndexOutOfBoundsException
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:619)
 Caused by: java.lang.IndexOutOfBoundsException
   at java.nio.Buffer.checkIndex(Buffer.java:514)
   at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209)
   at 
 org.apache.cassandra.io.util.MappedFileDataInput.read(MappedFileDataInput.java:104)
   at java.io.InputStream.read(InputStream.java:154)
   at 
 org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196)
   at 
 org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:61)
   at 
 org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:58)
   at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91)
   at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:67)
   at 
 org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66)
   at 
 org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1314)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1181)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1221)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1168)
   at org.apache.cassandra.db.Table.getRow(Table.java:385)
   at 
 org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58)
   at 
 org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:641)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-3060) Boostrapping node stuck after receiving file with version mismatch

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3060.
---

Resolution: Not A Problem

There's not much the bootstrapping node can do in that situation, other than 
wait for you to scrub and then retry the bootstrap.

 Boostrapping node stuck after receiving file with version mismatch
 --

 Key: CASSANDRA-3060
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3060
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.3
Reporter: Benjamin Schrauwen
Priority: Minor

 When adding a node to the cluster, the boostrapping process get's stuck when 
 it receives a file which is apparently from an older version. This is the 
 Exception
 java.lang.RuntimeException: Cannot recover SSTable 
 /var/lib/cassandra/data/Mollom/recent_sessions-tmp-f-1 due to version 
 mismatch. (current version is g).
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.createBuilder(SSTableWriter.java:240)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.submitSSTableBuild(CompactionManager.java:1090)
 at 
 org.apache.cassandra.streaming.StreamInSession.finished(StreamInSession.java:110)
 at 
 org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:104)
 at 
 org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:177)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:114)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-2997) Enhance human-readability of snapshot names created by drop column family

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-2997:
-

Assignee: Yuki Morishita

Eric's suggestion to include the CF name in single-CF snapshots is a good one.

 Enhance human-readability of snapshot names created by drop column family
 -

 Key: CASSANDRA-2997
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2997
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.8.2
Reporter: Eric Gilmore
Assignee: Yuki Morishita
Priority: Minor

 Currently when you drop a column family, a snapshot is automatically created 
 in a directory named with the timestamp of the drop.  
 Clever folk will find a way to understand the timestamps and locate 
 particular snapshots, but it is not as effortless as it could be if part or 
 all of the CF name was included in the snapshot name.
 Any strategy to make the snapshot name more user-friendly and easy to find 
 would be helpful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2932) Implement assume in cqlsh

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2932.
---

Resolution: Duplicate

being done in CASSANDRA-3188

 Implement assume in cqlsh
 ---

 Key: CASSANDRA-2932
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2932
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Priority: Minor
  Labels: lhf

 In the CLI there is a handy way to assume CF comparators/validators 
 (CASSANDRA-1693). It would be very nice to have the assume command in cqlsh 
 as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2944) Show the nodes' schemas in the nodetool ring result.

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2944.
---

Resolution: Not A Problem

Use the cli describe cluster command instead.

 Show the nodes' schemas in the nodetool ring result.
 

 Key: CASSANDRA-2944
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2944
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Affects Versions: 0.8.1
Reporter: Dikang Gu
  Labels: cassandra, nodetool, schema

 Sometimes, we get the schema does not agree error from the cassandra 
 cluster, it will be great if we can view the schemas directly on the nodetool 
 ring output.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2910) expose cluster availability

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2910.
---

Resolution: Won't Fix

ISTM these are more the responsibility of monitoring systems than Cassandra per 
se.

 expose cluster availability
 ---

 Key: CASSANDRA-2910
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2910
 Project: Cassandra
  Issue Type: New Feature
Reporter: Wojciech Meler

 Cluster should be able to:
  - return % of tokenspace available for given ConsistencyLevel
  - return highest available ConsistencyLevel for given token

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3292) creating column family sets durable_writes to true

[Jonathan Ellis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3292:
--

Reviewer: xedin  (was: tjake)

 creating column family sets durable_writes to true
 --

 Key: CASSANDRA-3292
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3292
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.5
Reporter: Radim Kolar
Assignee: Jonathan Ellis
Priority: Minor
  Labels: schema
 Fix For: 0.8.8

 Attachments: 0001-r-m-rename-migrations.patch, 
 0002-add-KSM.cloneWith.patch


 [default@rapidshare] describe keyspace rapidshare;
 Keyspace: rapidshare:
   Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
   Durable Writes: *false*
 Options: [datacenter1:1]
   Column Families:
 [default@rapidshare] create column family t1;
 1ba19300-ebfa-11e0--34912694d0bf
 Waiting for schema agreement...
 ... schemas agree across the cluster
 [default@rapidshare] describe keyspace rapidshare;
 Keyspace: rapidshare:
   Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
   Durable Writes: *true*
 Options: [datacenter1:1]
   Column Families:
 ColumnFamily: t1
   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
   Default column value validator: 
 org.apache.cassandra.db.marshal.BytesType
   Columns sorted by: org.apache.cassandra.db.marshal.BytesType
   Row cache size / save period in seconds: 0.0/0
   Key cache size / save period in seconds: 20.0/14400
   Memtable thresholds: 0.0281249997/1440/6 (millions of 
 ops/minutes/MB)
   GC grace seconds: 864000
   Compaction min/max thresholds: 4/32
   Read repair chance: 1.0
   Replicate on write: true
   Built indexes: []

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-2268) CQL-enabled stress.java

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-2268:
-

Assignee: paul cannon  (was: Aaron Morton)

 CQL-enabled stress.java
 ---

 Key: CASSANDRA-2268
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2268
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Eric Evans
Assignee: paul cannon
Priority: Minor
  Labels: cql
 Fix For: 0.8.8

 Attachments: 0001-2268-wip.patch


 It would be great if stress.java had a CQL mode.  For making the inevitable 
 RPC-CQL comparisons, but also as a basis for measuring optimizations, and 
 spotting performance regressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2963) Add a convenient way to reset a node's schema

[Jonathan Ellis (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128061#comment-13128061
 ] 

Jonathan Ellis commented on CASSANDRA-2963:
---

Pierre-Yves, are you still planning a v2 here?

 Add a convenient way to reset a node's schema
 -

 Key: CASSANDRA-2963
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2963
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 0.8.8

 Attachments: system_reset_schema.txt


 People often encounter a schema disagreement where just one node is out of 
 sync.  To get it back in sync, they shutdown the node, move the Schema* and 
 Migration* files out of the system ks, and then start it back up.  Rather 
 than go through this process, it would be nice if you could just tell the 
 node to reset its schema.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3088) Expose compaction_throughput_mb_per_sec for JMX tweaking

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3088:
-

Assignee: Brandon Williams

 Expose compaction_throughput_mb_per_sec for JMX tweaking
 

 Key: CASSANDRA-3088
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3088
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Affects Versions: 0.8.0
Reporter: Jonathan Ellis
Assignee: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 0.8.8




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3186) nodetool should not NPE when rack/dc info is not yet available

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3186:
-

Assignee: Brandon Williams

 nodetool should not NPE when rack/dc info is not yet available
 --

 Key: CASSANDRA-3186
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3186
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
  Labels: lhf
 Fix For: 0.8.8


 As the title says.  What happens is the persisted ring is loaded, but if 
 those nodes are down and you're using a snitch like ec2 that gets rack/dc 
 info from gossip, nodetool NPEs instead of showing that the node is down.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2453) node.js cql driver

[Jonathan Ellis (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2453.
---

   Resolution: Not A Problem
Fix Version/s: (was: 0.8.8)
 Assignee: (was: Gary Dusbabek)

Closing in favor of moving to apache extras.

 node.js cql driver
 --

 Key: CASSANDRA-2453
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2453
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Gary Dusbabek
Priority: Minor



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3126) unable to remove column metadata via CLI

[Jonathan Ellis (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-3126:
-

Assignee: Pavel Yaskevich

 unable to remove column metadata via CLI
 

 Key: CASSANDRA-3126
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3126
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Affects Versions: 0.7.0
Reporter: Radim Kolar
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: cassandra-cli
 Fix For: 0.8.8


 I cant find way how to remove all columns definitions without CF 
 import/export.
 [default@int4] update column family sipdb with column_metadata = [];
 Syntax error at position 51: required (...)+ loop did not match anything at 
 input ']'
 [default@int4] update column family sipdb with column_metadata = [{}];
 Command not found: `update column family sipdb with column_metadata = [{}];`. 
 Type 'help;' or '?' for help.
 [default@int4]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira