date:20110315


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006824#comment-13006824
 ] 

Sylvain Lebresne commented on CASSANDRA-2327:
-

I'll remark that in trunk the flusherLock don't block reads and writes anymore 
(see CASSANDRA-1954). We decided this was a change subtle enough to not put it 
in a stable version.

 Table.flusherLock is static final.. remove static.
 --

 Key: CASSANDRA-2327
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2327
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4, 0.7.5
 Environment: JVM
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Fix For: 0.7.5


 I see read and write latency spike when the system tables are 
 flushing(according to opscentral) 
 Only reason which i can come-up with is probably because of the 
 Table.flusherLock is static final... i think it should not be static because 
 the flush is per keyspace and this lock will lock all the read operations 
 because one table is going through a flush.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2272) Refactor Key and Row caches handling code


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-2272:


Attachment: 0001-Refactor-key-and-row-cache-handling-code-v3.patch

Attaching rebased v3

 Refactor Key and Row caches handling code 
 --

 Key: CASSANDRA-2272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2272
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Refactor-key-and-row-cache-handling-code-v2.patch, 
 0001-Refactor-key-and-row-cache-handling-code-v3.patch, 
 0001-Refactor-key-and-row-cache-handling-code.patch

   Original Estimate: 4h
  Remaining Estimate: 4h

 The code related to key and row caches has a bunch of code duplication. It's
 moreover a bit scattered, the caches are in SSTableTracker but the scheduled
 task to save them are in CFStore. And having the caches in SSTableTracker is
 not very logic, for row cache at least, since it does not only concern the
 sstables. Proposed patch refactor this to avoid the duplication and put all
 the cache handling code in one place.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2284) Make changes to the set of memtables and sstables of a cfstore atomic


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-2284:


Attachment: 0001-Make-memtable-and-sstable-switches-atomic-v2.patch

Attaching a v2 that is rebased against trunk and leave the query logic in 
CFStore.

 Make changes to the set of memtables and sstables of a cfstore atomic 
 --

 Key: CASSANDRA-2284
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2284
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 0.8

 Attachments: 0001-Make-memtable-and-sstable-switches-atomic-v2.patch, 
 0001-Make-memtable-and-sstable-switches-atomic.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 Some switches happen in the set of memtables and sstables of a cfstore:
   * when we switch the memtable, the current memtable is replaced by a new 
 one and add to the memtablesPendingFlush
   * when a memtable is fully flushed, it is removed from 
 memtablesPendingFlush and the newly created sstable is added to the set of 
 active sstables.
   * after compaction, compacted memtables are removed from the active 
 sstables and the compacted sstable is added instead
 Only the last of these operations is atomic. This ticket proposes to makes 
 all of them atomic, using the idea of the View idea hinted by Stu in the 
 comments of CASSANDRA-1954.
 The main reason for this is to fix CASSANDRA-2105. But I think that another 
 benefit of this is to make reasoning about those operations easier.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081758 - /cassandra/trunk/build.xml

2011-03-15 Thread gdusbabek

Author: gdusbabek
Date: Tue Mar 15 13:03:55 2011
New Revision: 1081758

URL: http://svn.apache.org/viewvc?rev=1081758view=rev
Log:
do not instrument the *Token classes. patch by gdusbabek, reviewed by jbellis. 
CASSANDRA-2258

Modified:
cassandra/trunk/build.xml

Modified: cassandra/trunk/build.xml
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/build.xml?rev=1081758r1=1081757r2=1081758view=diff
==
--- cassandra/trunk/build.xml (original)
+++ cassandra/trunk/build.xml Tue Mar 15 13:03:55 2011
@@ -703,6 +703,10 @@
 exclude name=**/*Test.class/
 exclude name=**/*TestCase.class/
 exclude name=**/test/*.class/
+!-- cobertura modifies the serialVersionUID of classes. Some of our 
unit tests rely on backward
+wire compatability of these classes.  It was easier to exlude them 
from instrumentation than to
+force their serialVersinUIDs. --
+exclude name=**/*Token.class/
 exclude name=${cobertura.excludes}/
   /fileset

svn commit: r1081760 - /cassandra/branches/cassandra-0.7/build.xml

2011-03-15 Thread gdusbabek

Author: gdusbabek
Date: Tue Mar 15 13:05:51 2011
New Revision: 1081760

URL: http://svn.apache.org/viewvc?rev=1081760view=rev
Log:
do not instrument *Token classes. patch by gdusbabek, reviewed by jbellis. 
CASSANDRA-2258

Modified:
cassandra/branches/cassandra-0.7/build.xml

Modified: cassandra/branches/cassandra-0.7/build.xml
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/build.xml?rev=1081760r1=1081759r2=1081760view=diff
==
--- cassandra/branches/cassandra-0.7/build.xml (original)
+++ cassandra/branches/cassandra-0.7/build.xml Tue Mar 15 13:05:51 2011
@@ -627,6 +627,10 @@
 exclude name=**/*Test.class/
 exclude name=**/*TestCase.class/
 exclude name=**/test/*.class/
+!-- cobertura modifies the serialVersionUID of classes. Some of our 
unit tests rely on backward
+wire compatability of these classes.  It was easier to exlude them 
from instrumentation than to
+force their serialVersinUIDs. --
+exclude name=**/*Token.class/
 exclude name=${cobertura.excludes}/
   /fileset

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL

2011-03-15 Thread Gary Dusbabek (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006914#comment-13006914
 ] 

Gary Dusbabek commented on CASSANDRA-2124:
--

bq. Should we replace rowmetadata with ResultsetMetaData?
The problem with using ResultSetMetaData is that it is a bad interface for 
Cassandra's sparse columns.  A good example is that column 0 in one row might 
map to a column named foo, while column 0 in another row might map to a 
column named bar.  The Wrappable interface exists so that JDBC driver 
implementations can provide features beyond the JDBC API without the ugliness 
of casting.  I think it is better to make the programmer fully aware that he is 
dealing with sparse columns and forcing him to use the RowMetaData API.

I'd like to hear your points on the benefits of using ResultSetMetaData though. 
 I admit that RowMetaData may not be the ideal way of accessing row-specific 
meta information.


 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v3-0001-first-pass-at-column-decoding.txt, 
 v3-0002-clean-up-JdbcDriverTest.txt, 
 v3-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v3-0004-more-comments-in-ColumnDecoder.txt, 
 v3-0005-move-all-java-cql-into-jdbc-package-and-only-expose-.patch


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006915#comment-13006915
 ] 

Jonathan Ellis commented on CASSANDRA-2124:
---

do we support resultsetmetadata for mostly static CFs?

 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v3-0001-first-pass-at-column-decoding.txt, 
 v3-0002-clean-up-JdbcDriverTest.txt, 
 v3-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v3-0004-more-comments-in-ColumnDecoder.txt, 
 v3-0005-move-all-java-cql-into-jdbc-package-and-only-expose-.patch


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006939#comment-13006939
 ] 

Jonathan Ellis commented on CASSANDRA-2124:
---

I like that idea a lot.

 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v3-0001-first-pass-at-column-decoding.txt, 
 v3-0002-clean-up-JdbcDriverTest.txt, 
 v3-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v3-0004-more-comments-in-ColumnDecoder.txt, 
 v3-0005-move-all-java-cql-into-jdbc-package-and-only-expose-.patch


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Reopened: (CASSANDRA-2302) JDBC driver needs ResultSet.getMetaData()

2011-03-15 Thread Gary Dusbabek (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek reopened CASSANDRA-2302:
--


 JDBC driver needs ResultSet.getMetaData()
 -

 Key: CASSANDRA-2302
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2302
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: 
 v2-0001-RowData-exposes-sparse-column-attributes-through-Cassa.txt, 
 v2-0002-make-RowMetaData-typed.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL

2011-03-15 Thread Gary Dusbabek (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006945#comment-13006945
 ] 

Gary Dusbabek commented on CASSANDRA-2124:
--

Ok. I reopened CASSANDRA-2302.

 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v3-0001-first-pass-at-column-decoding.txt, 
 v3-0002-clean-up-JdbcDriverTest.txt, 
 v3-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v3-0004-more-comments-in-ColumnDecoder.txt, 
 v3-0005-move-all-java-cql-into-jdbc-package-and-only-expose-.patch


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081806 - /cassandra/tags/cassandra-0.7.4/

Author: eevans
Date: Tue Mar 15 14:47:40 2011
New Revision: 1081806

URL: http://svn.apache.org/viewvc?rev=1081806view=rev
Log:
tag 0.7.4 release

Added:
cassandra/tags/cassandra-0.7.4/
  - copied from r1080811, cassandra/branches/cassandra-0.7/

[jira] Reopened: (CASSANDRA-1954) Double-check or replace RRW memtable lock


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-1954:
---


 Double-check or replace RRW memtable lock
 -

 Key: CASSANDRA-1954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 
 0001-Double-check-in-maybeSwitchMemtable-to-minimize-writeL.txt, 
 0001-Remove-flusherLock-readLock.patch, 1954-v2.txt, 1954_trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 {quote}...when a Memtable reaches its threshold, up to (all) N write threads 
 will often notice, and race to acquire the writeLock in order to freeze the 
 memtable. This means that we do way more writeLock acquisitions than we need 
 to...{quote}
 See CASSANDRA-1930 for backstory, but adding double checking inside a read 
 lock before trying to re-entrantly acquire the writelock would eliminate most 
 of these excess writelock acquisitions.
 Alternatively, we should explore removing locking from these structures 
 entirely, and replacing the writeLock acquisition with a per-memtable counter 
 of active threads.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2327) Table.flusherLock is static final.. remove static.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006978#comment-13006978
 ] 

Jonathan Ellis commented on CASSANDRA-2327:
---

Re-opened CASSANDRA-1954 to make the double-checked locking improvement to 0.7. 
 We shouldn't make more-invasive changes than that in a stable release series.

 Table.flusherLock is static final.. remove static.
 --

 Key: CASSANDRA-2327
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2327
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4, 0.7.5
 Environment: JVM
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Fix For: 0.7.5


 I see read and write latency spike when the system tables are 
 flushing(according to opscentral) 
 Only reason which i can come-up with is probably because of the 
 Table.flusherLock is static final... i think it should not be static because 
 the flush is per keyspace and this lock will lock all the read operations 
 because one table is going through a flush.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081824 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py

Author: eevans
Date: Tue Mar 15 15:58:25 2011
New Revision: 1081824

URL: http://svn.apache.org/viewvc?rev=1081824view=rev
Log:
update site versioning for 0.7.4 release

Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1081824r1=1081823r2=1081824view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Tue Mar 15 15:58:25 2011
@@ -73,31 +73,31 @@
   
 
   p
-  The latest stable release of Apache Cassandra is 0.7.3
-  (released on 2011-03-04).  iIf you're just
+  The latest stable release of Apache Cassandra is 0.7.4
+  (released on 2011-03-15).  iIf you're just
   starting out, download this one./i
   /p
 
   ul
 li
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/binary_download');
-  apache-cassandra-0.7.3-bin.tar.gz
+  apache-cassandra-0.7.4-bin.tar.gz
 /a
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz.sha;SHA1/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz.sha;SHA1/a]
 /li
 li
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.3/apache-cassandra-0.7.3-src.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.4/apache-cassandra-0.7.4-src.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/source_download');
-  apache-cassandra-0.7.3-src.tar.gz
+  apache-cassandra-0.7.4-src.tar.gz
 /a
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-src.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-src.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-src.tar.gz.sha;SHA1/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-src.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-src.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/0.7.4/apache-cassandra-0.7.4-src.tar.gz.sha;SHA1/a]
 /li
   /ul 
 
@@ -139,15 +139,15 @@ New users to Cassandra should be sure to
   h2Download/h2
   div class=inner rc
 p
-The latest release is b0.7.3/b
-span class=relnotes(a 
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.7.3/CHANGES.txt;Changes/a)/span
+The latest release is b0.7.4/b
+span class=relnotes(a 
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.7.4/CHANGES.txt;Changes/a)/span
 /p
 
 p
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/binary_download');
-  apache-cassandra-0.7.3-bin.tar.gz
+  apache-cassandra-0.7.4-bin.tar.gz
 /a
 /p
 

Modified: cassandra/site/publish/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/index.html?rev=1081824r1=1081823r2=1081824view=diff
==
--- cassandra/site/publish/index.html (original)
+++ cassandra/site/publish/index.html Tue Mar 15 15:58:25 2011
@@ -82,15 +82,15 @@
   h2Download/h2
   div class=inner rc
 p
-The latest release is b0.7.3/b
-span class=relnotes(a 
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.7.3/CHANGES.txt;Changes/a)/span
+The latest release is b0.7.4/b
+span class=relnotes(a 
href=https://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.7.4/CHANGES.txt;Changes/a)/span
 /p
 
 p
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.4/apache-cassandra-0.7.4-bin.tar.gz;
onclick=javascript:

[jira] Updated: (CASSANDRA-2321) Counter column values shows in hex values. Need to show it in string value.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-2321:


Attachment: 0001-Don-t-allow-normal-query-on-counter-CF.patch

Actually, having getString() return hexString is the right thing to do, because 
internally counters are bytes and getString is used by sstable2json in 
particular.

The problem however is that it is not disallowed to query a counter CF with 
non-counter operation. Attaching a patch to correct this. This is a bigger 
patch that one would hope because ThriftValidation doesn't help. So the patch 
does a bunch of refactoring to allow what it must do. As a side node, the 
refactoring makes it more efficient (We don't revalidate the column family for 
each mutation of a batch_mutate).

Btw, stress.py has no support for counters, but stress.java has it. 

 Counter column values shows in hex values. Need to show it in string value.
 ---

 Key: CASSANDRA-2321
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2321
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8
 Environment: Linux
Reporter: Mubarak Seyed
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Don-t-allow-normal-query-on-counter-CF.patch


 CounterColumnType.getString() returns hexString.
 {code}
 public String getString(ByteBuffer bytes)
 { 
return ByteBufferUtil.bytesToHex(bytes);
 }
 {code}
 and python stress.py reader returns
 [ColumnOrSuperColumn(column=None, super_column=SuperColumn(name='19', 
 columns=[Column(timestamp=1299984960277, name='56', 
 value='\x7f\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00,',
  ttl=None), Column(timestamp=1299985019923, name='57', 
 value='\x7f\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00;\x00\x00\x00\x00\x00\x00\x08\xfd',
  ttl=None))]

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1954) Double-check or replace RRW memtable lock


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006990#comment-13006990
 ] 

Sylvain Lebresne commented on CASSANDRA-1954:
-

+1 on the double-checkd locking patch for 0.7.

 Double-check or replace RRW memtable lock
 -

 Key: CASSANDRA-1954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 
 0001-Double-check-in-maybeSwitchMemtable-to-minimize-writeL.txt, 
 0001-Remove-flusherLock-readLock.patch, 1954-0.7-v2.txt, 1954-v2.txt, 
 1954_trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 {quote}...when a Memtable reaches its threshold, up to (all) N write threads 
 will often notice, and race to acquire the writeLock in order to freeze the 
 memtable. This means that we do way more writeLock acquisitions than we need 
 to...{quote}
 See CASSANDRA-1930 for backstory, but adding double checking inside a read 
 lock before trying to re-entrantly acquire the writelock would eliminate most 
 of these excess writelock acquisitions.
 Alternatively, we should explore removing locking from these structures 
 entirely, and replacing the writeLock acquisition with a per-memtable counter 
 of active threads.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081838 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/thrift/CassandraServer.java src/java/org/apache/cassandra/thrift/ThriftValidation.java test/syste

Author: jbellis
Date: Tue Mar 15 16:25:41 2011
New Revision: 1081838

URL: http://svn.apache.org/viewvc?rev=1081838view=rev
Log:
validate index names
patch by jhermes and jbellis for CASSANDRA-1761

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/ThriftValidation.java
cassandra/branches/cassandra-0.7/test/system/test_thrift_server.py

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081838r1=1081837r2=1081838view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 16:25:41 2011
@@ -2,6 +2,7 @@
  * Avoid seeking when sstable2json exports the entire file (CASSANDRA-2318)
  * fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
+ * validate index names (CASSANDRA-1761)
 
 
 0.7.4

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java?rev=1081838r1=1081837r2=1081838view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
 Tue Mar 15 16:25:41 2011
@@ -627,7 +627,7 @@ public class CassandraServer implements 
 
 // helper method to apply migration on the migration stage. typical 
migration failures will throw an 
 // InvalidRequestException. atypical failures will throw a 
RuntimeException.
-private static void applyMigrationOnStage(final Migration m) throws 
InvalidRequestException
+private static void applyMigrationOnStage(final Migration m)
 {
 Future f = StageManager.getStage(Stage.MIGRATION).submit(new Callable()
 {
@@ -644,23 +644,11 @@ public class CassandraServer implements 
 }
 catch (InterruptedException e)
 {
-throw new RuntimeException(e);
+throw new AssertionError(e);
 }
 catch (ExecutionException e)
 {
-// this means call() threw an exception. deal with it directly.
-if (e.getCause() != null)
-{
-InvalidRequestException ex = new 
InvalidRequestException(e.getCause().getMessage());
-ex.initCause(e.getCause());
-throw ex;
-}
-else
-{
-InvalidRequestException ex = new 
InvalidRequestException(e.getMessage());
-ex.initCause(e);
-throw ex;
-}
+throw new RuntimeException(e);
 }
 }
 
@@ -824,6 +812,7 @@ public class CassandraServer implements 
 {
 logger.debug(update_column_family);
 state().hasColumnFamilyListAccess(Permission.WRITE);
+ThriftValidation.validateCfDef(cf_def);
 if (cf_def.keyspace == null || cf_def.name == null)
 throw new InvalidRequestException(Keyspace and CF name must be 
set.);
 CFMetaData oldCfm = 
DatabaseDescriptor.getCFMetaData(CFMetaData.getId(cf_def.keyspace, 
cf_def.name));

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/ThriftValidation.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1081838r1=1081837r2=1081838view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/ThriftValidation.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/ThriftValidation.java
 Tue Mar 15 16:25:41 2011
@@ -21,15 +21,15 @@ package org.apache.cassandra.thrift;
  */
 
 import java.nio.ByteBuffer;
-import java.util.Arrays;
-import java.util.Comparator;
-import java.util.Set;
+import java.util.*;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ConfigurationException;
 import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.MarshalException;
@@ -422,8 +422,16 @@ public class ThriftValidation
 AbstractType comparator = cfType == ColumnFamilyType.Standard

[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL

2011-03-15 Thread Vivek Mishra (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006997#comment-13006997
 ] 

Vivek Mishra commented on CASSANDRA-2124:
-

Ok. Additionally, for methods like:
getTableName(int column) and getSchemaName(int column) should we implement them 
to return Column family and Keyspace respectively?

Not analyzed yet, but planning to add addBatch(), executeBatch calls on 
CassandraStatement.


 JDBC driver for CQL
 ---

 Key: CASSANDRA-2124
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Eric Evans
Assignee: Vivek Mishra
Priority: Minor
  Labels: cql
 Fix For: 0.8

 Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, 
 cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, 
 cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, 
 v3-0001-first-pass-at-column-decoding.txt, 
 v3-0002-clean-up-JdbcDriverTest.txt, 
 v3-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, 
 v3-0004-more-comments-in-ColumnDecoder.txt, 
 v3-0005-move-all-java-cql-into-jdbc-package-and-only-expose-.patch


 A simple connection class and corresponding pool was created for CQL as a 
 part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a 
 replacement for) would also be interesting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2329) comment topology file about ipv6 configuration

2011-03-15 Thread Eric Tamme (JIRA)

comment topology file about ipv6 configuration
--

 Key: CASSANDRA-2329
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2329
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.3
 Environment: all
Reporter: Eric Tamme
Priority: Trivial
 Fix For: 0.7.4


There is little documentation regarding ipv6 working with cassandra.  For the 
cassandra propertyfilesnitch to work with ipv6, the ipv6 addresses must have 
the colons escaped due to the parsing characteristics of 
java.util.Properties.load().

Adding this documentation to the cassandra-topology.properties file would be 
nice so that people dont have to hunt through source code and java doc to 
figure out how to configure topology.

Here is a simple patch that will alert users to how to config topology for ipv6 
use.

34a35,39
 # Native IPv6 is supported, however you must escape the : in the IPv6 Address
 # Also be sure to comment out JVM_OPTS=$JVM_OPTS 
 -Djava.net.preferIPv4Stack=true
 # in cassandra-env.sh
 fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3
 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2330) Queue indexes for flush before the parent

Queue indexes for flush before the parent
-

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4


Secondary indexes flush when the parent does.  This has an unfortunate side 
effect: a single CF flushing with a single secondary index fills the flush 
queue and blocks further writes until the first one completes.  A simple but 
naive optimization here would be to queue the indexes before the parent since 
they are generally going to be smaller, and thus flush faster, reducing the 
amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1761) Indexes: Auto-generating the CFname may collide with user-generated names


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007009#comment-13007009
 ] 

Hudson commented on CASSANDRA-1761:
---

Integrated in Cassandra-0.7 #382 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/382/])
validate index names
patch by jhermes and jbellis for CASSANDRA-1761


 Indexes: Auto-generating the CFname may collide with user-generated names
 -

 Key: CASSANDRA-1761
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1761
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7 beta 3
Reporter: Jon Hermes
Assignee: Jon Hermes
Priority: Minor
 Fix For: 0.7.5

 Attachments: 1761-0.7.txt, 1761.txt, repro.cli

   Original Estimate: 16h
  Remaining Estimate: 16h

 {noformat}column_families:
   - name: CF
 comparator: BytesType
 column_metadata: 
   - name: foo
 index_name: 626172
 index_type: KEYS
   - name: bar
 index_type: KEYS{noformat}
 Auto-generated versus user-supplied names collide in the YAML above. The code:
 {code}cfname = parentCf + . + (info.getIndexName() == null ? 
 FBUtilities.bytesToHex(info.name) : info.getIndexName()){code}
 From the first ColumnDefinition, we create cfname = CF.626172 (from the 
 fail clause of the ternany, user-supplied name)
 From the second ColumnDefinition, we create cfname = CF.626172 (from the 
 pass clause of the ternary, we generate the name)
 They're in hex form. This is possible, but fairly unlikely that someone will 
 do this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Issue Comment Edited: (CASSANDRA-2330) Queue indexes for flush before the parent


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007007#comment-13007007
 ] 

Brandon Williams edited comment on CASSANDRA-2330 at 3/15/11 4:46 PM:
--

Patch to concat indexes ahead of the parent.  stress.java blocks for 7s instead 
of 9, with an 8G heap.

  was (Author: brandon.williams):
Patch to concat indexes ahead of the parent.  stress.java blocks for 7s 
instead of 9.
  
 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2319) Promote row index

[
https://issues.apache.org/jira/browse/CASSANDRA-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007010#comment-13007010
]

Sylvain Lebresne commented on CASSANDRA-2319:
-

I'm reasonably in favor of this too, given that it would also give us a good
place to add checksumming (which should be a reasonably short term priority
imho) and compression.

There is however a few things to considered:
* This will remove as a consequence the row bloom filter (maybe it makes
sense to promote it too in some form to the sstable level, but I'm not sure).
Row bloom filter is not the most useful trick we have around (only useful for
names query for starter). Still something to keep in mind.
* There will be new trade-offs: either you index 'often' or the index becomes
potentially much bigger (in which case you push the problem to the sparse
in-memory index). I suspect it will be harder to have a 'one configuration fits
all'.

Promote row index
-

Key: CASSANDRA-2319
URL: https://issues.apache.org/jira/browse/CASSANDRA-2319
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Stu Hood
Assignee: Stu Hood
Labels: index, timeseries
Fix For: 0.8

The row index contains entries for configurably sized blocks of a wide row.
For a row of appreciable size, the row index ends up directing the third seek
(1. index, 2. row index, 3. content) to nearby the first column of a scan.
Since the row index is always used for wide rows, and since it contains
information that tells us whether or not the 3rd seek is necessary (the
column range or name we are trying to slice may not exist in a given
sstable), promoting the row index into the sstable index would allow us to
drop the maximum number of seeks for wide rows back to 2, and, more
importantly, would allow sstables to be eliminated using only the index.
An example usecase that benefits greatly from this change is time series data
in wide rows, where data is appended to the beginning or end of the row. Our
existing compaction strategy gets lucky and clusters the oldest data in the
oldest sstables: for queries to recently appended data, we would be able to
eliminate wide rows using only the sstable index, rather than needing to seek
into the data file to determine that it isn't interesting. For narrow rows,
this change would have no effect, as they will not reach the threshold for
indexing anyway.
A first cut design for this change would look very similar to the file format
design proposed on #674:
http://wiki.apache.org/cassandra/FileFormatDesignDoc: row keys clustered,
column names clustered, and offsets clustered and delta encoded.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2258) service.SerializationsTest failes under cobertura


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007008#comment-13007008
 ] 

Hudson commented on CASSANDRA-2258:
---

Integrated in Cassandra-0.7 #382 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/382/])


 service.SerializationsTest failes under cobertura
 -

 Key: CASSANDRA-2258
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2258
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Jonathan Ellis
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.7.5

 Attachments: v1-0001-do-not-instrument-the-Token-classes.txt, 
 v1-0001-fix-cobertura-brokenness-by-forcing-serialVersionUIDs.txt


 ant codecoverage -Dtest.name=SerializationsTest gives
 {noformat}
 [junit] Testcase: 
 testTreeResponseRead(org.apache.cassandra.service.SerializationsTest):  
 Caused an ERROR
 [junit] java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit] java.lang.RuntimeException: java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit]   at 
 org.apache.cassandra.service.AntiEntropyService$TreeResponseVerbHandler.deserialize(AntiEntropyService.java:634)
 [junit]   at 
 org.apache.cassandra.service.SerializationsTest.testTreeResponseRead(SerializationsTest.java:90)
 [junit] Caused by: java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit]   at 
 java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:562)
 [junit]   at 
 java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
 [junit]   at 
 java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
 [junit]   at 
 org.apache.cassandra.service.AntiEntropyService$TreeResponseVerbHandler.deserialize(AntiEntropyService.java:630)
 {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2330) Queue indexes for flush before the parent


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2330:


Attachment: 2330.txt

Patch to concat indexes ahead of the parent.  stress.java blocks for 7s instead 
of 9.

 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.1
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.4

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2331) Allow a Job Configuration to control the consistency level with which reads and writes occur for mapreduce jobs.

2011-03-15 Thread Eldon Stegall (JIRA)

Allow a Job Configuration to control the consistency level with which reads and 
writes occur for mapreduce jobs.


 Key: CASSANDRA-2331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2331
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Eldon Stegall
Priority: Minor
 Attachments: consistencylevelpatch.txt

Allow a Job Configuration to control the consistency level with which reads and 
writes occur for mapreduce jobs. This would be useful if we were doing 
statistical rollups and wanted to fail if consistency isn't met (W + R = N  as 
described by http://wiki.apache.org/cassandra/ArchitectureOverview). An example 
of a situation where this might happen would be if a set of cassandra nodes 
became partitioned from the other nodes, but were still accepting some writes 
with ConsistencyLevel 1/2 (in the case that writes were local to the 
partition). Then statistical rollup jobs might succeed without those accepted 
writes being included, and this would not be desirable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2331) Allow a Job Configuration to control the consistency level with which reads and writes occur for mapreduce jobs.

2011-03-15 Thread Eldon Stegall (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eldon Stegall updated CASSANDRA-2331:
-

Attachment: consistencylevelpatch.txt

 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs.
 

 Key: CASSANDRA-2331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2331
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Eldon Stegall
Priority: Minor
 Attachments: consistencylevelpatch.txt


 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs. This would be useful if we were doing 
 statistical rollups and wanted to fail if consistency isn't met (W + R = N  
 as described by http://wiki.apache.org/cassandra/ArchitectureOverview). An 
 example of a situation where this might happen would be if a set of cassandra 
 nodes became partitioned from the other nodes, but were still accepting some 
 writes with ConsistencyLevel 1/2 (in the case that writes were local to the 
 partition). Then statistical rollup jobs might succeed without those accepted 
 writes being included, and this would not be desirable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2330) Queue indexes for flush before the parent


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2330:


Affects Version/s: (was: 0.7.1)
   0.7.0
Fix Version/s: (was: 0.7.4)
   0.7.5

 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2329) comment topology file about ipv6 configuration


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2329:


Fix Version/s: (was: 0.7.4)
   0.7.5

 comment topology file about ipv6 configuration
 --

 Key: CASSANDRA-2329
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2329
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.3
 Environment: all
Reporter: Eric Tamme
Priority: Trivial
 Fix For: 0.7.5


 There is little documentation regarding ipv6 working with cassandra.  For the 
 cassandra propertyfilesnitch to work with ipv6, the ipv6 addresses must have 
 the colons escaped due to the parsing characteristics of 
 java.util.Properties.load().
 Adding this documentation to the cassandra-topology.properties file would be 
 nice so that people dont have to hunt through source code and java doc to 
 figure out how to configure topology.
 Here is a simple patch that will alert users to how to config topology for 
 ipv6 use.
 34a35,39
  # Native IPv6 is supported, however you must escape the : in the IPv6 
  Address
  # Also be sure to comment out JVM_OPTS=$JVM_OPTS 
  -Djava.net.preferIPv4Stack=true
  # in cassandra-env.sh
  fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3
  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081859 - /cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties

Author: brandonwilliams
Date: Tue Mar 15 17:06:24 2011
New Revision: 1081859

URL: http://svn.apache.org/viewvc?rev=1081859view=rev
Log:
Add comments to topology file about ipv6 configuration.
Patch by Eric Tamme, reviewed by brandonwilliams for CASSANDRA-2329

Modified:
cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties

Modified: cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties?rev=1081859r1=1081858r2=1081859view=diff
==
--- cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties 
(original)
+++ cassandra/branches/cassandra-0.7/conf/cassandra-topology.properties Tue Mar 
15 17:06:24 2011
@@ -35,3 +35,7 @@
 # default for unknown nodes
 default=DC1:r1
 
+# Native IPv6 is supported, however you must escape the colon in the IPv6 
Address
+# Also be sure to comment out JVM_OPTS=$JVM_OPTS 
-Djava.net.preferIPv4Stack=true
+# in cassandra-env.sh
+fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3

[jira] Resolved: (CASSANDRA-2329) comment topology file about ipv6 configuration


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-2329.
-

Resolution: Fixed

Committed, thanks.

 comment topology file about ipv6 configuration
 --

 Key: CASSANDRA-2329
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2329
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.3
 Environment: all
Reporter: Eric Tamme
Priority: Trivial
 Fix For: 0.7.5


 There is little documentation regarding ipv6 working with cassandra.  For the 
 cassandra propertyfilesnitch to work with ipv6, the ipv6 addresses must have 
 the colons escaped due to the parsing characteristics of 
 java.util.Properties.load().
 Adding this documentation to the cassandra-topology.properties file would be 
 nice so that people dont have to hunt through source code and java doc to 
 figure out how to configure topology.
 Here is a simple patch that will alert users to how to config topology for 
 ipv6 use.
 34a35,39
  # Native IPv6 is supported, however you must escape the : in the IPv6 
  Address
  # Also be sure to comment out JVM_OPTS=$JVM_OPTS 
  -Djava.net.preferIPv4Stack=true
  # in cassandra-env.sh
  fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3
  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081866 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java

Author: jbellis
Date: Tue Mar 15 17:21:20 2011
New Revision: 1081866

URL: http://svn.apache.org/viewvc?rev=1081866view=rev
Log:
r/m unused code

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java?rev=1081866r1=1081865r2=1081866view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableExport.java
 Tue Mar 15 17:21:20 2011
@@ -97,21 +97,16 @@ public class SSTableExport
  * @param out output stream
  * @return pair of (number of columns serialized, last column serialized)
  */
-private static PairInteger, ByteBuffer 
serializeColumns(IteratorIColumn columns, PrintStream out)
+private static void serializeColumns(IteratorIColumn columns, 
PrintStream out)
 {
-int n = 0;
-IColumn column = null;
 while (columns.hasNext())
 {
-column = columns.next();
-n++;
+IColumn column = columns.next();
 serializeColumn(column, out);
 
 if (columns.hasNext())
 out.print(, );
 }
-
-return new PairInteger, ByteBuffer(n, column == null ? null : 
column.name());
 }
 
 /**

svn commit: r1081867 - in /cassandra/branches/cassandra-0.7: ./ src/java/org/apache/cassandra/hadoop/

Author: brandonwilliams
Date: Tue Mar 15 17:22:45 2011
New Revision: 1081867

URL: http://svn.apache.org/viewvc?rev=1081867view=rev
Log:
Allow Hadoop jobs to set the consistency level.
Patch by Eldon Stegall, reviewed by brandonwilliams for CASSANDRA-2331

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ConfigHelper.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081867r1=1081866r2=1081867view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 17:22:45 2011
@@ -3,6 +3,7 @@
  * fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
  * validate index names (CASSANDRA-1761)
+ * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
 
 
 0.7.4

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java?rev=1081867r1=1081866r2=1081867view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
 Tue Mar 15 17:22:45 2011
@@ -57,6 +57,7 @@ public class ColumnFamilyRecordReader ex
 private String keyspace;
 private TSocket socket;
 private Cassandra.Client client;
+private ConsistencyLevel consistencyLevel;
 
 public void close() 
 {
@@ -92,6 +93,9 @@ public class ColumnFamilyRecordReader ex
 totalRowCount = ConfigHelper.getInputSplitSize(conf);
 batchRowCount = ConfigHelper.getRangeBatchSize(conf);
 cfName = ConfigHelper.getInputColumnFamily(conf);
+consistencyLevel = 
ConsistencyLevel.valueOf(ConfigHelper.getReadConsistencyLevel(conf));
+
+
 keyspace = ConfigHelper.getInputKeyspace(conf);
 
 try
@@ -238,7 +242,7 @@ public class ColumnFamilyRecordReader ex
 rows = client.get_range_slices(new ColumnParent(cfName),
predicate,
keyRange,
-   ConsistencyLevel.ONE);
+   consistencyLevel);
   
 // nothing new? reached the end
 if (rows.isEmpty())

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java?rev=1081867r1=1081866r2=1081867view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
 Tue Mar 15 17:22:45 2011
@@ -78,6 +78,9 @@ implements org.apache.hadoop.mapred.Reco
 // handles for clients for each range running in the threadpool
 private final MapRange,RangeClient clients;
 private final long batchThreshold;
+
+private final ConsistencyLevel consistencyLevel;
+
 
 /**
  * Upon construction, obtain the map that this writer will use to collect
@@ -101,6 +104,7 @@ implements org.apache.hadoop.mapred.Reco
 this.queueSize = conf.getInt(ColumnFamilyOutputFormat.QUEUE_SIZE, 32 * 
Runtime.getRuntime().availableProcessors());
 this.clients = new HashMapRange,RangeClient();
 batchThreshold = 
conf.getLong(ColumnFamilyOutputFormat.BATCH_THRESHOLD, 32);
+consistencyLevel = 
ConsistencyLevel.valueOf(ConfigHelper.getWriteConsistencyLevel(conf));
 }
 
 /**
@@ -347,7 +351,7 @@ implements org.apache.hadoop.mapred.Reco
 // send the mutation to the last-used endpoint.  first 
time through, this will NPE harmlessly.
 try
 {
-thriftClient.batch_mutate(batch, ConsistencyLevel.ONE);
+thriftClient.batch_mutate(batch, consistencyLevel);
 break;
 }
 catch

[jira] Updated: (CASSANDRA-2331) Allow a Job Configuration to control the consistency level with which reads and writes occur for mapreduce jobs.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2331:


Affects Version/s: 0.7.0

 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs.
 

 Key: CASSANDRA-2331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2331
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.7.0
Reporter: Eldon Stegall
Priority: Minor
 Fix For: 0.7.5

 Attachments: consistencylevelpatch.txt


 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs. This would be useful if we were doing 
 statistical rollups and wanted to fail if consistency isn't met (W + R = N  
 as described by http://wiki.apache.org/cassandra/ArchitectureOverview). An 
 example of a situation where this might happen would be if a set of cassandra 
 nodes became partitioned from the other nodes, but were still accepting some 
 writes with ConsistencyLevel 1/2 (in the case that writes were local to the 
 partition). Then statistical rollup jobs might succeed without those accepted 
 writes being included, and this would not be desirable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Issue Comment Edited: (CASSANDRA-2330) Queue indexes for flush before the parent


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007007#comment-13007007
 ] 

Brandon Williams edited comment on CASSANDRA-2330 at 3/15/11 5:26 PM:
--

Patch to concat indexes ahead of the parent.  stress.java blocks for 6s instead 
of 9, with an 8G heap.

  was (Author: brandon.williams):
Patch to concat indexes ahead of the parent.  stress.java blocks for 7s 
instead of 9, with an 8G heap.
  
 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2330) Queue indexes for flush before the parent


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007038#comment-13007038
 ] 

Jonathan Ellis commented on CASSANDRA-2330:
---

+1

 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081876 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/db/ColumnFamilyStore.java

Author: brandonwilliams
Date: Tue Mar 15 17:36:21 2011
New Revision: 1081876

URL: http://svn.apache.org/viewvc?rev=1081876view=rev
Log:
Queue secondary indexes for flush before the parent.
Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-2330

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081876r1=1081875r2=1081876view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 17:36:21 2011
@@ -4,6 +4,7 @@
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
  * validate index names (CASSANDRA-1761)
  * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
+ * queue secondary indexes for flush before the parent (CASSANDRA-2330)
 
 
 0.7.4

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1081876r1=1081875r2=1081876view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 Tue Mar 15 17:36:21 2011
@@ -2199,6 +2199,6 @@ public class ColumnFamilyStore implement
 
 public IterableColumnFamilyStore concatWithIndexes()
 {
-return Iterables.concat(Collections.singleton(this), 
indexedColumns.values());
+return Iterables.concat(indexedColumns.values(), 
Collections.singleton(this));
 }
 }

[jira] Resolved: (CASSANDRA-2330) Queue indexes for flush before the parent


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-2330.
-

Resolution: Fixed

Committed

 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2332) Insert to cluster from JSON Export

2011-03-15 Thread Paul Querna (JIRA)

Insert to cluster from JSON Export
--

 Key: CASSANDRA-2332
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2332
 Project: Cassandra
  Issue Type: Improvement
Reporter: Paul Querna


json2sstable isn't as useful if you are trying to take an exporterd sstable and 
import it into a different cluster of a different size.

This tool would take the JSON serialized SSTable and insert it into a live 
cluster.

It could use thrift or be a fat client; For a user the difference doesn't 
matter much.

The alternative that dritfx suggested is to copy these sstables to every 
machine in the new cluster, and then do a rolling cleanup so every node only 
has the data it wants, but this seems very inefficient compared to just 
re-inserting it from the dump.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2333) Clean up thread pool and queue sizes

Clean up thread pool and queue sizes


 Key: CASSANDRA-2333
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Priority: Minor


Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting 
with Core threads, adding threads up to Max as tasks arrive, then queuing any 
additional.  This is not correct:

{noformat}
If fewer than corePoolSize threads are running, the Executor always prefers 
adding a new thread rather than queuing.
If corePoolSize or more threads are running, the Executor always prefers 
queuing a request rather than adding a new thread.
If a request cannot be queued, a new thread is created unless this would 
exceed maximumPoolSize, in which case, the task will be rejected.
{noformat}

CASSANDRA-2178 fixed this in one place but made it worse by default since most 
people run with a single data dir, meaning as soon as you have multiple CFs 
flushing (or a single one with indexes) then you will start blocking writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2327) Table.flusherLock is static final.. remove static.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007059#comment-13007059
 ] 

Jonathan Ellis commented on CASSANDRA-2327:
---

also see CASSANDRA-2333.

 Table.flusherLock is static final.. remove static.
 --

 Key: CASSANDRA-2327
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2327
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4, 0.7.5
 Environment: JVM
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Fix For: 0.7.5


 I see read and write latency spike when the system tables are 
 flushing(according to opscentral) 
 Only reason which i can come-up with is probably because of the 
 Table.flusherLock is static final... i think it should not be static because 
 the flush is per keyspace and this lock will lock all the read operations 
 because one table is going through a flush.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2178) Memtable Flush writers doesn't actually flush in parallel


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007058#comment-13007058
 ] 

Jonathan Ellis commented on CASSANDRA-2178:
---

see CASSANDRA-2333 for related work.

 Memtable Flush writers doesn't actually flush in parallel
 -

 Key: CASSANDRA-2178
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2178
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.2, 0.8
Reporter: Chris Goffinet
Assignee: Chris Goffinet
Priority: Minor
 Fix For: 0.7.3, 0.8

 Attachments: 
 0001-Set-the-core-min-pool-size-to-memtable_flush_writers-v2.patch, 
 0001-Set-the-core-min-pool-size-to-memtable_flush_writers.patch


 The flushWriter JMXEnabledThreadPoolExecutor sets the core pool min to 1, and 
 sets the LBQ to DatabaseDescriptor.getFlushWriters(). Increasing 
 memtable_flush_writers should allow us to flush more in parallel. The pool 
 will not grow until LBQ fills up to DatabaseDescriptor.getFlushWriters(). 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2332) Insert to cluster from JSON Export


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2332:
--

Component/s: Tools
   Priority: Minor  (was: Major)

 Insert to cluster from JSON Export
 --

 Key: CASSANDRA-2332
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2332
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Paul Querna
Priority: Minor

 json2sstable isn't as useful if you are trying to take an exporterd sstable 
 and import it into a different cluster of a different size.
 This tool would take the JSON serialized SSTable and insert it into a live 
 cluster.
 It could use thrift or be a fat client; For a user the difference doesn't 
 matter much.
 The alternative that dritfx suggested is to copy these sstables to every 
 machine in the new cluster, and then do a rolling cleanup so every node only 
 has the data it wants, but this seems very inefficient compared to just 
 re-inserting it from the dump.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2327) Table.flusherLock is static final.. remove static.

2011-03-15 Thread Peter Schuller (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007062#comment-13007062
 ] 

Peter Schuller commented on CASSANDRA-2327:
---

Without checking in detail to what extent these may overlap (for time 
constraints), CASSANDRA-1955 and CASSANDRA-1991 might be relevant here.


 Table.flusherLock is static final.. remove static.
 --

 Key: CASSANDRA-2327
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2327
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4, 0.7.5
 Environment: JVM
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Fix For: 0.7.5


 I see read and write latency spike when the system tables are 
 flushing(according to opscentral) 
 Only reason which i can come-up with is probably because of the 
 Table.flusherLock is static final... i think it should not be static because 
 the flush is per keyspace and this lock will lock all the read operations 
 because one table is going through a flush.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2333) Clean up thread pool and queue sizes

[
https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-2333:
--

Attachment: 2333.txt

Patch to remove use of maximumpoolsize in favor of core size + timing out core
threads. Also adds memtable_flush_queue_size configuration directive,
defaulting to 4.

Clean up thread pool and queue sizes

Key: CASSANDRA-2333
URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
Project: Cassandra
Issue Type: Bug
Reporter: Jonathan Ellis
Priority: Minor
Fix For: 0.7.5

Attachments: 2333.txt

Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting
with Core threads, adding threads up to Max as tasks arrive, then queuing any
additional. This is not correct:
{noformat}
If fewer than corePoolSize threads are running, the Executor always
prefers adding a new thread rather than queuing.
If corePoolSize or more threads are running, the Executor always prefers
queuing a request rather than adding a new thread.
If a request cannot be queued, a new thread is created unless this would
exceed maximumPoolSize, in which case, the task will be rejected.
{noformat}
CASSANDRA-2178 fixed this in one place but made it worse by default since
most people run with a single data dir, meaning as soon as you have multiple
CFs flushing (or a single one with indexes) then you will start blocking
writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081899 - in /cassandra/trunk: ./ conf/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/hadoop/ src/java/or

Author: jbellis
Date: Tue Mar 15 18:42:11 2011
New Revision: 1081899

URL: http://svn.apache.org/viewvc?rev=1081899view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/conf/cassandra-topology.properties
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java

cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
cassandra/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
cassandra/trunk/src/java/org/apache/cassandra/tools/SSTableExport.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 18:42:11 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1071777,1076891
-/cassandra/branches/cassandra-0.7:1026516-1081840
+/cassandra/branches/cassandra-0.7:1026516-1081897
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1081899r1=1081898r2=1081899view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Tue Mar 15 18:42:11 2011
@@ -18,6 +18,8 @@
  * fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
  * validate index names (CASSANDRA-1761)
+ * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
+ * queue secondary indexes for flush before the parent (CASSANDRA-2330)
 
 
  .merge-right.r1081840

Modified: cassandra/trunk/conf/cassandra-topology.properties
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra-topology.properties?rev=1081899r1=1081898r2=1081899view=diff
==
--- cassandra/trunk/conf/cassandra-topology.properties (original)
+++ cassandra/trunk/conf/cassandra-topology.properties Tue Mar 15 18:42:11 2011
@@ -35,3 +35,7 @@
 # default for unknown nodes
 default=DC1:r1
 
+# Native IPv6 is supported, however you must escape the colon in the IPv6 
Address
+# Also be sure to comment out JVM_OPTS=$JVM_OPTS 
-Djava.net.preferIPv4Stack=true
+# in cassandra-env.sh
+fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 18:42:11 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1081840
+/cassandra/branches/cassandra-0.7/contrib:1026516-1081897
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 18:42:11 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1071777,1076891
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1081840
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1081897
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange:

[jira] Commented: (CASSANDRA-2333) Clean up thread pool and queue sizes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007102#comment-13007102
 ] 

Brandon Williams commented on CASSANDRA-2333:
-

+1, increased flush queue size is enough to prevent blocking writes at all with 
stress and an index.

 Clean up thread pool and queue sizes
 

 Key: CASSANDRA-2333
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2333.txt


 Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting 
 with Core threads, adding threads up to Max as tasks arrive, then queuing any 
 additional.  This is not correct:
 {noformat}
 If fewer than corePoolSize threads are running, the Executor always 
 prefers adding a new thread rather than queuing.
 If corePoolSize or more threads are running, the Executor always prefers 
 queuing a request rather than adding a new thread.
 If a request cannot be queued, a new thread is created unless this would 
 exceed maximumPoolSize, in which case, the task will be rejected.
 {noformat}
 CASSANDRA-2178 fixed this in one place but made it worse by default since 
 most people run with a single data dir, meaning as soon as you have multiple 
 CFs flushing (or a single one with indexes) then you will start blocking 
 writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081908 - in /cassandra/branches/cassandra-0.7: ./ conf/ src/java/org/apache/cassandra/concurrent/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/ test/unit/org/ap

Author: jbellis
Date: Tue Mar 15 19:08:11 2011
New Revision: 1081908

URL: http://svn.apache.org/viewvc?rev=1081908view=rev
Log:
add memtable_flush_queue_size defaulting to 4
patch by jbellis; reviewed by brandonwilliams for CASSANDRA-2333

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/conf/cassandra.yaml

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/JMXConfigurableThreadPoolExecutor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/JMXConfigurableThreadPoolExecutorMBean.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/StageManager.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/Config.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutorTest.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081908r1=1081907r2=1081908view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 19:08:11 2011
@@ -3,6 +3,7 @@
  * fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
  * validate index names (CASSANDRA-1761)
+ * add memtable_flush_queue_size defaulting to 4 (CASSANDRA-2333)
  * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
  * queue secondary indexes for flush before the parent (CASSANDRA-2330)
 

Modified: cassandra/branches/cassandra-0.7/conf/cassandra.yaml
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/conf/cassandra.yaml?rev=1081908r1=1081907r2=1081908view=diff
==
--- cassandra/branches/cassandra-0.7/conf/cassandra.yaml (original)
+++ cassandra/branches/cassandra-0.7/conf/cassandra.yaml Tue Mar 15 19:08:11 
2011
@@ -143,6 +143,11 @@ concurrent_writes: 32
 # By default this will be set to the amount of data directories defined.
 #memtable_flush_writers: 1
 
+# the number of full memtables to allow pending flush, that is,
+# waiting for a writer thread.  At a minimum, this should be set to
+# the maximum number of secondary indexes created on a single CF.
+memtable_flush_queue_size: 4
+
 # Buffer size to use when performing contiguous column slices. 
 # Increase this to the size of the column slices you typically perform
 sliced_buffer_size_in_kb: 64

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java?rev=1081908r1=1081907r2=1081908view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
 Tue Mar 15 19:08:11 2011
@@ -26,53 +26,63 @@ import java.util.concurrent.*;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+/**
+ * This class encorporates some Executor best practices for Cassandra.  Most 
of the executors in the system
+ * should use or extend this.  There are two main improvements over a vanilla 
TPE:
+ *
+ * - If a task throws an exception, the default uncaught exception handler 
will be invoked; if there is
+ *   no such handler, the exception will be logged.
+ * - MaximumPoolSize is not supported.  Here is what that means (quoting TPE 
javadoc):
+ *
+ * If fewer than corePoolSize threads are running, the Executor always 
prefers adding a new thread rather than queuing.
+ * If corePoolSize or more threads are running, the Executor always 
prefers queuing a request rather than adding a new thread.
+ * If a request cannot be queued, a new thread is created unless this 
would exceed maximumPoolSize, in which case, the task will be rejected.
+ *
+ *   We don't want this last stage of creating new threads if the queue is 
full; it makes it needlessly difficult to
+ *   reason about the system's behavior.  In other words, if DebuggableTPE has 
allocated our maximum number of (core)
+ *   threads and the queue

[jira] Commented: (CASSANDRA-2013) Add CL.TWO, CL.THREE; tweak CL documentation

2011-03-15 Thread Peter Schuller (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007119#comment-13007119
]

Peter Schuller commented on CASSANDRA-2013:
---

Sounds good to me.

Add CL.TWO, CL.THREE; tweak CL documentation

Key: CASSANDRA-2013
URL: https://issues.apache.org/jira/browse/CASSANDRA-2013
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
Fix For: 0.7.4

Attachments: 2013-assert.txt, 2013.txt

Attaching draft patch to add CL.TWO and CL.THREE.
Motivation for adding is that having to select between either ONE or QUORUM
is too narrow a choice for clusters with RF 3. In such a case, it makes
particular sense to want to do writes at e.g. CL.TWO for durability purposes
even though you are not looking to get strong consistency with QUORUM.
CL.THREE is the same argument. TWO and THREE felt reasonable; there is no
objective reason why stopping at THREE is the obvious choice.
Technically one would want to specify an arbitrary number, but that is a much
more significant change.
Two open questions:
(1) I adjusted the documentation of ConsistencyLevel to be more consistent
and also to reflect what I believe to be reality (for example, as far as I
can tell QUORUM doesn't send requests to all nodes as claimed in the .thrift
file). I'm not terribly confident that I have not missed something though.
(2) There is at least one unresolved issue, which is this assertion check
WriteResponseHandler:
assert 1 = blockFor blockFor = 2 *
Table.open(table).getReplicationStrategy().getReplicationFactor()
: String.format(invalid response count %d for replication factor
%d,
blockFor,
Table.open(table).getReplicationStrategy().getReplicationFactor());
At THREE, this causes an assertion failure on keyspace with RF=1. I would, as
a user, expect UnavailableException. However I am uncertain as to what to do
about this assertion. I think this highlights one TWO/THREE are different
from previously existing CL:s, in that they essentially hard-code replicate
counts rather than expressing them in terms that can by definition be served
by the cluster at any RF.
Given that with THREE (and not TWO, but that is only due to the
implementation detail that bootstrapping is involved) implies a replicate
count that is independent of the replication factor, there is essentially a
new failure mode. It is suddenly possible for a consistency level to be
fundamentally incompatible with the RF. My gut reaction is to want
UnavailableException still, and that the assertion check can essentially be
removed (other than the = 1 part).
If a different failure mode is desired, presumably it would not be an
assertion failure (which should indicate a Cassandra bug). Maybe
UnstisfiableConsistencyLevel? I propose just adjusting the assertion (which
has no equivalent in ReadCallback btw); giving a friendlier error message in
case of a CL/RF mismatch would be good, but doesn't feel worth introducing
extra complexity to deal with it.
'ant test' passes. I have tested w/ py_stress with a three-node cluster and
an RF=3 keyspace and with 1 and 2 nodes down, and get expected behavior
(available or unavailable as a function of nodes that are up).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2334) validateSchemaAgreement() should raise something besides InvalidRequestException

2011-03-15 Thread Eric Evans (JIRA)

validateSchemaAgreement() should raise something besides InvalidRequestException


 Key: CASSANDRA-2334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2334
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor


{{ThriftValidation.validateSchemaAgreement()}} throws 
{{InvalidRequestException}} when schemas disagree.  Since this is propagated to 
the client, a more specific exception would relay more context to clients.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2319) Promote row index

2011-03-15 Thread Stu Hood (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007128#comment-13007128
]

Stu Hood commented on CASSANDRA-2319:
-

I think this would also mean we can remove the back-seek from sstable writing.
In which case I am a huge fan in principle.
I suspect that this is more challenging: the back seek also writes the row
length. We'd need to move to a blocked design like the one on 674 to replace
the row length with a block length in the data file.

This will remove as a consequence the row bloom filter
Not necessarily, but I think this ticket does highlight the fact that the
column bloom filter is ill-positioned: they can prevent the 3rd seek, but only
for names queries which (I suspect) are less likely on wide rows. Nonetheless,
I can imagine wanting to do a point query for a secondary index to determine
whether a particular row matches the index, so we should probably consider
promoting it as well.

There will be new trade-offs: either you index 'often' or the index becomes
potentially much bigger
The important thing to remember is that the distinction between columns and
keys should be very fuzzy: columns are a suffix on keys, and treating them
otherwise leads to complications. In this case, we shouldn't be holding every
128th key in memory, but instead every 128th-512th _tuple_: that way wide
rows are handled naturally.

This also normalizes our indexing: the size of your index depends on the
_total_ number of columns, instead of on the width and number of your rows.

Promote row index
-

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081923 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/concurrent/ src/java/org/apache/cassandra/service/

Author: jbellis
Date: Tue Mar 15 20:08:42 2011
New Revision: 1081923

URL: http://svn.apache.org/viewvc?rev=1081923view=rev
Log:
merge from 0.6

Modified:
cassandra/branches/cassandra-0.7/   (props changed)
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java

Propchange: cassandra/branches/cassandra-0.7/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:08:42 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6:922689-1071777,1076891
+/cassandra/branches/cassandra-0.6:922689-1081914
 /cassandra/branches/cassandra-0.7:1026516,1035666,1050269
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081923r1=1081922r2=1081923view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 20:08:42 2011
@@ -6,6 +6,7 @@
  * add memtable_flush_queue_size defaulting to 4 (CASSANDRA-2333)
  * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
  * queue secondary indexes for flush before the parent (CASSANDRA-2330)
+ * shut down server for OOM on a Thrift thread (CASSANDRA-2269)
 
 
 0.7.4

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:08:42 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1071777,1076891
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1081914
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:08:42 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1071777,1076891
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1081914
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689

Propchange: 
cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:08:42 2011
@@ -1,4 +1,4 @@
-/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1071777,1076891
+/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1081914
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516,1035666,1050269
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654

svn commit: r1081924 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java

Author: jbellis
Date: Tue Mar 15 20:09:25 2011
New Revision: 1081924

URL: http://svn.apache.org/viewvc?rev=1081924view=rev
Log:
r/m obsolete assert
patch by jbellis; reviewed by Peter Schuller for CASSANDRA-2013

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java?rev=1081924r1=1081923r2=1081924view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/WriteResponseHandler.java
 Tue Mar 15 20:09:25 2011
@@ -74,36 +74,23 @@ public class WriteResponseHandler extend
 
 protected int determineBlockFor(String table)
 {
-int blockFor = 0;
 switch (consistencyLevel)
 {
 case ONE:
-blockFor = 1;
-break;
+return 1;
 case ANY:
-blockFor = 1;
-break;
+return 1;
 case TWO:
-blockFor = 2;
-break;
+return 2;
 case THREE:
-blockFor = 3;
-break;
+return 3;
 case QUORUM:
-blockFor = (writeEndpoints.size() / 2) + 1;
-break;
+return (writeEndpoints.size() / 2) + 1;
 case ALL:
-blockFor = writeEndpoints.size();
-break;
+return writeEndpoints.size();
 default:
 throw new UnsupportedOperationException(invalid consistency 
level:  + consistencyLevel.toString());
 }
-// at most one node per range can bootstrap at a time, and these will 
be added to the write until
-// bootstrap finishes (at which point we no longer need to write to 
the old ones).
-assert 1 = blockFor  blockFor = 2 * 
Table.open(table).getReplicationStrategy().getReplicationFactor()
-: String.format(invalid response count %d for replication factor 
%d,
-blockFor, 
Table.open(table).getReplicationStrategy().getReplicationFactor());
-return blockFor;
 }
 
 public void assureSufficientLiveNodes() throws UnavailableException

[jira] Commented: (CASSANDRA-2013) Add CL.TWO, CL.THREE; tweak CL documentation

[
https://issues.apache.org/jira/browse/CASSANDRA-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007135#comment-13007135
]

Jonathan Ellis commented on CASSANDRA-2013:
---

committed

Add CL.TWO, CL.THREE; tweak CL documentation

Attachments: 2013-assert.txt, 2013.txt

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081925 - in /cassandra/trunk: ./ conf/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/concurrent/ src/java/org/apache/cassandra/config/ src

Author: jbellis
Date: Tue Mar 15 20:12:03 2011
New Revision: 1081925

URL: http://svn.apache.org/viewvc?rev=1081925view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/conf/cassandra.yaml
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/trunk/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java

cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXConfigurableThreadPoolExecutor.java

cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXConfigurableThreadPoolExecutorMBean.java

cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
cassandra/trunk/src/java/org/apache/cassandra/concurrent/StageManager.java
cassandra/trunk/src/java/org/apache/cassandra/config/Config.java
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java

cassandra/trunk/src/java/org/apache/cassandra/service/WriteResponseHandler.java

cassandra/trunk/test/unit/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutorTest.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:12:03 2011
@@ -1,5 +1,5 @@
-/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1071777,1076891
-/cassandra/branches/cassandra-0.7:1026516-1081897
+/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914
+/cassandra/branches/cassandra-0.7:1026516-1081924
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1081925r1=1081924r2=1081925view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Tue Mar 15 20:12:03 2011
@@ -18,8 +18,10 @@
  * fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
  * clear Built flag in system table when dropping an index (CASSANDRA-2320)
  * validate index names (CASSANDRA-1761)
+ * add memtable_flush_queue_size defaulting to 4 (CASSANDRA-2333)
  * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
  * queue secondary indexes for flush before the parent (CASSANDRA-2330)
+ * shut down server for OOM on a Thrift thread (CASSANDRA-2269)
 
 
  .merge-right.r1081840

Modified: cassandra/trunk/conf/cassandra.yaml
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1081925r1=1081924r2=1081925view=diff
==
--- cassandra/trunk/conf/cassandra.yaml (original)
+++ cassandra/trunk/conf/cassandra.yaml Tue Mar 15 20:12:03 2011
@@ -148,6 +148,11 @@ concurrent_writes: 32
 # By default this will be set to the amount of data directories defined.
 #memtable_flush_writers: 1
 
+# the number of full memtables to allow pending flush, that is,
+# waiting for a writer thread.  At a minimum, this should be set to
+# the maximum number of secondary indexes created on a single CF.
+memtable_flush_queue_size: 4
+
 # Buffer size to use when performing contiguous column slices. 
 # Increase this to the size of the column slices you typically perform
 sliced_buffer_size_in_kb: 64

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Mar 15 20:12:03 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1081897
+/cassandra/branches/cassandra-0.7/contrib:1026516-1081924
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java

[jira] Created: (CASSANDRA-2335) Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED

Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED
--

 Key: CASSANDRA-2335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2335
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.4
 Environment: Windows
Reporter: Benjamin Coverston
Assignee: Gary Dusbabek
Priority: Minor




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081929 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/db/ColumnFamilyStore.java src/java/org/apache/cassandra/db/Memtable.java

Author: jbellis
Date: Tue Mar 15 20:27:29 2011
New Revision: 1081929

URL: http://svn.apache.org/viewvc?rev=1081929view=rev
Log:
reduce contention on Table.flusherLock
patch by Stu Hood and jbellis; reviewed by slebresne for CASSANDRA-1954

Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java

Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1081929r1=1081928r2=1081929view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Mar 15 20:27:29 2011
@@ -7,6 +7,7 @@
  * allow job configuration to set the CL used in Hadoop jobs (CASSANDRA-2331)
  * queue secondary indexes for flush before the parent (CASSANDRA-2330)
  * shut down server for OOM on a Thrift thread (CASSANDRA-2269)
+ * reduce contention on Table.flusherLock (CASSANDRA-1954)
 
 
 0.7.4

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1081929r1=1081928r2=1081929view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 Tue Mar 15 20:27:29 2011
@@ -678,6 +678,12 @@ public class ColumnFamilyStore implement
 /** flush the given memtable and swap in a new one for its CFS, if it 
hasn't been frozen already.  threadsafe. */
 Future? maybeSwitchMemtable(Memtable oldMemtable, final boolean 
writeCommitLog)
 {
+if (oldMemtable.isFrozen())
+{
+logger.debug(memtable is already frozen; another thread must be 
flushing it);
+return null;
+}
+
 /*
  * If we can get the writelock, that means no new updates can come in 
and
  * all ongoing updates to memtables have completed. We can get the tail
@@ -729,6 +735,10 @@ public class ColumnFamilyStore implement
 if (!icc.contains(this))
 memtable = new Memtable(this);
 
+if (memtableSwitchCount == Integer.MAX_VALUE)
+memtableSwitchCount = 0;
+memtableSwitchCount++;
+
 // when all the memtables have been written, including for 
indexes, mark the flush in the commitlog header.
 // a second executor makes sure the onMemtableFlushes get called 
in the right order,
 // while keeping the wait-for-flush (future.get) out of anything 
latency-sensitive.
@@ -749,11 +759,6 @@ public class ColumnFamilyStore implement
 finally
 {
 Table.flusherLock.writeLock().unlock();
-if (memtableSwitchCount == Integer.MAX_VALUE)
-{
-memtableSwitchCount = 0;
-}
-memtableSwitchCount++;
 }
 }
 

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java?rev=1081929r1=1081928r2=1081929view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java 
(original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java 
Tue Mar 15 20:27:29 2011
@@ -52,7 +52,7 @@ public class Memtable implements Compara
 {
 private static final Logger logger = 
LoggerFactory.getLogger(Memtable.class);
 
-private boolean isFrozen;
+private volatile boolean isFrozen;
 
 private final AtomicLong currentThroughput = new AtomicLong(0);
 private final AtomicLong currentOperations = new AtomicLong(0);

[jira] Resolved: (CASSANDRA-1954) Double-check or replace RRW memtable lock


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-1954.
---

Resolution: Fixed

committed

 Double-check or replace RRW memtable lock
 -

 Key: CASSANDRA-1954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 
 0001-Double-check-in-maybeSwitchMemtable-to-minimize-writeL.txt, 
 0001-Remove-flusherLock-readLock.patch, 1954-0.7-v2.txt, 1954-v2.txt, 
 1954_trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 {quote}...when a Memtable reaches its threshold, up to (all) N write threads 
 will often notice, and race to acquire the writeLock in order to freeze the 
 memtable. This means that we do way more writeLock acquisitions than we need 
 to...{quote}
 See CASSANDRA-1930 for backstory, but adding double checking inside a read 
 lock before trying to re-entrantly acquire the writelock would eliminate most 
 of these excess writelock acquisitions.
 Alternatively, we should explore removing locking from these structures 
 entirely, and replacing the writeLock acquisition with a per-memtable counter 
 of active threads.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2329) comment topology file about ipv6 configuration


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007156#comment-13007156
 ] 

Hudson commented on CASSANDRA-2329:
---

Integrated in Cassandra-0.7 #386 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])


 comment topology file about ipv6 configuration
 --

 Key: CASSANDRA-2329
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2329
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.3
 Environment: all
Reporter: Eric Tamme
Priority: Trivial
 Fix For: 0.7.5


 There is little documentation regarding ipv6 working with cassandra.  For the 
 cassandra propertyfilesnitch to work with ipv6, the ipv6 addresses must have 
 the colons escaped due to the parsing characteristics of 
 java.util.Properties.load().
 Adding this documentation to the cassandra-topology.properties file would be 
 nice so that people dont have to hunt through source code and java doc to 
 figure out how to configure topology.
 Here is a simple patch that will alert users to how to config topology for 
 ipv6 use.
 34a35,39
  # Native IPv6 is supported, however you must escape the : in the IPv6 
  Address
  # Also be sure to comment out JVM_OPTS=$JVM_OPTS 
  -Djava.net.preferIPv4Stack=true
  # in cassandra-env.sh
  fe80\:0\:0\:0\:202\:b3ff\:fe1e\:8329=DC1:RAC3
  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2330) Queue indexes for flush before the parent


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007157#comment-13007157
 ] 

Hudson commented on CASSANDRA-2330:
---

Integrated in Cassandra-0.7 #386 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])


 Queue indexes for flush before the parent
 -

 Key: CASSANDRA-2330
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2330
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.0
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2330.txt


 Secondary indexes flush when the parent does.  This has an unfortunate side 
 effect: a single CF flushing with a single secondary index fills the flush 
 queue and blocks further writes until the first one completes.  A simple but 
 naive optimization here would be to queue the indexes before the parent since 
 they are generally going to be smaller, and thus flush faster, reducing the 
 amount of time writes are blocked.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2331) Allow a Job Configuration to control the consistency level with which reads and writes occur for mapreduce jobs.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007159#comment-13007159
 ] 

Hudson commented on CASSANDRA-2331:
---

Integrated in Cassandra-0.7 #386 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])


 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs.
 

 Key: CASSANDRA-2331
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2331
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Affects Versions: 0.7.0
Reporter: Eldon Stegall
Priority: Minor
 Fix For: 0.7.5

 Attachments: consistencylevelpatch.txt


 Allow a Job Configuration to control the consistency level with which reads 
 and writes occur for mapreduce jobs. This would be useful if we were doing 
 statistical rollups and wanted to fail if consistency isn't met (W + R = N  
 as described by http://wiki.apache.org/cassandra/ArchitectureOverview). An 
 example of a situation where this might happen would be if a set of cassandra 
 nodes became partitioned from the other nodes, but were still accepting some 
 writes with ConsistencyLevel 1/2 (in the case that writes were local to the 
 partition). Then statistical rollup jobs might succeed without those accepted 
 writes being included, and this would not be desirable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2013) Add CL.TWO, CL.THREE; tweak CL documentation

[
https://issues.apache.org/jira/browse/CASSANDRA-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007158#comment-13007158
]

Hudson commented on CASSANDRA-2013:
---

Integrated in Cassandra-0.7 #386 (See
[https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])
r/m obsolete assert
patch by jbellis; reviewed by Peter Schuller for CASSANDRA-2013

Add CL.TWO, CL.THREE; tweak CL documentation

Attachments: 2013-assert.txt, 2013.txt

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2333) Clean up thread pool and queue sizes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007160#comment-13007160
 ] 

Hudson commented on CASSANDRA-2333:
---

Integrated in Cassandra-0.7 #386 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/386/])


 Clean up thread pool and queue sizes
 

 Key: CASSANDRA-2333
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2333
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 0.7.5

 Attachments: 2333.txt


 Most of Cassandra assumes that ThreadPoolExecutor handles tasks by starting 
 with Core threads, adding threads up to Max as tasks arrive, then queuing any 
 additional.  This is not correct:
 {noformat}
 If fewer than corePoolSize threads are running, the Executor always 
 prefers adding a new thread rather than queuing.
 If corePoolSize or more threads are running, the Executor always prefers 
 queuing a request rather than adding a new thread.
 If a request cannot be queued, a new thread is created unless this would 
 exceed maximumPoolSize, in which case, the task will be rejected.
 {noformat}
 CASSANDRA-2178 fixed this in one place but made it worse by default since 
 most people run with a single data dir, meaning as soon as you have multiple 
 CFs flushing (or a single one with indexes) then you will start blocking 
 writes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2335) Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007173#comment-13007173
 ] 

Benjamin Coverston commented on CASSANDRA-2335:
---

Including Stack Traces:
[junit] Testsuite: org.apache.cassandra.streaming.SerializationsTest
[junit] Tests run: 4, Failures: 3, Errors: 0, Time elapsed: 1.203 sec
[junit] 
[junit] Testcase: 
testPendingFileRead(org.apache.cassandra.streaming.SerializationsTest):   FAILED
[junit] Filename must include parent directory.
[junit] junit.framework.AssertionFailedError: Filename must include parent 
directory.
[junit] at 
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:116)
[junit] at 
org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.deserialize(PendingFile.java:126)
[junit] at 
org.apache.cassandra.streaming.SerializationsTest.testPendingFileRead(SerializationsTest.java:71)
[junit] 
[junit] 
[junit] Testcase: 
testStreamHeaderRead(org.apache.cassandra.streaming.SerializationsTest):  FAILED
[junit] Filename must include parent directory.
[junit] junit.framework.AssertionFailedError: Filename must include parent 
directory.
[junit] at 
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:116)
[junit] at 
org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.deserialize(PendingFile.java:126)
[junit] at 
org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.deserialize(StreamHeader.java:90)
[junit] at 
org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.deserialize(StreamHeader.java:72)
[junit] at 
org.apache.cassandra.streaming.SerializationsTest.testStreamHeaderRead(SerializationsTest.java:105)
[junit] 
[junit] 
[junit] Testcase: 
testStreamRequestMessageRead(org.apache.cassandra.streaming.SerializationsTest):
  FAILED
[junit] Filename must include parent directory.
[junit] junit.framework.AssertionFailedError: Filename must include parent 
directory.
[junit] at 
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:116)
[junit] at 
org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.deserialize(PendingFile.java:126)
[junit] at 
org.apache.cassandra.streaming.StreamRequestMessage$StreamRequestMessageSerializer.deserialize(StreamRequestMessage.java:153)
[junit] at 
org.apache.cassandra.streaming.StreamRequestMessage$StreamRequestMessageSerializer.deserialize(StreamRequestMessage.java:123)
[junit] at 
org.apache.cassandra.streaming.SerializationsTest.testStreamRequestMessageRead(SerializationsTest.java:170)
[junit] 

 Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED
 --

 Key: CASSANDRA-2335
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2335
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.4
 Environment: Windows
Reporter: Benjamin Coverston
Assignee: Gary Dusbabek
Priority: Minor



--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2336) Extract SSTable.Builder/IndexWriter

2011-03-15 Thread Stu Hood (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stu Hood updated CASSANDRA-2336:


Attachment: 0003-Move-statistics-writing-into-IndexWriter.txt
0002-Extract-Builder.txt
0001-Extract-IndexWriter.txt

Extracts {SSTableWriter.Builder = Rebuilder} and {SSTableWriter.IndexWriter = 
IndexWriter}. The IndexWriter name hasn't been accurate for a while since it 
writes all of the secondary (non-data) components of the sstable, but I can't 
think of a better name.

I've targeted this ticket to 0.8 because I'd like to try and have a patch ready 
for CASSANDRA-2319 in time (but definitely not 674).

 Extract SSTable.Builder/IndexWriter
 ---

 Key: CASSANDRA-2336
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2336
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Stu Hood
 Fix For: 0.8

 Attachments: 0001-Extract-IndexWriter.txt, 0002-Extract-Builder.txt, 
 0003-Move-statistics-writing-into-IndexWriter.txt


 The Builder and IndexWriter classes in SSTableWriter are static, and 
 independently useful. Additionally, we need the ability to subclass them for 
 CASSANDRA-674 and CASSANDRA-2319.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2284) Make changes to the set of memtables and sstables of a cfstore atomic

2011-03-15 Thread Stu Hood (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007201#comment-13007201
]

Stu Hood commented on CASSANDRA-2284:
-

+1
This looks awesome! Just needs a rebase. Thanks a lot Sylvain.

Make changes to the set of memtables and sstables of a cfstore atomic
--

Key: CASSANDRA-2284
URL: https://issues.apache.org/jira/browse/CASSANDRA-2284
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 0.8

Attachments: 0001-Make-memtable-and-sstable-switches-atomic-v2.patch,
0001-Make-memtable-and-sstable-switches-atomic.patch

Original Estimate: 8h
Remaining Estimate: 8h

Some switches happen in the set of memtables and sstables of a cfstore:
* when we switch the memtable, the current memtable is replaced by a new
one and add to the memtablesPendingFlush
* when a memtable is fully flushed, it is removed from
memtablesPendingFlush and the newly created sstable is added to the set of
active sstables.
* after compaction, compacted memtables are removed from the active
sstables and the compacted sstable is added instead
Only the last of these operations is atomic. This ticket proposes to makes
all of them atomic, using the idea of the View idea hinted by Stu in the
comments of CASSANDRA-1954.
The main reason for this is to fix CASSANDRA-2105. But I think that another
benefit of this is to make reasoning about those operations easier.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081955 - /cassandra/trunk/test/system/test_cql.py

Author: eevans
Date: Tue Mar 15 21:43:44 2011
New Revision: 1081955

URL: http://svn.apache.org/viewvc?rev=1081955view=rev
Log:
fix broken test that just now fails, why?

Patch by eevans

Modified:
cassandra/trunk/test/system/test_cql.py

Modified: cassandra/trunk/test/system/test_cql.py
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/test/system/test_cql.py?rev=1081955r1=1081954r2=1081955view=diff
==
--- cassandra/trunk/test/system/test_cql.py (original)
+++ cassandra/trunk/test/system/test_cql.py Tue Mar 15 21:43:44 2011
@@ -148,7 +148,7 @@ class TestCql(ThriftTester):
 r = conn.execute(
 SELECT 4 FROM StandardLongA WHERE KEY  'ad' AND KEY  'ag';
 )
-assert len(r) == 3
+assert len(r) == 4
 assert r[0].key == ad
 assert r[1].key == ae
 assert r[2].key == af

[jira] Updated: (CASSANDRA-2334) validateSchemaAgreement() should raise something besides InvalidRequestException

2011-03-15 Thread Eric Evans (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans updated CASSANDRA-2334:
--

Attachment: v1-0006-update-Python-driver-for-SchemaDisagreementException.txt
v1-0005-handle-SchemaDisagreementException-in-cli.txt
v1-0004-update-JDBC-driver-for-SchemaDisagreementException.txt
v1-0003-throw-SchemaDisagreementException-from-DDL-methods.txt
v1-0002-thrift-generated-code.txt

v1-0001-CASSANDRA-2334-add-SchemaDisagreementException-RPC-exc.txt

 validateSchemaAgreement() should raise something besides 
 InvalidRequestException
 

 Key: CASSANDRA-2334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2334
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
 Attachments: 
 v1-0001-CASSANDRA-2334-add-SchemaDisagreementException-RPC-exc.txt, 
 v1-0002-thrift-generated-code.txt, 
 v1-0003-throw-SchemaDisagreementException-from-DDL-methods.txt, 
 v1-0004-update-JDBC-driver-for-SchemaDisagreementException.txt, 
 v1-0005-handle-SchemaDisagreementException-in-cli.txt, 
 v1-0006-update-Python-driver-for-SchemaDisagreementException.txt


 {{ThriftValidation.validateSchemaAgreement()}} throws 
 {{InvalidRequestException}} when schemas disagree.  Since this is propagated 
 to the client, a more specific exception would relay more context to clients.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2334) validateSchemaAgreement() should raise something besides InvalidRequestException


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007224#comment-13007224
 ] 

Brandon Williams commented on CASSANDRA-2334:
-

+1

 validateSchemaAgreement() should raise something besides 
 InvalidRequestException
 

 Key: CASSANDRA-2334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2334
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
 Attachments: 
 v1-0001-CASSANDRA-2334-add-SchemaDisagreementException-RPC-exc.txt, 
 v1-0002-thrift-generated-code.txt, 
 v1-0003-throw-SchemaDisagreementException-from-DDL-methods.txt, 
 v1-0004-update-JDBC-driver-for-SchemaDisagreementException.txt, 
 v1-0005-handle-SchemaDisagreementException-in-cli.txt, 
 v1-0006-update-Python-driver-for-SchemaDisagreementException.txt


 {{ThriftValidation.validateSchemaAgreement()}} throws 
 {{InvalidRequestException}} when schemas disagree.  Since this is propagated 
 to the client, a more specific exception would relay more context to clients.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2022) Twisted driver for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2022:


Attachment: (was: txcql.txt)

 Twisted driver for CQL
 --

 Key: CASSANDRA-2022
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2022
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Eric Evans
Assignee: Brandon Williams
Priority: Minor

 In-tree CQL drivers should be reasonably consistent with one another 
 (wherever possible/practical), and implement a minimum of:
 • Query compression
 • Keyspace assignment on connection
 • Connection pooling / load-balancing
 The goal is not to supplant the idiomatic libraries, but to provide a 
 consistent, stable base for them to build upon.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2022) Twisted driver for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2022:


Attachment: txcql.txt

Update patch fixes validation_class, adds support for 
SchemaDisagreementException, allows a decoder to be passed to the conn pool, 
adds a setup.py and README.

 Twisted driver for CQL
 --

 Key: CASSANDRA-2022
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2022
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Eric Evans
Assignee: Brandon Williams
Priority: Minor
 Attachments: txcql.txt


 In-tree CQL drivers should be reasonably consistent with one another 
 (wherever possible/practical), and implement a minimum of:
 • Query compression
 • Keyspace assignment on connection
 • Connection pooling / load-balancing
 The goal is not to supplant the idiomatic libraries, but to provide a 
 consistent, stable base for them to build upon.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1081970 - /cassandra/trunk/interface/cassandra.thrift

Author: eevans
Date: Tue Mar 15 22:51:40 2011
New Revision: 1081970

URL: http://svn.apache.org/viewvc?rev=1081970view=rev
Log:
add SchemaDisagreementException RPC exception

Patch by eevans for CASSANDRA-2334; reviewed by brandon.williams

Modified:
cassandra/trunk/interface/cassandra.thrift

Modified: cassandra/trunk/interface/cassandra.thrift
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/interface/cassandra.thrift?rev=1081970r1=1081969r2=1081970view=diff
==
--- cassandra/trunk/interface/cassandra.thrift (original)
+++ cassandra/trunk/interface/cassandra.thrift Tue Mar 15 22:51:40 2011
@@ -126,6 +126,10 @@ exception AuthorizationException {
 1: required string why
 }
 
+/** schemas are not in agreement across all nodes */
+exception SchemaDisagreementException {
+}
+
 
 #
 # service api
@@ -659,32 +663,35 @@ service Cassandra {
 
   /** adds a column family. returns the new schema id. */
   string system_add_column_family(1:required CfDef cf_def)
-throws (1:InvalidRequestException ire),
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde),
 
   /** drops a column family. returns the new schema id. */
   string system_drop_column_family(1:required string column_family)
-throws (1:InvalidRequestException ire), 
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde), 
   
   /** adds a keyspace and any column families that are part of it. returns the 
new schema id. */
   string system_add_keyspace(1:required KsDef ks_def)
-throws (1:InvalidRequestException ire),
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde),
   
   /** drops a keyspace and any column families that are part of it. returns 
the new schema id. */
   string system_drop_keyspace(1:required string keyspace)
-throws (1:InvalidRequestException ire),
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde),
   
   /** updates properties of a keyspace. returns the new schema id. */
   string system_update_keyspace(1:required KsDef ks_def)
-throws (1:InvalidRequestException ire),
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde),
 
   /** updates properties of a column family. returns the new schema id. */
   string system_update_column_family(1:required CfDef cf_def)
-throws (1:InvalidRequestException ire),
+throws (1:InvalidRequestException ire, 2:SchemaDisagreementException sde),
   
   /**
* Executes a CQL (Cassandra Query Language) statement and returns a
* CqlResult containing the results.
*/
   CqlResult execute_cql_query(1:required binary query, 2:required Compression 
compression)
-throws (1:InvalidRequestException ire, 2:UnavailableException ue, 
3:TimedOutException te)
+throws (1:InvalidRequestException ire,
+2:UnavailableException ue,
+3:TimedOutException te,
+4:SchemaDisagreementException sde)
 }

svn commit: r1081972 - in /cassandra/trunk/src/java/org/apache/cassandra: cql/QueryProcessor.java thrift/CassandraServer.java

Author: eevans
Date: Tue Mar 15 22:52:24 2011
New Revision: 1081972

URL: http://svn.apache.org/viewvc?rev=1081972view=rev
Log:
throw SchemaDisagreementException from DDL methods

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334

Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java

Modified: cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java?rev=1081972r1=1081971r2=1081972view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java Tue 
Mar 15 22:52:24 2011
@@ -425,17 +425,17 @@ public class QueryProcessor
 }
 
 // Copypasta from CassandraServer (where it is private).
-private static void validateSchemaAgreement() throws 
InvalidRequestException
+private static void validateSchemaAgreement() throws 
SchemaDisagreementException
 {
 // unreachable hosts don't count towards disagreement
 MapString, ListString versions = 
Maps.filterKeys(StorageProxy.describeSchemaVersions(),
  
Predicates.not(Predicates.equalTo(StorageProxy.UNREACHABLE)));
 if (versions.size()  1)
-throw new InvalidRequestException(Cluster schema does not yet 
agree);
+throw new SchemaDisagreementException();
 }
 
 public static CqlResult process(String queryString, ClientState 
clientState)
-throws RecognitionException, UnavailableException, 
InvalidRequestException, TimedOutException
+throws RecognitionException, UnavailableException, 
InvalidRequestException, TimedOutException, SchemaDisagreementException
 {
 logger.trace(CQL QUERY: {}, queryString);
 

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java?rev=1081972r1=1081971r2=1081972view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java 
Tue Mar 15 22:52:24 2011
@@ -703,7 +703,8 @@ public class CassandraServer implements 
 }
 }
 
-public synchronized String system_add_column_family(CfDef cf_def) throws 
InvalidRequestException, TException
+public synchronized String system_add_column_family(CfDef cf_def)
+throws InvalidRequestException, SchemaDisagreementException, TException
 {
 logger.debug(add_column_family);
 state().hasColumnFamilyListAccess(Permission.WRITE);
@@ -729,7 +730,8 @@ public class CassandraServer implements 
 }
 }
 
-public synchronized String system_drop_column_family(String column_family) 
throws InvalidRequestException, TException
+public synchronized String system_drop_column_family(String column_family)
+throws InvalidRequestException, SchemaDisagreementException, TException
 {
 logger.debug(drop_column_family);
 state().hasColumnFamilyListAccess(Permission.WRITE);
@@ -754,7 +756,8 @@ public class CassandraServer implements 
 }
 }
 
-public synchronized String system_add_keyspace(KsDef ks_def) throws 
InvalidRequestException, TException
+public synchronized String system_add_keyspace(KsDef ks_def)
+throws InvalidRequestException, SchemaDisagreementException, TException
 {
 logger.debug(add_keyspace);
 state().hasKeyspaceListAccess(Permission.WRITE);
@@ -800,7 +803,8 @@ public class CassandraServer implements 
 }
 }
 
-public synchronized String system_drop_keyspace(String keyspace) throws 
InvalidRequestException, TException
+public synchronized String system_drop_keyspace(String keyspace)
+throws InvalidRequestException, SchemaDisagreementException, TException
 {
 logger.debug(drop_keyspace);
 state().hasKeyspaceListAccess(Permission.WRITE);
@@ -825,8 +829,10 @@ public class CassandraServer implements 
 }
 }
 
-/** update an existing keyspace, but do not allow column family 
modifications. */
-public synchronized String system_update_keyspace(KsDef ks_def) throws 
InvalidRequestException, TException
+/** update an existing keyspace, but do not allow column family 
modifications. 
+ * @throws SchemaDisagreementException */
+public synchronized String system_update_keyspace(KsDef ks_def)
+throws InvalidRequestException, SchemaDisagreementException, TException
 {
 logger.debug(update_keyspace);

svn commit: r1081973 - in /cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc: CassandraConnection.java CassandraStatement.java Connection.java

Author: eevans
Date: Tue Mar 15 22:52:36 2011
New Revision: 1081973

URL: http://svn.apache.org/viewvc?rev=1081973view=rev
Log:
update JDBC driver for SchemaDisagreementException

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334

Modified:

cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java

cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java

cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/Connection.java

Modified: 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java?rev=1081973r1=1081972r2=1081973view=diff
==
--- 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java
 (original)
+++ 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraConnection.java
 Tue Mar 15 22:52:36 2011
@@ -41,6 +41,7 @@ import java.util.Properties;
 import org.apache.cassandra.thrift.AuthenticationException;
 import org.apache.cassandra.thrift.AuthorizationException;
 import org.apache.cassandra.thrift.InvalidRequestException;
+import org.apache.cassandra.thrift.SchemaDisagreementException;
 import org.apache.cassandra.thrift.TimedOutException;
 import org.apache.cassandra.thrift.UnavailableException;
 import org.apache.thrift.TException;
@@ -113,6 +114,10 @@ public class CassandraConnection impleme
 {
 throw new DriverResolverException(e.getMessage());
 }
+catch (SchemaDisagreementException e)
+{
+throw new DriverResolverException(schema does not match across 
nodes, (try again later).);
+}
 
 }
 

Modified: 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java?rev=1081973r1=1081972r2=1081973view=diff
==
--- 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java
 (original)
+++ 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/CassandraStatement.java
 Tue Mar 15 22:52:36 2011
@@ -47,6 +47,7 @@ import java.util.regex.Pattern;
 import org.apache.cassandra.thrift.CqlResult;
 import org.apache.cassandra.thrift.CqlResultType;
 import org.apache.cassandra.thrift.InvalidRequestException;
+import org.apache.cassandra.thrift.SchemaDisagreementException;
 import org.apache.cassandra.thrift.TimedOutException;
 import org.apache.cassandra.thrift.UnavailableException;
 import org.apache.thrift.TException;
@@ -182,6 +183,10 @@ class CassandraStatement implements Prep
 {
 throw new SQLException(e.getMessage());
 }
+catch (SchemaDisagreementException e)
+{
+throw new SQLException(schema does not match across nodes, (try 
again later).);
+}
 catch (TException e)
 {
 throw new SQLException(e.getMessage());
@@ -260,6 +265,10 @@ class CassandraStatement implements Prep
 {
 throw new SQLException(e.getMessage());
 }
+catch (SchemaDisagreementException e)
+{
+throw new SQLException(schema does not match across nodes, (try 
again later).);
+}
 catch (TException e)
 {
 throw new SQLException(e.getMessage());
@@ -294,6 +303,10 @@ class CassandraStatement implements Prep
 {
 throw new SQLException(e.getMessage());
 }
+catch (SchemaDisagreementException e)
+{
+throw new SQLException(schema does not match across nodes, (try 
again later).);
+}
 catch (TException e)
 {
 throw new SQLException(e.getMessage());

Modified: 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/Connection.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/Connection.java?rev=1081973r1=1081972r2=1081973view=diff
==
--- 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/Connection.java 
(original)
+++ 
cassandra/trunk/drivers/java/src/org/apache/cassandra/cql/jdbc/Connection.java 
Tue Mar 15 22:52:36 2011
@@ -33,6 +33,7 @@ import org.apache.cassandra.thrift.Cassa
 import org.apache.cassandra.thrift.Compression;
 import org.apache.cassandra.thrift.CqlResult;
 import org.apache.cassandra.thrift.InvalidRequestException;
+import org.apache.cassandra.thrift.SchemaDisagreementException;
 import org.apache.cassandra.thrift.TimedOutException;
 import org.apache.cassandra.thrift.UnavailableException;
 import

svn commit: r1081975 - /cassandra/trunk/drivers/py/cql/connection.py

Author: eevans
Date: Tue Mar 15 22:53:00 2011
New Revision: 1081975

URL: http://svn.apache.org/viewvc?rev=1081975view=rev
Log:
update Python driver for SchemaDisagreementException

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334

Modified:
cassandra/trunk/drivers/py/cql/connection.py

Modified: cassandra/trunk/drivers/py/cql/connection.py
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/py/cql/connection.py?rev=1081975r1=1081974r2=1081975view=diff
==
--- cassandra/trunk/drivers/py/cql/connection.py (original)
+++ cassandra/trunk/drivers/py/cql/connection.py Tue Mar 15 22:53:00 2011
@@ -28,7 +28,8 @@ import zlib, re
 try:
 from cassandra import Cassandra
 from cassandra.ttypes import Compression, InvalidRequestException, \
- CqlResultType, AuthenticationRequest
+ CqlResultType, AuthenticationRequest, \
+ SchemaDisagreementException
 except ImportError:
 # Hack to run from a source tree
 import sys
@@ -41,7 +42,8 @@ except ImportError:
  'gen-py'))
 from cassandra import Cassandra
 from cassandra.ttypes import Compression, InvalidRequestException, \
-  CqlResultType, AuthenticationRequest
+ CqlResultType, AuthenticationRequest, \
+ SchemaDisagreementException
 
 COMPRESSION_SCHEMES = ['GZIP']
 DEFAULT_COMPRESSION = 'GZIP'
@@ -156,6 +158,8 @@ class Connection(object):
  request_compression)
 except InvalidRequestException, ire:
 raise CQLException(Bad Request: %s % ire.why)
+except SchemaDisagreementException, sde:
+raise CQLException(schema versions disagree, (try again later).)
 except TApplicationException, tapp:
 raise CQLException(Internal application error)
 except Exception, exc:

[jira] Resolved: (CASSANDRA-2334) validateSchemaAgreement() should raise something besides InvalidRequestException

2011-03-15 Thread Eric Evans (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans resolved CASSANDRA-2334.
---

Resolution: Fixed

committed

 validateSchemaAgreement() should raise something besides 
 InvalidRequestException
 

 Key: CASSANDRA-2334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2334
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
 Attachments: 
 v1-0001-CASSANDRA-2334-add-SchemaDisagreementException-RPC-exc.txt, 
 v1-0002-thrift-generated-code.txt, 
 v1-0003-throw-SchemaDisagreementException-from-DDL-methods.txt, 
 v1-0004-update-JDBC-driver-for-SchemaDisagreementException.txt, 
 v1-0005-handle-SchemaDisagreementException-in-cli.txt, 
 v1-0006-update-Python-driver-for-SchemaDisagreementException.txt


 {{ThriftValidation.validateSchemaAgreement()}} throws 
 {{InvalidRequestException}} when schemas disagree.  Since this is propagated 
 to the client, a more specific exception would relay more context to clients.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2328) Index predicate values used in get_indexed_slice() are not validated

2011-03-15 Thread Aaron Morton (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Morton updated CASSANDRA-2328:


Attachment: 0001-validate-index-predicate-name-and-value.patch

Attached patch validates the expression column name and value for 
get_indexed_slice(). 

Also adds a regression test in the (thrift) system tests.

 Index predicate values used in get_indexed_slice() are not validated
 

 Key: CASSANDRA-2328
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2328
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.3
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Minor
 Fix For: 0.7.5, 0.8

 Attachments: 0001-validate-index-predicate-name-and-value.patch


 If a client makes a get_indexed_slice() request with malformed predicate 
 values we get an assertion failing rather than InvalidRequestException.
 {noformat}
 ERROR 14:47:56,842 Error in ThreadPoolExecutor
 java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: 6
 at 
 org.apache.cassandra.service.IndexScanVerbHandler.doVerb(IndexScanVer
 bHandler.java:51)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.
 java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
 utor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
 .java:908)
 at java.lang.Thread.run(Thread.java:619)
 Caused by: java.lang.IndexOutOfBoundsException: 6
 at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
 at 
 org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(Ti
 meUUIDType.java:56)
 at 
 org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.jav
 a:45)
 at 
 org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.jav
 a:29)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.satisfies(ColumnFamilyStore
 .java:1608)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.scan(ColumnFamilyStore.java
 :1552)
 at 
 org.apache.cassandra.service.IndexScanVerbHandler.doVerb(IndexScanVer
 bHandler.java:42)
 ... 4 more
 {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2334) validateSchemaAgreement() should raise something besides InvalidRequestException


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007299#comment-13007299
 ] 

Hudson commented on CASSANDRA-2334:
---

Integrated in Cassandra #781 (See 
[https://hudson.apache.org/hudson/job/Cassandra/781/])
update Python driver for SchemaDisagreementException

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334
handle SchemaDisagreementException in cli

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334
update JDBC driver for SchemaDisagreementException

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334
throw SchemaDisagreementException from DDL methods

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334
thrift generated code

Patch by eevans; reviewed by brandon.williams for CASSANDRA-2334
add SchemaDisagreementException RPC exception

Patch by eevans for CASSANDRA-2334; reviewed by brandon.williams


 validateSchemaAgreement() should raise something besides 
 InvalidRequestException
 

 Key: CASSANDRA-2334
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2334
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Affects Versions: 0.8
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
 Attachments: 
 v1-0001-CASSANDRA-2334-add-SchemaDisagreementException-RPC-exc.txt, 
 v1-0002-thrift-generated-code.txt, 
 v1-0003-throw-SchemaDisagreementException-from-DDL-methods.txt, 
 v1-0004-update-JDBC-driver-for-SchemaDisagreementException.txt, 
 v1-0005-handle-SchemaDisagreementException-in-cli.txt, 
 v1-0006-update-Python-driver-for-SchemaDisagreementException.txt


 {{ThriftValidation.validateSchemaAgreement()}} throws 
 {{InvalidRequestException}} when schemas disagree.  Since this is propagated 
 to the client, a more specific exception would relay more context to clients.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2258) service.SerializationsTest failes under cobertura


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007298#comment-13007298
 ] 

Hudson commented on CASSANDRA-2258:
---

Integrated in Cassandra #781 (See 
[https://hudson.apache.org/hudson/job/Cassandra/781/])


 service.SerializationsTest failes under cobertura
 -

 Key: CASSANDRA-2258
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2258
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Jonathan Ellis
Assignee: Gary Dusbabek
Priority: Minor
 Fix For: 0.7.5

 Attachments: v1-0001-do-not-instrument-the-Token-classes.txt, 
 v1-0001-fix-cobertura-brokenness-by-forcing-serialVersionUIDs.txt


 ant codecoverage -Dtest.name=SerializationsTest gives
 {noformat}
 [junit] Testcase: 
 testTreeResponseRead(org.apache.cassandra.service.SerializationsTest):  
 Caused an ERROR
 [junit] java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit] java.lang.RuntimeException: java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit]   at 
 org.apache.cassandra.service.AntiEntropyService$TreeResponseVerbHandler.deserialize(AntiEntropyService.java:634)
 [junit]   at 
 org.apache.cassandra.service.SerializationsTest.testTreeResponseRead(SerializationsTest.java:90)
 [junit] Caused by: java.io.InvalidClassException: 
 org.apache.cassandra.dht.BigIntegerToken; local class incompatible: stream 
 classdesc serialVersionUID = -5833589141319293006, local class 
 serialVersionUID = 2280189098581028124
 [junit]   at 
 java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:562)
 [junit]   at 
 java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
 [junit]   at 
 java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
 [junit]   at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
 [junit]   at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
 [junit]   at 
 java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
 [junit]   at 
 java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
 [junit]   at 
 org.apache.cassandra.service.AntiEntropyService$TreeResponseVerbHandler.deserialize(AntiEntropyService.java:630)
 {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1954) Double-check or replace RRW memtable lock

2011-03-15 Thread Vijay (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007310#comment-13007310
 ] 

Vijay commented on CASSANDRA-1954:
--

+1 but a invalid check which was not created by this patch though... assert 
memtable == oldMemtable; if it is really needed...

 Double-check or replace RRW memtable lock
 -

 Key: CASSANDRA-1954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 
 0001-Double-check-in-maybeSwitchMemtable-to-minimize-writeL.txt, 
 0001-Remove-flusherLock-readLock.patch, 1954-0.7-v2.txt, 1954-v2.txt, 
 1954_trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 {quote}...when a Memtable reaches its threshold, up to (all) N write threads 
 will often notice, and race to acquire the writeLock in order to freeze the 
 memtable. This means that we do way more writeLock acquisitions than we need 
 to...{quote}
 See CASSANDRA-1930 for backstory, but adding double checking inside a read 
 lock before trying to re-entrantly acquire the writelock would eliminate most 
 of these excess writelock acquisitions.
 Alternatively, we should explore removing locking from these structures 
 entirely, and replacing the writeLock acquisition with a per-memtable counter 
 of active threads.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (CASSANDRA-2337) Windows: CliTest broken because of /r/n

Windows: CliTest broken because of /r/n
---

 Key: CASSANDRA-2337
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2337
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.4
 Environment: Windows
Reporter: Benjamin Coverston


Somebody thought that windows should emulate a telex machine and we ended up 
with /r/n.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2337) Windows: CliTest broken because of /r/n


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Coverston reassigned CASSANDRA-2337:
-

Assignee: Benjamin Coverston

 Windows: CliTest broken because of /r/n
 ---

 Key: CASSANDRA-2337
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2337
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.4
 Environment: Windows
Reporter: Benjamin Coverston
Assignee: Benjamin Coverston
  Labels: windows

 Somebody thought that windows should emulate a telex machine and we ended up 
 with /r/n.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2337) Windows: CliTest broken because of /r/n


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Coverston updated CASSANDRA-2337:
--

Attachment: 2337.patch

 Windows: CliTest broken because of /r/n
 ---

 Key: CASSANDRA-2337
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2337
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.4
 Environment: Windows
Reporter: Benjamin Coverston
Assignee: Benjamin Coverston
  Labels: windows
 Fix For: 0.7.4

 Attachments: 2337.patch


 Somebody thought that windows should emulate a telex machine and we ended up 
 with /r/n.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-1954) Double-check or replace RRW memtable lock


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007320#comment-13007320
 ] 

Jonathan Ellis commented on CASSANDRA-1954:
---

are you saying that the assert is obviously correct and hence redundant, or the 
assert is incorrect and will throw exceptions under some conditions?

 Double-check or replace RRW memtable lock
 -

 Key: CASSANDRA-1954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 
 0001-Double-check-in-maybeSwitchMemtable-to-minimize-writeL.txt, 
 0001-Remove-flusherLock-readLock.patch, 1954-0.7-v2.txt, 1954-v2.txt, 
 1954_trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 {quote}...when a Memtable reaches its threshold, up to (all) N write threads 
 will often notice, and race to acquire the writeLock in order to freeze the 
 memtable. This means that we do way more writeLock acquisitions than we need 
 to...{quote}
 See CASSANDRA-1930 for backstory, but adding double checking inside a read 
 lock before trying to re-entrantly acquire the writelock would eliminate most 
 of these excess writelock acquisitions.
 Alternatively, we should explore removing locking from these structures 
 entirely, and replacing the writeLock acquisition with a per-memtable counter 
 of active threads.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2272) Refactor Key and Row caches handling code

2011-03-15 Thread Matthew F. Dennis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew F. Dennis updated CASSANDRA-2272:
-

Attachment: 2272_trunk_v4.txt

attached v4 patch:

1) prefers explicit [key|row] cache objects that extend AutoSavingCache in 
preference to branching on CacheType (e.g. if (cacheType == blah))

2) minor formatting changes in generic declarations

3) minor name changes

+1 otherwise

 Refactor Key and Row caches handling code 
 --

 Key: CASSANDRA-2272
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2272
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Refactor-key-and-row-cache-handling-code-v2.patch, 
 0001-Refactor-key-and-row-cache-handling-code-v3.patch, 
 0001-Refactor-key-and-row-cache-handling-code.patch, 2272_trunk_v4.txt

   Original Estimate: 4h
  Remaining Estimate: 4h

 The code related to key and row caches has a bunch of code duplication. It's
 moreover a bit scattered, the caches are in SSTableTracker but the scheduled
 task to save them are in CFStore. And having the caches in SSTableTracker is
 not very logic, for row cache at least, since it does not only concern the
 sstables. Proposed patch refactor this to avoid the duplication and put all
 the cache handling code in one place.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1082034 - in /cassandra/trunk: build.xml drivers/txpy/ drivers/txpy/README drivers/txpy/example.py drivers/txpy/setup.py drivers/txpy/txcql/ drivers/txpy/txcql/init.py drivers/txpy/tx

Author: brandonwilliams
Date: Wed Mar 16 03:05:21 2011
New Revision: 1082034

URL: http://svn.apache.org/viewvc?rev=1082034view=rev
Log:
Twisted driver for CQL
Patch by brandonwilliams, reviewed by eevans for CASSANDRA-2022

Added:
cassandra/trunk/drivers/txpy/
cassandra/trunk/drivers/txpy/README
cassandra/trunk/drivers/txpy/example.py   (with props)
cassandra/trunk/drivers/txpy/setup.py   (with props)
cassandra/trunk/drivers/txpy/txcql/
cassandra/trunk/drivers/txpy/txcql/__init__.py   (with props)
cassandra/trunk/drivers/txpy/txcql/connection.py   (with props)
cassandra/trunk/drivers/txpy/txcql/connection_pool.py   (with props)
Modified:
cassandra/trunk/build.xml

Modified: cassandra/trunk/build.xml
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/build.xml?rev=1082034r1=1082033r2=1082034view=diff
==
--- cassandra/trunk/build.xml (original)
+++ cassandra/trunk/build.xml Wed Mar 16 03:05:21 2011
@@ -337,6 +337,11 @@
 arg line=-o ${interface.thrift.dir} /
 arg line=cassandra.thrift /
   /exec
+  exec executable=thrift dir=${basedir}/interface failonerror=true
+arg line=--gen py:twisted /
+arg line=-o ${interface.thrift.dir} /
+arg line=cassandra.thrift /
+  /exec
 /target
 
 !-- create properties file with C version --

Added: cassandra/trunk/drivers/txpy/README
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/txpy/README?rev=1082034view=auto
==
--- cassandra/trunk/drivers/txpy/README (added)
+++ cassandra/trunk/drivers/txpy/README Wed Mar 16 03:05:21 2011
@@ -0,0 +1,2 @@
+Python (Twisted) language driver for CQL, a SQL-alike query language for Apache
+Cassandra.

Added: cassandra/trunk/drivers/txpy/example.py
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/txpy/example.py?rev=1082034view=auto
==
--- cassandra/trunk/drivers/txpy/example.py (added)
+++ cassandra/trunk/drivers/txpy/example.py Wed Mar 16 03:05:21 2011
@@ -0,0 +1,29 @@
+#!/usr/bin/python
+from txcql.connection_pool import ConnectionPool
+from twisted.internet import defer, reactor
+
+HOST = 'localhost'
+PORT = 9160
+KEYSPACE = 'txcql_test'
+CF = 'test'
+
+@defer.inlineCallbacks
+def testcql(client):
+yield client.execute(CREATE KEYSPACE ? with replication_factor=1 and 
strategy_class='SimpleStrategy', KEYSPACE)
+yield client.execute(CREATE COLUMNFAMILY ? with comparator=ascii and 
default_validation=ascii, CF)
+yield client.execute(UPDATE ? set foo = bar, bar = baz WHERE key = test, 
CF)
+res = yield client.execute(SELECT foo, bar from ? WHERE key = test, CF)
+for r in res:
+print r.__dict__
+yield client.execute(DROP KEYSPACE ?, KEYSPACE)
+reactor.stop() 
+
+if __name__ == '__main__':
+from twisted.python import log
+import sys
+log.startLogging(sys.stdout)
+
+pool = ConnectionPool(KEYSPACE)
+testcql(pool)
+reactor.connectTCP(HOST, PORT, pool)
+reactor.run()

Propchange: cassandra/trunk/drivers/txpy/example.py
--
svn:eol-style = native

Added: cassandra/trunk/drivers/txpy/setup.py
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/txpy/setup.py?rev=1082034view=auto
==
--- cassandra/trunk/drivers/txpy/setup.py (added)
+++ cassandra/trunk/drivers/txpy/setup.py Wed Mar 16 03:05:21 2011
@@ -0,0 +1,37 @@
+#!/usr/bin/python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# License); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from distutils.core import setup
+from os.path import abspath, join, dirname
+
+setup(
+name=txcql,
+version=1.0,
+description=Cassandra Query Language driver,
+long_description=open(abspath(join(dirname(__file__), 'README'))).read(),
+url=http://cassandra.apache.org;,
+packages=[txcql],
+requires=[thrift],
+provides=[txcql],
+classifiers=[
+Development Status :: 4 - Beta,
+License :: OSI Approved :: Apache Software License,
+

[jira] Resolved: (CASSANDRA-2022) Twisted driver for CQL


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-2022.
-

   Resolution: Fixed
Fix Version/s: 0.8

Committed

 Twisted driver for CQL
 --

 Key: CASSANDRA-2022
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2022
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Eric Evans
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.8

 Attachments: txcql.txt


 In-tree CQL drivers should be reasonably consistent with one another 
 (wherever possible/practical), and implement a minimum of:
 • Query compression
 • Keyspace assignment on connection
 • Connection pooling / load-balancing
 The goal is not to supplant the idiomatic libraries, but to provide a 
 consistent, stable base for them to build upon.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1082035 - /cassandra/site/src/README