[jira] Created: (CASSANDRA-1946) cassandra-cli always uses port 8080 (show keyspaces brings up a WARNing), no matter which port is defined in cassandra-env.sh
cassandra-cli always uses port 8080 (show keyspaces brings up a WARNing), no matter which port is defined in cassandra-env.sh - Key: CASSANDRA-1946 URL: https://issues.apache.org/jira/browse/CASSANDRA-1946 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.0 Environment: Ubuntu 10.10, JDK 1.6.0_23 Reporter: Markus Wiesenbacher Priority: Trivial In cassandra-cli I do a "show keyspaces;", this bring up this WARNing: WARNING: Could not connect to the JMX on 10.1.1.106:8080, information won't be shown. JMX (mx4j-tools) is up and running, but on port 18080 (defined in cassandra-env.sh), so the problem should be that the CLI uses 8080 statically? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1896) Improve throughput by adding buffering to the inter-node TCP communication
[
https://issues.apache.org/jira/browse/CASSANDRA-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978606#action_12978606
]
Hudson commented on CASSANDRA-1896:
---
Integrated in Cassandra-0.6 #48 (See
[https://hudson.apache.org/hudson/job/Cassandra-0.6/48/])
> Improve throughput by adding buffering to the inter-node TCP communication
> --
>
> Key: CASSANDRA-1896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1896
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.6
>Reporter: Tsiki
>Assignee: Brandon Williams
> Fix For: 0.6.10, 0.7.1
>
> Attachments: 1896-v2.txt, 1896-v3.txt, 1896-v4.txt, 1896.txt
>
>
> The inbound and outbound TCP implementation under org.apache.cassandra.net
> does not buffer the socket streams. A simple change in IncomingTcpConnection
> and OutboundTcpConnection may give a rather big throughput increase. In my
> tests, I got up t o 30% more out of my cluster. Below is the diff of these
> two files with buffering included. The diff is over release 0.6.5 but can be
> quite simply applied also to 0.7. I suggest perhaps to limit the buffered
> input stream I added in IncomingTcpConnection to 4K. The Outbound
> implementation can surely be implemented a bit better (remove some of the
> code I duplicated there).
> diff -r
> apache-cassandra-0.6.5-src/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
>
> fix/apache-cassandra-0.6.5-src/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
> 44c44
> < input = new DataInputStream(new
> BufferedInputStream(socket.getInputStream()));
> ---
> > input = new DataInputStream(socket.getInputStream());
> diff -r
> apache-cassandra-0.6.5-src/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
>
> fix/apache-cassandra-0.6.5-src/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
> 77d76
> < byte[] buf = new byte[4096];
> 80,112c79,89
> < int l = 0;
> < ByteBuffer bb;
> < while ((bb = queue.peek()) != null && l+bb.limit() <
> buf.length) {
> < bb = take();
> < System.arraycopy(bb.array(), 0, buf, l, bb.limit());
> < l += bb.limit();
> < }
> < if (l == 0) {
> < bb = take();
> < if (bb == CLOSE_SENTINEL)
> < {
> < disconnect();
> < continue;
> < }
> < if (socket != null || connect())
> < writeConnected(bb);
> < else
> < // clear out the queue, else gossip messages back up.
> < queue.clear();
> < } else {
> < if (socket != null || connect()) {
> < try {
> < output.write(buf, 0, l);
> < if (queue.peek() == null)
> < output.flush();
> < } catch (IOException e) {
> < logger.info("error writing to " + endpoint);
> < disconnect();
> < }
> < } else {
> < queue.clear();
> < }
> < }
> ---
> > ByteBuffer bb = take();
> > if (bb == CLOSE_SENTINEL)
> > {
> > disconnect();
> > continue;
> > }
> > if (socket != null || connect())
> > writeConnected(bb);
> > else
> > // clear out the queue, else gossip messages back up.
> > queue.clear();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1943) Addition of internode buffering broke Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978607#action_12978607 ] Hudson commented on CASSANDRA-1943: --- Integrated in Cassandra-0.6 #48 (See [https://hudson.apache.org/hudson/job/Cassandra-0.6/48/]) Don't begin buffering a connection until we've determined the type. Patch by Stu Hood, reviewed by gdusbabek for CASSANDRA-1943 > Addition of internode buffering broke Streaming > --- > > Key: CASSANDRA-1943 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1943 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.6.10, 0.7.1 >Reporter: Stu Hood >Priority: Critical > Fix For: 0.6.10, 0.7.1, 0.8 > > Attachments: > 0.6-0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt, > 0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt > > > Adding internode buffering broke StreamingTransferTest in the 0.7 branch. > Bisected to r1055313 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1711) Python driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978600#action_12978600 ] Hudson commented on CASSANDRA-1711: --- Integrated in Cassandra #663 (See [https://hudson.apache.org/hudson/job/Cassandra/663/]) > Python driver for CQL > - > > Key: CASSANDRA-1711 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1711 > Project: Cassandra > Issue Type: Sub-task > Components: API >Affects Versions: 0.8 >Reporter: Eric Evans >Priority: Minor > Fix For: 0.8 > > Attachments: > v2-0001-CASSANDRA-1711-basic-connection-pooling-for-python-dri.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > In-tree CQL drivers should be reasonably consistent with one another > (wherever possible/practical), and implement a minimum of: > * Query compression > * Keyspace assignment on connection > * Connection pooling / load-balancing > The goal is not to supplant the idiomatic libraries, but to provide a > consistent, stable base for them to build upon. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1705) CQL writes (aka UPDATE)
[
https://issues.apache.org/jira/browse/CASSANDRA-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978601#action_12978601
]
Hudson commented on CASSANDRA-1705:
---
Integrated in Cassandra #663 (See
[https://hudson.apache.org/hudson/job/Cassandra/663/])
> CQL writes (aka UPDATE)
> ---
>
> Key: CASSANDRA-1705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1705
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Priority: Minor
> Fix For: 0.8
>
> Attachments: CASSANDRA-1705.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> CQL specification and implementation for data manipulation.
> This corresponds to the following RPC methods:
> * insert()
> * batch_mutate() (writes, not deletes)
> The initial check-in to trunk/ uses a syntax that looks like:
> {code:SQL}
> UPDATE [USING CONSISTENCY.] WITH ROW(, COLUMN(,
> )[, COLUMN(...)])[ AND ROW(...)];
> {code}
> Where:
> * is the column family name.
> * Rows are a parenthesized expressions with comma separated arguments for a
> key and one or more columns.
> * Columns are a parenthesized expressions with comma separated arguments for
> the name and value (timestamp is inaccessible).
> What is still undone:
> * Complete test coverage
> And of course, all of this is still very much open to further discussion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1710) Java driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978599#action_12978599 ] Hudson commented on CASSANDRA-1710: --- Integrated in Cassandra #663 (See [https://hudson.apache.org/hudson/job/Cassandra/663/]) > Java driver for CQL > --- > > Key: CASSANDRA-1710 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1710 > Project: Cassandra > Issue Type: Sub-task > Components: API >Affects Versions: 0.8 >Reporter: Eric Evans >Priority: Minor > Fix For: 0.8 > > Attachments: jdbc-ish.diff, > v3-0001-CASSANDRA-1710-basic-connection-pooling-for-java-drive.txt, > v3-0002-compile-driver-source.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > In-tree CQL drivers should be reasonably consistent with one another > (wherever possible/practical), and implement a minimum of: > * Query compression > * Keyspace assignment on connection > * Connection pooling / load-balancing > The goal is not to supplant the idiomatic libraries, but to provide a > consistent, stable base for them to build upon. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[Cassandra Wiki] Trivial Update of "FileFormatDesignDoc " by StuHood
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "FileFormatDesignDoc" page has been changed by StuHood.
http://wiki.apache.org/cassandra/FileFormatDesignDoc?action=diff&rev1=19&rev2=20
--
|| havarti || muenster || 0 || 0 ||
|| || || 1 || 1 ||
- This example shows a range tombstone for values at level "name1" between
'havarti' and 'muenster': the chunk for the "name1" level stores a pair of
range tuples for the 'cheese' parent and a nulls are stored for parents without
any range metadata. The end result is that the span stores a tombstone from
('cheese', 'havarti', ) to ('cheese', 'muenster', null), where
is the smallest value, and null is the largest value.
+ This example shows a range tombstone for values at level "name1" between
'havarti' and 'muenster': the chunk for the "name1" level stores a pair of
range tuples for the 'cheese' parent and nulls are stored for parents without
any range metadata. The end result is that the span stores a tombstone from
('cheese', 'havarti', ) to ('cheese', 'muenster', null), where
is the smallest value, and null is the largest value.
Note that it is not possible for ranges for a parent to overlap: in this
case, the ranges would be resolved such that the intersection was given the
winning timestamp, and the two remainders would use their original timestamps.
[jira] Commented: (CASSANDRA-1427) Optimize loadbalance/move for moves within the current range
[ https://issues.apache.org/jira/browse/CASSANDRA-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978595#action_12978595 ] Ryan King commented on CASSANDRA-1427: -- I think we should generalize it to cover all cases. > Optimize loadbalance/move for moves within the current range > > > Key: CASSANDRA-1427 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1427 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 0.7 beta 1 >Reporter: Nick Bailey >Assignee: Brandon Williams > Fix For: 0.8 > > > Currently our move/loadbalance operations only implement case 2 of the Ruhl > algorithm described at > https://issues.apache.org/jira/browse/CASSANDRA-192#action_12713079. > We should add functionality to optimize moves that take/give ranges to a > node's direct neighbors. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/918 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1056128 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
svn commit: r1056128 - in /cassandra/trunk: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/net/
Author: jbellis Date: Fri Jan 7 00:03:53 2011 New Revision: 1056128 URL: http://svn.apache.org/viewvc?rev=1056128&view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:03:53 2011 @@ -1,5 +1,5 @@ -/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7:1026516-1056109 +/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1055311,1056121 +/cassandra/branches/cassandra-0.7:1026516-1056127 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:03:53 2011 @@ -1,5 +1,5 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1056109 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1055311,1056121 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1056127 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:03:53 2011 @@ -1,5 +1,5 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1056109 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1055311,1056121 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1056127 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:03:53 2011 @@ -1,5 +1,5 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1056109 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1055311,1056121 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1056127 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/t
svn commit: r1056127 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/net/
Author: jbellis Date: Fri Jan 7 00:02:55 2011 New Revision: 1056127 URL: http://svn.apache.org/viewvc?rev=1056127&view=rev Log: merge from 0.6 Modified: cassandra/branches/cassandra-0.7/ (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/net/IncomingTcpConnection.java Propchange: cassandra/branches/cassandra-0.7/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:02:55 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6:922689-1055311 +/cassandra/branches/cassandra-0.6:922689-1055311,1056121 /cassandra/branches/cassandra-0.7:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:02:55 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1055311 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1055311,1056121 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:02:55 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1055311 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1055311,1056121 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:02:55 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1055311 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1055311,1056121 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jan 7 00:02:55 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1055311 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1055311,1056121 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassand
buildbot success in ASF Buildbot on cassandra-0.6
The Buildbot has detected a restored build of cassandra-0.6 on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-0.6/builds/273 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/branches/cassandra-0.6] 1056121 Blamelist: brandonwilliams Build succeeded! sincerely, -The Buildbot
svn commit: r1056121 - /cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Author: brandonwilliams
Date: Thu Jan 6 23:33:45 2011
New Revision: 1056121
URL: http://svn.apache.org/viewvc?rev=1056121&view=rev
Log:
Don't begin buffering a connection until we've determined the type.
Patch by Stu Hood, reviewed by gdusbabek for CASSANDRA-1943
Modified:
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Modified:
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java?rev=1056121&r1=1056120&r2=1056121&view=diff
==
---
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
(original)
+++
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Thu Jan 6 23:33:45 2011
@@ -32,40 +32,48 @@ public class IncomingTcpConnection exten
{
private static Logger logger =
Logger.getLogger(IncomingTcpConnection.class);
-private final DataInputStream input;
private Socket socket;
public IncomingTcpConnection(Socket socket)
{
assert socket != null;
this.socket = socket;
+}
+
+/**
+ * A new connection will either stream or message for its entire lifetime:
because streaming
+ * bypasses the InputStream implementations to use sendFile, we cannot
begin buffering until
+ * we've determined the type of the connection.
+ */
+@Override
+public void run()
+{
+DataInputStream input;
+boolean isStream;
try
{
-input = new DataInputStream(new
BufferedInputStream(socket.getInputStream(), 4096));
+// determine the connection type to decide whether to buffer
+input = new DataInputStream(socket.getInputStream());
+MessagingService.validateMagic(input.readInt());
+int header = input.readInt();
+isStream = MessagingService.getBits(header, 3, 1) == 1;
+if (!isStream)
+// we should buffer
+input = new DataInputStream(new
BufferedInputStream(socket.getInputStream(), 4096));
}
catch (IOException e)
{
close();
throw new IOError(e);
}
-}
-
-@Override
-public void run()
-{
while (true)
{
try
{
-MessagingService.validateMagic(input.readInt());
-int header = input.readInt();
-int type = MessagingService.getBits(header, 1, 2);
-boolean isStream = MessagingService.getBits(header, 3, 1) == 1;
-int version = MessagingService.getBits(header, 15, 8);
-
if (isStream)
{
new IncomingStreamReader(socket.getChannel()).read();
+break;
}
else
{
@@ -76,6 +84,10 @@ public class IncomingTcpConnection exten
Message message = Message.serializer().deserialize(new
DataInputStream(new ByteArrayInputStream(contentBytes)));
MessagingService.receive(message);
}
+// prepare to read the next message
+MessagingService.validateMagic(input.readInt());
+int header = input.readInt();
+assert isStream == (MessagingService.getBits(header, 3, 1) ==
1) : "Connections cannot change type: " + isStream;
}
catch (EOFException e)
{
[jira] Updated: (CASSANDRA-1943) Addition of internode buffering broke Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1943: Attachment: 0.6-0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt And a version rebased for 0.6 > Addition of internode buffering broke Streaming > --- > > Key: CASSANDRA-1943 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1943 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.6.10, 0.7.1 >Reporter: Stu Hood >Priority: Critical > Fix For: 0.6.10, 0.7.1, 0.8 > > Attachments: > 0.6-0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt, > 0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt > > > Adding internode buffering broke StreamingTransferTest in the 0.7 branch. > Bisected to r1055313 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1930) db.Table flusherLock write lock fairness policy is sub-optimal
[ https://issues.apache.org/jira/browse/CASSANDRA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1930. --- Resolution: Fixed Fix Version/s: 0.7.1 Reviewer: jbellis bq. The reason we originally switched to a fair policy was, without it there is no guarantee that the writer (the flushing thread) will ever get the lock, and we were seeing OOMing from the memtable exhausting memory instead of getting switched out. This reasoning is obsolete; as Stu points out, we try to flush whenever we do a write that violates the memtable threshold. So you should see at most concurrent-writes extra updates before the flush. Committed, with comment on RRWL instantiation. (Still open to another ticket for a new approach entirely, but this fix seems better than the status quo.) > db.Table flusherLock write lock fairness policy is sub-optimal > -- > > Key: CASSANDRA-1930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1930 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Kelvin Kakugawa >Assignee: Kelvin Kakugawa > Fix For: 0.7.1, 0.8 > > Attachments: CASSANDRA-1930.0001.patch > > > h4. scenario: > 1) high write throughput cluster > 2) external services adding material cpu contention > h4. symptoms: > The row mutation stage falls *very* behind. > h4. cause: > ReentrantReadWriteLock's fair policy causes write lock acquisition / release > to require a material amount of CPU time. > h4. summary: > When there are other services contending for the CPU, the RRW lock's fair > policy causes write lock acquisition / release to take enough time to > eventually put threads waiting for read lock acquisition very behind. We > repro'd this scenario by reducing the scope of the write lock to: 1 boolean > check, 1 boolean assignment, and 1 db.Memtable instantiation (itself, just: 2 > variable assignments) w/ the same performance. Modifying the fairness policy > to be the default policy allowed the row mutation stage to keep up. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1056110 - in /cassandra/trunk: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/
Author: jbellis Date: Thu Jan 6 22:58:29 2011 New Revision: 1056110 URL: http://svn.apache.org/viewvc?rev=1056110&view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/Table.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Jan 6 22:58:29 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7:1026516-1055668 +/cassandra/branches/cassandra-0.7:1026516-1056109 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1056110&r1=1056109&r2=1056110&view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Thu Jan 6 22:58:29 2011 @@ -18,6 +18,7 @@ * make read repair behave as expected for ConsistencyLevel > ONE (CASSANDRA-982) * distributed test harness (CASSANDRA-1859) + * reduce flush lock contention (CASSANDRA-1930) 0.7.0-dev Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Jan 6 22:58:29 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1055668 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1056109 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Jan 6 22:58:29 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1055668 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1056109 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Jan 6 22:58:29 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1055311 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1055668 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1056109 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689
svn commit: r1056109 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/db/Table.java
Author: jbellis Date: Thu Jan 6 22:57:18 2011 New Revision: 1056109 URL: http://svn.apache.org/viewvc?rev=1056109&view=rev Log: reduce flush lock contention patch by Kelvin Kakugawa; reviewed by jbellis for CASSANDRA-1930 Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1056109&r1=1056108&r2=1056109&view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Jan 6 22:57:18 2011 @@ -13,6 +13,7 @@ * make read repair behave as expected for ConsistencyLevel > ONE (CASSANDRA-982) * distributed test harness (CASSANDRA-1859) + * reduce flush lock contention (CASSANDRA-1930) 0.7.0-dev Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java?rev=1056109&r1=1056108&r2=1056109&view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java Thu Jan 6 22:57:18 2011 @@ -64,7 +64,7 @@ public class Table * accesses to CFS.memtable should acquire this for thread safety. * Table.maybeSwitchMemtable should aquire the writeLock; see that method for the full explanation. */ -static final ReentrantReadWriteLock flusherLock = new ReentrantReadWriteLock(true); +static final ReentrantReadWriteLock flusherLock = new ReentrantReadWriteLock(); // It is possible to call Table.open without a running daemon, so it makes sense to ensure // proper directories here as well as in CassandraDaemon.
[jira] Commented: (CASSANDRA-1930) db.Table flusherLock write lock fairness policy is sub-optimal
[ https://issues.apache.org/jira/browse/CASSANDRA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978543#action_12978543 ] Stu Hood commented on CASSANDRA-1930: - I think there is more going on here: while swapping the log entries around, we noticed that when a Memtable reaches its threshold, up to (all) N write threads will often notice, and race to acquire the writeLock in order to freeze the memtable. This means that we do way more writeLock acquisitions than we need to. This is probably just more reason to replace this lock. > db.Table flusherLock write lock fairness policy is sub-optimal > -- > > Key: CASSANDRA-1930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1930 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Kelvin Kakugawa >Assignee: Kelvin Kakugawa > Fix For: 0.8 > > Attachments: CASSANDRA-1930.0001.patch > > > h4. scenario: > 1) high write throughput cluster > 2) external services adding material cpu contention > h4. symptoms: > The row mutation stage falls *very* behind. > h4. cause: > ReentrantReadWriteLock's fair policy causes write lock acquisition / release > to require a material amount of CPU time. > h4. summary: > When there are other services contending for the CPU, the RRW lock's fair > policy causes write lock acquisition / release to take enough time to > eventually put threads waiting for read lock acquisition very behind. We > repro'd this scenario by reducing the scope of the write lock to: 1 boolean > check, 1 boolean assignment, and 1 db.Memtable instantiation (itself, just: 2 > variable assignments) w/ the same performance. Modifying the fairness policy > to be the default policy allowed the row mutation stage to keep up. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1943) Addition of internode buffering broke Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1943: -- Reviewer: gdusbabek Component/s: Core Affects Version/s: 0.7.1 0.6.10 > Addition of internode buffering broke Streaming > --- > > Key: CASSANDRA-1943 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1943 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.6.10, 0.7.1 >Reporter: Stu Hood >Priority: Critical > Fix For: 0.6.10, 0.7.1, 0.8 > > Attachments: > 0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt > > > Adding internode buffering broke StreamingTransferTest in the 0.7 branch. > Bisected to r1055313 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1943) Addition of internode buffering broke Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978520#action_12978520 ] Gary Dusbabek commented on CASSANDRA-1943: -- +1 > Addition of internode buffering broke Streaming > --- > > Key: CASSANDRA-1943 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1943 > Project: Cassandra > Issue Type: Bug >Reporter: Stu Hood >Priority: Critical > Fix For: 0.6.10, 0.7.1, 0.8 > > Attachments: > 0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt > > > Adding internode buffering broke StreamingTransferTest in the 0.7 branch. > Bisected to r1055313 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: CASSANDRA-1714-v3.patch branch: trunk (latest commit d8c4f7ce5fb2b5d2d1e75aee99426eec77ceadc2) version 3 with refactoring and renames removed. > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, > CASSANDRA-1714.patch, zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1848) Separate thrift and avro classes from cassandra's jar
[ https://issues.apache.org/jira/browse/CASSANDRA-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978498#action_12978498 ] Gary Dusbabek commented on CASSANDRA-1848: -- +1 on reorganizing the code like that. > Separate thrift and avro classes from cassandra's jar > - > > Key: CASSANDRA-1848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1848 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Affects Versions: 0.7.0 rc 2 >Reporter: Tristan Tarrant >Assignee: Eric Evans >Priority: Trivial > Fix For: 0.8 > > Attachments: CASSANDRA-1848.patch, CASSANDRA-1848_with_hadoop.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > Most client applications written in Java include the full > apache-cassandra-x.y.z.jar in their classpath. I propose to separate the avro > and thrift classes into separate jars. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1848) Separate thrift and avro classes from cassandra's jar
[
https://issues.apache.org/jira/browse/CASSANDRA-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978489#action_12978489
]
Eric Evans commented on CASSANDRA-1848:
---
I suggest we create the following jars (with corresponding source and javadoc):
* cassandra-jmx-$version.jar (mbeans)
* cassandra-thrift-$version.jar (generated thrift code (clients))
* cassandra-cql-$version.jar (CQL driver)
* cassandra-tools-$version.jar (cli, nodetool, clustertool)
* cassandra-hadoop-$version.jar (hadoop support)
* cassandra-main-$version.jar (basically everything else)
To accomplish this, I'd like to reogranize the source folders to look something
like the following diagram. Classes would be compiled to corresponding
directories under {{build/classes}}, (i.e. {{build/classes/main}},
{{build/classes/mbeans}}, etc).
{noformat}
src
|- main
| \_ gen-java
| \_ java
|
|- mbeans
| \_ java
|
|- thrift
| \_ gen-java
|
|- cql
| \_ java
|
|- hadoop
| \_ java
|
|- tools
\_ java
{noformat}
A logical separation like this will make it easier to get the right files into
the right jars without a lot of complex excludes/includes.
> Separate thrift and avro classes from cassandra's jar
> -
>
> Key: CASSANDRA-1848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1848
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Affects Versions: 0.7.0 rc 2
>Reporter: Tristan Tarrant
>Assignee: Eric Evans
>Priority: Trivial
> Fix For: 0.8
>
> Attachments: CASSANDRA-1848.patch, CASSANDRA-1848_with_hadoop.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Most client applications written in Java include the full
> apache-cassandra-x.y.z.jar in their classpath. I propose to separate the avro
> and thrift classes into separate jars.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1923) unit tests that validate that message serialization isn't broken in the current version.
[ https://issues.apache.org/jira/browse/CASSANDRA-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978487#action_12978487 ] Gary Dusbabek commented on CASSANDRA-1923: -- the txt patches are against trunk, the 0.7 patches are labeled as such. serializations can be generated with `ant msg-ser-gen-test`. serialization tests can be run (after generating serializations) with `ant msg-ser-test`. 1923-0.7-binary.zip contains the serialized messages for 0.7. unzip them into test/data. It looks like we already have to wire incompatibilities to deal with. :) > unit tests that validate that message serialization isn't broken in the > current version. > > > Key: CASSANDRA-1923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1923 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Gary Dusbabek >Assignee: Gary Dusbabek >Priority: Minor > Fix For: 0.7.1 > > Attachments: 1923-0.7-binary.zip, 1923-0.7-code.diff, > v1-0001-ICompactSerializer-assumptions-test.txt, > v1-0002-Serialization-tests.txt, v1-0003-build-changes-to-run-test.txt, > v1-0004-notes.txt > > > There are two components to this. First, code that will generate the > serialized messages. Second, code that will attempt to read the serialized > messages. > My plan is to commit this to 0.7.1 with generated serialized messages. Then > I will merge that into trunk sans the generation code. A similar process > will need to take place when we branch trunk to create 0.8, etc. On second > thought, maybe it makes sense to keep the generation code and let it morph as > the message formats change. > If the tests ever break in the 0.7 branch, that means we've created a message > incompatibility regression that needs to be fixed. If the tests ever break > in trunk (post CASSANDRA-1015), it means that something in trunk has changed > message serialization compatibility that will need to be restored (via > whatever process is used for 1015). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1923) unit tests that validate that message serialization isn't broken in the current version.
[ https://issues.apache.org/jira/browse/CASSANDRA-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-1923: - Attachment: 1923-0.7-binary.zip 1923-0.7-code.diff > unit tests that validate that message serialization isn't broken in the > current version. > > > Key: CASSANDRA-1923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1923 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Gary Dusbabek >Assignee: Gary Dusbabek >Priority: Minor > Fix For: 0.7.1 > > Attachments: 1923-0.7-binary.zip, 1923-0.7-code.diff, > v1-0001-ICompactSerializer-assumptions-test.txt, > v1-0002-Serialization-tests.txt, v1-0003-build-changes-to-run-test.txt, > v1-0004-notes.txt > > > There are two components to this. First, code that will generate the > serialized messages. Second, code that will attempt to read the serialized > messages. > My plan is to commit this to 0.7.1 with generated serialized messages. Then > I will merge that into trunk sans the generation code. A similar process > will need to take place when we branch trunk to create 0.8, etc. On second > thought, maybe it makes sense to keep the generation code and let it morph as > the message formats change. > If the tests ever break in the 0.7 branch, that means we've created a message > incompatibility regression that needs to be fixed. If the tests ever break > in trunk (post CASSANDRA-1015), it means that something in trunk has changed > message serialization compatibility that will need to be restored (via > whatever process is used for 1015). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1923) unit tests that validate that message serialization isn't broken in the current version.
[ https://issues.apache.org/jira/browse/CASSANDRA-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-1923: - Attachment: v1-0004-notes.txt v1-0003-build-changes-to-run-test.txt v1-0002-Serialization-tests.txt v1-0001-ICompactSerializer-assumptions-test.txt > unit tests that validate that message serialization isn't broken in the > current version. > > > Key: CASSANDRA-1923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1923 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Gary Dusbabek >Assignee: Gary Dusbabek >Priority: Minor > Fix For: 0.7.1 > > Attachments: 1923-0.7-binary.zip, 1923-0.7-code.diff, > v1-0001-ICompactSerializer-assumptions-test.txt, > v1-0002-Serialization-tests.txt, v1-0003-build-changes-to-run-test.txt, > v1-0004-notes.txt > > > There are two components to this. First, code that will generate the > serialized messages. Second, code that will attempt to read the serialized > messages. > My plan is to commit this to 0.7.1 with generated serialized messages. Then > I will merge that into trunk sans the generation code. A similar process > will need to take place when we branch trunk to create 0.8, etc. On second > thought, maybe it makes sense to keep the generation code and let it morph as > the message formats change. > If the tests ever break in the 0.7 branch, that means we've created a message > incompatibility regression that needs to be fixed. If the tests ever break > in trunk (post CASSANDRA-1015), it means that something in trunk has changed > message serialization compatibility that will need to be restored (via > whatever process is used for 1015). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978459#action_12978459 ] Peter Schuller commented on CASSANDRA-1902: --- Recommend is a strong word, but it was at least my initial reaction for the simplicity of it. A threshold expressed in pages or bytes. I am definitely in favor of an option above a hard-coded threshold, but that's coming from the perspective of expecting to deploy in cache sensitive scenarios where CPU will not be the main issue. As for WILLNEED: I don't know enough about the kernel's page cache eviction to say whether a write + WILLNEED will affect eviction relative to a write only. Either way is plausible I think, without knowing. (e.g. if both write() and WILLNEED is essentially counted as a 'hit' it might help; on the other hand it might not if only page accesses are counted...). In either case, it's another syscall. > Migrate cached pages during compaction > --- > > Key: CASSANDRA-1902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.1 >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 0.7.1 > > Original Estimate: 32h > Remaining Estimate: 32h > > Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a > pre-compacted CF during the compaction process. > First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that > uses the posix mincore() function to detect the offsets of pages for this > file currently in page cache. > Then add getActiveKeys() which uses underlying pagesInPageCache() to get the > keys actually in the page cache. > use getActiveKeys() to detect which SSTables being compacted are in the os > cache and make sure the subsequent pages in the new compacted SSTable are > kept in the page cache for these keys. This will minimize the impact of > compacting a "hot" SSTable. > A simpler yet similar approach is described here: > http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978450#action_12978450 ] T Jake Luciani edited comment on CASSANDRA-1902 at 1/6/11 1:46 PM: --- Right, *not* DONTNEEDing the pages needed should prob be fine just thought WILLNEED might ensure these pages are kept over the presumably active older sstables. So you are recommending a binary flag like "compaction.hotness.migration.thresh = .5"? I was thinking the same for simplicity sake. was (Author: tjake): Right, *not* DONTNEEDing the pages needed should prob be fine just though will need might ensure these pages are kept over the presumably active older sstables. So you are recommending a binary flag like "compaction.hotness.migration.thresh = .5"? I was thinking the same for simplicity sake. > Migrate cached pages during compaction > --- > > Key: CASSANDRA-1902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.1 >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 0.7.1 > > Original Estimate: 32h > Remaining Estimate: 32h > > Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a > pre-compacted CF during the compaction process. > First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that > uses the posix mincore() function to detect the offsets of pages for this > file currently in page cache. > Then add getActiveKeys() which uses underlying pagesInPageCache() to get the > keys actually in the page cache. > use getActiveKeys() to detect which SSTables being compacted are in the os > cache and make sure the subsequent pages in the new compacted SSTable are > kept in the page cache for these keys. This will minimize the impact of > compacting a "hot" SSTable. > A simpler yet similar approach is described here: > http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978450#action_12978450 ] T Jake Luciani commented on CASSANDRA-1902: --- Right, *not* DONTNEEDing the pages needed should prob be fine just though will need might ensure these pages are kept over the presumably active older sstables. So you are recommending a binary flag like "compaction.hotness.migration.thresh = .5"? I was thinking the same for simplicity sake. > Migrate cached pages during compaction > --- > > Key: CASSANDRA-1902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.1 >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 0.7.1 > > Original Estimate: 32h > Remaining Estimate: 32h > > Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a > pre-compacted CF during the compaction process. > First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that > uses the posix mincore() function to detect the offsets of pages for this > file currently in page cache. > Then add getActiveKeys() which uses underlying pagesInPageCache() to get the > keys actually in the page cache. > use getActiveKeys() to detect which SSTables being compacted are in the os > cache and make sure the subsequent pages in the new compacted SSTable are > kept in the page cache for these keys. This will minimize the impact of > compacting a "hot" SSTable. > A simpler yet similar approach is described here: > http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978446#action_12978446 ] Jonathan Ellis commented on CASSANDRA-1714: --- Sorry; it really makes it difficult when reading e.g. svn annotate to see what was actually relevant to the zero-copy read change. Please re-submit with the other changes backed out. You're welcome to create a ticket for the refactorings. (FWIW, a git-based workflow like the one I described in http://spyced.blogspot.com/2009/06/patch-oriented-development-made-sane.html makes it much easier to keep these kinds of changes separated out.) > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1945) Use EHcache for row cache + disk spillover
Use EHcache for row cache + disk spillover -- Key: CASSANDRA-1945 URL: https://issues.apache.org/jira/browse/CASSANDRA-1945 Project: Cassandra Issue Type: Improvement Components: Core Reporter: T Jake Luciani Ehcache does a nice job of spilling over to disk and its ASL2 http://www.ehcache.org/license.html Also, the commercial version supports Smaller JVM sizes via it's BigMemory product http://www.terracotta.org/bigmemory -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1902) Migrate cached pages during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978435#action_12978435
]
Peter Schuller commented on CASSANDRA-1902:
---
Should it not be a matter of *not* DONTNEED:ing the relevant ranges during
write, rather than using WILLNEED? WILLNEED seems to result in read-ahead and
is presumably meant for cases where you're reading data in that is expected to
not be in cache and want to inform the kernel about the desire for read-ahead.
With respect to contiguous ranges: If possible it would be nice if the desired
contiguousness would be a configuration option. For workloads where disk I/O is
very critical, I would not be surprised if even the worst-case overhead of very
frequent posix_fadvise() calls is going to be worth it since the cost of cache
misses is so extremely high relative to the cost of a syscall.
It seems (looking again at
http://lxr.free-electrons.com/source/mm/fadvise.c#L118) that posix_fadvise()
overhead should be the usual syscall overhead plus O(n) on the number of pages.
As to accomplishing advice on contiguous ranges, I'm not sure what the best
course of action is. On a sparsely in-core sstable there is no issue. On a very
in-core sstable, DONTNEED:s are probably very often going to be so small as to
mean that any thresholding just turns it off almost completely.
I suppose there is some middle-ground somewhere where being smart about
fadvise() may be advisable.
For simplicity's sake, how about having a simple threshold; "posix_fadvise() in
such a way that no call applies to less than N pages of data". The
implementation could be as simple as skipping ("skip" meaning, "dont avoid
DONTNEED" - double negation...) ranges that do not fulfill the criteria. The
expected result I see from an operational perspective is:
* For very hot tables, hotness remains reasonable because in effect, no
DONTNEED is done. You're less efficient of course, but no worse than currently.
* For less hot tables, you start seeing an effect, with the effect being
largest on large sparsely cached tables (which is also where it is most
important).
* If you are in a position where you really want to squeeze that last bit of
hotness out of compaction even if it costs lots of syscalls, you can set the
threshold to 0.
I am operating on the assumption that the only motivation for preferring
contiguous ranges is the performance penalty of the fadvise().
> Migrate cached pages during compaction
> ---
>
> Key: CASSANDRA-1902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1902
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.7.1
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 0.7.1
>
> Original Estimate: 32h
> Remaining Estimate: 32h
>
> Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a
> pre-compacted CF during the compaction process.
> First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that
> uses the posix mincore() function to detect the offsets of pages for this
> file currently in page cache.
> Then add getActiveKeys() which uses underlying pagesInPageCache() to get the
> keys actually in the page cache.
> use getActiveKeys() to detect which SSTables being compacted are in the os
> cache and make sure the subsequent pages in the new compacted SSTable are
> kept in the page cache for these keys. This will minimize the impact of
> compacting a "hot" SSTable.
> A simpler yet similar approach is described here:
> http://insights.oetiker.ch/linux/fadvise/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978436#action_12978436 ] Peter Schuller commented on CASSANDRA-1902: --- Okay. JIRA's web interface renders "O ( n )" as an O followed by thumbs-down... How insightful :) > Migrate cached pages during compaction > --- > > Key: CASSANDRA-1902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.1 >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 0.7.1 > > Original Estimate: 32h > Remaining Estimate: 32h > > Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a > pre-compacted CF during the compaction process. > First, add a method to MmappedSegmentFile: long[] pagesInPageCache() that > uses the posix mincore() function to detect the offsets of pages for this > file currently in page cache. > Then add getActiveKeys() which uses underlying pagesInPageCache() to get the > keys actually in the page cache. > use getActiveKeys() to detect which SSTables being compacted are in the os > cache and make sure the subsequent pages in the new compacted SSTable are > kept in the page cache for these keys. This will minimize the impact of > compacting a "hot" SSTable. > A simpler yet similar approach is described here: > http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1943) Addition of internode buffering broke Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-1943: - Fix Version/s: 0.6.10 > Addition of internode buffering broke Streaming > --- > > Key: CASSANDRA-1943 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1943 > Project: Cassandra > Issue Type: Bug >Reporter: Stu Hood >Priority: Critical > Fix For: 0.6.10, 0.7.1, 0.8 > > Attachments: > 0001-Don-t-begin-buffering-a-connection-until-we-ve-determi.txt > > > Adding internode buffering broke StreamingTransferTest in the 0.7 branch. > Bisected to r1055313 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1055934 - /cassandra/branches/cassandra-0.7.0/.rat-excludes
Author: eevans Date: Thu Jan 6 16:37:38 2011 New Revision: 1055934 URL: http://svn.apache.org/viewvc?rev=1055934&view=rev Log: additional rat excludes Modified: cassandra/branches/cassandra-0.7.0/.rat-excludes Modified: cassandra/branches/cassandra-0.7.0/.rat-excludes URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7.0/.rat-excludes?rev=1055934&r1=1055933&r2=1055934&view=diff == --- cassandra/branches/cassandra-0.7.0/.rat-excludes (original) +++ cassandra/branches/cassandra-0.7.0/.rat-excludes Thu Jan 6 16:37:38 2011 @@ -20,3 +20,5 @@ redhat/cassandra redhat/cassandra.conf redhat/cassandra.in.sh redhat/default +.externalToolBuilders/** +**/*.wpr
svn commit: r1055933 - in /cassandra/branches/cassandra-0.7.0: build.xml debian/changelog
Author: eevans Date: Thu Jan 6 16:37:34 2011 New Revision: 1055933 URL: http://svn.apache.org/viewvc?rev=1055933&view=rev Log: update versioning for 7.0 release Modified: cassandra/branches/cassandra-0.7.0/build.xml cassandra/branches/cassandra-0.7.0/debian/changelog Modified: cassandra/branches/cassandra-0.7.0/build.xml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7.0/build.xml?rev=1055933&r1=1055932&r2=1055933&view=diff == --- cassandra/branches/cassandra-0.7.0/build.xml (original) +++ cassandra/branches/cassandra-0.7.0/build.xml Thu Jan 6 16:37:34 2011 @@ -47,7 +47,7 @@ - + Modified: cassandra/branches/cassandra-0.7.0/debian/changelog URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7.0/debian/changelog?rev=1055933&r1=1055932&r2=1055933&view=diff == --- cassandra/branches/cassandra-0.7.0/debian/changelog (original) +++ cassandra/branches/cassandra-0.7.0/debian/changelog Thu Jan 6 16:37:34 2011 @@ -1,3 +1,9 @@ +cassandra (0.7.0) unstable; urgency=low + + * New stable release. + + -- Eric Evans Thu, 06 Jan 2011 10:28:11 -0600 + cassandra (0.7.0~rc4) unstable; urgency=low * Release candidate release.
[jira] Commented: (CASSANDRA-1472) Add bitmap secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978394#action_12978394 ] T Jake Luciani commented on CASSANDRA-1472: --- This breaks the secondaryCleanupTest Testcase: testSecondaryCleanup(org.apache.cassandra.db.SecondaryIndexCleanupTest): Caused an ERROR [junit] org.apache.cassandra.db.ColumnFamilyStore.buildSecondaryIndexes(Ljava/util/Collection;Ljava/util/SortedSet;)V [junit] java.lang.NoSuchMethodError: org.apache.cassandra.db.ColumnFamilyStore.buildSecondaryIndexes(Ljava/util/Collection;Ljava/util/SortedSet;)V [junit] at org.apache.cassandra.db.SecondaryIndexCleanupTest.fillCF(SecondaryIndexCleanupTest.java:142) [junit] at org.apache.cassandra.db.SecondaryIndexCleanupTest.testSecondaryCleanup(SecondaryIndexCleanupTest.java:91) > Add bitmap secondary indexes > > > Key: CASSANDRA-1472 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1472 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood > Fix For: 0.7.1 > > Attachments: 0.7-1472-v5.tgz, 1472-v3.tgz, 1472-v4.tgz, 1472-v5.tgz, > anatomy.png, v4-bench-c32.txt > > > Bitmap indexes are a very efficient structure for dealing with immutable > data. We can take advantage of the fact that SSTables are immutable by > attaching them directly to SSTables as a new component (supported by > CASSANDRA-1471). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978388#action_12978388 ] Pavel Yaskevich commented on CASSANDRA-1714: The biggest of unrelated changes in removal of "_" in attributes of SuperColumn class and few code style tweaks... Can't we just commit it as is because it will cost me good time of injecting them off, if not - where can I post a patch with those refactorings? > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978373#action_12978373 ] Jonathan Ellis commented on CASSANDRA-1714: --- we need to clean the renames and other unrelated changes out of this patch and submit separately. > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978360#action_12978360 ] Pavel Yaskevich edited comment on CASSANDRA-1714 at 1/6/11 10:15 AM: - I concur! Can you create that task and assign it on me, please? was (Author: xedin): I concur! Can you create that task and assign it on me, pelase? > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978360#action_12978360 ] Pavel Yaskevich commented on CASSANDRA-1714: I concur! Can you create that task and assign it on me, pelase? > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978358#action_12978358 ] T Jake Luciani commented on CASSANDRA-1714: --- I think it's worth adding then, and do ICS2 in another ticket +1 > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: CASSANDRA-1714-v2.patch branch: trunk (latest commit d8c4f7ce5fb2b5d2d1e75aee99426eec77ceadc2) benchmark using stress.py (-S 8192 -n 10): writes are equal, reads are 16 secs. faster (250 comparing to 266) > zero-copy reads > --- > > Key: CASSANDRA-1714 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 0.7.1 > > Attachments: CASSANDRA-1714-v2.patch, CASSANDRA-1714.patch, > zerocopy.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Since we are already using mmap'd ByteBuffers in MappedFileDataInput we > should be able to do zero-copy reads (via buffer.slice()), which would give > us better performance than CASSANDRA-1651 without having to worry about > buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[Cassandra Wiki] Update of "CassandraHardware" by Jonat hanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraHardware" page has been changed by JonathanEllis. The comment on this change is: ec2 setup recommendations. http://wiki.apache.org/cassandra/CassandraHardware?action=diff&rev1=16&rev2=17 -- On ext2/ext3 the maximum file size is 2TB, even on a 64 bit kernel. On ext4 that goes up to 16TB. Since Cassandra can use almost half your disk space on a single file, if you are raiding large disks together you may want to use XFS instead, particularly if you are using a 32-bit kernel. XFS file size limits are 16TB max on a 32 bit kernel, and basically unlimited on 64 bit. === Cloud === - Several heavy users of Cassandra deploy in the cloud, e.g. CloudKick on Rackspace Cloud Servers and SimpleGeo on Amazon EC2. The general consensus in the community seems to be that Rackspace's VMs offer better performance for Cassandra because of CPU bursting, raided local disks, and separate public/private network interfaces. + Several heavy users of Cassandra deploy in the cloud, e.g. CloudKick on Rackspace Cloud Servers and SimpleGeo on Amazon EC2. + On EC2, the best practice is to use L or XL instances with local storage. I/o performance is proportionately much worse on S and M sizes, and EBS essentially doubles your dependence on the already-overcrowded EC2 network (https://www.cloudkick.com/blog/2010/jan/12/visual-ec2-latency/), so you want to avoid that. Put the Cassandra commitlog on the root volume, and the data directory on the raid0'd ephemeral disks. +
[Cassandra Wiki] Update of "CassandraHardware" by Jonat hanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraHardware" page has been changed by JonathanEllis. http://wiki.apache.org/cassandra/CassandraHardware?action=diff&rev1=15&rev2=16 -- === Memory === - The most recently written data resides in memory tables (aka [[MemtableThresholds|memtables]]), but older data that has been flushed to disk can be kept in the OS's file-system cache. In other words, ''the more memory, the better'', with 1GB being the minimum we typically recommended in a virtualized environment. Obviously there is no benefit to having more RAM than your hot data set, but with dedicated hardware there is no reason to use less than 8GB or 16GB, and you often see clusters with 32 GB or more per node. + The most recently written data resides in memory tables (aka [[MemtableThresholds|memtables]]), but older data that has been flushed to disk can be kept in the OS's file-system cache. In other words, ''the more memory, the better'', with 4GB being the minimum we typically recommended in a virtualized environment (e.g., EC2 Large instances). Obviously there is no benefit to having more RAM than your hot data set, but with dedicated hardware there is no reason to use less than 8GB or 16GB, and you often see clusters with 32 GB or more per node. RAM can also be useful for the key cache (introduced in 0.5) and row cache (in 0.6).
[jira] Commented: (CASSANDRA-1944) Support CL.QUORUM and CL.ALL for counter writes
[ https://issues.apache.org/jira/browse/CASSANDRA-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978275#action_12978275 ] Sylvain Lebresne commented on CASSANDRA-1944: - I intend to borrow the code from CASSANDRA-1456 for this so more details can be found there. > Support CL.QUORUM and CL.ALL for counter writes > > > Key: CASSANDRA-1944 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1944 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.8 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Original Estimate: 16h > Remaining Estimate: 16h > > This ticket basically proposes to make the counter ReplicateOnWrite mechanism > synchronous instead of asynchronous. This will allow to support CL.QUORUM and > CL.ALL for counter writes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1944) Support CL.QUORUM and CL.ALL for counter writes
Support CL.QUORUM and CL.ALL for counter writes Key: CASSANDRA-1944 URL: https://issues.apache.org/jira/browse/CASSANDRA-1944 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8 This ticket basically proposes to make the counter ReplicateOnWrite mechanism synchronous instead of asynchronous. This will allow to support CL.QUORUM and CL.ALL for counter writes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
