[jira] [Commented] (CASSANDRA-3188) cqlsh 2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103956#comment-13103956 ] Jonathan Ellis commented on CASSANDRA-3188: --- bq. cassandra-cli uses JMX for a few of its cluster-inspection duties If this is important, we can look at making mx4j an official dependency instead of just supported-if-you-happen-to-have-it-around. cqlsh 2.0 - Key: CASSANDRA-3188 URL: https://issues.apache.org/jira/browse/CASSANDRA-3188 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: paul cannon Fix For: 1.0.1 I'd like to see some improvements to cqlsh to bring it to feature parity w/ the old cli: - describe [KS | CF | cluster | schema] - assume - connect / don't crap out w/ stacktrace if C* isn't currently running on localhost - help It may be easier to do this by forking the cli and replacing its one-off api with CQL, or it may be easier to add these features to cqlsh. Either is fine, but if it's a close call my inclination would be to build it in Java for the reasoning over on CASSANDRA-3010. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3198) debian packaging installation problem when installing for the first time
[ https://issues.apache.org/jira/browse/CASSANDRA-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3198: -- Fix Version/s: 0.8.6 debian packaging installation problem when installing for the first time Key: CASSANDRA-3198 URL: https://issues.apache.org/jira/browse/CASSANDRA-3198 Project: Cassandra Issue Type: Bug Components: Packaging Affects Versions: 0.8.5 Reporter: Jérémy Sevellec Assignee: Jérémy Sevellec Fix For: 0.8.6 Attachments: trunk-3198-v1.patch when installing cassandra through the debian packaging for the first time, there is permission problem when starting Cassandra. Normally, the postinst script change owner of /var/log/cassandra and /var/lib/cassandra from root to cassandra user. there is a problem with the test which verify if threre is a need to change the owner of these directory or not. On a new install, the $2 parameter is not set and the the test is false and the owner is not changed. (simply, i think replace with || might work) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103960#comment-13103960 ] Brandon Williams commented on CASSANDRA-3179: - I'm unable to discern any read performance difference between the performance test and v2 patches. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1170333 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
Author: jbellis
Date: Tue Sep 13 20:28:12 2011
New Revision: 1170333
URL: http://svn.apache.org/viewvc?rev=1170333view=rev
Log:
revert CASSANDRA-2388 (again)
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1170333r1=1170332r2=1170333view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Tue Sep 13 20:28:12 2011
@@ -1,5 +1,6 @@
0.7.10
* Fix divide by zero error in GCInspector (CASSANDRA-3164)
+ * revert CASSANDRA-2388
0.7.9
@@ -8,7 +9,7 @@
* prune index scan resultset back to original request for lazy
resultset expansion case (CASSANDRA-2964)
* (Hadoop) fail jobs when Cassandra node has failed but TaskTracker
-has not (CASSANDRA-2388)
+ has not (CASSANDRA-2388)
* avoid retaining references to dropped CFS objects in
CompactionManager.estimatedCompactions (CASSANDRA-2708)
* remove gossip state when a new IP takes over a token (CASSANDRA-3071)
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java?rev=1170333r1=1170332r2=1170333view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
Tue Sep 13 20:28:12 2011
@@ -138,6 +138,8 @@ public class ColumnFamilyRecordReader ex
return true;
}
+// we don't use endpointsnitch since we are trying to support hadoop nodes
that are
+// not necessarily on Cassandra machines, too. This should be adequate
for single-DC clusters, at least.
private String getLocation()
{
InetAddress[] localAddresses;
@@ -168,7 +170,7 @@ public class ColumnFamilyRecordReader ex
}
}
}
-throw new UnsupportedOperationException(no local connection
available);
+return split.getLocations()[0];
}
private class RowIterator extends AbstractIteratorPairByteBuffer,
SortedMapByteBuffer, IColumn
svn commit: r1170335 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/hadoop/
Author: jbellis Date: Tue Sep 13 20:29:05 2011 New Revision: 1170335 URL: http://svn.apache.org/viewvc?rev=1170335view=rev Log: merge #2388 revert from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:29:05 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1170309 +/cassandra/branches/cassandra-0.7:1026516-1170333 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/branches/cassandra-0.8:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0:1125021-1130369 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1170335r1=1170334r2=1170335view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Tue Sep 13 20:29:05 2011 @@ -16,6 +16,7 @@ * Avoid streaming data to nodes that already have it, on move as well as decommission (CASSANDRA-3041) * Fix divide by zero error in GCInspector + * revert CASSANDRA-2388 0.8.5 @@ -42,7 +43,7 @@ * prune index scan resultset back to original request for lazy resultset expansion case (CASSANDRA-2964) * (Hadoop) fail jobs when Cassandra node has failed but TaskTracker -has not (CASSANDRA-2388) + has not (CASSANDRA-2388) * fix dynamic snitch ignoring nodes when read_repair_chance is zero (CASSANDRA-2662) * avoid retaining references to dropped CFS objects in Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:29:05 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1170309 +/cassandra/branches/cassandra-0.7/contrib:1026516-1170333 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:29:05 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1170309 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1170333 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125041 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:29:05 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1170309
svn commit: r1170336 - in /cassandra/branches/cassandra-1.0.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/hadoop/
Author: jbellis Date: Tue Sep 13 20:34:42 2011 New Revision: 1170336 URL: http://svn.apache.org/viewvc?rev=1170336view=rev Log: merge #2388 revert from 0.8 Modified: cassandra/branches/cassandra-1.0.0/ (props changed) cassandra/branches/cassandra-1.0.0/CHANGES.txt cassandra/branches/cassandra-1.0.0/contrib/ (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java Propchange: cassandra/branches/cassandra-1.0.0/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:34:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7:1026516-1170309 +/cassandra/branches/cassandra-0.7:1026516-1170333 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1170318 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1170336r1=1170335r2=1170336view=diff == --- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Tue Sep 13 20:34:42 2011 @@ -92,6 +92,7 @@ * Avoid streaming data to nodes that already have it, on move as well as decommission (CASSANDRA-3041) * Fix divide by zero error in GCInspector + * revert CASSANDRA-2388 0.8.5 @@ -116,7 +117,7 @@ * prune index scan resultset back to original request for lazy resultset expansion case (CASSANDRA-2964) * (Hadoop) fail jobs when Cassandra node has failed but TaskTracker -has not (CASSANDRA-2388) + has not (CASSANDRA-2388) * fix dynamic snitch ignoring nodes when read_repair_chance is zero (CASSANDRA-2662) * avoid retaining references to dropped CFS objects in Propchange: cassandra/branches/cassandra-1.0.0/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:34:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1170309 +/cassandra/branches/cassandra-0.7/contrib:1026516-1170333 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1170318 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 20:34:42 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1163782,1170147 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1170333 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1166782,1166809,1169823,1170148,1170219,1170236 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
svn commit: r1170341 [2/2] - in /cassandra/branches/cassandra-1.0: ./ conf/ contrib/ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/ src/
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/dht/BootStrapper.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/dht/BootStrapper.java?rev=1170341r1=1170340r2=1170341view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/dht/BootStrapper.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/dht/BootStrapper.java
Tue Sep 13 20:56:44 2011
@@ -165,7 +165,12 @@ public class BootStrapper
}
if (endpoints.isEmpty())
-throw new RuntimeException(No other nodes seen! Unable to
bootstrap);
+throw new RuntimeException(No other nodes seen! Unable to
bootstrap.
+ + If you intended to start a
single-node cluster, you should make sure
+ + your broadcast_address (or
listen_address) is listed as a seed.
+ + Otherwise, you need to determine why
the seed being contacted
+ + has no knowledge of the rest of the
cluster. Usually, this can be solved
+ + by giving all nodes the same seed
list.);
Collections.sort(endpoints, new ComparatorInetAddress()
{
public int compare(InetAddress ia1, InetAddress ia2)
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java?rev=1170341r1=1170340r2=1170341view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
Tue Sep 13 20:56:44 2011
@@ -143,6 +143,8 @@ public class ColumnFamilyRecordReader ex
return true;
}
+// we don't use endpointsnitch since we are trying to support hadoop nodes
that are
+// not necessarily on Cassandra machines, too. This should be adequate
for single-DC clusters, at least.
private String getLocation()
{
InetAddress[] localAddresses;
@@ -173,7 +175,7 @@ public class ColumnFamilyRecordReader ex
}
}
}
-throw new UnsupportedOperationException(no local connection
available);
+return split.getLocations()[0];
}
private class RowIterator extends AbstractIteratorPairByteBuffer,
SortedMapByteBuffer, IColumn
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java?rev=1170341r1=1170340r2=1170341view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Tue Sep 13 20:56:44 2011
@@ -67,6 +67,8 @@ public class IncomingTcpConnection exten
int header = input.readInt();
isStream = MessagingService.getBits(header, 3, 1) == 1;
version = MessagingService.getBits(header, 15, 8);
+if (logger.isDebugEnabled())
+logger.debug(Version for + socket.getInetAddress() + is
+ version);
if (isStream)
{
if (version == MessagingService.version_)
@@ -98,6 +100,7 @@ public class IncomingTcpConnection exten
else if (msg != null)
{
Gossiper.instance.setVersion(msg.getFrom(), version);
+logger.debug(set version for {} to {},
socket.getInetAddress(), version);
}
// loop to get the next message.
@@ -108,6 +111,7 @@ public class IncomingTcpConnection exten
header = input.readInt();
assert isStream == (MessagingService.getBits(header, 3, 1) ==
1) : Connections cannot change type: + isStream;
version = MessagingService.getBits(header, 15, 8);
+logger.debug(Version is now {}, version);
receiveMessage(input, version);
}
}
@@ -153,7 +157,7 @@ public class IncomingTcpConnection exten
MessagingService.instance().receive(message, id);
return message;
}
-logger.info(Received connection from newer protocol version.
Ignorning message.);
+logger.debug(Received connection from newer protocol version {}.
svn commit: r1170342 - in /cassandra/branches/cassandra-1.0.0: CHANGES.txt src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
Author: jbellis
Date: Tue Sep 13 20:59:07 2011
New Revision: 1170342
URL: http://svn.apache.org/viewvc?rev=1170342view=rev
Log:
copy buffers from mmap'd source so we don't segfault if the source is unmapped
before we're done with it
patch by jbellis; tested by brandonwilliams for CASSANDRA-3179
Modified:
cassandra/branches/cassandra-1.0.0/CHANGES.txt
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1170342r1=1170341r2=1170342view=diff
==
--- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Tue Sep 13 20:59:07 2011
@@ -14,7 +14,7 @@
* optimize away seek when compacting wide rows (CASSANDRA-2879)
* single-pass streaming (CASSANDRA-2677, 3003)
* use reference counting for deleting sstables instead of relying on GC
- (CASSANDRA-2521)
+ (CASSANDRA-2521, 3179)
* store hints as serialized mutations instead of pointers to data row
(CASSANDRA-2045)
* store hints in the coordinator node instead of in the closest replica
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java?rev=1170342r1=1170341r2=1170342view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
Tue Sep 13 20:59:07 2011
@@ -25,6 +25,8 @@ import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
+import org.apache.cassandra.utils.ByteBufferUtil;
+
public class MappedFileDataInput extends AbstractDataInput implements
FileDataInput
{
private final MappedByteBuffer buffer;
@@ -117,11 +119,18 @@ public class MappedFileDataInput extends
throw new IOException(String.format(mmap segment underflow;
remaining is %d but %d requested,
remaining, length));
+if (length == 0)
+return ByteBufferUtil.EMPTY_BYTE_BUFFER;
+
ByteBuffer bytes = buffer.duplicate();
bytes.position(buffer.position() + position).limit(buffer.position() +
position + length);
position += length;
-return bytes;
+// we have to copy the data in case we unreference the underlying
sstable. See CASSANDRA-3179
+ByteBuffer clone = ByteBuffer.allocate(bytes.remaining());
+clone.put(bytes);
+clone.flip();
+return clone;
}
@Override
[jira] [Commented] (CASSANDRA-3188) cqlsh 2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103977#comment-13103977 ] paul cannon commented on CASSANDRA-3188: I'm super -1 on making mx4j an official dependency, btw. Would not want for it to become the de facto standard for non-java jmx access, when it's really only useful with a browser. Screen scraping sucks. Maybe some more appropriate jmx adapter, though. cqlsh 2.0 - Key: CASSANDRA-3188 URL: https://issues.apache.org/jira/browse/CASSANDRA-3188 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: paul cannon Fix For: 1.0.1 I'd like to see some improvements to cqlsh to bring it to feature parity w/ the old cli: - describe [KS | CF | cluster | schema] - assume - connect / don't crap out w/ stacktrace if C* isn't currently running on localhost - help It may be easier to do this by forking the cli and replacing its one-off api with CQL, or it may be easier to add these features to cqlsh. Either is fine, but if it's a close call my inclination would be to build it in Java for the reasoning over on CASSANDRA-3010. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1170347 - in /cassandra/branches/cassandra-1.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/io/util/
Author: jbellis Date: Tue Sep 13 21:02:53 2011 New Revision: 1170347 URL: http://svn.apache.org/viewvc?rev=1170347view=rev Log: merge #3179 from 1.0.0 Modified: cassandra/branches/cassandra-1.0/ (props changed) cassandra/branches/cassandra-1.0/CHANGES.txt cassandra/branches/cassandra-1.0/contrib/ (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java Propchange: cassandra/branches/cassandra-1.0/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:02:53 2011 @@ -5,7 +5,7 @@ /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/branches/cassandra-1.0:1167106,1167185 -/cassandra/branches/cassandra-1.0.0:1167104-1170340 +/cassandra/branches/cassandra-1.0.0:1167104-1170344 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 /cassandra/trunk:1167085-1167102,1169870 Modified: cassandra/branches/cassandra-1.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1170347r1=1170346r2=1170347view=diff == --- cassandra/branches/cassandra-1.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0/CHANGES.txt Tue Sep 13 21:02:53 2011 @@ -18,7 +18,7 @@ * optimize away seek when compacting wide rows (CASSANDRA-2879) * single-pass streaming (CASSANDRA-2677, 3003) * use reference counting for deleting sstables instead of relying on GC - (CASSANDRA-2521) + (CASSANDRA-2521, 3179) * store hints as serialized mutations instead of pointers to data row (CASSANDRA-2045) * store hints in the coordinator node instead of in the closest replica Propchange: cassandra/branches/cassandra-1.0/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:02:53 2011 @@ -5,7 +5,7 @@ /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/branches/cassandra-1.0/contrib:1167106,1167185 -/cassandra/branches/cassandra-1.0.0/contrib:1167104-1170340 +/cassandra/branches/cassandra-1.0.0/contrib:1167104-1170344 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 /cassandra/trunk/contrib:1167085-1167102,1169870 Propchange: cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:02:53 2011 @@ -5,7 +5,7 @@ /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167106,1167185 -/cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1170340 +/cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1170344 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1102511-1125020 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1167102,1169870 Propchange: cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:02:53 2011 @@ -5,7 +5,7 @@ /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1101014-1125018
[jira] [Resolved] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3179. --- Resolution: Fixed Reviewer: brandon.williams Assignee: Jonathan Ellis (was: Sylvain Lebresne) committed v2 JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103985#comment-13103985 ] Jonathan Ellis commented on CASSANDRA-3179: --- bq. SSTableReader.releaseReference() could see a refcount of 2, and not call the dfile.cleanup() That means a read thread has a reference still, so cleanup will run when THAT thread releases it. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1170350 [2/2] - in /cassandra/trunk: ./ conf/ contrib/ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/ src/java/org/apache/c
Modified: cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java?rev=1170350r1=1170349r2=1170350view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java Tue Sep
13 21:08:48 2011
@@ -165,7 +165,12 @@ public class BootStrapper
}
if (endpoints.isEmpty())
-throw new RuntimeException(No other nodes seen! Unable to
bootstrap);
+throw new RuntimeException(No other nodes seen! Unable to
bootstrap.
+ + If you intended to start a
single-node cluster, you should make sure
+ + your broadcast_address (or
listen_address) is listed as a seed.
+ + Otherwise, you need to determine why
the seed being contacted
+ + has no knowledge of the rest of the
cluster. Usually, this can be solved
+ + by giving all nodes the same seed
list.);
Collections.sort(endpoints, new ComparatorInetAddress()
{
public int compare(InetAddress ia1, InetAddress ia2)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java?rev=1170350r1=1170349r2=1170350view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
Tue Sep 13 21:08:48 2011
@@ -143,6 +143,8 @@ public class ColumnFamilyRecordReader ex
return true;
}
+// we don't use endpointsnitch since we are trying to support hadoop nodes
that are
+// not necessarily on Cassandra machines, too. This should be adequate
for single-DC clusters, at least.
private String getLocation()
{
InetAddress[] localAddresses;
@@ -173,7 +175,7 @@ public class ColumnFamilyRecordReader ex
}
}
}
-throw new UnsupportedOperationException(no local connection
available);
+return split.getLocations()[0];
}
private class RowIterator extends AbstractIteratorPairByteBuffer,
SortedMapByteBuffer, IColumn
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java?rev=1170350r1=1170349r2=1170350view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
Tue Sep 13 21:08:48 2011
@@ -25,6 +25,8 @@ import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
+import org.apache.cassandra.utils.ByteBufferUtil;
+
public class MappedFileDataInput extends AbstractDataInput implements
FileDataInput
{
private final MappedByteBuffer buffer;
@@ -117,11 +119,18 @@ public class MappedFileDataInput extends
throw new IOException(String.format(mmap segment underflow;
remaining is %d but %d requested,
remaining, length));
+if (length == 0)
+return ByteBufferUtil.EMPTY_BYTE_BUFFER;
+
ByteBuffer bytes = buffer.duplicate();
bytes.position(buffer.position() + position).limit(buffer.position() +
position + length);
position += length;
-return bytes;
+// we have to copy the data in case we unreference the underlying
sstable. See CASSANDRA-3179
+ByteBuffer clone = ByteBuffer.allocate(bytes.remaining());
+clone.put(bytes);
+clone.flip();
+return clone;
}
@Override
Modified:
cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java?rev=1170350r1=1170349r2=1170350view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Tue Sep 13 21:08:48 2011
@@ -67,6 +67,8 @@ public class IncomingTcpConnection exten
int header = input.readInt();
isStream = MessagingService.getBits(header,
svn commit: r1170360 - in /cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra: concurrent/DebuggableThreadPoolExecutor.java net/IncomingTcpConnection.java
Author: jbellis
Date: Tue Sep 13 21:38:33 2011
New Revision: 1170360
URL: http://svn.apache.org/viewvc?rev=1170360view=rev
Log:
avoid logging spurious errors in tests due to gossip task shutdown race
patch by jbellis for CASSANDRA-3193
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java?rev=1170360r1=1170359r2=1170360view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
Tue Sep 13 21:38:33 2011
@@ -126,6 +126,10 @@ public class DebuggableThreadPoolExecuto
{
throw new AssertionError(e);
}
+catch (CancellationException e)
+{
+logger.debug(Task cancelled, e);
+}
catch (ExecutionException e)
{
if (Thread.getDefaultUncaughtExceptionHandler() == null)
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java?rev=1170360r1=1170359r2=1170360view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/net/IncomingTcpConnection.java
Tue Sep 13 21:38:33 2011
@@ -22,6 +22,7 @@ package org.apache.cassandra.net;
import java.io.*;
+import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
@@ -41,11 +42,13 @@ public class IncomingTcpConnection exten
private static final int CHUNK_SIZE = 1024 * 1024;
private Socket socket;
+public InetAddress from;
public IncomingTcpConnection(Socket socket)
{
assert socket != null;
this.socket = socket;
+from = socket.getInetAddress(); // maximize chance of this not being
nulled by disconnect
}
/**
@@ -67,8 +70,7 @@ public class IncomingTcpConnection exten
int header = input.readInt();
isStream = MessagingService.getBits(header, 3, 1) == 1;
version = MessagingService.getBits(header, 15, 8);
-if (logger.isDebugEnabled())
-logger.debug(Version for + socket.getInetAddress() + is
+ version);
+logger.debug(Version for {} is {}, from, version);
if (isStream)
{
if (version == MessagingService.version_)
@@ -94,13 +96,13 @@ public class IncomingTcpConnection exten
if (version MessagingService.version_)
{
// save the endpoint so gossip will reconnect to it
-Gossiper.instance.addSavedEndpoint(socket.getInetAddress());
+Gossiper.instance.addSavedEndpoint(from);
logger.info(Received + (isStream ? streaming : ) +
connection from newer protocol version. Ignorning);
}
else if (msg != null)
{
Gossiper.instance.setVersion(msg.getFrom(), version);
-logger.debug(set version for {} to {},
socket.getInetAddress(), version);
+logger.debug(set version for {} to {}, from, version);
}
// loop to get the next message.
@@ -164,7 +166,8 @@ public class IncomingTcpConnection exten
private void close()
{
// reset version here, since we set when starting an incoming socket
-Gossiper.instance.resetVersion(socket.getInetAddress());
+if (from != null)
+Gossiper.instance.resetVersion(from);
try
{
socket.close();
svn commit: r1170362 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/concurrent/ src/java/org/apache/cassandra/net/
Author: jbellis Date: Tue Sep 13 21:40:40 2011 New Revision: 1170362 URL: http://svn.apache.org/viewvc?rev=1170362view=rev Log: merge #3193 from 1.0 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java cassandra/trunk/src/java/org/apache/cassandra/net/IncomingTcpConnection.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:40:40 2011 @@ -4,8 +4,8 @@ /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1170347 -/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1170344 +/cassandra/branches/cassandra-1.0:1167085-1170361 +/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1170360 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:40:40 2011 @@ -4,8 +4,8 @@ /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1170347 -/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1170344 +/cassandra/branches/cassandra-1.0/contrib:1167085-1170361 +/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1170360 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:40:40 2011 @@ -4,8 +4,8 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1170347 -/cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1167229,1167232-1170344 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1170361 +/cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1167229,1167232-1170360 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1102511-1125020 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Sep 13 21:40:40 2011 @@ -4,8 +4,8 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1170335 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167085-1170347
[jira] [Created] (CASSANDRA-3203) Odd flush behavior
Odd flush behavior
--
Key: CASSANDRA-3203
URL: https://issues.apache.org/jira/browse/CASSANDRA-3203
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Reporter: Brandon Williams
Priority: Critical
Fix For: 1.0.0
Given the same workload against 0.8, trunk is creating more than twice the
amount of sstables. Even though a uniform stress workload is being generated,
flush size degrades quickly:
{noformat}
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:22,878 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@2058235391(7741
035/110172631 serialized/live bytes, 151785 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:24,888 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@1520390052(3887
220/72403158 serialized/live bytes, 76220 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:26,890 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@1868496516(4097
085/76255481 serialized/live bytes, 80335 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:28,893 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@498232521(43513
20/80922269 serialized/live bytes, 85320 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:29,895 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@1592308290(2310
810/44514839 serialized/live bytes, 45310 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:30,897 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@775439677(22684
80/64984390 serialized/live bytes, 44480 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:31,899 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@928217914(26741
85/76231422 serialized/live bytes, 52435 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:32,901 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@158103119(27511
95/77317732 serialized/live bytes, 53945 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:33,903 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@2035169258(3132
420/88934701 serialized/live bytes, 61420 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:34,905 ColumnFamilyStore.java (line
658) Enqueuing flush of Memtable-Standard1@1097314626(2979
675/83651699 serialized/live bytes, 58425 ops)
{noformat}
The serialized to live size ratio appears completely out of whack.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3181) Compaction fails to occur
[ https://issues.apache.org/jira/browse/CASSANDRA-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3181: -- Attachment: 3181.txt Attached patch does a couple things: - Schedules a compaction submission for each CFS 5 minutes after startup. My reasoning is, five minutes is (a) enough time for most caches to warm up under load and (b) when it is not, at least it is enough time to reduce the compaction i/o limit. - removes the permanent check-for-compactions-every-3s task from leveled compaction; I don't like spinning that up for no reason, when we already kick off a check on each flush and end-of-compaction, which should be adequate. (Every 3s for 1 CFS = every 0.0003s for 10K CFS.) - makes Leveled getMaximal return a normal, leveling compaction, if any needs to be done, allowing users of leveldb compaction to kick things off earlier than 5m via nodetool compact, if desired Compaction fails to occur - Key: CASSANDRA-3181 URL: https://issues.apache.org/jira/browse/CASSANDRA-3181 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Brandon Williams Assignee: Benjamin Coverston Labels: compaction Fix For: 1.0.0 Attachments: 3181.txt Compaction just stops running at some point. To repro, insert like 20M rows with a 1G heap and you'll get around 1k sstables. Restarting doesn't help, you have to invoke a major to get anything to happen. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104035#comment-13104035 ] Yang Yang commented on CASSANDRA-3179: -- thanks. unfortunately I got the SEGV again, after updating to the latest version. I tried to confirm so ran it again, so far seen 2 times. INFO 14:49:57,806 Compacting Minor: [SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h -76-Data.db'), SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-77-Data.db'), SSTableR eader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-79-Data.db'), SSTableReader(path='/usr/scratc h/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-78-Data.db')] 9135.963: [GC 9135.963: [ParNew: 2550601K-133528K(2764800K), 1.2821340 secs] 5163440K-2756799K(5836800K) icms_dc=0 , 1.2824040 secs] [Times: user=1.06 sys=0.04, real=1.28 secs] INFO 14:50:07,017 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/measuredSession-h-88-Data.db ( 21435483 bytes) INFO 14:50:07,018 Writing Memtable-session_limit_filter@839347940(21205810/281849841 serialized/live bytes, 286565 ops) INFO 14:50:15,987 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/session_limit_filter-h-88-Data .db (23326502 bytes) INFO 14:50:15,999 Writing Memtable-ad_impression_session@100750437(21205662/296528055 serialized/live bytes, 286563 ops) 9154.456: [GC 9154.456: [ParNew: 2591128K-125067K(2764800K), 0.2846910 secs] 5214399K-2756607K(5836800K) icms_dc=0 , 0.2849470 secs] [Times: user=0.71 sys=0.01, real=0.29 secs] INFO 14:50:26,220 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/ad_impression_session-h-89-Data.db (52154466 bytes) INFO 14:50:26,221 Writing Memtable-ad_ip_agent@1095480823(21205662/286042295 serialized/live bytes, 286563 ops) INFO 14:50:26,463 Compacted to [/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-80-Data.db,]. 94,782,508 to 83,577,108 (~88% of original) bytes for 6,368 keys at 2.783688MBPS. Time: 28,633ms. INFO 14:50:26,464 CF Total Bytes Compacted: 3,494,202,400 # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2b7b9c88, pid=6461, tid=1253022016 # # JRE version: 7.0-b147 # Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode linux-amd64 compressed oops) # Problematic frame: # J org.apache.cassandra.io.util.AbstractDataInput.readByte()B # # Core dump written. Default location: /usr/scratch/yyang/whisky/core or core.6461 attaching the err file separately JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated CASSANDRA-3179: - Attachment: hs_err_pid6461.log failure log after using updated code JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104043#comment-13104043
]
Mikko Koppanen commented on CASSANDRA-3025:
---
Hi,
didn't remove it yet but I will do it now.
PHP/PDO driver for Cassandra CQL
Key: CASSANDRA-3025
URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Mikko Koppanen
Labels: php
Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz,
pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, pdo_cassandra-0.2.0.tgz,
php_test_results_20110818_2317.txt
Hello,
attached is the initial version of the PDO driver for Cassandra CQL language.
This is a native PHP extension written in what I would call a combination of
C and C++, due to PHP being C. The thrift API used is the C++.
The API looks roughly following:
{code}
?php
$db = new PDO('cassandra:host=127.0.0.1;port=9160');
$db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor=1;);
$db-exec (USE mytest);
$db-exec (CREATE COLUMNFAMILY users (
my_key varchar PRIMARY KEY,
full_name varchar ););
$stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key,
:full_name););
$stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
{code}
Currently prepared statements are emulated on the client side but I
understand that there is a plan to add prepared statements to Cassandra CQL
API as well. I will add this feature in to the extension as soon as they are
implemented.
Additional documentation can be found in github
https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered
MarkDown file. Tests are currently not included in the package file and they
can be found in the github for now as well.
I have created documentation in docbook format as well, but have not yet
rendered it.
Comments and feedback are welcome.
Thanks,
Mikko
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikko Koppanen updated CASSANDRA-3025:
--
Attachment: pdo_cassandra-0.2.1.tgz
Adding here for reference. Rowset iterators removed.
PHP/PDO driver for Cassandra CQL
Key: CASSANDRA-3025
URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Mikko Koppanen
Labels: php
Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz,
pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, pdo_cassandra-0.2.0.tgz,
pdo_cassandra-0.2.1.tgz, php_test_results_20110818_2317.txt
Hello,
attached is the initial version of the PDO driver for Cassandra CQL language.
This is a native PHP extension written in what I would call a combination of
C and C++, due to PHP being C. The thrift API used is the C++.
The API looks roughly following:
{code}
?php
$db = new PDO('cassandra:host=127.0.0.1;port=9160');
$db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor=1;);
$db-exec (USE mytest);
$db-exec (CREATE COLUMNFAMILY users (
my_key varchar PRIMARY KEY,
full_name varchar ););
$stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key,
:full_name););
$stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
{code}
Currently prepared statements are emulated on the client side but I
understand that there is a plan to add prepared statements to Cassandra CQL
API as well. I will add this feature in to the extension as soon as they are
implemented.
Additional documentation can be found in github
https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered
MarkDown file. Tests are currently not included in the package file and they
can be found in the github for now as well.
I have created documentation in docbook format as well, but have not yet
rendered it.
Comments and feedback are welcome.
Thanks,
Mikko
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104055#comment-13104055 ] Brandon Williams commented on CASSANDRA-3179: - Can you reproduce w/o compressed oops? JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3181) Compaction fails to occur
[ https://issues.apache.org/jira/browse/CASSANDRA-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104059#comment-13104059 ] Benjamin Coverston commented on CASSANDRA-3181: --- Was a little concerned about removing the scheduled compaction from leveldb, but the mechanics are really no different from the tiered compaction in terms of it will stop when it's finished assuming that nothing goes wrong with the running compactions. To be (probably overly) pedantic another advantage this has is that you are essentially kicking off only a single compaction where when the server was brought down there were probably multiple compactions in flight. +1 Compaction fails to occur - Key: CASSANDRA-3181 URL: https://issues.apache.org/jira/browse/CASSANDRA-3181 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Brandon Williams Assignee: Jonathan Ellis Labels: compaction Fix For: 1.0.0 Attachments: 3181.txt Compaction just stops running at some point. To repro, insert like 20M rows with a 1G heap and you'll get around 1k sstables. Restarting doesn't help, you have to invoke a major to get anything to happen. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104061#comment-13104061 ] Yang Yang commented on CASSANDRA-3179: -- let me try ... On Tue, Sep 13, 2011 at 3:47 PM, Brandon Williams (JIRA) JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3198) debian packaging installation problem when installing for the first time
[ https://issues.apache.org/jira/browse/CASSANDRA-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shyamal Prasad updated CASSANDRA-3198: -- Attachment: debian-postinst-fixperms.patch I believe this is the correct patch. It sets permissions on first install, or failing that, when upgrading from earlier than 0.6.4-2 per CASSANDRA-1004 debian packaging installation problem when installing for the first time Key: CASSANDRA-3198 URL: https://issues.apache.org/jira/browse/CASSANDRA-3198 Project: Cassandra Issue Type: Bug Components: Packaging Affects Versions: 0.8.5 Reporter: Jérémy Sevellec Assignee: Jérémy Sevellec Fix For: 0.8.6 Attachments: debian-postinst-fixperms.patch, trunk-3198-v1.patch when installing cassandra through the debian packaging for the first time, there is permission problem when starting Cassandra. Normally, the postinst script change owner of /var/log/cassandra and /var/lib/cassandra from root to cassandra user. there is a problem with the test which verify if threre is a need to change the owner of these directory or not. On a new install, the $2 parameter is not set and the the test is false and the owner is not changed. (simply, i think replace with || might work) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104106#comment-13104106 ] Yang Yang commented on CASSANDRA-3179: -- confirmed with the -XX:-UseCompressedOops arg, has generated 1 SEGV so far btw, it would be really helpful to add a simple dumb stress test testcase, some bugs are not easily exposed with the unit tests. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104110#comment-13104110 ] Brandon Williams commented on CASSANDRA-3179: - What process are you following to reproduce? I can't do it with compaction+reads, but I'm also not using JRE 7. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104119#comment-13104119 ] Yang Yang commented on CASSANDRA-3179: -- I'm just writing + reading on the cassandra server, and doing this on the side: while : ;do nodetool flush ; sleep 20;done (disclaimer: the cassandra server is a modified one, where I have a thread that takes in client request through a custom AVRO server, and then call the storageProxy.batch_mutate() and StorageProxy.get() directly. it should not be materially different from the pure cassandra server as far as this bug is concerned, since it's basically just swapping out the thrift server ) On Tue, Sep 13, 2011 at 5:42 PM, Brandon Williams (JIRA) JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3203) Odd flush behavior
[
https://issues.apache.org/jira/browse/CASSANDRA-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104124#comment-13104124
]
Brandon Williams commented on CASSANDRA-3203:
-
Here is what I've noticed:
Things proceed normally for a while:
{noformat}
INFO 00:50:16,801 Enqueuing flush of
Memtable-Standard1@304913075(36443580/45554475 serialized/live bytes, 714580
ops)
{noformat}
Eventually, something bad happens:
{noformat}
INFO 00:51:00,001 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Counter1') (estimated 0
bytes)
INFO 00:51:00,002 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Super1') (estimated 0 bytes)
INFO 00:51:00,002 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='SuperCounter1') (estimated 0
bytes)
INFO 00:51:00,002 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Standard1') (estimated
88291581 bytes)
INFO 00:51:00,004 Enqueuing flush of
Memtable-Standard1@741697653(2646900/88291581 serialized/live bytes, 51900 ops)
INFO 00:51:00,004 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='NodeIdInfo') (estimated 0 bytes)
INFO 00:51:00,004 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='IndexInfo') (estimated 0 bytes)
INFO 00:51:00,004 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='LocationInfo') (estimated 87
bytes)
INFO 00:51:00,005 Enqueuing flush of Memtable-LocationInfo@1706933590(70/87
serialized/live bytes, 2 ops)
INFO 00:51:00,005 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='Migrations') (estimated 0 bytes)
INFO 00:51:00,006 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='HintsColumnFamily') (estimated
0 bytes)
INFO 00:51:00,006 flushing high-traffic column family
ColumnFamilyStore(table='system', columnFamily='Schema') (estimated 0 bytes)
INFO 00:51:00,006 estimated 0 bytes used by all memtables pre-flush
INFO 00:51:00,008 flushing ColumnFamilyStore(table='Keyspace1',
columnFamily='Standard1') to free up 459320 bytes
INFO 00:51:00,009 Enqueuing flush of Memtable-Standard1@456936648(14280/476332
serialized/live bytes, 280 ops)
INFO 00:51:00,010 flushing ColumnFamilyStore(table='system',
columnFamily='Schema') to free up 0 bytes
INFO 00:51:00,010 flushing ColumnFamilyStore(table='system',
columnFamily='HintsColumnFamily') to free up 0 bytes
INFO 00:51:00,011 flushing ColumnFamilyStore(table='system',
columnFamily='Migrations') to free up 0 bytes
INFO 00:51:00,011 flushing ColumnFamilyStore(table='system',
columnFamily='LocationInfo') to free up 0 bytes
INFO 00:51:00,012 flushing ColumnFamilyStore(table='system',
columnFamily='IndexInfo') to free up 0 bytes
INFO 00:51:00,013 flushing ColumnFamilyStore(table='system',
columnFamily='NodeIdInfo') to free up 0 bytes
INFO 00:51:00,013 flushing ColumnFamilyStore(table='Keyspace1',
columnFamily='SuperCounter1') to free up 0 bytes
INFO 00:51:00,014 flushing ColumnFamilyStore(table='Keyspace1',
columnFamily='Super1') to free up 0 bytes
INFO 00:51:00,014 flushing ColumnFamilyStore(table='Keyspace1',
columnFamily='Counter1') to free up 0 bytes
INFO 00:51:01,016 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Counter1') (estimated 0
bytes)
INFO 00:51:01,016 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Super1') (estimated 0 bytes)
INFO 00:51:01,016 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='SuperCounter1') (estimated 0
bytes)
INFO 00:51:01,016 flushing high-traffic column family
ColumnFamilyStore(table='Keyspace1', columnFamily='Standard1') (estimated
83808954 bytes)
INFO 00:51:01,017 Enqueuing flush of
Memtable-Standard1@911886300(2512770/83817460 serialized/live bytes, 49270 ops)
{noformat}
After this point, it stays bad:
{noformat}
INFO 00:51:01,879 Enqueuing flush of
Memtable-Standard1@937412989(2670870/96531021 serialized/live bytes, 52370 ops)
INFO 00:51:02,029 Enqueuing flush of
Memtable-Standard1@282282499(2668065/88997573 serialized/live bytes, 52315 ops)
{noformat}
Odd flush behavior
--
Key: CASSANDRA-3203
URL: https://issues.apache.org/jira/browse/CASSANDRA-3203
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Reporter: Brandon Williams
Priority: Critical
Fix For: 1.0.0
Given the same workload against 0.8, trunk is creating more than twice the
amount of sstables. Even though a uniform stress workload is being
generated, flush size degrades quickly:
{noformat}
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104140#comment-13104140 ] Jonathan Ellis commented on CASSANDRA-3179: --- You're sure you're on 1.0.0 = r1170342? JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104141#comment-13104141 ] Yang Yang commented on CASSANDRA-3179: -- very interesting, I switched from mmap mode to standard, and go the following errors on compaction, I believe they follow the same path as the mmap route, but since it gave an exception here, instead of siliently SEGV, this could provide a useful hint to what caused the SEGV Caused by: java.nio.channels.ClosedChannelException at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268) at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640) at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356) at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79) ... 21 more ERROR 01:19:22,415 Fatal exception in thread Thread[ReadStage:246,5,main] java.lang.RuntimeException: java.lang.RuntimeException: error reading 1 of 1 at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1165) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.RuntimeException: error reading 1 of 1 at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107) at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:194) at org.apache.cassandra.utils.MergeIterator.init(MergeIterator.java:47) at org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:142) at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:66) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:96) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:249) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:61) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1276) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1171) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1127) at org.apache.cassandra.db.Table.getRow(Table.java:388) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:61) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:694) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1161) ... 3 more Caused by: java.nio.channels.ClosedChannelException at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268) at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640) at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356) at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79) ... 21 more INFO 01:19:23,738 Compacted to [/mnt/cass/lib/cassandra/data/testBudget_items/measuredSession-h-10-Data.db,]. 16,840,264 to 16,840,036 (~99% of original) bytes for 22,531 keys at 3.247707MBPS. Time: 4,945ms. INFO 01:19:23,738 CF Total Bytes Compacted:
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104143#comment-13104143 ] Yang Yang commented on CASSANDRA-3179: -- sorry for the misunderstanding. I applied the v2 patch on 4dbda612f49c97fd5e3f66e7875a20ec9a0dc829 (Aug 26 version) I could test the latest one if you think that's going to be different. On Tue, Sep 13, 2011 at 6:22 PM, Jonathan Ellis (JIRA) j...@apache.org wrote: JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1170408 - in /cassandra/branches/cassandra-1.0.0: CHANGES.txt NEWS.txt src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java src/java/org/apache/cassandra/db/compact
Author: jbellis
Date: Wed Sep 14 01:27:16 2011
New Revision: 1170408
URL: http://svn.apache.org/viewvc?rev=1170408view=rev
Log:
Add five-minute delay before starting compactions on a restarted server
patch by jbellis; reviewed by bcoverston for CASSANDRA-3181
Modified:
cassandra/branches/cassandra-1.0.0/CHANGES.txt
cassandra/branches/cassandra-1.0.0/NEWS.txt
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1170408r1=1170407r2=1170408view=diff
==
--- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original)
+++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Wed Sep 14 01:27:16 2011
@@ -72,6 +72,8 @@
* Add RowLevel support to secondary index API (CASSANDRA-3147)
* Make SerializingCacheProvider the default if JNA is available
(CASSANDRA-3183)
* Fix backwards compatibilty for CQL memtable properties (CASSANDRA-3190)
+ * Add five-minute delay before starting compactions on a restarted server
+ (CASSANDRA-3181)
0.8.6
Modified: cassandra/branches/cassandra-1.0.0/NEWS.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/NEWS.txt?rev=1170408r1=1170407r2=1170408view=diff
==
--- cassandra/branches/cassandra-1.0.0/NEWS.txt (original)
+++ cassandra/branches/cassandra-1.0.0/NEWS.txt Wed Sep 14 01:27:16 2011
@@ -60,6 +60,10 @@ Other
This does not affect client compatibility -- the old options are
still allowed, but have no effect. These options may be removed
entirely in a future release.
+- Backlogged compactions will begin five minutes after startup. The 0.8
+ behavior of never starting compaction until a flush happens is usually
+ not what is desired, but a short grace period is useful to allow caches
+ to warm up first.
0.8.5
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java?rev=1170408r1=1170407r2=1170408view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
Wed Sep 14 01:27:16 2011
@@ -18,21 +18,14 @@
package org.apache.cassandra.db.compaction;
-import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
+import java.util.concurrent.TimeUnit;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.db.Table;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.io.sstable.SSTableReader;
-import org.apache.cassandra.utils.DefaultInteger;
-import org.apache.cassandra.utils.Pair;
+import org.apache.cassandra.service.StorageService;
/**
@@ -53,6 +46,19 @@ public abstract class AbstractCompaction
assert cfs != null;
this.cfs = cfs;
this.options = options;
+
+// start compactions in five minutes (if no flushes have occurred by
then to do so)
+Runnable runnable = new Runnable()
+{
+public void run()
+{
+if (CompactionManager.instance.getActiveCompactions() == 0)
+{
+
CompactionManager.instance.submitBackground(AbstractCompactionStrategy.this.cfs);
+}
+}
+};
+StorageService.tasks.schedule(runnable, 5 * 60, TimeUnit.SECONDS);
}
/**
Modified:
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java?rev=1170408r1=1170407r2=1170408view=diff
==
---
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
(original)
+++
cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
Wed Sep 14 01:27:16 2011
@@ -29,24 +29,6 @@ public class LeveledCompactionStrategy e
private final int maxSSTableSize;
private final
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104148#comment-13104148 ] Jonathan Ellis commented on CASSANDRA-3179: --- Please test 1.0.0 branch to make sure we are doing the same thing. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2388) ColumnFamilyRecordReader fails for a given split because a host is down, even if records could reasonably be read from other replica.
[ https://issues.apache.org/jira/browse/CASSANDRA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104179#comment-13104179 ] T Jake Luciani commented on CASSANDRA-2388: --- I just want to confirm what this ticket is about. The JT has a list of endpoints for a given split. When a task runs it may or may not be on one of those nodes If other tasks are running on all those replicas the JT may put them on a remote node. So we need to decide which endpoint to connect to given the chance that nodes are down. 1. Check if the node running CFRR is one of the replicas (we have this) this means JT has assigned a data-local task (good) 2. If none of these nodes are local then pick another. 3. If connection fails try the one other nodes. 4. Try to avoid endpoints in a different DC. The biggest problem is 4. Maybe the way todo this is change getSplits logic to never return replicas in another DC. I think this would require adding DC info to the describe_ring call. Then we only need to worry about 1-3. ColumnFamilyRecordReader fails for a given split because a host is down, even if records could reasonably be read from other replica. - Key: CASSANDRA-2388 URL: https://issues.apache.org/jira/browse/CASSANDRA-2388 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 0.6 Reporter: Eldon Stegall Assignee: Mck SembWever Labels: hadoop, inputformat Fix For: 0.8.6 Attachments: 0002_On_TException_try_next_split.patch, CASSANDRA-2388-addition1.patch, CASSANDRA-2388-extended.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch, CASSANDRA-2388.patch ColumnFamilyRecordReader only tries the first location for a given split. We should try multiple locations for a given split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3204) stress cannot set the compaction strategy
stress cannot set the compaction strategy - Key: CASSANDRA-3204 URL: https://issues.apache.org/jira/browse/CASSANDRA-3204 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.0.0 Reporter: Brandon Williams Assignee: Pavel Yaskevich Fix For: 1.0.0 stress can't set the compaction strategy, so testing leveldb-style compaction is more difficult than it should be, especially with lots of cluster setup/teardown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3203) Odd flush behavior
[
https://issues.apache.org/jira/browse/CASSANDRA-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-3203:
--
Attachment: 3203-prelim.txt
patch that fixes two problems:
- removes double-counting of ByteBuffer overhead in jamm 0.2.4
- removes 25% fudge factor that is no longer useful with Slab allocation
That still leaves us with at least one other problem, though.
Odd flush behavior
--
Key: CASSANDRA-3203
URL: https://issues.apache.org/jira/browse/CASSANDRA-3203
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Reporter: Brandon Williams
Priority: Critical
Fix For: 1.0.0
Attachments: 3203-prelim.txt
Given the same workload against 0.8, trunk is creating more than twice the
amount of sstables. Even though a uniform stress workload is being
generated, flush size degrades quickly:
{noformat}
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:22,878 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@2058235391(7741
035/110172631 serialized/live bytes, 151785 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:24,888 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1520390052(3887
220/72403158 serialized/live bytes, 76220 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:26,890 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1868496516(4097
085/76255481 serialized/live bytes, 80335 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:28,893 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@498232521(43513
20/80922269 serialized/live bytes, 85320 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:29,895 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1592308290(2310
810/44514839 serialized/live bytes, 45310 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:30,897 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@775439677(22684
80/64984390 serialized/live bytes, 44480 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:31,899 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@928217914(26741
85/76231422 serialized/live bytes, 52435 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:32,901 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@158103119(27511
95/77317732 serialized/live bytes, 53945 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:33,903 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@2035169258(3132
420/88934701 serialized/live bytes, 61420 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:34,905 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1097314626(2979
675/83651699 serialized/live bytes, 58425 ops)
{noformat}
The serialized to live size ratio appears completely out of whack.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3203) Odd flush behavior
[
https://issues.apache.org/jira/browse/CASSANDRA-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104258#comment-13104258
]
Brandon Williams commented on CASSANDRA-3203:
-
Something strange is going on here, as this patch actually exacerbated the
problem slightly.
Odd flush behavior
--
Key: CASSANDRA-3203
URL: https://issues.apache.org/jira/browse/CASSANDRA-3203
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Reporter: Brandon Williams
Priority: Critical
Fix For: 1.0.0
Attachments: 3203-prelim.txt
Given the same workload against 0.8, trunk is creating more than twice the
amount of sstables. Even though a uniform stress workload is being
generated, flush size degrades quickly:
{noformat}
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:22,878 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@2058235391(7741
035/110172631 serialized/live bytes, 151785 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:24,888 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1520390052(3887
220/72403158 serialized/live bytes, 76220 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:26,890 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1868496516(4097
085/76255481 serialized/live bytes, 80335 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:28,893 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@498232521(43513
20/80922269 serialized/live bytes, 85320 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:29,895 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1592308290(2310
810/44514839 serialized/live bytes, 45310 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:30,897 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@775439677(22684
80/64984390 serialized/live bytes, 44480 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:31,899 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@928217914(26741
85/76231422 serialized/live bytes, 52435 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:32,901 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@158103119(27511
95/77317732 serialized/live bytes, 53945 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:33,903 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@2035169258(3132
420/88934701 serialized/live bytes, 61420 ops)
INFO [NonPeriodicTasks:1] 2011-09-09 18:24:34,905 ColumnFamilyStore.java
(line 658) Enqueuing flush of Memtable-Standard1@1097314626(2979
675/83651699 serialized/live bytes, 58425 ops)
{noformat}
The serialized to live size ratio appears completely out of whack.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
