[jira] [Updated] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Aymé updated CASSANDRA-5587:
---
Labels: patch (was: )
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Aymé updated CASSANDRA-5587:
---
Attachment: cassandra-1.2-5587.txt
The proposed patch
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663831#comment-13663831
]
Julien Aymé edited comment on CASSANDRA-5587 at 5/22/13 6:24 AM:
-
The proposed patch, made against branch cassandra-1.2
was (Author: julien.a...@gmail.com):
The proposed patch
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Deleted] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Aymé updated CASSANDRA-5587:
---
Comment: was deleted
(was: diff --git a/src/java/org/apache/cassandra/io/sstable/SSTable.java
b/src/java/org/apache/cassandra/io/sstable/SSTable.java
index c7486ba..47b2612 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTable.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTable.java
@@ -191,7 +191,16 @@ public abstract class SSTable
*/
public static PairDescriptor,Component tryComponentFromFilename(File
dir, String name)
{
-return Component.fromFilename(dir, name);
+try
+{
+return Component.fromFilename(dir, name);
+}
+catch (NoSuchElementException e)
+{
+// A NoSuchElementException is thrown if the name does not match
the Descriptor format
+// This is the less impacting change (all calls to this method
test for null return)
+return null;
+}
}
/**
diff --git a/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
b/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
index 007e0ca..9d31d7e 100644
--- a/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
@@ -21,6 +21,7 @@ package org.apache.cassandra.io.sstable;
*/
import org.apache.cassandra.utils.FilterFactory;
+import org.apache.cassandra.utils.Pair;
import org.junit.Test;
import static org.junit.Assert.*;
@@ -64,4 +65,11 @@ public class DescriptorTest
assertEquals(ia, desc.version.toString());
assertEquals(desc.version.filterType, FilterFactory.Type.MURMUR3);
}
+
+@Test
+public void testInvalidnameFormat()
+{
+PairDescriptor, Component p = SSTable.tryComponentFromFilename(null,
snapshot);
+assertNull(p);
+}
}
)
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4693) CQL Protocol should allow multiple PreparedStatements to be atomically executed
[ https://issues.apache.org/jira/browse/CASSANDRA-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4693: Attachment: (was: 0001-Binary-protocol-adds-message-to-batch-prepared-or-not-.txt) CQL Protocol should allow multiple PreparedStatements to be atomically executed --- Key: CASSANDRA-4693 URL: https://issues.apache.org/jira/browse/CASSANDRA-4693 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Michaël Figuière Assignee: Sylvain Lebresne Labels: cql, protocol Fix For: 2.0 Attachments: 0001-Binary-protocol-adds-message-to-batch-prepared-or-not-.txt Currently the only way to insert multiple records on the same partition key, atomically and using PreparedStatements is to use a CQL BATCH command. Unfortunately when doing so the amount of records to be inserted must be known prior to prepare the statement which is rarely the case. Thus the only workaround if one want to keep atomicity is currently to use unprepared statements which send a bulk of CQL strings and is fairly inefficient. Therefore CQL Protocol should allow clients to send multiple PreparedStatements to be executed with similar guarantees and semantic as CQL BATCH command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4693) CQL Protocol should allow multiple PreparedStatements to be atomically executed
[ https://issues.apache.org/jira/browse/CASSANDRA-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4693: Attachment: 0001-Binary-protocol-adds-message-to-batch-prepared-or-not-.txt Rebased version attached. CQL Protocol should allow multiple PreparedStatements to be atomically executed --- Key: CASSANDRA-4693 URL: https://issues.apache.org/jira/browse/CASSANDRA-4693 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Michaël Figuière Assignee: Sylvain Lebresne Labels: cql, protocol Fix For: 2.0 Attachments: 0001-Binary-protocol-adds-message-to-batch-prepared-or-not-.txt Currently the only way to insert multiple records on the same partition key, atomically and using PreparedStatements is to use a CQL BATCH command. Unfortunately when doing so the amount of records to be inserted must be known prior to prepare the statement which is rarely the case. Thus the only workaround if one want to keep atomicity is currently to use unprepared statements which send a bulk of CQL strings and is fairly inefficient. Therefore CQL Protocol should allow clients to send multiple PreparedStatements to be executed with similar guarantees and semantic as CQL BATCH command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663866#comment-13663866
]
Dave Brosius commented on CASSANDRA-5587:
-
the SSTableLoader.openSSTables filenameFilter could immediately ignore
directories before even trying SSTable.tryComponentFromFilename. as well.
ie
if (new File(dir, name).isDirectory()) return false;
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663870#comment-13663870
]
Julien Aymé commented on CASSANDRA-5587:
Yes, this is also a valid solution, but there is one use case I could think of
which would fail with the same symptom: if a user deliberatly adds other files
(like metainfo, ...) in the directory (yes, this is a bad thing to do).
And nothing prevents us from doing both: immediatly ignore directories, and
keep the second safe guard catch NSEE.
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Aymé updated CASSANDRA-5587:
---
Attachment: cassandra-1.2-5587.txt
Updated patch: both checks are done
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5587) BulkLoader fails with NoSuchElementException
[
https://issues.apache.org/jira/browse/CASSANDRA-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Aymé updated CASSANDRA-5587:
---
Attachment: (was: cassandra-1.2-5587.txt)
BulkLoader fails with NoSuchElementException
Key: CASSANDRA-5587
URL: https://issues.apache.org/jira/browse/CASSANDRA-5587
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.4, 1.2.5
Reporter: Julien Aymé
Labels: patch
Attachments: cassandra-1.2-5587.txt
Original Estimate: 4h
Remaining Estimate: 4h
When using BulkLoader tool (sstableloader command) to transfer data from a
cluster to another,
a java.util.NoSuchElementException is thrown whenever the directory contains
a snapshot sub directory,
and the bulk load fails.
The fix should be quite simple:
Catch any NoSuchElementException thrown in {{SSTableLoader#openSSTables()}}
The directory structure:
{noformat}
user@cassandrasrv01:~$ ls /var/lib/cassandra/data/Keyspace1/CF1/
Keyspace1-CF1-ib-1872-CompressionInfo.db
Keyspace1-CF1-ib-1872-Data.db
Keyspace1-CF1-ib-1872-Filter.db
Keyspace1-CF1-ib-1872-Index.db
Keyspace1-CF1-ib-1872-Statistics.db
Keyspace1-CF1-ib-1872-Summary.db
Keyspace1-CF1-ib-1872-TOC.txt
Keyspace1-CF1-ib-2166-CompressionInfo.db
Keyspace1-CF1-ib-2166-Data.db
Keyspace1-CF1-ib-2166-Filter.db
Keyspace1-CF1-ib-2166-Index.db
Keyspace1-CF1-ib-2166-Statistics.db
Keyspace1-CF1-ib-2166-Summary.db
Keyspace1-CF1-ib-2166-TOC.txt
Keyspace1-CF1-ib-5-CompressionInfo.db
Keyspace1-CF1-ib-5-Data.db
Keyspace1-CF1-ib-5-Filter.db
Keyspace1-CF1-ib-5-Index.db
Keyspace1-CF1-ib-5-Statistics.db
Keyspace1-CF1-ib-5-Summary.db
Keyspace1-CF1-ib-5-TOC.txt
...
snapshots
{noformat}
The stacktrace:
{noformat}
user@cassandrasrv01:~$ ./cassandra/bin/sstableloader -v --debug -d
cassandrabck01 /var/lib/cassandra/data/Keyspace1/CF1/
null
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
at
org.apache.cassandra.io.sstable.Descriptor.fromFilename(Descriptor.java:265)
at
org.apache.cassandra.io.sstable.Component.fromFilename(Component.java:122)
at
org.apache.cassandra.io.sstable.SSTable.tryComponentFromFilename(SSTable.java:194)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:71)
at java.io.File.list(File.java:1087)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Updates news file, version and missing licenses for 1.1.12 release
Updated Branches: refs/heads/cassandra-1.1 93cfbc187 - 0db940695 Updates news file, version and missing licenses for 1.1.12 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0db94069 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0db94069 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0db94069 Branch: refs/heads/cassandra-1.1 Commit: 0db94069550b9c38b9f749e2087b196bb519664e Parents: 93cfbc1 Author: Sylvain Lebresne sylv...@datastax.com Authored: Wed May 22 09:10:40 2013 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Wed May 22 09:10:40 2013 +0200 -- NEWS.txt |9 ++ build.xml |2 +- debian/changelog |6 .../org/apache/cassandra/MethodComparator.java | 21 + .../apache/cassandra/OrderedJUnit4ClassRunner.java | 23 ++- 5 files changed, 59 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0db94069/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index a768ddd..d5ba882 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -8,6 +8,15 @@ upgrade, just in case you need to roll back to the previous version. (Cassandra version X + 1 will always be able to read data files created by version X, but the inverse is not necessarily the case.) +1.1.12 +== + +Upgrading +- +- Nothing specific to this release, but please see the previous instructions + if you are not upgrading from 1.1.11. + + 1.1.11 == http://git-wip-us.apache.org/repos/asf/cassandra/blob/0db94069/build.xml -- diff --git a/build.xml b/build.xml index b77c417..945fff7 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ property name=debuglevel value=source,lines,vars/ !-- default version and SCM information -- -property name=base.version value=1.1.11/ +property name=base.version value=1.1.12/ property name=scm.connection value=scm:git://git.apache.org/cassandra.git/ property name=scm.developerConnection value=scm:git://git.apache.org/cassandra.git/ property name=scm.url value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/ http://git-wip-us.apache.org/repos/asf/cassandra/blob/0db94069/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 76bac83..9e33d7c 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.1.12) unstable; urgency=low + + * New release + + -- Sylvain Lebresne slebre...@apache.org Wed, 22 May 2013 08:54:45 +0200 + cassandra (1.1.11) unstable; urgency=low * New release http://git-wip-us.apache.org/repos/asf/cassandra/blob/0db94069/test/unit/org/apache/cassandra/MethodComparator.java -- diff --git a/test/unit/org/apache/cassandra/MethodComparator.java b/test/unit/org/apache/cassandra/MethodComparator.java index 690ae57..8cc163a 100644 --- a/test/unit/org/apache/cassandra/MethodComparator.java +++ b/test/unit/org/apache/cassandra/MethodComparator.java @@ -1,4 +1,25 @@ package org.apache.cassandra; +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ + import org.junit.Ignore; import org.junit.runners.model.FrameworkMethod; http://git-wip-us.apache.org/repos/asf/cassandra/blob/0db94069/test/unit/org/apache/cassandra/OrderedJUnit4ClassRunner.java -- diff --git a/test/unit/org/apache/cassandra/OrderedJUnit4ClassRunner.java b/test/unit/org/apache/cassandra/OrderedJUnit4ClassRunner.java index d84aedb..d0dec24 100644 --- a/test/unit/org/apache/cassandra/OrderedJUnit4ClassRunner.java +++
Git Push Summary
Updated Tags: refs/tags/1.1.12-tentative [created] 0db940695
[jira] [Reopened] (CASSANDRA-5488) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true'
[
https://issues.apache.org/jira/browse/CASSANDRA-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna reopened CASSANDRA-5488:
-
There ended up being a secondary problem that was hidden by the first NPE. It
seems to be related to getting the AbstractType. The NPE was for this line:
https://github.com/apache/cassandra/blob/cassandra-1.1/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java#L307
which I decomposed to find out what it was NPEing on, and got this:
{code}
ListAbstractType atList = getDefaultMarshallers(cfDef);
AbstractType at = atList.get(2);
Object o = at.compose(key); //NPE from this line
setTupleValue(tuple, 0, o);
//setTupleValue(tuple, 0,
getDefaultMarshallers(cfDef).get(2).compose(key));
{code}
So it seems unrelated to the original NPE, but still matches the description of
this ticket.
To reproduce, here is my schema:
{code}
CREATE KEYSPACE circus
with placement_strategy = 'SimpleStrategy'
and strategy_options = {replication_factor:1};
use circus;
CREATE COLUMN FAMILY acrobats
WITH comparator = UTF8Type
AND key_validation_class=UTF8Type
AND default_validation_class = UTF8Type;
{code}
Here is a pycassa script to create the data:
{code}
from pycassa.pool import ConnectionPool
from pycassa.columnfamily import ColumnFamily
pool = ConnectionPool('circus')
col_fam = pycassa.ColumnFamily(pool, 'acrobats')
for i in range(1, 10):
for j in range(1, 20):
col_fam.insert('row_key' + str(i), {str(j): 'val'})
{code}
Here is the pig (0.9.2) that I'm running in local mode:
{code}
rows = LOAD 'cassandra://circus/acrobats?widerows=truelimit=20' USING
CassandraStorage();
filtered = filter rows by key == 'row_key1';
columns = foreach filtered generate flatten(columns);
counted = foreach (group columns all) generate COUNT($1);
dump counted;
{code}
CassandraStorage throws NullPointerException (NPE) when widerows is set to
'true'
-
Key: CASSANDRA-5488
URL: https://issues.apache.org/jira/browse/CASSANDRA-5488
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.9, 1.2.4
Environment: Ubuntu 12.04.1 x64, Cassandra 1.2.4
Reporter: Sheetal Gosrani
Assignee: Sheetal Gosrani
Priority: Minor
Labels: cassandra, hadoop, pig
Fix For: 1.1.12, 1.2.6
Attachments: 5488-2.txt, 5488.txt
CassandraStorage throws NPE when widerows is set to 'true'.
2 problems in getNextWide:
1. Creation of tuple without specifying size
2. Calling addKeyToTuple on lastKey instead of key
java.lang.NullPointerException
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167)
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:124)
at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:73)
at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:93)
at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:34)
at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:26)
at
org.apache.cassandra.hadoop.pig.CassandraStorage.addKeyToTuple(CassandraStorage.java:313)
at
org.apache.cassandra.hadoop.pig.CassandraStorage.getNextWide(CassandraStorage.java:196)
at
org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(CassandraStorage.java:224)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
2013-04-16 12:28:03,671 INFO org.apache.hadoop.mapred.Task: Runnning cleanup
for the task
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-5488) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true'
[ https://issues.apache.org/jira/browse/CASSANDRA-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko reassigned CASSANDRA-5488: Assignee: Aleksey Yeschenko (was: Sheetal Gosrani) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true' - Key: CASSANDRA-5488 URL: https://issues.apache.org/jira/browse/CASSANDRA-5488 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.9, 1.2.4 Environment: Ubuntu 12.04.1 x64, Cassandra 1.2.4 Reporter: Sheetal Gosrani Assignee: Aleksey Yeschenko Priority: Minor Labels: cassandra, hadoop, pig Fix For: 1.1.12, 1.2.6 Attachments: 5488-2.txt, 5488.txt CassandraStorage throws NPE when widerows is set to 'true'. 2 problems in getNextWide: 1. Creation of tuple without specifying size 2. Calling addKeyToTuple on lastKey instead of key java.lang.NullPointerException at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167) at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:124) at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:73) at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:93) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:34) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:26) at org.apache.cassandra.hadoop.pig.CassandraStorage.addKeyToTuple(CassandraStorage.java:313) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNextWide(CassandraStorage.java:196) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(CassandraStorage.java:224) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) 2013-04-16 12:28:03,671 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5545) Add SASL authentication to CQL native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-5545: --- Attachment: 0001-Add-SASL-hooks-to-CQL-native-protocol-v3.patch Version 3 patch with comments addressed Add SASL authentication to CQL native protocol -- Key: CASSANDRA-5545 URL: https://issues.apache.org/jira/browse/CASSANDRA-5545 Project: Cassandra Issue Type: Improvement Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe Fix For: 2.0 Attachments: 0001-Add-SASL-authentication-to-CQL-native-protocol.patch, 0001-Add-SASL-hooks-to-CQL-native-protocol.patch, 0001-Add-SASL-hooks-to-CQL-native-protocol-v3.patch Adding hooks for SASL authentication would make it much easier to integrate with external auth providers, such as Kerberos NTLM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5273) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling
[ https://issues.apache.org/jira/browse/CASSANDRA-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5273: -- Attachment: 5273-v2.txt Thinking about it more, I think adding a lock doesn't change anything. System.exit already locks/synchronizes the important parts. So we still have the deadlock problem, which we can hack around with timeouts but I'd rather not. Patch attached against 1.2 to call System.exit from a new thread instead. Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling - Key: CASSANDRA-5273 URL: https://issues.apache.org/jira/browse/CASSANDRA-5273 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.1 Environment: linux, 64 bit Reporter: Ignace Desimpel Assignee: Marcus Eriksson Priority: Minor Fix For: 2.0 Attachments: 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 5273-v2.txt, CassHangs.txt On out of memory exception, there is an uncaughtexception handler that is calling System.exit(). However, multiple threads are calling this handler causing a deadlock and the server cannot stop working. See http://www.mail-archive.com/user@cassandra.apache.org/msg27898.html. And see stack trace in attachement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5442) Add support for specifying CAS commit CL
[ https://issues.apache.org/jira/browse/CASSANDRA-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5442: -- Reviewer: iamaleksey (was: slebresne) Add support for specifying CAS commit CL Key: CASSANDRA-5442 URL: https://issues.apache.org/jira/browse/CASSANDRA-5442 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 2.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5514) Allow timestamp hints
[ https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5514: -- Reviewer: jbellis (was: slebresne) Allow timestamp hints - Key: CASSANDRA-5514 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Marcus Eriksson Fix For: 2.0 Attachments: 0001-CASSANDRA-5514-v1.patch, 0001-CASSANDRA-5514-v2.patch Slice queries can't optimize based on timestamp except for rare cases (CASSANDRA-4116). However, many common queries involve an implicit time component, where the application author knows that he is only interested in data more recent than X, or older than Y. We could use the per-sstable max and min timestamps we track to avoid touching cold data if we could pass a hint to Cassandra about the time range we care about. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4388) Use MMap for CompressedSegmentFile with Native Checksum
[ https://issues.apache.org/jira/browse/CASSANDRA-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664146#comment-13664146 ] Jonathan Ellis commented on CASSANDRA-4388: --- Should we wontfix this or are you looking to pick it back up [~vijay2...@yahoo.com]? Use MMap for CompressedSegmentFile with Native Checksum --- Key: CASSANDRA-4388 URL: https://issues.apache.org/jira/browse/CASSANDRA-4388 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Vijay Assignee: Vijay Fix For: 2.0 Attachments: 0001-CASSANDRA-4388.patch Use MMap for CompressedSegmentFile (Something similar to CASSANDRA-3623) and use Native Checksum (HDFS-2080) to avoid memcpy and be faster in its calculations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4658) Explore improved vnode-aware replication strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4658. --- Resolution: Duplicate Fix Version/s: (was: 2.0) Explore improved vnode-aware replication strategy - Key: CASSANDRA-4658 URL: https://issues.apache.org/jira/browse/CASSANDRA-4658 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 beta 1 Reporter: Nick Bailey It doesn't look like the current vnode placement strategy will work with people using NTS and multiple racks. For reasons also described on CASSANDRA-3810, using racks and NTS requires tokens to alternate racks around the ring in order to get an even distribution of data. The current solution for upgrading/placing vnodes won't take this into account and will likely generate some hotspots around the ring. Not sure what the best solution is. The two immediately obvious approaches appear to be quite complicated at first. * Fixing NTS to remove the requirement for rack ordering ** No idea how this would be accomplished ** Presents challenges for people upgrading. Would need to deprecate NTS for a new strategy that replaces it, then have a clear upgrade path to that strategy which would need to be in a pre 1.2 release. * Changing vnode placement strategy ** Ordering vnodes would require quite a bit of additional logic. Adding a new node or rack would also need to maintain ordering which could cause enough data movement to remove any benefits vnodes have already. ** We could potentially adjust vnode token placement to offset any imbalances caused by NTS but this would require a fairly intelligent mechanism for determining vnode placement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5265) operation to add/remove virtual nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5265: -- Affects Version/s: (was: 1.2.1) 1.2.0 Assignee: (was: Eric Evans) Issue Type: Improvement (was: Bug) operation to add/remove virtual nodes - Key: CASSANDRA-5265 URL: https://issues.apache.org/jira/browse/CASSANDRA-5265 Project: Cassandra Issue Type: Improvement Affects Versions: 1.2.0 Reporter: Eric Evans Labels: vnodes Fix For: 2.0 It should be possible to alter the proportion of data a node owns by increasing, or decreasing the number of tokens assigned to it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4603) use Map internally in schema_ tables where appropriate
[
https://issues.apache.org/jira/browse/CASSANDRA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4603:
--
Priority: Minor (was: Major)
Assignee: (was: Pavel Yaskevich)
use Map internally in schema_ tables where appropriate
--
Key: CASSANDRA-4603
URL: https://issues.apache.org/jira/browse/CASSANDRA-4603
Project: Cassandra
Issue Type: Improvement
Components: API, Core
Affects Versions: 1.2.0
Reporter: Jonathan Ellis
Priority: Minor
Labels: cql3
Fix For: 2.0
{replication, compression, compaction}_parameters should be stored as Map
type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3929) Support row size limits
[ https://issues.apache.org/jira/browse/CASSANDRA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3929. --- Resolution: Won't Fix Fix Version/s: (was: 2.0) Reviewer: (was: slebresne) Wontfixing for now, although still open to a pluggable solution as above. Support row size limits --- Key: CASSANDRA-3929 URL: https://issues.apache.org/jira/browse/CASSANDRA-3929 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Priority: Minor Labels: ponies Attachments: 3929_b.txt, 3929_c.txt, 3929_d.txt, 3929_e.txt, 3929_f.txt, 3929_g_tests.txt, 3929_g.txt, 3929.txt We currently support expiring columns by time-to-live; we've also had requests for keeping the most recent N columns in a row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3943) Too many small size sstables after loading data using sstableloader or BulkOutputFormat increases compaction time.
[
https://issues.apache.org/jira/browse/CASSANDRA-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-3943.
---
Resolution: Won't Fix
Fix Version/s: (was: 2.0)
Happy to see this picked up again but I think CASSANDRA-5371 addresses the
low-hanging fruit here.
Too many small size sstables after loading data using sstableloader or
BulkOutputFormat increases compaction time.
--
Key: CASSANDRA-3943
URL: https://issues.apache.org/jira/browse/CASSANDRA-3943
Project: Cassandra
Issue Type: Wish
Components: Hadoop, Tools
Affects Versions: 0.8.2, 1.1.0
Reporter: Samarth Gahire
Priority: Minor
Labels: bulkloader, hadoop, sstableloader, streaming, tools
Original Estimate: 168h
Remaining Estimate: 168h
When we create sstables using SimpleUnsortedWriter or BulkOutputFormat,the
size of sstables created is around the buffer size provided.
But After loading , sstables created in the cluster nodes are of size around
{code}( (sstable_size_before_loading) * replication_factor ) /
No_Of_Nodes_In_Cluster{code}
As the no of nodes in cluster goes increasing, size of each sstable loaded to
cassandra node decreases.Such small size sstables take too much time to
compact (minor compaction) as compare to relatively large size sstables.
One solution that we have tried is to increase the buffer size while
generating sstables.But as we increase the buffer size ,time taken to
generate sstables increases.Is there any solution to this in existing
versions or are you fixing this in future version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4450) CQL3: Allow preparing the consistency level, timestamp and ttl
[
https://issues.apache.org/jira/browse/CASSANDRA-4450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4450:
--
Reviewer: iamaleksey
Oops, didn't tag this w/ a reviewer and it slipped through the cracks a bit.
CQL3: Allow preparing the consistency level, timestamp and ttl
--
Key: CASSANDRA-4450
URL: https://issues.apache.org/jira/browse/CASSANDRA-4450
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
Labels: cql3
Fix For: 2.0
It could be useful to allow the preparation of the consitency level, the
timestamp and the ttl. I.e. to allow:
{noformat}
UPDATE foo SET .. USING CONSISTENCY ? AND TIMESTAMP ? AND TTL ?
{noformat}
A slight concern is that when preparing a statement we return the names of
the prepared variables, but none of timestamp, ttl and consistency are
reserved names currently, so returning those as names could conflict with a
column name. We can either:
* make these reserved identifier (I have to add that I'm not a fan because at
least for timestamp, I think that's a potentially useful and common column
name).
* use some specific special character to indicate those are not column names,
like returning [timestamp], [ttl], [consistency].
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664183#comment-13664183 ] Ryan McGuire commented on CASSANDRA-5586: - Doing a few greps, it looks like the following tests use the cli: * configuration_test * super_column_cache_test * cql_tests I notice that some of the tests are specifically in regards to dealing with data that was originally created with cassandra-cli (see cql_tests:rename_Test.) Part of me feels reluctant to remove tests that were once valid scenarios. Remove cli usage from dtests Key: CASSANDRA-5586 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Ryan McGuire Priority: Minor The dtests in some situations fork the cli. With the cli essentially stagnant now, there's no need to do this when the same thing can be accomplished with a thrift or cql call. (ccm's convenience api for invoking the cli could probably also be removed at this point) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5586) Remove cli usage from dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664183#comment-13664183 ] Ryan McGuire edited comment on CASSANDRA-5586 at 5/22/13 3:25 PM: -- Doing a few greps, it looks like the following tests use the cli: * configuration_test * super_column_cache_test * cql_tests I notice that some of the tests are specifically in regards to dealing with data that was originally created with cassandra-cli (see cql_tests:rename_test.) Part of me feels reluctant to remove tests that were once valid scenarios. was (Author: enigmacurry): Doing a few greps, it looks like the following tests use the cli: * configuration_test * super_column_cache_test * cql_tests I notice that some of the tests are specifically in regards to dealing with data that was originally created with cassandra-cli (see cql_tests:rename_Test.) Part of me feels reluctant to remove tests that were once valid scenarios. Remove cli usage from dtests Key: CASSANDRA-5586 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Ryan McGuire Priority: Minor The dtests in some situations fork the cli. With the cli essentially stagnant now, there's no need to do this when the same thing can be accomplished with a thrift or cql call. (ccm's convenience api for invoking the cli could probably also be removed at this point) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5273) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling
[ https://issues.apache.org/jira/browse/CASSANDRA-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5273: -- Attachment: 5273-v3.txt v3 preallocates the Thread. Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling - Key: CASSANDRA-5273 URL: https://issues.apache.org/jira/browse/CASSANDRA-5273 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.1 Environment: linux, 64 bit Reporter: Ignace Desimpel Assignee: Marcus Eriksson Priority: Minor Fix For: 2.0 Attachments: 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 5273-v2.txt, 5273-v3.txt, CassHangs.txt On out of memory exception, there is an uncaughtexception handler that is calling System.exit(). However, multiple threads are calling this handler causing a deadlock and the server cannot stop working. See http://www.mail-archive.com/user@cassandra.apache.org/msg27898.html. And see stack trace in attachement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5273) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling
[ https://issues.apache.org/jira/browse/CASSANDRA-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664186#comment-13664186 ] Marcus Eriksson commented on CASSANDRA-5273: lgtm Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling - Key: CASSANDRA-5273 URL: https://issues.apache.org/jira/browse/CASSANDRA-5273 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.1 Environment: linux, 64 bit Reporter: Ignace Desimpel Assignee: Marcus Eriksson Priority: Minor Fix For: 2.0 Attachments: 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 5273-v2.txt, 5273-v3.txt, CassHangs.txt On out of memory exception, there is an uncaughtexception handler that is calling System.exit(). However, multiple threads are calling this handler causing a deadlock and the server cannot stop working. See http://www.mail-archive.com/user@cassandra.apache.org/msg27898.html. And see stack trace in attachement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-5273) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling
[ https://issues.apache.org/jira/browse/CASSANDRA-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-5273. --- Resolution: Fixed Reviewer: mericsson Assignee: Jonathan Ellis (was: Marcus Eriksson) committed Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling - Key: CASSANDRA-5273 URL: https://issues.apache.org/jira/browse/CASSANDRA-5273 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.1 Environment: linux, 64 bit Reporter: Ignace Desimpel Assignee: Jonathan Ellis Priority: Minor Fix For: 2.0 Attachments: 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 5273-v2.txt, 5273-v3.txt, CassHangs.txt On out of memory exception, there is an uncaughtexception handler that is calling System.exit(). However, multiple threads are calling this handler causing a deadlock and the server cannot stop working. See http://www.mail-archive.com/user@cassandra.apache.org/msg27898.html. And see stack trace in attachement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/3] git commit: Move System.exit on OOM into a separate thread patch by jbellis; reviewed by marcuse for CASSANDRA-5273
Updated Branches:
refs/heads/cassandra-1.2 de2ee6e28 - bf28e8d4b
refs/heads/trunk bac41da61 - 44827b4a8
Move System.exit on OOM into a separate thread
patch by jbellis; reviewed by marcuse for CASSANDRA-5273
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf28e8d4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf28e8d4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf28e8d4
Branch: refs/heads/cassandra-1.2
Commit: bf28e8d4b84afd5f9821ad965274768b8d4ca179
Parents: de2ee6e
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed May 22 10:32:36 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed May 22 10:33:09 2013 -0500
--
CHANGES.txt|1 +
.../apache/cassandra/service/CassandraDaemon.java | 12 +++-
2 files changed, 12 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf28e8d4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3902dec..21faf5a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.6
+ * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
* Write row markers when serializing schema (CASSANDRA-5572)
* Check only SSTables for the requested range when streaming (CASSANDRA-5569)
* Improve batchlog replay behavior and hint ttl handling (CASSANDRA-5314)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf28e8d4/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 40c453d..343d497 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -58,6 +58,16 @@ public class CassandraDaemon
initLog4j();
}
+// Have a dedicated thread to call exit to avoid deadlock in the case
where the thread that wants to invoke exit
+// belongs to an executor that our shutdown hook wants to wait to exit
gracefully. See CASSANDRA-5273.
+private static final Thread exitThread = new Thread(new Runnable()
+{
+public void run()
+{
+System.exit(100);
+}
+}, Exit invoker);
+
/**
* Initialize logging in such a way that it checks for config changes
every 10 seconds.
*/
@@ -178,7 +188,7 @@ public class CassandraDaemon
{
// some code, like FileChannel.map, will wrap an
OutOfMemoryError in another exception
if (e2 instanceof OutOfMemoryError)
-System.exit(100);
+exitThread.start();
if (e2 instanceof FSError)
{
[2/3] git commit: Move System.exit on OOM into a separate thread patch by jbellis; reviewed by marcuse for CASSANDRA-5273
Move System.exit on OOM into a separate thread
patch by jbellis; reviewed by marcuse for CASSANDRA-5273
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf28e8d4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf28e8d4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf28e8d4
Branch: refs/heads/trunk
Commit: bf28e8d4b84afd5f9821ad965274768b8d4ca179
Parents: de2ee6e
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed May 22 10:32:36 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed May 22 10:33:09 2013 -0500
--
CHANGES.txt|1 +
.../apache/cassandra/service/CassandraDaemon.java | 12 +++-
2 files changed, 12 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf28e8d4/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3902dec..21faf5a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.6
+ * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
* Write row markers when serializing schema (CASSANDRA-5572)
* Check only SSTables for the requested range when streaming (CASSANDRA-5569)
* Improve batchlog replay behavior and hint ttl handling (CASSANDRA-5314)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf28e8d4/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 40c453d..343d497 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -58,6 +58,16 @@ public class CassandraDaemon
initLog4j();
}
+// Have a dedicated thread to call exit to avoid deadlock in the case
where the thread that wants to invoke exit
+// belongs to an executor that our shutdown hook wants to wait to exit
gracefully. See CASSANDRA-5273.
+private static final Thread exitThread = new Thread(new Runnable()
+{
+public void run()
+{
+System.exit(100);
+}
+}, Exit invoker);
+
/**
* Initialize logging in such a way that it checks for config changes
every 10 seconds.
*/
@@ -178,7 +188,7 @@ public class CassandraDaemon
{
// some code, like FileChannel.map, will wrap an
OutOfMemoryError in another exception
if (e2 instanceof OutOfMemoryError)
-System.exit(100);
+exitThread.start();
if (e2 instanceof FSError)
{
[3/3] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44827b4a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44827b4a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44827b4a Branch: refs/heads/trunk Commit: 44827b4a824611cdbcd152b6240ccd688ac2ba82 Parents: bac41da bf28e8d Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 22 10:33:20 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 22 10:33:20 2013 -0500 -- CHANGES.txt|1 + .../apache/cassandra/service/CassandraDaemon.java | 12 +++- 2 files changed, 12 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/44827b4a/CHANGES.txt -- diff --cc CHANGES.txt index 570c867,21faf5a..1ad2f25 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,57 -1,5 +1,58 @@@ +2.0 + * use nanotime consistently for node-local timeouts (CASSANDRA-5581) + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577) + * Experimental triggers (CASSANDRA-1311) + * JEMalloc support for off-heap allocation (CASSANDRA-3997) + * Single-pass compaction (CASSANDRA-4180) + * Removed token range bisection (CASSANDRA-5518) + * Removed compatibility with pre-1.2.5 sstables and network messages + (CASSANDRA-5511) + * removed PBSPredictor (CASSANDRA-5455) + * CAS support (CASSANDRA-5062, 5441, 5443) + * Leveled compaction performs size-tiered compactions in L0 + (CASSANDRA-5371, 5439) + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339) + * Log when a node is down longer than the hint window (CASSANDRA-4554) + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917) + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407) + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430) + * Change Message IDs to ints (CASSANDRA-5307) + * Move sstable level information into the Stats component, removing the + need for a separate Manifest file (CASSANDRA-4872) + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199) + * make index_interval configurable per columnfamily (CASSANDRA-3961) + * add default_time_to_live (CASSANDRA-3974) + * add memtable_flush_period_in_ms (CASSANDRA-4237) + * replace supercolumns internally by composites (CASSANDRA-3237, 5123) + * upgrade thrift to 0.9.0 (CASSANDRA-3719) + * drop unnecessary keyspace parameter from user-defined compaction API + (CASSANDRA-5139) + * more robust solution to incomplete compactions + counters (CASSANDRA-5151) + * Change order of directory searching for c*.in.sh (CASSANDRA-3983) + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271) + * Allow custom configuration loader (CASSANDRA-5045) + * Remove memory emergency pressure valve logic (CASSANDRA-3534) + * Reduce request latency with eager retry (CASSANDRA-4705) + * cqlsh: Remove ASSUME command (CASSANDRA-5331) + * Rebuild BF when loading sstables if bloom_filter_fp_chance + has changed since compaction (CASSANDRA-5015) + * remove row-level bloom filters (CASSANDRA-4885) + * Change Kernel Page Cache skipping into row preheating (disabled by default) + (CASSANDRA-4937) + * Improve repair by deciding on a gcBefore before sending + out TreeRequests (CASSANDRA-4932) + * Add an official way to disable compactions (CASSANDRA-5074) + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919) + * Add binary protocol versioning (CASSANDRA-5436) + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530) + * Add alias support to SELECT statement (CASSANDRA-5075) + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541) + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579) + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585) + + 1.2.6 + * Move System.exit on OOM into a separate thread (CASSANDRA-5273) * Write row markers when serializing schema (CASSANDRA-5572) * Check only SSTables for the requested range when streaming (CASSANDRA-5569) * Improve batchlog replay behavior and hint ttl handling (CASSANDRA-5314) http://git-wip-us.apache.org/repos/asf/cassandra/blob/44827b4a/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[jira] [Updated] (CASSANDRA-5273) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling
[ https://issues.apache.org/jira/browse/CASSANDRA-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-5273: --- Reviewer: krummas (was: mericsson) Hanging system after OutOfMemory. Server cannot die due to uncaughtException handling - Key: CASSANDRA-5273 URL: https://issues.apache.org/jira/browse/CASSANDRA-5273 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.1 Environment: linux, 64 bit Reporter: Ignace Desimpel Assignee: Jonathan Ellis Priority: Minor Fix For: 2.0 Attachments: 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 0001-CASSANDRA-5273-add-timeouts-to-the-blocking-commitlo.patch, 5273-v2.txt, 5273-v3.txt, CassHangs.txt On out of memory exception, there is an uncaughtexception handler that is calling System.exit(). However, multiple threads are calling this handler causing a deadlock and the server cannot stop working. See http://www.mail-archive.com/user@cassandra.apache.org/msg27898.html. And see stack trace in attachement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5498) Possible NPE on EACH_QUORUM writes
[ https://issues.apache.org/jira/browse/CASSANDRA-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664199#comment-13664199 ] Jonathan Ellis commented on CASSANDRA-5498: --- But I don't see how 0.7 would return an exception; it would just close the connection uncleanly. Possible NPE on EACH_QUORUM writes -- Key: CASSANDRA-5498 URL: https://issues.apache.org/jira/browse/CASSANDRA-5498 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.10 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: each_quorum, ec2 Fix For: 1.2.6 Attachments: 5498-v1.patch, 5498-v2.patch When upgrading from 1.0 to 1.1, we observed that DatacenterSyncWriteResponseHandler.assureSufficientLiveNodes() can throw an NPE if one of the writeEndpoints has a DC that is not listed in the keyspace while one of the nodes is down. We observed this while running in EC2, and using the Ec2Snitch. The exception typically was was brief, but a certain segment of writes (using EACH_QUORUM) failed during that time. This ticket will address the NPE in DSWRH, while a followup ticket will be created once we get to the bottom of the incorrect DC being reported from Ec2Snitch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5498) Possible NPE on EACH_QUORUM writes
[ https://issues.apache.org/jira/browse/CASSANDRA-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664199#comment-13664199 ] Jonathan Ellis edited comment on CASSANDRA-5498 at 5/22/13 3:46 PM: But I don't see how 0.7 would return an exception at all; it would just close the connection uncleanly. was (Author: jbellis): But I don't see how 0.7 would return an exception; it would just close the connection uncleanly. Possible NPE on EACH_QUORUM writes -- Key: CASSANDRA-5498 URL: https://issues.apache.org/jira/browse/CASSANDRA-5498 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.10 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: each_quorum, ec2 Fix For: 1.2.6 Attachments: 5498-v1.patch, 5498-v2.patch When upgrading from 1.0 to 1.1, we observed that DatacenterSyncWriteResponseHandler.assureSufficientLiveNodes() can throw an NPE if one of the writeEndpoints has a DC that is not listed in the keyspace while one of the nodes is down. We observed this while running in EC2, and using the Ec2Snitch. The exception typically was was brief, but a certain segment of writes (using EACH_QUORUM) failed during that time. This ticket will address the NPE in DSWRH, while a followup ticket will be created once we get to the bottom of the incorrect DC being reported from Ec2Snitch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-5171) Enhance Ec2Snitch gossip info.
[ https://issues.apache.org/jira/browse/CASSANDRA-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reopened CASSANDRA-5171: --- Enhance Ec2Snitch gossip info. -- Key: CASSANDRA-5171 URL: https://issues.apache.org/jira/browse/CASSANDRA-5171 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.0 Environment: EC2 Reporter: Vijay Assignee: Vijay Priority: Trivial Fix For: 1.2.1 Attachments: 0001-CASSANDRA-5171.patch EC2Snitch currently waits for the Gossip information to understand the cluster information every time we restart. It will be nice to use already available system table info similar to GPFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5171) Save EC2Snitch topology information in system table
[
https://issues.apache.org/jira/browse/CASSANDRA-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-5171:
--
Priority: Critical (was: Trivial)
Affects Version/s: (was: 1.2.0)
0.7.1
Fix Version/s: (was: 1.2.1)
1.2.6
Issue Type: Bug (was: Improvement)
Summary: Save EC2Snitch topology information in system table
(was: Enhance Ec2Snitch gossip info.)
This was reverted in CASSANDRA-5432, but I think the problem it solves is
actually pretty severe, so I'm reopening it.
The problem is that pretty much everything from TokenMetadata to
NetworkTopologyStrategy assumes that once we see a node, the snitch can tell us
where it lives, and in particular that once the snitch tells us where a node
lives it won't change its answer.
So this is problematic:
{code}
public String getDatacenter(InetAddress endpoint)
{
if (endpoint.equals(FBUtilities.getBroadcastAddress()))
return ec2region;
EndpointState state =
Gossiper.instance.getEndpointStateForEndpoint(endpoint);
if (state == null || state.getApplicationState(ApplicationState.DC) ==
null)
return DEFAULT_DC;
return state.getApplicationState(ApplicationState.DC).value;
}
{code}
That is, if we don't know where a node belongs (e.g., we just restarted and
haven't been gosipped to yet), assume it's in {{DEFAULT_DC}}.
This can lead to data loss. Consider node X in DC1, where keyspace KS is
replicated. Suddenly X is yanked out of DC1 and placed in DC2, where KS is not
replicated. Nobody will bother querying X for the data in KS that was formerly
replicated to it. Even repair will not see it.
Save EC2Snitch topology information in system table
---
Key: CASSANDRA-5171
URL: https://issues.apache.org/jira/browse/CASSANDRA-5171
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.1
Environment: EC2
Reporter: Vijay
Assignee: Vijay
Priority: Critical
Fix For: 1.2.6
Attachments: 0001-CASSANDRA-5171.patch
EC2Snitch currently waits for the Gossip information to understand the
cluster information every time we restart. It will be nice to use already
available system table info similar to GPFS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Fix 5488 round 2
Updated Branches:
refs/heads/cassandra-1.1 0db940695 - 2dd73d171
Fix 5488 round 2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2dd73d17
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2dd73d17
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2dd73d17
Branch: refs/heads/cassandra-1.1
Commit: 2dd73d171068d743befcd445a14751032d232e4e
Parents: 0db9406
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed May 22 11:18:59 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed May 22 11:19:05 2013 -0500
--
.../cassandra/hadoop/pig/CassandraStorage.java | 34 ++-
1 files changed, 23 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd73d17/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
index b681ee3..cf1c08f 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
@@ -130,7 +130,7 @@ public class CassandraStorage extends LoadFunc implements
StoreFuncInterface, Lo
{
CfDef cfDef = getCfDef(loadSignature);
ByteBuffer key = null;
-Tuple tuple = TupleFactory.getInstance().newTuple();
+Tuple tuple = null;
DefaultDataBag bag = new DefaultDataBag();
try
{
@@ -139,12 +139,15 @@ public class CassandraStorage extends LoadFunc implements
StoreFuncInterface, Lo
hasNext = reader.nextKeyValue();
if (!hasNext)
{
+if (tuple == null)
+tuple = TupleFactory.getInstance().newTuple();
+
if (lastRow != null)
{
if (tuple.size() == 0) // lastRow is a new one
{
key = (ByteBuffer)reader.getCurrentKey();
-tuple = addKeyToTuple(tuple, key, cfDef,
parseType(cfDef.getKey_validation_class()));
+tuple = keyToTuple(key, cfDef,
parseType(cfDef.getKey_validation_class()));
}
for (Map.EntryByteBuffer, IColumn entry :
lastRow.entrySet())
{
@@ -180,7 +183,10 @@ public class CassandraStorage extends LoadFunc implements
StoreFuncInterface, Lo
key = (ByteBuffer)reader.getCurrentKey();
if (lastKey != null !(key.equals(lastKey))) // last key
only had one value
{
-tuple = addKeyToTuple(tuple, lastKey, cfDef,
parseType(cfDef.getKey_validation_class()));
+if (tuple == null)
+tuple = keyToTuple(lastKey, cfDef,
parseType(cfDef.getKey_validation_class()));
+else
+addKeyToTuple(tuple, lastKey, cfDef,
parseType(cfDef.getKey_validation_class()));
for (Map.EntryByteBuffer, IColumn entry :
lastRow.entrySet())
{
bag.add(columnToTuple(entry.getValue(), cfDef,
parseType(cfDef.getComparator_type(;
@@ -190,7 +196,10 @@ public class CassandraStorage extends LoadFunc implements
StoreFuncInterface, Lo
lastRow =
(SortedMapByteBuffer,IColumn)reader.getCurrentValue();
return tuple;
}
-tuple = addKeyToTuple(tuple, lastKey, cfDef,
parseType(cfDef.getKey_validation_class()));
+if (tuple == null)
+tuple = keyToTuple(key, cfDef,
parseType(cfDef.getKey_validation_class()));
+else
+addKeyToTuple(tuple, lastKey, cfDef,
parseType(cfDef.getKey_validation_class()));
}
SortedMapByteBuffer,IColumn row =
(SortedMapByteBuffer,IColumn)reader.getCurrentValue();
if (lastRow != null) // prepend what was read last time
@@ -233,7 +242,7 @@ public class CassandraStorage extends LoadFunc implements
StoreFuncInterface, Lo
// output tuple, will hold the key, each indexed column in a
tuple, then a bag of the rest
// NOTE: we're setting the tuple size here only for the key so we
can use setTupleValue on it
-Tuple tuple = addKeyToTuple(null, key, cfDef,
parseType(cfDef.getKey_validation_class()));
+Tuple tuple = keyToTuple(key, cfDef,
parseType(cfDef.getKey_validation_class()));
[jira] [Resolved] (CASSANDRA-5488) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true'
[ https://issues.apache.org/jira/browse/CASSANDRA-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-5488. - Resolution: Fixed v2 was a little too aggressive in function consolidation. I reverted it and applied v1. CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true' - Key: CASSANDRA-5488 URL: https://issues.apache.org/jira/browse/CASSANDRA-5488 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.9, 1.2.4 Environment: Ubuntu 12.04.1 x64, Cassandra 1.2.4 Reporter: Sheetal Gosrani Assignee: Aleksey Yeschenko Priority: Minor Labels: cassandra, hadoop, pig Fix For: 1.1.12, 1.2.6 Attachments: 5488-2.txt, 5488.txt CassandraStorage throws NPE when widerows is set to 'true'. 2 problems in getNextWide: 1. Creation of tuple without specifying size 2. Calling addKeyToTuple on lastKey instead of key java.lang.NullPointerException at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167) at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:124) at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:73) at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:93) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:34) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:26) at org.apache.cassandra.hadoop.pig.CassandraStorage.addKeyToTuple(CassandraStorage.java:313) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNextWide(CassandraStorage.java:196) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(CassandraStorage.java:224) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) 2013-04-16 12:28:03,671 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5586) Remove cli usage from dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664230#comment-13664230 ] Brandon Williams commented on CASSANDRA-5586: - I don't think rename_test was truly cli-specific; we can do the equivalent over thrift without cql. Remove cli usage from dtests Key: CASSANDRA-5586 URL: https://issues.apache.org/jira/browse/CASSANDRA-5586 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Ryan McGuire Priority: Minor The dtests in some situations fork the cli. With the cli essentially stagnant now, there's no need to do this when the same thing can be accomplished with a thrift or cql call. (ccm's convenience api for invoking the cli could probably also be removed at this point) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5538) Reduce Empty Map allocations in StorageProxy.sendToHintedEndpoints
[ https://issues.apache.org/jira/browse/CASSANDRA-5538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5538: -- Attachment: 5538-2.txt 2nd patch to send dc-local replicas inline instead of putting them in a Map. (Note that I ninja-committed 62f429337caf0aa83b68720a5904e8527b840c80 ahead of this, might want to review that too. :) Reduce Empty Map allocations in StorageProxy.sendToHintedEndpoints -- Key: CASSANDRA-5538 URL: https://issues.apache.org/jira/browse/CASSANDRA-5538 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 2.0 Reporter: Dave Brosius Assignee: Dave Brosius Priority: Trivial Fix For: 2.0 Attachments: 5538-2.txt, 5538.txt StorageProxy.sendToHintedEndpoints allocates HashMaps consistently that are very often not used. See output: http://pastebin.com/jEaBxz1h Format is Type Date SourceLine CollectionSize NumBuckets NumBucketsUsed The snapshot is taken after the size of the collection hasn't changed for 5 seconds. Postpone creation of hashmap until you need it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4388) Use MMap for CompressedSegmentFile with Native Checksum
[ https://issues.apache.org/jira/browse/CASSANDRA-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay resolved CASSANDRA-4388. -- Resolution: Won't Fix Hi Jonathan, The main idea was to reduce the system calls which is better served currently with the pool... hence closing the ticket. Use MMap for CompressedSegmentFile with Native Checksum --- Key: CASSANDRA-4388 URL: https://issues.apache.org/jira/browse/CASSANDRA-4388 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Vijay Assignee: Vijay Fix For: 2.0 Attachments: 0001-CASSANDRA-4388.patch Use MMap for CompressedSegmentFile (Something similar to CASSANDRA-3623) and use Native Checksum (HDFS-2080) to avoid memcpy and be faster in its calculations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-5488) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true'
[ https://issues.apache.org/jira/browse/CASSANDRA-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-5488: --- Assignee: Sheetal Gosrani (was: Brandon Williams) CassandraStorage throws NullPointerException (NPE) when widerows is set to 'true' - Key: CASSANDRA-5488 URL: https://issues.apache.org/jira/browse/CASSANDRA-5488 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.9, 1.2.4 Environment: Ubuntu 12.04.1 x64, Cassandra 1.2.4 Reporter: Sheetal Gosrani Assignee: Sheetal Gosrani Priority: Minor Labels: cassandra, hadoop, pig Fix For: 1.1.12, 1.2.6 Attachments: 5488-2.txt, 5488.txt CassandraStorage throws NPE when widerows is set to 'true'. 2 problems in getNextWide: 1. Creation of tuple without specifying size 2. Calling addKeyToTuple on lastKey instead of key java.lang.NullPointerException at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167) at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:124) at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:73) at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:93) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:34) at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:26) at org.apache.cassandra.hadoop.pig.CassandraStorage.addKeyToTuple(CassandraStorage.java:313) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNextWide(CassandraStorage.java:196) at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(CassandraStorage.java:224) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) 2013-04-16 12:28:03,671 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[3/3] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2fc450a0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2fc450a0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2fc450a0 Branch: refs/heads/trunk Commit: 2fc450a0bf85fb3caaa1b81274f40552a7b59ba4 Parents: 62f4293 c48c7ef Author: Yuki Morishita yu...@apache.org Authored: Wed May 22 14:08:31 2013 -0500 Committer: Yuki Morishita yu...@apache.org Committed: Wed May 22 14:08:31 2013 -0500 -- CHANGES.txt|1 + .../org/apache/cassandra/db/DeletedColumn.java | 21 +++ .../org/apache/cassandra/db/RangeTombstone.java|1 - 3 files changed, 22 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2fc450a0/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2fc450a0/src/java/org/apache/cassandra/db/DeletedColumn.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2fc450a0/src/java/org/apache/cassandra/db/RangeTombstone.java --
[2/3] git commit: Exclude localTimestamp from merkle tree calculation for tombstones patch by Christian Spriegel; reviewed by yukim for CASSANDRA-5398
Exclude localTimestamp from merkle tree calculation for tombstones patch by
Christian Spriegel; reviewed by yukim for CASSANDRA-5398
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c48c7ef1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c48c7ef1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c48c7ef1
Branch: refs/heads/trunk
Commit: c48c7ef16610b1ef86fc3c784a0659de9b5effd4
Parents: bf28e8d
Author: Christian Spriegel hors...@googlemail.com
Authored: Wed May 22 11:51:45 2013 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed May 22 14:05:56 2013 -0500
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/DeletedColumn.java | 21 +++
.../org/apache/cassandra/db/RangeTombstone.java|1 -
3 files changed, 22 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 21faf5a..6f1127a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Write row markers when serializing schema (CASSANDRA-5572)
* Check only SSTables for the requested range when streaming (CASSANDRA-5569)
* Improve batchlog replay behavior and hint ttl handling (CASSANDRA-5314)
+ * Exclude localTimestamp from validation for tombstones (CASSANDRA-5398)
Merged from 1.1:
* Remove buggy thrift max message length option (CASSANDRA-5529)
* Fix NPE in Pig's widerow mode (CASSANDRA-5488)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/src/java/org/apache/cassandra/db/DeletedColumn.java
--
diff --git a/src/java/org/apache/cassandra/db/DeletedColumn.java
b/src/java/org/apache/cassandra/db/DeletedColumn.java
index 18faeef..9814a3d 100644
--- a/src/java/org/apache/cassandra/db/DeletedColumn.java
+++ b/src/java/org/apache/cassandra/db/DeletedColumn.java
@@ -17,10 +17,13 @@
*/
package org.apache.cassandra.db;
+import java.io.IOException;
import java.nio.ByteBuffer;
+import java.security.MessageDigest;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.marshal.MarshalException;
+import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.HeapAllocator;
@@ -52,6 +55,24 @@ public class DeletedColumn extends Column
}
@Override
+public void updateDigest(MessageDigest digest)
+{
+digest.update(name.duplicate());
+
+DataOutputBuffer buffer = new DataOutputBuffer();
+try
+{
+buffer.writeLong(timestamp);
+buffer.writeByte(serializationFlags());
+}
+catch (IOException e)
+{
+throw new RuntimeException(e);
+}
+digest.update(buffer.getData(), 0, buffer.getLength());
+}
+
+@Override
public int getLocalDeletionTime()
{
return value.getInt(value.position());
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/src/java/org/apache/cassandra/db/RangeTombstone.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstone.java
b/src/java/org/apache/cassandra/db/RangeTombstone.java
index 1d472c3..5e87847 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstone.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstone.java
@@ -96,7 +96,6 @@ public class RangeTombstone extends IntervalByteBuffer,
DeletionTime implement
try
{
buffer.writeLong(data.markedForDeleteAt);
-buffer.writeInt(data.localDeletionTime);
}
catch (IOException e)
{
[1/3] git commit: Exclude localTimestamp from merkle tree calculation for tombstones patch by Christian Spriegel; reviewed by yukim for CASSANDRA-5398
Updated Branches:
refs/heads/cassandra-1.2 bf28e8d4b - c48c7ef16
refs/heads/trunk 62f429337 - 2fc450a0b
Exclude localTimestamp from merkle tree calculation for tombstones patch by
Christian Spriegel; reviewed by yukim for CASSANDRA-5398
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c48c7ef1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c48c7ef1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c48c7ef1
Branch: refs/heads/cassandra-1.2
Commit: c48c7ef16610b1ef86fc3c784a0659de9b5effd4
Parents: bf28e8d
Author: Christian Spriegel hors...@googlemail.com
Authored: Wed May 22 11:51:45 2013 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed May 22 14:05:56 2013 -0500
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/DeletedColumn.java | 21 +++
.../org/apache/cassandra/db/RangeTombstone.java|1 -
3 files changed, 22 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 21faf5a..6f1127a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Write row markers when serializing schema (CASSANDRA-5572)
* Check only SSTables for the requested range when streaming (CASSANDRA-5569)
* Improve batchlog replay behavior and hint ttl handling (CASSANDRA-5314)
+ * Exclude localTimestamp from validation for tombstones (CASSANDRA-5398)
Merged from 1.1:
* Remove buggy thrift max message length option (CASSANDRA-5529)
* Fix NPE in Pig's widerow mode (CASSANDRA-5488)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/src/java/org/apache/cassandra/db/DeletedColumn.java
--
diff --git a/src/java/org/apache/cassandra/db/DeletedColumn.java
b/src/java/org/apache/cassandra/db/DeletedColumn.java
index 18faeef..9814a3d 100644
--- a/src/java/org/apache/cassandra/db/DeletedColumn.java
+++ b/src/java/org/apache/cassandra/db/DeletedColumn.java
@@ -17,10 +17,13 @@
*/
package org.apache.cassandra.db;
+import java.io.IOException;
import java.nio.ByteBuffer;
+import java.security.MessageDigest;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.marshal.MarshalException;
+import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.HeapAllocator;
@@ -52,6 +55,24 @@ public class DeletedColumn extends Column
}
@Override
+public void updateDigest(MessageDigest digest)
+{
+digest.update(name.duplicate());
+
+DataOutputBuffer buffer = new DataOutputBuffer();
+try
+{
+buffer.writeLong(timestamp);
+buffer.writeByte(serializationFlags());
+}
+catch (IOException e)
+{
+throw new RuntimeException(e);
+}
+digest.update(buffer.getData(), 0, buffer.getLength());
+}
+
+@Override
public int getLocalDeletionTime()
{
return value.getInt(value.position());
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c48c7ef1/src/java/org/apache/cassandra/db/RangeTombstone.java
--
diff --git a/src/java/org/apache/cassandra/db/RangeTombstone.java
b/src/java/org/apache/cassandra/db/RangeTombstone.java
index 1d472c3..5e87847 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstone.java
+++ b/src/java/org/apache/cassandra/db/RangeTombstone.java
@@ -96,7 +96,6 @@ public class RangeTombstone extends IntervalByteBuffer,
DeletionTime implement
try
{
buffer.writeLong(data.markedForDeleteAt);
-buffer.writeInt(data.localDeletionTime);
}
catch (IOException e)
{
[jira] [Updated] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size
[ https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5272: -- Attachment: 5272.txt Makes sense to me. Patch attached against 1.2. Hinted Handoff Throttle based on cluster size - Key: CASSANDRA-5272 URL: https://issues.apache.org/jira/browse/CASSANDRA-5272 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.1 Reporter: Rick Branson Priority: Minor Labels: lhf Fix For: 2.0 Attachments: 5272.txt For a 12-node EC2 m1.xlarge cluster, restarting a node causes it to get completely overloaded with the default 2-thread, 1024KB setting in 1.2.x. This seemed to be a smaller problem when it was 6-nodes, but still required us to abort handoffs. The old defaults in 1.1.x were WAY more conservative. I've dropped this way down to 128KB on our production cluster which is really conservative, but appears to have solved it. The default seems way too high on any cluster that is non-trivial in size. After putting some thought to this, it seems that this should really be based on cluster size, making the throttle a target for how much write load a single node can swallow. As the cluster grows, the amount of hints that can be delivered by each other node in the cluster goes down, so the throttle should self-adjust to take that into account. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size
[ https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5272: -- Reviewer: rbranson Affects Version/s: (was: 1.2.1) 1.2.0 Fix Version/s: (was: 2.0) 1.2.6 Assignee: Jonathan Ellis Hinted Handoff Throttle based on cluster size - Key: CASSANDRA-5272 URL: https://issues.apache.org/jira/browse/CASSANDRA-5272 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.0 Reporter: Rick Branson Assignee: Jonathan Ellis Priority: Minor Labels: lhf Fix For: 1.2.6 Attachments: 5272.txt For a 12-node EC2 m1.xlarge cluster, restarting a node causes it to get completely overloaded with the default 2-thread, 1024KB setting in 1.2.x. This seemed to be a smaller problem when it was 6-nodes, but still required us to abort handoffs. The old defaults in 1.1.x were WAY more conservative. I've dropped this way down to 128KB on our production cluster which is really conservative, but appears to have solved it. The default seems way too high on any cluster that is non-trivial in size. After putting some thought to this, it seems that this should really be based on cluster size, making the throttle a target for how much write load a single node can swallow. As the cluster grows, the amount of hints that can be delivered by each other node in the cluster goes down, so the throttle should self-adjust to take that into account. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size
[ https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664482#comment-13664482 ] Jonathan Ellis commented on CASSANDRA-5272: --- Correction; patch is against trunk but should be trivial to retrofit if necessary. Hinted Handoff Throttle based on cluster size - Key: CASSANDRA-5272 URL: https://issues.apache.org/jira/browse/CASSANDRA-5272 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.0 Reporter: Rick Branson Assignee: Jonathan Ellis Priority: Minor Labels: lhf Fix For: 1.2.6 Attachments: 5272.txt For a 12-node EC2 m1.xlarge cluster, restarting a node causes it to get completely overloaded with the default 2-thread, 1024KB setting in 1.2.x. This seemed to be a smaller problem when it was 6-nodes, but still required us to abort handoffs. The old defaults in 1.1.x were WAY more conservative. I've dropped this way down to 128KB on our production cluster which is really conservative, but appears to have solved it. The default seems way too high on any cluster that is non-trivial in size. After putting some thought to this, it seems that this should really be based on cluster size, making the throttle a target for how much write load a single node can swallow. As the cluster grows, the amount of hints that can be delivered by each other node in the cluster goes down, so the throttle should self-adjust to take that into account. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4421) Support cql3 table definitions in Hadoop InputFormat
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664499#comment-13664499
]
Mike Schrag commented on CASSANDRA-4421:
I spoke too soon (I accidentally had my override classes still in place).
reachEndRange is consuming the partition key ByteBuffers, so they're basically
empty values every time, so the split just keeps repeating starting at the
beginning.
At the top of reachEndRange, if you:
{code}for (Key k : partitionKeys) k.value.mark();{code}
and at the bottom, you can reset them:
{code}for (Key k : partitionKeys) k.value.reset();{code}
that will fix it.
Support cql3 table definitions in Hadoop InputFormat
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Labels: cql3
Fix For: 1.2.6
Attachments: 4421-1.txt, 4421-2.txt, 4421-3.txt, 4421-4.txt, 4421.txt
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5348) Remove row cache until we can replace it with something better
[ https://issues.apache.org/jira/browse/CASSANDRA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5348: -- Attachment: 5348.txt Patch to remove on-heap row cache. Remove row cache until we can replace it with something better -- Key: CASSANDRA-5348 URL: https://issues.apache.org/jira/browse/CASSANDRA-5348 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Vijay Fix For: 2.0 Attachments: 5348.txt The row (partition) cache easily does more harm than good. People expect it to act like a query cache but it is very different than that, especially for the wide partitions that are so common in Cassandra data models. Making it off-heap by default only helped a little; we still have to deserialize the partition to the heap to query it. Ultimately we can add a better cache based on the ideas in CASSANDRA-1956 or CASSANDRA-2864, but even if we don't get to that until 2.1, removing the old row cache for 2.0 is a good idea. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5348) Remove on-heap row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5348: -- Summary: Remove on-heap row cache (was: Remove row cache until we can replace it with something better) Remove on-heap row cache Key: CASSANDRA-5348 URL: https://issues.apache.org/jira/browse/CASSANDRA-5348 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 2.0 Attachments: 5348.txt The row (partition) cache easily does more harm than good. People expect it to act like a query cache but it is very different than that, especially for the wide partitions that are so common in Cassandra data models. Making it off-heap by default only helped a little; we still have to deserialize the partition to the heap to query it. Ultimately we can add a better cache based on the ideas in CASSANDRA-1956 or CASSANDRA-2864, but even if we don't get to that until 2.1, removing the old row cache for 2.0 is a good idea. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5348) Remove on-heap row cache
[
https://issues.apache.org/jira/browse/CASSANDRA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664507#comment-13664507
]
Jason Brown commented on CASSANDRA-5348:
code lgtm, but tests couldn't compile
{code}[javac]
/usr/local/src/cassandra/test/unit/org/apache/cassandra/db/CollationControllerTest.java:73:
error: constructor CollationController in class CollationController cannot be
applied to given types;
[javac] CollationController controller = new
CollationController(store, false, filter, Integer.MIN_VALUE);
[javac] ^
[javac] required: ColumnFamilyStore,QueryFilter,int
[javac] found: ColumnFamilyStore,boolean,QueryFilter,int
[javac] reason: actual and formal argument lists differ in length
[javac]
/usr/local/src/cassandra/test/unit/org/apache/cassandra/db/CollationControllerTest.java:81:
error: constructor CollationController in class CollationController cannot be
applied to given types;
[javac] controller = new CollationController(store, false, filter,
Integer.MIN_VALUE);
[javac] ^
[javac] required: ColumnFamilyStore,QueryFilter,int
[javac] found: ColumnFamilyStore,boolean,QueryFilter,int
[javac] reason: actual and formal argument lists differ in length
{code}
Once I removed the boolean 'false' argument to the method, it compiled. Running
tests now.
Remove on-heap row cache
Key: CASSANDRA-5348
URL: https://issues.apache.org/jira/browse/CASSANDRA-5348
Project: Cassandra
Issue Type: Task
Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Fix For: 2.0
Attachments: 5348.txt
The row (partition) cache easily does more harm than good. People expect it
to act like a query cache but it is very different than that, especially for
the wide partitions that are so common in Cassandra data models.
Making it off-heap by default only helped a little; we still have to
deserialize the partition to the heap to query it.
Ultimately we can add a better cache based on the ideas in CASSANDRA-1956 or
CASSANDRA-2864, but even if we don't get to that until 2.1, removing the old
row cache for 2.0 is a good idea.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3706) Back up configuration files on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3706. --- Resolution: Won't Fix Fix Version/s: (was: 2.0) Reviewer: (was: brandon.williams) I think the discomfort we ran into here is a good sign that this is something better managed outside of C*. Back up configuration files on startup -- Key: CASSANDRA-3706 URL: https://issues.apache.org/jira/browse/CASSANDRA-3706 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Assignee: Dave Brosius Priority: Minor Labels: lhf Attachments: save_configuration_2.diff, save_configuration_3.diff, save_configuration_4.diff, save_configuration_6.diff, save_configuration_7.diff, save_configuration_8.diff, save_configuration_9.diff, save_configuration.diff Snapshot can backup user data, but it's also nice to be able to have known-good configurations saved as well in case of accidental snafus or even catastrophic loss of a cluster. If we check for changes to cassandra.yaml, cassandra-env.sh, and maybe log4j-server.properties on startup, we can back them up to a columnfamily that can then be handled by normal snapshot/backup procedures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: remove on-heap row cache
Updated Branches:
refs/heads/trunk 2fc450a0b - a3734e54b
remove on-heap row cache
Removed on-heap row cache
patch by jbellis; reviewed by jasobrown for CASSANDRA-5348
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a3734e54
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a3734e54
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a3734e54
Branch: refs/heads/trunk
Commit: a3734e54bc9032dedad5d44394cca8dc7e400a43
Parents: 2fc450a
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed May 22 15:31:26 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed May 22 15:59:52 2013 -0500
--
CHANGES.txt|1 +
conf/cassandra.yaml| 18 --
.../cassandra/cache/ConcurrentLinkedHashCache.java |5 --
.../cache/ConcurrentLinkedHashCacheProvider.java | 26 -
src/java/org/apache/cassandra/cache/ICache.java|6 --
.../apache/cassandra/cache/IRowCacheProvider.java | 26 -
.../apache/cassandra/cache/InstrumentingCache.java |5 --
.../apache/cassandra/cache/SerializingCache.java |5 --
.../cassandra/cache/SerializingCacheProvider.java |2 +-
src/java/org/apache/cassandra/config/Config.java |2 -
.../cassandra/config/DatabaseDescriptor.java |8 ---
.../apache/cassandra/db/CollationController.java | 11 +---
.../org/apache/cassandra/db/ColumnFamilyStore.java | 41 ---
.../db/compaction/CompactionController.java| 14 -
.../cassandra/db/compaction/CompactionTask.java|3 -
.../org/apache/cassandra/service/CacheService.java | 10 ++--
.../org/apache/cassandra/utils/FBUtilities.java|8 ---
.../org/apache/cassandra/utils/StatusLogger.java | 29 +-
.../cassandra/db/CollationControllerTest.java |4 +-
19 files changed, 35 insertions(+), 189 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a3734e54/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fd85dab..b765896 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0
+ * Removed on-heap row cache (CASSANDRA-5348)
* use nanotime consistently for node-local timeouts (CASSANDRA-5581)
* Avoid unnecessary second pass on name-based queries (CASSANDRA-5577)
* Experimental triggers (CASSANDRA-1311)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a3734e54/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 1192442..f8cf4f7 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -170,24 +170,6 @@ row_cache_save_period: 0
# Disabled by default, meaning all keys are going to be saved
# row_cache_keys_to_save: 100
-# The provider for the row cache to use.
-#
-# Supported values are: ConcurrentLinkedHashCacheProvider,
SerializingCacheProvider
-#
-# SerializingCacheProvider serialises the contents of the row and stores
-# it in native memory, i.e., off the JVM Heap. Serialized rows take
-# significantly less memory than live rows in the JVM, so you can cache
-# more rows in a given memory footprint. And storing the cache off-heap
-# means you can use smaller heap sizes, reducing the impact of GC pauses.
-# Note however that when a row is requested from the row cache, it must be
-# deserialized into the heap for use.
-#
-# It is also valid to specify the fully-qualified class name to a class
-# that implements org.apache.cassandra.cache.IRowCacheProvider.
-#
-# Defaults to SerializingCacheProvider
-row_cache_provider: SerializingCacheProvider
-
# The off-heap memory allocator. Affects storage engine metadata as
# well as caches. Experiments show that JEMAlloc saves some memory
# than the native GCC allocator (i.e., JEMalloc is more
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a3734e54/src/java/org/apache/cassandra/cache/ConcurrentLinkedHashCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/ConcurrentLinkedHashCache.java
b/src/java/org/apache/cassandra/cache/ConcurrentLinkedHashCache.java
index af6549d..f1e0466 100644
--- a/src/java/org/apache/cassandra/cache/ConcurrentLinkedHashCache.java
+++ b/src/java/org/apache/cassandra/cache/ConcurrentLinkedHashCache.java
@@ -130,9 +130,4 @@ public class ConcurrentLinkedHashCacheK extends
IMeasurableMemory, V extends IM
{
return map.containsKey(key);
}
-
-public boolean isPutCopying()
-{
-return false;
-}
}
[jira] [Commented] (CASSANDRA-5348) Remove on-heap row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664521#comment-13664521 ] Jason Brown commented on CASSANDRA-5348: tests completed and passed. Remove on-heap row cache Key: CASSANDRA-5348 URL: https://issues.apache.org/jira/browse/CASSANDRA-5348 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 2.0 Attachments: 5348.txt The row (partition) cache easily does more harm than good. People expect it to act like a query cache but it is very different than that, especially for the wide partitions that are so common in Cassandra data models. Making it off-heap by default only helped a little; we still have to deserialize the partition to the heap to query it. Ultimately we can add a better cache based on the ideas in CASSANDRA-1956 or CASSANDRA-2864, but even if we don't get to that until 2.1, removing the old row cache for 2.0 is a good idea. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4421) Support cql3 table definitions in Hadoop InputFormat
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Liu updated CASSANDRA-4421:
Attachment: 4421-5.txt
Version 5 is attached to add the fix by Mike
Support cql3 table definitions in Hadoop InputFormat
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Labels: cql3
Fix For: 1.2.6
Attachments: 4421-1.txt, 4421-2.txt, 4421-3.txt, 4421-4.txt,
4421-5.txt, 4421.txt
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5588) Add get commands to nodetool for things with set
Jeremiah Jordan created CASSANDRA-5588: -- Summary: Add get commands to nodetool for things with set Key: CASSANDRA-5588 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Priority: Minor Can we add: nodetool getcompactionthroughput nodetool getstreamthroughput To go with the set commands? You currently have to fire up a JMX client to know what the current values are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5588) Add get commands to nodetool for things with set
[ https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-5588: Labels: lhf (was: ) Add get commands to nodetool for things with set Key: CASSANDRA-5588 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Priority: Minor Labels: lhf Can we add: nodetool getcompactionthroughput nodetool getstreamthroughput To go with the set commands? You currently have to fire up a JMX client to know what the current values are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-5588) Add get commands to nodetool for things with set
[ https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-5588: --- Assignee: Michał Michalski Add get commands to nodetool for things with set Key: CASSANDRA-5588 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Assignee: Michał Michalski Priority: Minor Labels: lhf Can we add: nodetool getcompactionthroughput nodetool getstreamthroughput To go with the set commands? You currently have to fire up a JMX client to know what the current values are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5588) Add get commands to nodetool for things with set
[ https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-5588: Reviewer: brandon.williams Add get commands to nodetool for things with set Key: CASSANDRA-5588 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Assignee: Michał Michalski Priority: Minor Labels: lhf Can we add: nodetool getcompactionthroughput nodetool getstreamthroughput To go with the set commands? You currently have to fire up a JMX client to know what the current values are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5588) Add get commands to nodetool for things with set
[ https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-5588: Fix Version/s: 1.2.6 Add get commands to nodetool for things with set Key: CASSANDRA-5588 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Jeremiah Jordan Assignee: Michał Michalski Priority: Minor Labels: lhf Fix For: 1.2.6 Can we add: nodetool getcompactionthroughput nodetool getstreamthroughput To go with the set commands? You currently have to fire up a JMX client to know what the current values are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3868) Remove or nullify replicate_on_write option
[ https://issues.apache.org/jira/browse/CASSANDRA-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664583#comment-13664583 ] Jonathan Ellis commented on CASSANDRA-3868: --- I think we should either remove r_o_w for 2.0 or close this as wontfix (and hopefully obsoleted by CASSANDRA-4775 for 2.1). Does removing it buy us much in terms of code cleanup? Remove or nullify replicate_on_write option --- Key: CASSANDRA-3868 URL: https://issues.apache.org/jira/browse/CASSANDRA-3868 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.0 Reporter: Brandon Williams Fix For: 2.0 Attachments: 3868.txt My understanding from Sylvain is that setting this option to false is rather dangerous/stupid, and you should basically never do it. So 1.1 is a good time to get rid of it, or make it a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1337: -- Fix Version/s: (was: 2.0) 2.1 parallelize fetching rows for low-cardinality indexes - Key: CASSANDRA-1337 URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Priority: Minor Fix For: 2.1 Attachments: 1137-bugfix.patch, 1337.patch, 1337-v4.patch, ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, CASSANDRA-1337.patch Original Estimate: 8h Remaining Estimate: 8h currently, we read the indexed rows from the first node (in partitioner order); if that does not have enough matching rows, we read the rows from the next, and so forth. we should use the statistics fom CASSANDRA-1155 to query multiple nodes in parallel, such that we have a high chance of getting enough rows w/o having to do another round of queries (but, if our estimate is incorrect, we do need to loop and do more rounds until we have enough data or we have fetched from each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2308) add tracing of cell name index effectiveness
[ https://issues.apache.org/jira/browse/CASSANDRA-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2308: -- Assignee: Jonathan Ellis Summary: add tracing of cell name index effectiveness (was: track row indexes in sstable stats) I think we have a pretty good idea of index size from the existing row size stats, but it would be useful to include in query traces how many columns we had to deserialize post-seek. (Probably the index deserialize too.) add tracing of cell name index effectiveness Key: CASSANDRA-2308 URL: https://issues.apache.org/jira/browse/CASSANDRA-2308 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 2.0 Exposing row index sizes would give us the information to tune column_index_size correctly (see CASSANDRA-2297). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2986) Fix short reads in range (and index?) scans
[ https://issues.apache.org/jira/browse/CASSANDRA-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2986: -- Fix Version/s: (was: 2.0) 2.1 Fix short reads in range (and index?) scans --- Key: CASSANDRA-2986 URL: https://issues.apache.org/jira/browse/CASSANDRA-2986 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jason Brown Priority: Minor Fix For: 2.1 See CASSANDRA-2643 for the [multi]get fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3512) Getting Started instructions don't work in README.txt - wrong version of jamm, wrong path
[
https://issues.apache.org/jira/browse/CASSANDRA-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-3512:
-
Assignee: Dave Brosius (was: Brandon Williams)
Can you take a look, [~dbrosius]?
Getting Started instructions don't work in README.txt - wrong version of
jamm, wrong path
-
Key: CASSANDRA-3512
URL: https://issues.apache.org/jira/browse/CASSANDRA-3512
Project: Cassandra
Issue Type: Bug
Components: Packaging
Environment: Ubuntu 11.04
Reporter: David Allsopp
Assignee: Dave Brosius
Priority: Minor
Fix For: 2.0
Download latest release from
http://www.apache.org/dyn/closer.cgi?path=/cassandra/1.0.3/apache-cassandra-1.0.3-bin.tar.gz
Unpack the tarball.
Follow instructions in README.txt, concluding with:
{noformat}
dna@master:~/code/apache-cassandra-1.0.3$ bin/cassandra -f
Error opening zip file or JAR manifest missing : /lib/jamm-0.2.1.jar
Error occurred during initialization of VM
agent library failed to init: instrument
{noformat}
Firstly, the version of jamm packaged with Cassandra 1.0.3 is jamm-0.2.5, not
jamm-0.2.1.
Both bin/cassandra.bat and conf/cassandra-env.sh reference jamm-0.2.5 so not
sure where jamm-0.2.1 is being referenced from - nothing obvious using grep.
Secondly, /lib/jamm-0.2.1.jar is the wrong path - should be set relative to
working directory, not filesystem root
(Incidentally, Cassandra v1.0.3 is still listed as unreleased on JIRA.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4511) Secondary index support for CQL3 collections
[ https://issues.apache.org/jira/browse/CASSANDRA-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4511: -- Fix Version/s: (was: 2.0) 2.1 Secondary index support for CQL3 collections - Key: CASSANDRA-4511 URL: https://issues.apache.org/jira/browse/CASSANDRA-4511 Project: Cassandra Issue Type: Improvement Affects Versions: 1.2.0 beta 1 Reporter: Sylvain Lebresne Priority: Minor Fix For: 2.1 We should allow to 2ndary index on collections. A typical use case would be to add a 'tag setString' to say a user profile and to query users based on what tag they have. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4655) Truncate operation doesn't delete rows from HintsColumnFamily.
[ https://issues.apache.org/jira/browse/CASSANDRA-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4655: -- Attachment: 4655-v3.txt v3 attached. I've retained removing obsolete truncation information; otherwise, in the case of creating many temporary tables we will cause problems eventually. Moved caching to HHOM. Also combined the new hint time with the existing truncated_at blob for simplicity and to make sure both get updated as a group. Truncate operation doesn't delete rows from HintsColumnFamily. -- Key: CASSANDRA-4655 URL: https://issues.apache.org/jira/browse/CASSANDRA-4655 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Environment: Centos 6.2, Cassandra 1.1.4 (DataStax distribution), three-nodes cluster. Reporter: Alexey Zotov Assignee: Alexey Zotov Priority: Minor Labels: hintedhandoff, truncate Fix For: 2.0 Attachments: 4655-v3.txt, cassandra-1.2-4655-hints_truncation.txt, cassandra-1.2-4655-hints_truncation-v2.txt Steps to reproduce: 1. Start writing of data to some column family, let name it 'MyCF' 2. Stop 1 node 3. Wait some time (until some data will be collected in HintsColumnFamily) 4. Start node (HintedHandoff will be started automatically for 'MyCF') 5. Run 'truncate' command for 'MyCF' column family from command from cli 6. Wait until truncate will be finished 7. You will see that 'MyCF' is not empty because HintedHandoff is copying data So, I suggest to clean HintsColumnFamily (for truncated column family) before we had started to discard sstables. I think it should be done in CompactionManager#submitTrucate() method. I can try to create patch but I need to know right way of cleaning HintsColumnFamily. Could you clarify it? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4655) Truncate operation doesn't delete rows from HintsColumnFamily.
[ https://issues.apache.org/jira/browse/CASSANDRA-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-4655: - Assignee: Jonathan Ellis (was: Alexey Zotov) Truncate operation doesn't delete rows from HintsColumnFamily. -- Key: CASSANDRA-4655 URL: https://issues.apache.org/jira/browse/CASSANDRA-4655 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Environment: Centos 6.2, Cassandra 1.1.4 (DataStax distribution), three-nodes cluster. Reporter: Alexey Zotov Assignee: Jonathan Ellis Priority: Minor Labels: hintedhandoff, truncate Fix For: 2.0 Attachments: 4655-v3.txt, cassandra-1.2-4655-hints_truncation.txt, cassandra-1.2-4655-hints_truncation-v2.txt Steps to reproduce: 1. Start writing of data to some column family, let name it 'MyCF' 2. Stop 1 node 3. Wait some time (until some data will be collected in HintsColumnFamily) 4. Start node (HintedHandoff will be started automatically for 'MyCF') 5. Run 'truncate' command for 'MyCF' column family from command from cli 6. Wait until truncate will be finished 7. You will see that 'MyCF' is not empty because HintedHandoff is copying data So, I suggest to clean HintsColumnFamily (for truncated column family) before we had started to discard sstables. I think it should be done in CompactionManager#submitTrucate() method. I can try to create patch but I need to know right way of cleaning HintsColumnFamily. Could you clarify it? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4655) Truncate operation doesn't delete rows from HintsColumnFamily.
[ https://issues.apache.org/jira/browse/CASSANDRA-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4655: -- Fix Version/s: (was: 2.0) 1.2.6 Truncate operation doesn't delete rows from HintsColumnFamily. -- Key: CASSANDRA-4655 URL: https://issues.apache.org/jira/browse/CASSANDRA-4655 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Environment: Centos 6.2, Cassandra 1.1.4 (DataStax distribution), three-nodes cluster. Reporter: Alexey Zotov Assignee: Jonathan Ellis Priority: Minor Labels: hintedhandoff, truncate Fix For: 1.2.6 Attachments: 4655-v3.txt, cassandra-1.2-4655-hints_truncation.txt, cassandra-1.2-4655-hints_truncation-v2.txt Steps to reproduce: 1. Start writing of data to some column family, let name it 'MyCF' 2. Stop 1 node 3. Wait some time (until some data will be collected in HintsColumnFamily) 4. Start node (HintedHandoff will be started automatically for 'MyCF') 5. Run 'truncate' command for 'MyCF' column family from command from cli 6. Wait until truncate will be finished 7. You will see that 'MyCF' is not empty because HintedHandoff is copying data So, I suggest to clean HintsColumnFamily (for truncated column family) before we had started to discard sstables. I think it should be done in CompactionManager#submitTrucate() method. I can try to create patch but I need to know right way of cleaning HintsColumnFamily. Could you clarify it? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor
[ https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5582: --- Attachment: (was: disruptor-thrift-0.1-SNAPSHOT.jar) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor --- Key: CASSANDRA-5582 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Fix For: 2.0 Attachments: CASSANDRA-5530-invoker-fix.patch, Pavel's Patch.rtf I have been working on https://github.com/xedin/disruptor_thrift_server and consider it as stable and performant enough for integration with Cassandra. Proposed replacement can work in both on/off Heap modes (depending if JNA is available) and doesn't blindly reallocate things, which allows to resolve CASSANDRA-4265 as Won't Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor
[ https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5582: --- Attachment: (was: disruptor-3.0.0.beta3.jar) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor --- Key: CASSANDRA-5582 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Fix For: 2.0 Attachments: CASSANDRA-5530-invoker-fix.patch, Pavel's Patch.rtf I have been working on https://github.com/xedin/disruptor_thrift_server and consider it as stable and performant enough for integration with Cassandra. Proposed replacement can work in both on/off Heap modes (depending if JNA is available) and doesn't blindly reallocate things, which allows to resolve CASSANDRA-4265 as Won't Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor
[ https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5582: --- Attachment: (was: CASSANDRA-5582.patch) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor --- Key: CASSANDRA-5582 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Fix For: 2.0 Attachments: CASSANDRA-5530-invoker-fix.patch, Pavel's Patch.rtf I have been working on https://github.com/xedin/disruptor_thrift_server and consider it as stable and performant enough for integration with Cassandra. Proposed replacement can work in both on/off Heap modes (depending if JNA is available) and doesn't blindly reallocate things, which allows to resolve CASSANDRA-4265 as Won't Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor
[ https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5582: --- Attachment: disruptor-thrift-0.1-SNAPSHOT.jar disruptor-3.0.1.jar CASSANDRA-5582.patch updated patch/server that uses ring buffer per selector and updated disruptor to 3.0.1 Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor --- Key: CASSANDRA-5582 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Fix For: 2.0 Attachments: CASSANDRA-5530-invoker-fix.patch, CASSANDRA-5582.patch, disruptor-3.0.1.jar, disruptor-thrift-0.1-SNAPSHOT.jar, Pavel's Patch.rtf I have been working on https://github.com/xedin/disruptor_thrift_server and consider it as stable and performant enough for integration with Cassandra. Proposed replacement can work in both on/off Heap modes (depending if JNA is available) and doesn't blindly reallocate things, which allows to resolve CASSANDRA-4265 as Won't Fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3512) Getting Started instructions don't work in README.txt - wrong version of jamm, wrong path
[
https://issues.apache.org/jira/browse/CASSANDRA-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664752#comment-13664752
]
Dave Brosius commented on CASSANDRA-3512:
-
i don't see anything wrong on trunk here.
cassandra.in.sh does
if [ x$CASSANDRA_HOME = x ]; then
CASSANDRA_HOME=`dirname $0`/..
fi
which works correctly whether you have it explicitly set (EXPORTED) or not.
Getting Started instructions don't work in README.txt - wrong version of
jamm, wrong path
-
Key: CASSANDRA-3512
URL: https://issues.apache.org/jira/browse/CASSANDRA-3512
Project: Cassandra
Issue Type: Bug
Components: Packaging
Environment: Ubuntu 11.04
Reporter: David Allsopp
Assignee: Dave Brosius
Priority: Minor
Fix For: 2.0
Download latest release from
http://www.apache.org/dyn/closer.cgi?path=/cassandra/1.0.3/apache-cassandra-1.0.3-bin.tar.gz
Unpack the tarball.
Follow instructions in README.txt, concluding with:
{noformat}
dna@master:~/code/apache-cassandra-1.0.3$ bin/cassandra -f
Error opening zip file or JAR manifest missing : /lib/jamm-0.2.1.jar
Error occurred during initialization of VM
agent library failed to init: instrument
{noformat}
Firstly, the version of jamm packaged with Cassandra 1.0.3 is jamm-0.2.5, not
jamm-0.2.1.
Both bin/cassandra.bat and conf/cassandra-env.sh reference jamm-0.2.5 so not
sure where jamm-0.2.1 is being referenced from - nothing obvious using grep.
Secondly, /lib/jamm-0.2.1.jar is the wrong path - should be set relative to
working directory, not filesystem root
(Incidentally, Cassandra v1.0.3 is still listed as unreleased on JIRA.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: remove unused field RandomPartitioner.DELIMITER_BYTE
Updated Branches:
refs/heads/trunk a3734e54b - a583123e8
remove unused field RandomPartitioner.DELIMITER_BYTE
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a583123e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a583123e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a583123e
Branch: refs/heads/trunk
Commit: a583123e8a8c07d5cd1fee06cb1a15af2670
Parents: a3734e5
Author: Dave Brosius dbros...@apache.org
Authored: Wed May 22 21:55:35 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Wed May 22 21:55:35 2013 -0400
--
.../apache/cassandra/dht/RandomPartitioner.java| 10 --
1 files changed, 4 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a583123e/src/java/org/apache/cassandra/dht/RandomPartitioner.java
--
diff --git a/src/java/org/apache/cassandra/dht/RandomPartitioner.java
b/src/java/org/apache/cassandra/dht/RandomPartitioner.java
index 15453ba..c9ddccf 100644
--- a/src/java/org/apache/cassandra/dht/RandomPartitioner.java
+++ b/src/java/org/apache/cassandra/dht/RandomPartitioner.java
@@ -40,8 +40,6 @@ public class RandomPartitioner extends
AbstractPartitionerBigIntegerToken
public static final BigIntegerToken MINIMUM = new BigIntegerToken(-1);
public static final BigInteger MAXIMUM = new BigInteger(2).pow(127);
-private static final byte DELIMITER_BYTE = :.getBytes()[0];
-
public DecoratedKey decorateKey(ByteBuffer key)
{
return new DecoratedKey(getToken(key), key);
@@ -128,23 +126,23 @@ public class RandomPartitioner extends
AbstractPartitionerBigIntegerToken
public MapToken, Float describeOwnership(ListToken sortedTokens)
{
MapToken, Float ownerships = new HashMapToken, Float();
-Iterator i = sortedTokens.iterator();
+IteratorToken i = sortedTokens.iterator();
// 0-case
if (!i.hasNext()) { throw new RuntimeException(No nodes present in
the cluster. Has this node finished starting up?); }
// 1-case
if (sortedTokens.size() == 1) {
-ownerships.put((Token)i.next(), new Float(1.0));
+ownerships.put(i.next(), new Float(1.0));
}
// n-case
else {
// NOTE: All divisions must take place in BigDecimals, and all
modulo operators must take place in BigIntegers.
final BigInteger ri = MAXIMUM;
// (used for addition later)
final BigDecimal r = new BigDecimal(ri);
// The entire range, 2**127
-Token start = (Token)i.next(); BigInteger ti =
((BigIntegerToken)start).token; // The first token and its value
+Token start = i.next(); BigInteger ti =
((BigIntegerToken)start).token; // The first token and its value
Token t; BigInteger tim1 = ti;
// The last token and its value (after loop)
while (i.hasNext()) {
-t = (Token)i.next(); ti = ((BigIntegerToken)t).token;
// The next token and its value
+t = i.next(); ti = ((BigIntegerToken)t).token;
// The next token and its value
float x = new
BigDecimal(ti.subtract(tim1).add(ri).mod(ri)).divide(r).floatValue(); // %age =
((T(i) - T(i-1) + R) % R) / R
ownerships.put(t, x);
// save (T(i) - %age)
tim1 = ti;
// - advance loop
[jira] [Commented] (CASSANDRA-4655) Truncate operation doesn't delete rows from HintsColumnFamily.
[ https://issues.apache.org/jira/browse/CASSANDRA-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664804#comment-13664804 ] Vijay commented on CASSANDRA-4655: -- +1 Truncate operation doesn't delete rows from HintsColumnFamily. -- Key: CASSANDRA-4655 URL: https://issues.apache.org/jira/browse/CASSANDRA-4655 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Environment: Centos 6.2, Cassandra 1.1.4 (DataStax distribution), three-nodes cluster. Reporter: Alexey Zotov Assignee: Jonathan Ellis Priority: Minor Labels: hintedhandoff, truncate Fix For: 1.2.6 Attachments: 4655-v3.txt, cassandra-1.2-4655-hints_truncation.txt, cassandra-1.2-4655-hints_truncation-v2.txt Steps to reproduce: 1. Start writing of data to some column family, let name it 'MyCF' 2. Stop 1 node 3. Wait some time (until some data will be collected in HintsColumnFamily) 4. Start node (HintedHandoff will be started automatically for 'MyCF') 5. Run 'truncate' command for 'MyCF' column family from command from cli 6. Wait until truncate will be finished 7. You will see that 'MyCF' is not empty because HintedHandoff is copying data So, I suggest to clean HintsColumnFamily (for truncated column family) before we had started to discard sstables. I think it should be done in CompactionManager#submitTrucate() method. I can try to create patch but I need to know right way of cleaning HintsColumnFamily. Could you clarify it? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
