svn commit: r1136304 - in /cassandra/branches/cassandra-0.8: CHANGES.txt NEWS.txt
Author: slebresne Date: Thu Jun 16 07:35:19 2011 New Revision: 1136304 URL: http://svn.apache.org/viewvc?rev=1136304view=rev Log: Update CHANGES.txt and NEWS.txt for 0.8.1 release Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/NEWS.txt Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1136304r1=1136303r2=1136304view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Thu Jun 16 07:35:19 2011 @@ -5,7 +5,6 @@ - timestamp support for INSERT, UPDATE, and BATCH (CASSANDRA-2555) - TTL support (CASSANDRA-2476) - counter support (CASSANDRA-2473) - - improve JDBC spec compliance (CASSANDRA-2720) - ALTER COLUMNFAMILY (CASSANDRA-1709) - DROP INDEX (CASSANDRA-2617) - add SCHEMA/TABLE as aliases for KS/CF (CASSANDRA-2743) @@ -45,6 +44,7 @@ by nio sockets (CASSANDRA-2654) * restrict repair streaming to specific columnfamilies (CASSANDRA-2280) * fix nodetool ring use with Ec2Snitch (CASSANDRA-2733) + * fix inconsistency window during bootstrap (CASSANDRA-833) * fix removing columns and subcolumns that are supressed by a row or supercolumn tombstone during replica resolution (CASSANDRA-2590) * support sstable2json against snapshot sstables (CASSANDRA-2386) @@ -58,6 +58,11 @@ * use threadsafe collections for StreamInSession (CASSANDRA-2766) * avoid infinite loop when creating merkle tree (CASSANDRA-2758) * avoids unmarking compacting sstable prematurely in cleanup (CASSANDRA-2769) + * fix NPE when the commit log is bypassed (CASSANDRA-2718) + * don't throw an exception in SS.isRPCServerRunning (CASSANDRA-2721) + * make stress.jar executable (CASSANDRA-2744) + * add daemon mode to java stress (CASSANDRA-2267) + * expose the DC and rack of a node through JMX and nodetool ring (CASSANDRA-2531) 0.8.0-final Modified: cassandra/branches/cassandra-0.8/NEWS.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/NEWS.txt?rev=1136304r1=1136303r2=1136304view=diff == --- cassandra/branches/cassandra-0.8/NEWS.txt (original) +++ cassandra/branches/cassandra-0.8/NEWS.txt Thu Jun 16 07:35:19 2011 @@ -1,3 +1,28 @@ +0.8.1 += + +Upgrading +- +- 0.8.1 is backwards compatible with 0.8, upgrade can be achieved by a + simple rolling restart. +- If upgrading for earlier version (0.7), please refer to the 0.8 section + for instructions. + +Features + +- Numerous additions/improvements to CQL (support for counters, TTL, batch + inserts/deletes, index dropping, ...). +- Add two new AbstractTypes (comparator) to support compound keys + (CompositeType and DynamicCompositeType), as well as a ReverseType to + reverse the order of any existing comparator. +- New option to bypass the commit log on some keyspaces (for advanced + users). + +Tools +- +- Add new data bulk loading utility (sstableloader). + + 0.8 ===
svn commit: r1136307 - in /cassandra/branches/cassandra-0.8: ./ debian/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/commitlog/ test/unit/org/apache/cassandra/locator/
Author: slebresne
Date: Thu Jun 16 07:42:11 2011
New Revision: 1136307
URL: http://svn.apache.org/viewvc?rev=1136307view=rev
Log:
Add missing headers and bump build version
Removed:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/commitlog/CommitLogHeader.java
Modified:
cassandra/branches/cassandra-0.8/build.xml
cassandra/branches/cassandra-0.8/debian/changelog
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/EchoedRow.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/locator/EC2SnitchTest.java
Modified: cassandra/branches/cassandra-0.8/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/build.xml?rev=1136307r1=1136306r2=1136307view=diff
==
--- cassandra/branches/cassandra-0.8/build.xml (original)
+++ cassandra/branches/cassandra-0.8/build.xml Thu Jun 16 07:42:11 2011
@@ -25,8 +25,8 @@
property name=debuglevel value=source,lines,vars/
!-- default version and SCM information (we need the default SCM info as
people may checkout with git-svn) --
-property name=base.version value=0.8.0/
-property name=scm.default.path
value=cassandra/branches/cassandra-0.7/
+property name=base.version value=0.8.1/
+property name=scm.default.path
value=cassandra/branches/cassandra-0.8/
property name=scm.default.connection
value=scm:svn:http://svn.apache.org/repos/asf/${scm.default.path}/
property name=scm.default.developerConnection
value=scm:svn:https://svn.apache.org/repos/asf/${scm.default.path}/
property name=scm.default.url
value=http://svn.apache.org/viewvc/${scm.default.path}/
Modified: cassandra/branches/cassandra-0.8/debian/changelog
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/debian/changelog?rev=1136307r1=1136306r2=1136307view=diff
==
--- cassandra/branches/cassandra-0.8/debian/changelog (original)
+++ cassandra/branches/cassandra-0.8/debian/changelog Thu Jun 16 07:42:11 2011
@@ -1,3 +1,9 @@
+cassandra (0.8.1) unstable; urgency=low
+
+ * New release
+
+ -- Sylvain Lebresne slebre...@apache.org Thu, 16 Jun 2011 09:37:27 +0200
+
cassandra (0.8.0) unstable; urgency=low
* New release
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/EchoedRow.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/EchoedRow.java?rev=1136307r1=1136306r2=1136307view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/EchoedRow.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/EchoedRow.java
Thu Jun 16 07:42:11 2011
@@ -1,4 +1,25 @@
package org.apache.cassandra.db;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
import java.io.DataOutput;
import java.io.IOException;
Modified:
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/locator/EC2SnitchTest.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/locator/EC2SnitchTest.java?rev=1136307r1=1136306r2=1136307view=diff
==
---
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/locator/EC2SnitchTest.java
(original)
+++
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/locator/EC2SnitchTest.java
Thu Jun 16 07:42:11 2011
@@ -1,4 +1,25 @@
package org.apache.cassandra.locator;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law
[jira] [Created] (CASSANDRA-2780) sstable json convertion needs to encode quotes
sstable json convertion needs to encode quotes
--
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code 102)):
was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2780) sstable json conversion needs to encode quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timo Nentwig updated CASSANDRA-2780:
Summary: sstable json conversion needs to encode quotes (was: sstable json
convertion needs to encode quotes)
sstable json conversion needs to encode quotes
--
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2780) sstable json conversion needs to encode quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timo Nentwig updated CASSANDRA-2780:
Priority: Critical (was: Major)
sstable json conversion needs to encode quotes
--
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Priority: Critical
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2780) sstable json conversion needs to encode quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050274#comment-13050274
]
Timo Nentwig commented on CASSANDRA-2780:
-
Fix is easy:
$ cat /tmp/json
{
74657374: [[data, {\foo\:\bar\}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json
/var/lib/cassandra/data/foo/transactions-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
Importing 1 keys...
1 keys imported successfully.
[default@foo] get transactions[test][data];
= (column=data, value={foo:bar}, timestamp=1308209845388000)
sstable json conversion needs to encode quotes
--
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050287#comment-13050287 ] Sylvain Lebresne commented on CASSANDRA-2521: - Actually I started working on this yesterday evening and I think I'm almost done. So re-assigning to myself for now :) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136328 - /cassandra/branches/cassandra-0.8/debian/rules
Author: slebresne Date: Thu Jun 16 08:54:42 2011 New Revision: 1136328 URL: http://svn.apache.org/viewvc?rev=1136328view=rev Log: Remove CQL drivers from debian packaging Modified: cassandra/branches/cassandra-0.8/debian/rules Modified: cassandra/branches/cassandra-0.8/debian/rules URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/debian/rules?rev=1136328r1=1136327r2=1136328view=diff == --- cassandra/branches/cassandra-0.8/debian/rules (original) +++ cassandra/branches/cassandra-0.8/debian/rules Thu Jun 16 08:54:42 2011 @@ -40,8 +40,6 @@ install: build usr/share/cassandra dh_install build/apache-cassandra-thrift-$(VERSION).jar \ usr/share/cassandra - dh_install build/apache-cassandra-cql-*.jar \ - usr/share/cassandra dh_link usr/share/cassandra/apache-cassandra-$(VERSION).jar \ usr/share/cassandra/apache-cassandra.jar
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050309#comment-13050309 ] Terje Marthinussen commented on CASSANDRA-2521: --- Fabulous! If you backport that to 0.8 (or if it is back-portable), I will volunteer to give it a decent doze of real life testing asap. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2675) java.io.IOError: java.io.EOFException with version 0.7.6
[ https://issues.apache.org/jira/browse/CASSANDRA-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050311#comment-13050311 ] rene kochen commented on CASSANDRA-2675: According to the changes of Cassandra 0.8.0, this is fixed in 0.8.0. The fixed versions in this issue says 0.7.7 and 0.8.1 java.io.IOError: java.io.EOFException with version 0.7.6 - Key: CASSANDRA-2675 URL: https://issues.apache.org/jira/browse/CASSANDRA-2675 Project: Cassandra Issue Type: Bug Components: Core Environment: Reproduced on single Cassandra node (CentOS 5.5) Reproduced on single Cassandra node (Windows Server 2008) Reporter: rene kochen Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.7, 0.8.1 Attachments: 0001-Don-t-remove-columns-from-super-columns-in-memtable.patch, 0002-Avoid-modifying-super-column-in-memtable-being-flush-v2.patch, 0002-Avoid-modifying-super-column-in-memtable-being-flush.patch, CassandraIssue.zip, CassandraIssueJava.zip I use the following data-model column_metadata: [] name: Customers column_type: Super gc_grace_seconds: 60 I have a super-column-family with a single row. Within this row I have a single super-column. Within this super-column, I concurrently create, read and delete columns. I have three threads: - Do in a loop: add a column to the super-column. - Do in a loop: delete a random column from the super-column. - Do in a loop: read the super-column (with all columns). After running the above threads concurrently, I always receive one of the following errors: ERROR 17:09:57,036 Fatal exception in thread Thread[ReadStage:81,5,main] java.io.IOError: java.io.EOFException at org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:252) at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:268) at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:227) at java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(Unknown Source) at java.util.concurrent.ConcurrentSkipListMap.init(Unknown Source) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:379) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:362) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:322) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108) at org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:283) at org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326) at org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:69) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:116) at org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:130) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1390) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1267) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1195) at org.apache.cassandra.db.Table.getRow(Table.java:324) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:451) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.EOFException at java.io.RandomAccessFile.readByte(Unknown Source) at
[jira] [Commented] (CASSANDRA-2779) files not cleaned up by GC?
[ https://issues.apache.org/jira/browse/CASSANDRA-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050314#comment-13050314 ] Terje Marthinussen commented on CASSANDRA-2779: --- If we get CASSANDRA-2521 in for 0.8 then first thing would be to check if this magically fixes itself. If not, I think I can volunteer to tune things further. Starting to get familiar with the code and I already have a setup to test and reproduce. files not cleaned up by GC? --- Key: CASSANDRA-2779 URL: https://issues.apache.org/jira/browse/CASSANDRA-2779 Project: Cassandra Issue Type: Bug Reporter: Terje Marthinussen This is 0.8.0 + a few 0.8.1 patches on repair. We tested repair on 2 nodes in the cluster last night. Interestingly enough, I don't believe the node described here is in any way neighbour of the nodes we tested repair on so I am not sure why it is streaming data both in and out, but in any case, it has joined the streaming party. We now see: ERROR [CompactionExecutor:5] 2011-06-16 09:12:23,928 CompactionManager.java (line 510) insufficient space to compact even the two smallest files, aborting INFO [CompactionExecutor:5] 2011-06-16 09:12:23,929 StorageService.java (line 2071) requesting GC to free disk space And we see a lot of them: INFO [CompactionExecutor:5] 2011-06-16 09:11:59,164 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:5] 2011-06-16 09:12:23,929 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:5] 2011-06-16 09:12:46,489 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:17:53,299 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:18:17,782 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:18:42,078 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:19:06,984 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:19:32,079 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:19:57,265 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:20:22,706 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:20:47,331 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:21:13,062 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:21:38,288 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:22:03,500 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:22:29,407 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:22:55,577 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:23:20,951 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:23:46,448 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:3] 2011-06-16 09:24:12,030 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 09:48:00,633 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 09:48:26,119 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 09:48:49,002 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 10:10:20,196 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 10:10:45,322 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:6] 2011-06-16 10:11:07,619 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:7] 2011-06-16 11:01:45,562 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:7] 2011-06-16 11:02:10,236 StorageService.java (line 2071) requesting GC to free disk space INFO [CompactionExecutor:7] 2011-06-16 11:05:31,297 StorageService.java (line 2071) requesting GC to free disk space Available disk is 105GB and it is trying to compact a set of the largest sstables. There is probably easily enough disk to do so, but the
[jira] [Updated] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timo Nentwig updated CASSANDRA-2780:
Summary: sstable2json needs to escape quotes (was: sstable json conversion
needs to encode quotes)
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Priority: Critical
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-2405: --- Attachment: CASSANDRA-2405-v4.patch Made two CFs: SuccessfulRepairs (to use with `./bin/nodetool time` command) and RepairDetailedInfo - with the structure from my previous comment. should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2521: Attachment: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch Attaching patch against trunk. Tests are passing and it seems to work, at least with small tests. I started a stress on a 3 node cluster with a repair and a major compaction started towards the end and compacted files did wait to be fully streamed to be removed and I didn't hit any bump (I did hit CASSANDRA-2769 a bunch of time but that's another story). Still, this is a fairly tricky problem so it could use other eyes. The basics are fairly simple though: each time a thread want to do something with a SSTableReader, it acquires a reference to that sstable and releases it when done. SSTableReader just keep a counter of acquired references. When the sstable has been marked compacted, we start looking until all acquired reference has been released. When that's the case, the file can be removed. Obviously the main drawback of this approach (compared to the phantomReference one) is that there is room for error. If a consumer forgot to acquire a reference (or do it in a non-thread-safe manner), the sstable can be removed. Thankfully there is not so many place in the code that needs to do this so hopefully I haven't missed any place. The other thing is that if a reference on a sstable is acquired, it should be released (otherwise the sstable will not be removed until next restart). I've try to ensure this using try-catch block, but it's not really possible with the way streaming works. However, if streaming fails, it's not really worst than before since the files where not cleaned due to the (failed) session staying in the global map of streaming sessions. CASSANDRA-2433 should fix that in most cases anyway. Last thing is http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4715154. In other words, the deletion of a file won't work until the mmapping is finalized (aka, GC, Where art thou), at least on windows. For that reason, when the deletion of file fails (after the usual number of retries, which btw may make less sense now), the deletion task is saved in a global list. If Cassandra is low on disk, it will still trigger a GC, after which it will reschedule all failed files in the hope they can now be deleted. There is also a JMX call to retry this rescheduling. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2774) one way to make counter delete work better
[ https://issues.apache.org/jira/browse/CASSANDRA-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050016#comment-13050016 ] Yang Yang edited comment on CASSANDRA-2774 at 6/16/11 1:40 PM: --- ---Once compaction has compacted the deletes, all node will reach common agreement. we probably need to look at this more closely. I was going to use the example of a node always changing its report due to the different merging order in each read, but since you pointed out that we should allow all compaction to finish before making the judgement, let's look at another ill-case: the way I contrive the following case is very similar to the often-mentioned compaction ill-case, and we move that effect onto network messages, because changing the order of merging sstables in compaction is very similar to changing the order of message deliveries. let's say we have 4 nodes, A B C D. all the traffic we observe is, with increasing timestamp(): A leader add 1 ts=100 B leader delete ts=200 C leader add 2 ts=300 now the updates so far start to replicate to D: assume that D sees the following order: A.(add 1), C.(add 2), B.(delete), after these, D's state is: [A:1 C:2, last_delete=200, timestamp=300] now let's all the traffic between A,B,C go through, and they fully resolve (receiving pair-wise messages and etc), so A B C all come to state: [A:nil C:2, last_delete=200 timestamp=300] now A's state and D's state are different, let's say we let A repair D, A's A-shard has a lower clock, so D will win; if we let D repair A, A's A-shard is isDelta(), so it trumps D. as a result it seems we *never reach agreement between A and D*, even though traffic is allowed to flow freely. I just started looking inside the CounterContext logic, so I could very well be wrong. Thanks for your time looking through this. as to performance, I don't think it will be a significant increase: 1) most application use cases will increment the same counter for many times, while it is in memtable. it's hard to imagine that most counters will be incremented only once before being dumped out to sstable. only the first increment in memtable for each counter will suffer a read into disk, if on average each counter is incremented 1000 times before being flushed, the disk read cost is amortized over 1000 increments; 2) even if we do the disk read, any realistic counter setup already needs ROW and CLONE, so a disk read is needed anyway before the client is acked. here we do an extra disk read, but when we do the regular disk read for CounterMutation.makeReplicationMutation() , the file blocks are already brought into cache by the new extra read, so it saves time, and total disk access time remains roughly the same. was (Author: yangyangyyy): ---Once compaction has compacted the deletes, all node will reach common agreement. we probably need to look at this more closely. I was going to use the example of a node always changing its report due to the different merging order in each read, but since you pointed out that we should allow all compaction to finish before making the judgement, let's look at another ill-case: the way I contrive the following case is very similar to the often-mentioned compaction ill-case, and we move that effect onto network messages, because changing the order of merging sstables in compaction is very similar to changing the order of message deliveries. let's say we have 4 nodes, A B C D. all the traffic we observe is, with increasing timestamp(): A leader add 1 ts=100 B leader delete ts=200 C leader add 2 ts=300 now the updates so far start to replicate to D: assume that D sees the following order: A.(add 1), C.(add 2), B.(delete), after these, D's state is: [A:1 C:2, last_delete=200, timestamp=300] now let's all the traffic between A,B,C go through, and they fully resolve (receiving pair-wise messages and etc), so A B C all come to state: [A:nil C:2, last_delete=200 timestamp=300] now A's state and D's state are different, let's say we let A repair D, A's A-shard has a lower clock, so D will win; if we let D repair A, A's A-shard is isDelta(), so it trumps D. as a result we never reach agreement between A and D, even though traffic is allowed to flow freely. I just started looking inside the CounterContext logic, so I could very well be wrong. Thanks for your time looking through this. as to performance, I don't think it will be a significant increase: 1) most application use cases will increment the same counter for many times, while it is in memtable. it's hard to imagine that most counters will be incremented only once before being dumped out to sstable. only the first increment in memtable for each counter will suffer a read into disk, if on average each counter is incremented 1000 times before being flushed, the disk read
[jira] [Updated] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2780:
--
Priority: Minor (was: Critical)
Fix Version/s: 0.8.2
Assignee: Pavel Yaskevich
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Burroughs updated CASSANDRA-1966: --- Attachment: 1966-v1.txt Rough draft against trunk, not quite ready for full review yet. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2781) regression: exposing cache size through MBean
regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2781) regression: exposing cache size through MBean
[ https://issues.apache.org/jira/browse/CASSANDRA-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Burroughs updated CASSANDRA-2781: --- Attachment: 2781-v1.txt Just size -- getSize in InstrumentingCacheMBean regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 2781-v1.txt Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2782) Create a debian package for the CQL drivers
Create a debian package for the CQL drivers --- Key: CASSANDRA-2782 URL: https://issues.apache.org/jira/browse/CASSANDRA-2782 Project: Cassandra Issue Type: Wish Components: Packaging Reporter: Sylvain Lebresne Priority: Minor Since the CQL drivers are not release in lockstep with Cassandra, they are excluded from the Cassandra debian package. Creating a debian package for them could make debian user's live a bit easier. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136472 - /cassandra/trunk/test/conf/cassandra.yaml
Author: jbellis Date: Thu Jun 16 15:00:04 2011 New Revision: 1136472 URL: http://svn.apache.org/viewvc?rev=1136472view=rev Log: fix tests Modified: cassandra/trunk/test/conf/cassandra.yaml Modified: cassandra/trunk/test/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/trunk/test/conf/cassandra.yaml?rev=1136472r1=1136471r2=1136472view=diff == --- cassandra/trunk/test/conf/cassandra.yaml (original) +++ cassandra/trunk/test/conf/cassandra.yaml Thu Jun 16 15:00:04 2011 @@ -14,7 +14,6 @@ rpc_port: 9170 column_index_size_in_kb: 4 commitlog_directory: build/test/cassandra/commitlog saved_caches_directory: build/test/cassandra/saved_caches -commitlog_rotation_threshold_in_mb: 128 data_file_directories: - build/test/cassandra/data disk_access_mode: mmap
svn commit: r1136514 - in /cassandra/trunk: lib/commons-collections-3.2.1.jar src/java/org/apache/cassandra/utils/MergeIterator.java
Author: jbellis
Date: Thu Jun 16 16:10:09 2011
New Revision: 1136514
URL: http://svn.apache.org/viewvc?rev=1136514view=rev
Log:
add missing src/java/org/apache/cassandra/utils/MergeIterator.java
Added:
cassandra/trunk/src/java/org/apache/cassandra/utils/MergeIterator.java
Removed:
cassandra/trunk/lib/commons-collections-3.2.1.jar
Added: cassandra/trunk/src/java/org/apache/cassandra/utils/MergeIterator.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/utils/MergeIterator.java?rev=1136514view=auto
==
--- cassandra/trunk/src/java/org/apache/cassandra/utils/MergeIterator.java
(added)
+++ cassandra/trunk/src/java/org/apache/cassandra/utils/MergeIterator.java Thu
Jun 16 16:10:09 2011
@@ -0,0 +1,224 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* License); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*/
+package org.apache.cassandra.utils;
+
+import java.io.IOException;
+import java.io.IOError;
+import java.util.ArrayDeque;
+import java.util.Comparator;
+import java.util.List;
+import java.util.PriorityQueue;
+
+import com.google.common.collect.AbstractIterator;
+import com.google.common.collect.Ordering;
+
+/** Merges sorted input iterators which individually contain unique items. */
+public abstract class MergeIteratorIn,Out extends AbstractIteratorOut
implements CloseableIteratorOut
+{
+public final ComparatorIn comp;
+protected final List? extends CloseableIteratorIn iterators;
+// a queue for return: all candidates must be open and have at least one
item
+protected final PriorityQueueCandidateIn queue;
+
+protected MergeIterator(List? extends CloseableIteratorIn iters,
ComparatorIn comp)
+{
+this.iterators = iters;
+this.comp = comp;
+this.queue = new PriorityQueueCandidateIn(Math.max(1,
iters.size()));
+for (CloseableIteratorIn iter : iters)
+{
+CandidateIn candidate = new CandidateIn(iter, comp);
+if (!candidate.advance())
+// was empty
+continue;
+this.queue.add(candidate);
+}
+}
+
+public static E MergeIteratorE,E get(List? extends
CloseableIteratorE iters)
+{
+return get(iters, (ComparatorE)Ordering.natural());
+}
+
+public static E MergeIteratorE,E get(List? extends
CloseableIteratorE iters, ComparatorE comp)
+{
+return new OneToOneE(iters, comp);
+}
+
+public static In,Out MergeIteratorIn,Out get(List? extends
CloseableIteratorIn iters, ComparatorIn comp, ReducerIn,Out reducer)
+{
+return new ManyToOneIn,Out(iters, comp, reducer);
+}
+
+public Iterable? extends CloseableIteratorIn iterators()
+{
+return iterators;
+}
+
+/**
+ * Consumes sorted items from the queue: should only remove items from the
queue,
+ * not add them.
+ */
+protected abstract Out consume();
+
+/**
+ * Returns consumed items to the queue.
+ */
+protected abstract void advance();
+
+protected final Out computeNext()
+{
+advance();
+return consume();
+}
+
+public void close()
+{
+for (CloseableIteratorIn iterator : this.iterators)
+{
+try
+{
+iterator.close();
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+}
+}
+
+/** A MergeIterator that returns a single value for each one consumed. */
+private static final class OneToOneE extends MergeIteratorE,E
+{
+// the last returned candidate, so that we can lazily call 'advance()'
+protected CandidateE candidate;
+public OneToOne(List? extends CloseableIteratorE iters,
ComparatorE comp)
+{
+super(iters, comp);
+}
+
+protected final E consume()
+{
+candidate = queue.poll();
+if (candidate == null)
+return endOfData();
+return candidate.item;
+}
+
+protected final void advance()
+{
+if (candidate != null candidate.advance())
+// has more items
+
[jira] [Created] (CASSANDRA-2783) Explore adding replay ID for counters
Explore adding replay ID for counters - Key: CASSANDRA-2783 URL: https://issues.apache.org/jira/browse/CASSANDRA-2783 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Sylvain Lebresne If a counter write returns a TimeoutException, the client cannot retry its write without risking an overcount. One idea to fix this would be to allow the client to specify a replay ID with each counter write unique to the write. If the write timeout, the client would resubmit the write with the same replay ID and the system would ensure that write is only replayed if the previous write was not persisted. Of course, the last part of this (the system would ensure ...) is the hard part. Still worth exploring I believe. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1373 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1136514 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-2781) regression: exposing cache size through MBean
[ https://issues.apache.org/jira/browse/CASSANDRA-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050532#comment-13050532 ] Ryan King commented on CASSANDRA-2781: -- It would be nice if we had some tests around these things, but I'm +1 on this patch. regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 2781-v1.txt Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2761) JDBC driver does not build
[
https://issues.apache.org/jira/browse/CASSANDRA-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050534#comment-13050534
]
Eric Evans commented on CASSANDRA-2761:
---
{quote}
I can try an get them all in but I think we should seriously rethink that
strategy for now. With this many dependancies they should probably be put in
their own jar(s). It would really make the driver jar have to keep up with
detailed dependency changes in the server code base. Miss one and it's messy
and the errors are non-obvious.
{quote}
What would you call such a jar? When I looked at this, there didn't seem to be
any delineation that made sense.
{quote}
I was a big whiner about not carrying around the whole server just for client
access, but until it is re-factored, I can now appreciate the horror that will
commence if we piece-meal drag over classes from the server into the JAR.
{quote}
What are these 50 other class dependencies? Where are they being drug in?
JDBC driver does not build
--
Key: CASSANDRA-2761
URL: https://issues.apache.org/jira/browse/CASSANDRA-2761
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Rick Shaw
Fix For: 1.0
Attachments: jdbc-driver-build-v1.txt
Need a way to build (and run tests for) the Java driver.
Also: still some vestigal references to drivers/ in trunk build.xml.
Should we remove drivers/ from the 0.8 branch as well?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-2780:
---
Attachment: CASSANDRA-2780.patch
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
Attachments: CASSANDRA-2780.patch
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136529 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/cache/InstrumentingCache.java src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java sr
Author: jbellis
Date: Thu Jun 16 16:23:03 2011
New Revision: 1136529
URL: http://svn.apache.org/viewvc?rev=1136529view=rev
Log:
fix cache mbean getSize
patch by Chris Burroughs; reviewed by Ryan King for CASSANDRA-2781
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1136529r1=1136528r2=1136529view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Thu Jun 16 16:23:03 2011
@@ -1,3 +1,7 @@
+0.8.2
+ * fix cache mbean getSize (CASSANDRA-2781)
+
+
0.8.1
* CQL:
- support for insert, delete in BATCH (CASSANDRA-2537)
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java?rev=1136529r1=1136528r2=1136529view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java
Thu Jun 16 16:23:03 2011
@@ -108,6 +108,11 @@ public class InstrumentingCacheK, V im
return map.size();
}
+public int getSize()
+{
+return size();
+}
+
public long getHits()
{
return hits.get();
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java?rev=1136529r1=1136528r2=1136529view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java
Thu Jun 16 16:23:03 2011
@@ -25,7 +25,7 @@ public interface InstrumentingCacheMBean
{
public int getCapacity();
public void setCapacity(int capacity);
-public int size();
+public int getSize();
/** total request count since cache creation */
public long getRequests();
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java?rev=1136529r1=1136528r2=1136529view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java
Thu Jun 16 16:23:03 2011
@@ -450,7 +450,7 @@ public class NodeCmd
if (keyCacheMBean.getCapacity() 0)
{
outs.println(\t\tKey cache capacity: +
keyCacheMBean.getCapacity());
-outs.println(\t\tKey cache size: +
keyCacheMBean.size());
+outs.println(\t\tKey cache size: +
keyCacheMBean.getSize());
outs.println(\t\tKey cache hit rate: +
keyCacheMBean.getRecentHitRate());
}
else
@@ -462,7 +462,7 @@ public class NodeCmd
if (rowCacheMBean.getCapacity() 0)
{
outs.println(\t\tRow cache capacity: +
rowCacheMBean.getCapacity());
-outs.println(\t\tRow cache size: +
rowCacheMBean.size());
+outs.println(\t\tRow cache size: +
rowCacheMBean.getSize());
outs.println(\t\tRow cache hit rate: +
rowCacheMBean.getRecentHitRate());
}
else
[jira] [Updated] (CASSANDRA-2781) regression: exposing cache size through MBean
[ https://issues.apache.org/jira/browse/CASSANDRA-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2781: -- Affects Version/s: 0.8 beta 1 Fix Version/s: 0.8.2 regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 beta 1 Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Fix For: 0.8.2 Attachments: 2781-v1.txt Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2521: -- Reviewer: bcoverston Component/s: Core Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050551#comment-13050551 ] Jonathan Ellis commented on CASSANDRA-2521: --- bq. the deletion of a file won't work until the mmapping is finalized (aka, GC, Where art thou), at least on windows Ugh, totally forgot about that. Probably one of the reasons we went with might as well use a phantom reference in the first place. Linux does allow the delete but I suspect the space doesn't actually get freed up until the munmap still. So not really an improvement there either. Is it possible to do mmap/munmap via JNA for libc-based systems? I think I'd rather fall back to non-mmap'd i/o on windows, than continue to hack around the GC like this. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136541 - /cassandra/drivers/py/setup.py
Author: eevans Date: Thu Jun 16 16:48:32 2011 New Revision: 1136541 URL: http://svn.apache.org/viewvc?rev=1136541view=rev Log: bump version to 1.0.4 Modified: cassandra/drivers/py/setup.py Modified: cassandra/drivers/py/setup.py URL: http://svn.apache.org/viewvc/cassandra/drivers/py/setup.py?rev=1136541r1=1136540r2=1136541view=diff == --- cassandra/drivers/py/setup.py (original) +++ cassandra/drivers/py/setup.py Thu Jun 16 16:48:32 2011 @@ -20,7 +20,7 @@ from os.path import abspath, join, dirna setup( name=cql, -version=1.0.3, +version=1.0.4, description=Cassandra Query Language driver, long_description=open(abspath(join(dirname(__file__), 'README'))).read(), url=http://cassandra.apache.org;,
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050560#comment-13050560
]
T Jake Luciani commented on CASSANDRA-2521:
---
We could use this trick:
{noformat}
try {
// Manually free the old buffer using undocumented unsafe APIs.
// If this fails, we'll drop the reference and hope GC finds it
// eventually.
Object cleaner = buf.getClass().getMethod(cleaner).invoke(buf);
cleaner.getClass().getMethod(clean).invoke(cleaner);
} catch (Exception e) {
// Perhaps a non-sun-derived JVM - contributions welcome
LOG.warn(Couldn't realloc bytebuffer, e);
}
{noformat}
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050563#comment-13050563 ] Jonathan Ellis commented on CASSANDRA-1966: --- Do we want to specify in terms of % or absolute key count? Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2477) CQL support for describing keyspaces / column familes
[
https://issues.apache.org/jira/browse/CASSANDRA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050567#comment-13050567
]
Jonathan Ellis commented on CASSANDRA-2477:
---
{quote}
What I want to avoid is a special query type (i.e. anything not SELECT)
because it makes the language less orthogonal because of implementation details
that are subject to change (in an ideal world, we'd move away from avro and
store schema information in real columns, with indexes so you could easily
say give me all the columns for CF X at schema version Y.)
You'd also be limited to basically an RPC style call – no specifying which
columns to select, or which rows you're interested in. Not without reinventing
that wheel on a LOT of code (because SELECT right now relies on CFS to perform
the actual queries).
{quote}
CQL support for describing keyspaces / column familes
-
Key: CASSANDRA-2477
URL: https://issues.apache.org/jira/browse/CASSANDRA-2477
Project: Cassandra
Issue Type: Sub-task
Components: API, Core
Reporter: Eric Evans
Labels: cql
Fix For: 0.8.2
Attachments: 2477-virtual-cfs-false-start.txt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136547 - /cassandra/trunk/build.xml
Author: jbellis
Date: Thu Jun 16 17:10:34 2011
New Revision: 1136547
URL: http://svn.apache.org/viewvc?rev=1136547view=rev
Log:
s/1.1/1.2/ for clhm pom
Modified:
cassandra/trunk/build.xml
Modified: cassandra/trunk/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/trunk/build.xml?rev=1136547r1=1136546r2=1136547view=diff
==
--- cassandra/trunk/build.xml (original)
+++ cassandra/trunk/build.xml Thu Jun 16 17:10:34 2011
@@ -352,7 +352,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
dependency groupId=commons-codec artifactId=commons-codec
version=1.2/
dependency groupId=commons-collections
artifactId=commons-collections version=3.2.1/
dependency groupId=commons-lang artifactId=commons-lang
version=2.4/
- dependency groupId=com.googlecode.concurrentlinkedhashmap
artifactId=concurrentlinkedhashmap-lru version=1.1/
+ dependency groupId=com.googlecode.concurrentlinkedhashmap
artifactId=concurrentlinkedhashmap-lru version=1.2/
dependency groupId=org.antlr artifactId=antlr version=3.2/
dependency groupId=org.slf4j artifactId=slf4j-api
version=1.6.1/
dependency groupId=org.slf4j artifactId=slf4j-log4j12
version=1.6.1/
[jira] [Resolved] (CASSANDRA-2681) Upgrade ConcurrentLinkedHashMap (v1.2)
[ https://issues.apache.org/jira/browse/CASSANDRA-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2681. --- Resolution: Fixed s/1.1/1.2/ in r1136547. If it's more complicated than that please submit a patch. :) Upgrade ConcurrentLinkedHashMap (v1.2) -- Key: CASSANDRA-2681 URL: https://issues.apache.org/jira/browse/CASSANDRA-2681 Project: Cassandra Issue Type: Task Components: Core Reporter: Ben Manes Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0 This release has numerous performance improvements. See the change log for details. It also includes a few useful features that may be of interest, - Snapshot iteration in order of hotness (CASSANDRA-1966) - Optionally defer LRU maintenance penalty to a background executor (instead of amortized on caller threads) (Note that this setting is not advised if write storms are not rate limited, since it defers eviction until the executor runs.) http://code.google.com/p/concurrentlinkedhashmap/ http://code.google.com/p/concurrentlinkedhashmap/wiki/ExampleUsage Verified compatibility with NonBlockingHashMap. Cassandra may want to consider adding the java_util_concurrent_chm.jar to the bootclasspath to swap all CHM usages with NBHM (CLHM is a decorator on top of CHM). http://high-scale-lib.cvs.sourceforge.net/viewvc/high-scale-lib/high-scale-lib/README -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions per-key
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050572#comment-13050572 ] Jonathan Ellis commented on CASSANDRA-2369: --- bq. There is a similar problem on bootstrap and node movement. Instead of asking a single replica to stream data from the range a new node is assuming, it will have to ask all replicas that may have rows for that range to make sure it doesn't miss any. Additionally, each source replica would have to deserialize each row in the range to make sure it's actually supposed to be streamed, which would slow streaming down significantly. Starting to think this is not worth trying to support. support replication decisions per-key - Key: CASSANDRA-2369 URL: https://issues.apache.org/jira/browse/CASSANDRA-2369 Project: Cassandra Issue Type: New Feature Reporter: Jonathan Ellis Assignee: Vijay Priority: Minor Fix For: 1.0 Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2614) create Column and CounterColumn in the same column family
[ https://issues.apache.org/jira/browse/CASSANDRA-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050153#comment-13050153 ] Dave Rav edited comment on CASSANDRA-2614 at 6/16/11 5:28 PM: -- Is it possible (inside a thrift Mutation) to loop through all the columns that need to be deleted find there type, 'validation_class' (regular or CounterColumnType) and delete them base on there type? was (Author: daver...@yahoo.com): Is it possible to loop through all the columns that need to be deleted find there type, 'validation_class' (regular or CounterColumnType) and delete them base on there type? create Column and CounterColumn in the same column family - Key: CASSANDRA-2614 URL: https://issues.apache.org/jira/browse/CASSANDRA-2614 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Dave Rav Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 create Column and CounterColumn in the same column family -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate Compaction Marker
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050579#comment-13050579 ] Jonathan Ellis commented on CASSANDRA-2769: --- For trunk patches, I'm not comfortable w/ 0001 reassigning the sstables field on general principles either. We could have the compaction proceed using smallerSSTables as a simpler alternative, but in general this organization feels like negative progress from the 0.8 doCompaction/doCompactionWithoutSizeEstimation. trunk 0002 looks fine. Cannot Create Duplicate Compaction Marker - Key: CASSANDRA-2769 URL: https://issues.apache.org/jira/browse/CASSANDRA-2769 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.8.2 Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully. java.io.IOError: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638) at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321) at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294) at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255) at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634) ... 12 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2732) StringIndexOutOfBoundsException when specifying JDBC connection string without user and password
[
https://issues.apache.org/jira/browse/CASSANDRA-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050584#comment-13050584
]
Jonathan Ellis commented on CASSANDRA-2732:
---
Is this subsumed by CASSANDRA-2754?
StringIndexOutOfBoundsException when specifying JDBC connection string
without user and password
Key: CASSANDRA-2732
URL: https://issues.apache.org/jira/browse/CASSANDRA-2732
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8 beta 1
Reporter: Cathy Daw
Assignee: Rick Shaw
Priority: Trivial
Labels: cql
*PASS: specify connection string user and password*
_connection =
DriverManager.getConnection(jdbc:cassandra:root/root@localhost:9170/default)_
*FAIL: specify connection string without user and password*
_connection =
DriverManager.getConnection(jdbc:cassandra://localhost:9170/default)_
{code}
[junit] String index out of range: -1
[junit] java.lang.StringIndexOutOfBoundsException: String index out of range:
-1
[junit] at java.lang.String.substring(String.java:1937)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraConnection.init(CassandraConnection.java:74)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraConnection.init(CassandraConnection.java:74)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraDriver.connect(CassandraDriver.java:86)
[junit] at java.sql.DriverManager.getConnection(DriverManager.java:582)
[junit] at java.sql.DriverManager.getConnection(DriverManager.java:207)
[junit] at
com.datastax.cql.runJDBCSmokeTest.setUpBeforeClass(runJDBCSmokeTest.java:45)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2766) ConcurrentModificationException during node recovery
[ https://issues.apache.org/jira/browse/CASSANDRA-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2766. --- Resolution: Fixed Fix Version/s: 0.8.1 Reviewer: slebresne (committed) ConcurrentModificationException during node recovery Key: CASSANDRA-2766 URL: https://issues.apache.org/jira/browse/CASSANDRA-2766 Project: Cassandra Issue Type: Bug Components: Core Reporter: Terje Marthinussen Assignee: Jonathan Ellis Fix For: 0.8.1 Attachments: 2766-v2.txt, 2766.txt Testing some node recovery operations. In this case: 1. Data is being added/updated as it would in production 2. repair is running on other nodes in the cluster 3. we wiped data on this node and started up again, but before repair was actually started on this node (but it had gotten data through the regular data feed) we got this error. I see no indication in the logs that outgoing streams has been started, but the node have finished one incoming stream before this (I guess from some other node doing repair). INFO [CompactionExecutor:11] 2011-06-14 14:15:09,078 SSTableReader.java (line 155) Opening /data/cassandra/node1/data/JP/test-g-8 INFO [CompactionExecutor:13] 2011-06-14 14:15:09,079 SSTableReader.java (line 155) Opening /data/cassandra/node1/data/JP/test-g-10 INFO [HintedHandoff:1] 2011-06-14 14:15:26,623 HintedHandOffManager.java (line 302) Started hinted handoff for endpoint /1.10.42.216 INFO [HintedHandoff:1] 2011-06-14 14:15:26,623 HintedHandOffManager.java (line 358) Finished hinted handoff of 0 rows to endpoint /1.10.42.216 INFO [CompactionExecutor:9] 2011-06-14 14:15:29,417 SSTableReader.java (line 155) Opening /data/cassandra/node1/data/JP/Datetest-g-2 ERROR [Thread-84] 2011-06-14 14:15:36,755 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[Thread-84,5,main] java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:132) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:155) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:93) ERROR [Thread-79] 2011-06-14 14:15:36,755 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[Thread-79,5,main] java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:132) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:155) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:93) ERROR [Thread-83] 2011-06-14 14:15:36,755 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[Thread-83,5,main] java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:132) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:155) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:93) ERROR [Thread-85] 2011-06-14 14:15:36,755 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[Thread-85,5,main] java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:132) at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:155) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:93) -- This message is automatically generated by JIRA. For more information on JIRA,
[jira] [Commented] (CASSANDRA-2045) Simplify HH to decrease read load when nodes come back
[ https://issues.apache.org/jira/browse/CASSANDRA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050589#comment-13050589 ] Jonathan Ellis commented on CASSANDRA-2045: --- Code style is generally fine, though there's a few violations of brace-on-newline. :) Simplify HH to decrease read load when nodes come back -- Key: CASSANDRA-2045 URL: https://issues.apache.org/jira/browse/CASSANDRA-2045 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Nicholas Telford Fix For: 1.0 Attachments: CASSANDRA-2045-simplify-hinted-handoff-001.diff, CASSANDRA-2045-simplify-hinted-handoff-002.diff Currently when HH is enabled, hints are stored, and when a node comes back, we begin sending that node data. We do a lookup on the local node for the row to send. To help reduce read load (if a node is offline for long period of time) we should store the data we want forward the node locally instead. We wouldn't have to do any lookups, just take byte[] and send to the destination. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050597#comment-13050597 ] Ryan King commented on CASSANDRA-2521: -- +1 for less hacking around the GC Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2614) create Column and CounterColumn in the same column family
[ https://issues.apache.org/jira/browse/CASSANDRA-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050153#comment-13050153 ] Dave Rav edited comment on CASSANDRA-2614 at 6/16/11 6:12 PM: -- Is it possible (when Deleting, inside a thrift Mutation) to loop through all the columns that need to be deleted find there type, 'validation_class' (regular or CounterColumnType) and delete them base on there type? was (Author: daver...@yahoo.com): Is it possible (inside a thrift Mutation) to loop through all the columns that need to be deleted find there type, 'validation_class' (regular or CounterColumnType) and delete them base on there type? create Column and CounterColumn in the same column family - Key: CASSANDRA-2614 URL: https://issues.apache.org/jira/browse/CASSANDRA-2614 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Dave Rav Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 create Column and CounterColumn in the same column family -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2498) Improve read performance in update-intensive workload
[ https://issues.apache.org/jira/browse/CASSANDRA-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050617#comment-13050617 ] Jonathan Ellis commented on CASSANDRA-2498: --- bq. Column Families with wide rows need some love on the latency side too. Yes, and (I think this is where you were going with this, but to be explicit) this will help with those too, IF the queries are for most-recent-data. Which is the most common kind of wide row from what I've seen. Improve read performance in update-intensive workload - Key: CASSANDRA-2498 URL: https://issues.apache.org/jira/browse/CASSANDRA-2498 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor Labels: ponies Fix For: 1.0 Read performance in an update-heavy environment relies heavily on compaction to maintain good throughput. (This is not the case for workloads where rows are only inserted once, because the bloom filter keeps us from having to check sstables unnecessarily.) Very early versions of Cassandra attempted to mitigate this by checking sstables in descending generation order (mostly equivalent to descending mtime): once all the requested columns were found, it would not check any older sstables. This was incorrect, because data timestamp will not correspond to sstable timestamp, both because compaction has the side effect of refreshing data to a newer sstable, and because hintead handoff may send us data older than what we already have. Instead, we could create a per-sstable piece of metadata containing the most recent (client-specified) timestamp for any column in the sstable. We could then sort sstables by this timestamp instead, and perform a similar optimization (if the remaining sstable client-timestamps are older than the oldest column found in the desired result set so far, we don't need to look further). Since under almost every workload, client timestamps of data in a given sstable will tend to be similar, we expect this to cut the number of sstables down proportionally to how frequently each column in the row is updated. (If each column is updated with each write, we only have to check a single sstable.) This may also be useful information when deciding which SSTables to compact. (Note that this optimization is only appropriate for named-column queries, not slice queries, since we don't know what non-overlapping columns may exist in older sstables.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050628#comment-13050628 ] Chris Burroughs commented on CASSANDRA-1966: My view is that if you are using this it's because you have decided on a constraint like Need to be able to restart within t time units and are choosing how many row keys to save based on that, not as a function of total row cache capacity. I don't mind working backwards from a % configuration if there is a case when that makes more sense. I don't think it's worth the complication to support both. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050611#comment-13050611 ] Peter Schuller commented on CASSANDRA-2521: --- @jbellis The *unlink* is fine (just like with files that are opened w/o mmap), but the actual deletion has to be postponed for fundamental reasons: How would the userland application be informed of an attempted access? You could segfault of course, or similar delivery of an asynchronous event, but that's not very useful to most applications. This closely relates to why Java doesn't allow forced munmap to begin with; having to do some kind of synchronization to allow safe access to munmapped memory in a way that doesn't violate the Java sandbox, would be very costly given that a large point of mmap is to utilize the CPU's MMU in the non-faulting case for a minimum of overhead. (I'm very much +1 on not using the GC for external resources like sstables (for trunk).) @jbellis And yes we can do our own mmap:ing with JNA as an alternative, but given that tjake's solution should work for Sun JVM:s even without JNA that seems preferable to me. Longer term support for non-hotspot JVM:s seems like a lesser concern. Cassandra isn't really something you want to run on an arbitrary JVM anyway. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050637#comment-13050637
]
Jonathan Ellis commented on CASSANDRA-2521:
---
bq. We could use this trick
It's not obvious that this works with mapped buffers (as opposed to normal
malloc'd direct buffers) but it looks like it does:
DirectByteBuffer constructor, used via reflection to create MappedByteBuffer
objects (!)
{code}
// For memory-mapped buffers -- invoked by FileChannelImpl via reflection
protected DirectByteBuffer(int cap, long addr, Runnable unmapper) {
super(-1, 0, cap, cap, true);
address = addr;
viewedBuffer = null;
cleaner = Cleaner.create(this, unmapper);
}
{code}
Here's the unmapper that gets passed, from FCI:
{code}
private static class Unmapper
implements Runnable
{
private long address;
private long size;
private Unmapper(long address, long size) {
assert (address != 0);
this.address = address;
this.size = size;
}
public void run() {
if (address == 0)
return;
unmap0(address, size);
address = 0;
}
}
{code}
(And of course Cleaner.clean just calls run on its Runnable, with a second
layer of only-run-this-once wrapping.)
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050640#comment-13050640 ] Jonathan Ellis commented on CASSANDRA-1966: --- Sounds good. Comments: - row_cache_keys_to_save should be an int, not double - if we use Integer.MAX_VALUE for default we don't need to special case -1 - the purpose of DefaultInteger is to preserve temporary changes made via jmx, when the schema is modified (see DefaultX.isModified and CFS.reload) Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050644#comment-13050644 ] Chris Burroughs commented on CASSANDRA-1966: bq. row_cache_keys_to_save should be an int, not double Makes sense, I was confused as to why [row|key]_cache_size are both doubles but followed the pattern. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of ThirdPartySupport by manumarchal
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The ThirdPartySupport page has been changed by manumarchal:
http://wiki.apache.org/cassandra/ThirdPartySupport?action=diffrev1=10rev2=11
{{http://www.datastax.com/sites/all/themes/datastax20110201/logo.png}}
[[http://datastax.com|Datastax]] provides professional Cassandra support and
services.
+ {{http://acunu.com/images/logo.png}} [[http://www.acunu.com|Acunu]] provides
software products for Cassandra applications, as well as Cassandra support and
professional services.
+
{{http://www.onzra.com/images/Small-Logo.gif}} [[http://www.ONZRA.com|ONZRA]]
has been around for over 10 years and specializes on enterprise grade
architecture, development and security consulting services utilizing many large
scale database technologies such as Cassandra, Oracle, Alegro Graph, and much
more.
-
- {{http://acunu.com/images/logo.png}} [[http://www.acunu.com|Acunu]] provides
software products for Cassandra applications, as well as Cassandra support and
professional services.
{{http://www.impetus.com/sites/impetus.com/impetus/gifs/logo_impetus.png}}
[[http://www.impetus.com/ |Impetus]] provides expertise in Cassandra, Hbase,
[jira] [Commented] (CASSANDRA-1966) Option to control how many items are read on cache load
[ https://issues.apache.org/jira/browse/CASSANDRA-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050655#comment-13050655 ] Jonathan Ellis commented on CASSANDRA-1966: --- that's b/c they support the fractional sizing as well. Option to control how many items are read on cache load --- Key: CASSANDRA-1966 URL: https://issues.apache.org/jira/browse/CASSANDRA-1966 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Attachments: 1966-v1.txt CASSANDRA-1417 added an option to save the key and/or row cache keys which is cool. However, for a row large cache it can take a long time to read all of the rows. For example I have a 400,000 item row cache, and loading that on restart takes a little under an hour. In addition to configuring the size of the row cache, and how often it should be saved to disk, I propose an option to control how many items are loaded on startup (or alternately only saving n items out of the full row cache to begin with). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2732) StringIndexOutOfBoundsException when specifying JDBC connection string without user and password
[
https://issues.apache.org/jira/browse/CASSANDRA-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050667#comment-13050667
]
Rick Shaw commented on CASSANDRA-2732:
--
Yes. that is where the patch is. (i'll have an updated patch today)
StringIndexOutOfBoundsException when specifying JDBC connection string
without user and password
Key: CASSANDRA-2732
URL: https://issues.apache.org/jira/browse/CASSANDRA-2732
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8 beta 1
Reporter: Cathy Daw
Assignee: Rick Shaw
Priority: Trivial
Labels: cql
*PASS: specify connection string user and password*
_connection =
DriverManager.getConnection(jdbc:cassandra:root/root@localhost:9170/default)_
*FAIL: specify connection string without user and password*
_connection =
DriverManager.getConnection(jdbc:cassandra://localhost:9170/default)_
{code}
[junit] String index out of range: -1
[junit] java.lang.StringIndexOutOfBoundsException: String index out of range:
-1
[junit] at java.lang.String.substring(String.java:1937)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraConnection.init(CassandraConnection.java:74)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraConnection.init(CassandraConnection.java:74)
[junit] at
org.apache.cassandra.cql.jdbc.CassandraDriver.connect(CassandraDriver.java:86)
[junit] at java.sql.DriverManager.getConnection(DriverManager.java:582)
[junit] at java.sql.DriverManager.getConnection(DriverManager.java:207)
[junit] at
com.datastax.cql.runJDBCSmokeTest.setUpBeforeClass(runJDBCSmokeTest.java:45)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2754) Consolidating Ticket for JDBC Semantic Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rick Shaw updated CASSANDRA-2754: - Attachment: jdbc-consoidated-v2.txt Rebased. Completed semantic revisions for this round. Fixed some more bugs. This should complete this ticket. Additional tickets will be cut for pooling, additional data type support (BigNumber), Row Navigation and additional semantics on PreparedStatement, DataBaseMetaData, and DataSource. Consolidating Ticket for JDBC Semantic Improvements --- Key: CASSANDRA-2754 URL: https://issues.apache.org/jira/browse/CASSANDRA-2754 Project: Cassandra Issue Type: Improvement Affects Versions: 0.8.0 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: CQL, JDBC Fix For: 0.8.2 Attachments: jdbc-consoidated-v1.txt, jdbc-consoidated-v2.txt First round of improved semantics for the JDBC Suite require a coordinated patch that covers multiple Classes in o.a.c.cql.jdbc package. This ticket is meant to house the consolidated patch and to organize the multiple existing tickets as sub-tickets relating to this topic. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-2780:
---
Reviewer: brandon.williams (was: jbellis)
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
Attachments: CASSANDRA-2780.patch
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136637 - in /cassandra/branches/cassandra-0.8: src/java/org/apache/cassandra/tools/SSTableExport.java test/unit/org/apache/cassandra/SchemaLoader.java test/unit/org/apache/cassandra/tool
Author: brandonwilliams
Date: Thu Jun 16 20:03:57 2011
New Revision: 1136637
URL: http://svn.apache.org/viewvc?rev=1136637view=rev
Log:
sstable2json escapes quotes.
Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-2780
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java?rev=1136637r1=1136636r2=1136637view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java
Thu Jun 16 20:03:57 2011
@@ -81,7 +81,12 @@ public class SSTableExport
*/
private static String quote(String val)
{
-return String.format(\%s\, val);
+return String.format(\%s\, escapeQuotes(val));
+}
+
+private static String escapeQuotes(String val)
+{
+return val.replace(\, \\\);
}
/**
Modified:
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java?rev=1136637r1=1136636r2=1136637view=diff
==
---
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java
(original)
+++
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java
Thu Jun 16 20:03:57 2011
@@ -113,6 +113,12 @@ public class SchemaLoader
standardCFMD(ks1, Standard4),
standardCFMD(ks1, StandardLong1),
standardCFMD(ks1, StandardLong2),
+ new CFMetaData(ks1,
+ ValuesWithQuotes,
+ st,
+ BytesType.instance,
+ null)
+
.defaultValidator(UTF8Type.instance),
superCFMD(ks1, Super1, LongType.instance),
superCFMD(ks1, Super2, LongType.instance),
superCFMD(ks1, Super3, LongType.instance),
Modified:
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java?rev=1136637r1=1136636r2=1136637view=diff
==
---
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
(original)
+++
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
Thu Jun 16 20:03:57 2011
@@ -22,17 +22,15 @@ import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.PrintStream;
-import java.nio.ByteBuffer;
-import java.util.Arrays;
import org.apache.cassandra.SchemaLoader;
-import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.CounterColumn;
import org.apache.cassandra.db.ExpiringColumn;
+import org.apache.cassandra.db.Column;
import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.filter.QueryPath;
-import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.SSTableReader;
import org.apache.cassandra.io.sstable.SSTableWriter;
@@ -243,4 +241,30 @@ public class SSTableExportTest extends S
assert ((String) colA.get(3)).equals(c);
assert (Long) colA.get(4) == Long.MIN_VALUE;
}
+
+@Test
+public void testEscapingDoubleQuotes() throws IOException
+{
+File tempSS = tempSSTableFile(Keyspace1, ValuesWithQuotes);
+ColumnFamily cfamily = ColumnFamily.create(Keyspace1,
ValuesWithQuotes);
+SSTableWriter writer = new SSTableWriter(tempSS.getPath(), 2);
+
+// Add rowA
+cfamily.addColumn(null, new Column(ByteBufferUtil.bytes(data),
UTF8Type.instance.fromString({\foo\:\bar\})));
+writer.append(Util.dk(rowA), cfamily);
+cfamily.clear();
+
+SSTableReader reader =
[jira] [Updated] (CASSANDRA-2778) Unable to set compaction strategy in cli using create column family command
[
https://issues.apache.org/jira/browse/CASSANDRA-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Liang updated CASSANDRA-2778:
--
Attachment: 0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch
Unable to set compaction strategy in cli using create column family command
---
Key: CASSANDRA-2778
URL: https://issues.apache.org/jira/browse/CASSANDRA-2778
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Attachments:
0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch,
0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch
The following command does not set compaction strategy and its options:
{code}
create column family Standard1
with comparator = BytesType
and compaction_strategy =
'org.apache.cassandra.db.compaction.TimestampBucketedCompactionStrategy'
and compaction_strategy_options = [{max_sstable_size:504857600,
retention_in_seconds:60}];
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2778) Unable to set compaction strategy in cli using create column family command
[
https://issues.apache.org/jira/browse/CASSANDRA-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Liang updated CASSANDRA-2778:
--
Attachment: (was:
0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch)
Unable to set compaction strategy in cli using create column family command
---
Key: CASSANDRA-2778
URL: https://issues.apache.org/jira/browse/CASSANDRA-2778
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Attachments:
0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch
The following command does not set compaction strategy and its options:
{code}
create column family Standard1
with comparator = BytesType
and compaction_strategy =
'org.apache.cassandra.db.compaction.TimestampBucketedCompactionStrategy'
and compaction_strategy_options = [{max_sstable_size:504857600,
retention_in_seconds:60}];
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2778) Unable to set compaction strategy in cli using create column family command
[
https://issues.apache.org/jira/browse/CASSANDRA-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050715#comment-13050715
]
Chris Goffinet commented on CASSANDRA-2778:
---
+1
Unable to set compaction strategy in cli using create column family command
---
Key: CASSANDRA-2778
URL: https://issues.apache.org/jira/browse/CASSANDRA-2778
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Alan Liang
Assignee: Alan Liang
Attachments:
0001-2778-allow-for-dynamic-changes-to-compaction-strateg.patch
The following command does not set compaction strategy and its options:
{code}
create column family Standard1
with comparator = BytesType
and compaction_strategy =
'org.apache.cassandra.db.compaction.TimestampBucketedCompactionStrategy'
and compaction_strategy_options = [{max_sstable_size:504857600,
retention_in_seconds:60}];
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2784) Fix build for removal of commons-collections
Fix build for removal of commons-collections Key: CASSANDRA-2784 URL: https://issues.apache.org/jira/browse/CASSANDRA-2784 Project: Cassandra Issue Type: Bug Reporter: Stu Hood Assignee: Stu Hood Priority: Minor -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136662 - in /cassandra/trunk/src/java/org/apache/cassandra: cli/CliClient.java config/CFMetaData.java db/ColumnFamilyStore.java
Author: goffinet
Date: Thu Jun 16 20:44:50 2011
New Revision: 1136662
URL: http://svn.apache.org/viewvc?rev=1136662view=rev
Log:
Fixed the ability to set compaction strategy in cli using create column family
command
patch by alanliang; reviewed by goffinet for CASSANDRA-2778
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1136662r1=1136661r2=1136662view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Thu Jun 16
20:44:50 2011
@@ -1731,7 +1731,7 @@ public class CliClient
sessionState.out.printf( Compaction Strategy: %s%n,
cf_def.compaction_strategy);
if (!cf_def.compaction_strategy_options.isEmpty())
{
-sessionState.out.printf( Compaction Strategy
Options: %s%n, cf_def.compaction_strategy);
+sessionState.out.println( Compaction Strategy
Options:);
for (Map.EntryString, String e :
cf_def.compaction_strategy_options.entrySet())
sessionState.out.printf(%s: %s%n,
e.getKey(), e.getValue());
}
Modified: cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java?rev=1136662r1=1136661r2=1136662view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/config/CFMetaData.java Thu
Jun 16 20:44:50 2011
@@ -705,6 +705,19 @@ public final class CFMetaData
if (cf_def.isSetRow_cache_provider()) {
newCFMD.rowCacheProvider(FBUtilities.newCacheProvider(cf_def.row_cache_provider));
}
if (cf_def.isSetKey_alias()) { newCFMD.keyAlias(cf_def.key_alias); }
if (cf_def.isSetKey_validation_class()) {
newCFMD.keyValidator(TypeParser.parse(cf_def.key_validation_class)); }
+if (cf_def.isSetCompaction_strategy())
+{
+try
+{
+ newCFMD.compactionStrategyClass((Class? extends
AbstractCompactionStrategy)Class.forName(cf_def.compaction_strategy));
+}
+catch (Exception e)
+{
+throw new ConfigurationException(Unable to set Compaction
Strategy Class of + cf_def.compaction_strategy, e);
+}
+}
+if (cf_def.isSetCompaction_strategy_options())
+newCFMD.compactionStrategyOptions(new HashMapString,
String(cf_def.compaction_strategy_options));
return newCFMD.comment(cf_def.comment)
.rowCacheSize(cf_def.row_cache_size)
@@ -817,6 +830,7 @@ public final class CFMetaData
if (null != cf_def.compaction_strategy_options)
{
+compactionStrategyOptions = new HashMapString, String();
for (Map.EntryCharSequence, CharSequence e :
cf_def.compaction_strategy_options.entrySet())
compactionStrategyOptions.put(e.getKey().toString(),
e.getValue().toString());
}
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1136662r1=1136661r2=1136662view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java Thu
Jun 16 20:44:50 2011
@@ -195,7 +195,9 @@ public class ColumnFamilyStore implement
rowCacheSaveInSeconds = new
DefaultInteger(metadata.getRowCacheSavePeriodInSeconds());
if (!keyCacheSaveInSeconds.isModified())
keyCacheSaveInSeconds = new
DefaultInteger(metadata.getKeyCacheSavePeriodInSeconds());
-
+
+compactionStrategy = metadata.createCompactionStrategyInstance(this);
+
updateCacheSizes();
scheduleCacheSaving(rowCacheSaveInSeconds.value(),
keyCacheSaveInSeconds.value());
svn commit: r1136663 - /cassandra/trunk/CHANGES.txt
Author: goffinet Date: Thu Jun 16 20:45:16 2011 New Revision: 1136663 URL: http://svn.apache.org/viewvc?rev=1136663view=rev Log: Updated CHANGES.txt Modified: cassandra/trunk/CHANGES.txt Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1136663r1=1136662r2=1136663view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Thu Jun 16 20:45:16 2011 @@ -5,7 +5,7 @@ * make AbstractBounds.normalize de-overlapp overlapping ranges (CASSANDRA-2641) * replace CollatingIterator, ReducingIterator with MergeIterator (CASSANDRA-2062) - + * Fixed the ability to set compaction strategy in cli using create column family command (CASSANDRA-2778) 0.8.1 * CQL:
[jira] [Updated] (CASSANDRA-2784) Fix build for removal of commons-collections
[ https://issues.apache.org/jira/browse/CASSANDRA-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2784: Attachment: 0001-Remove-references-to-the-commons.collections-package.txt 0001 Removes the last references to the commons.collections package. Fix build for removal of commons-collections Key: CASSANDRA-2784 URL: https://issues.apache.org/jira/browse/CASSANDRA-2784 Project: Cassandra Issue Type: Bug Reporter: Stu Hood Assignee: Stu Hood Priority: Minor Attachments: 0001-Remove-references-to-the-commons.collections-package.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1136679 - in /cassandra/trunk: build.xml src/java/org/apache/cassandra/db/compaction/CompactionManager.java src/java/org/apache/cassandra/db/compaction/CompactionTask.java src/java/org/ap
Author: jbellis
Date: Thu Jun 16 21:03:15 2011
New Revision: 1136679
URL: http://svn.apache.org/viewvc?rev=1136679view=rev
Log:
r/m last references to commons-collections
patch by stuhood; reviewed by jbellis for CASSANDRA-2784
Modified:
cassandra/trunk/build.xml
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
cassandra/trunk/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
Modified: cassandra/trunk/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/trunk/build.xml?rev=1136679r1=1136678r2=1136679view=diff
==
--- cassandra/trunk/build.xml (original)
+++ cassandra/trunk/build.xml Thu Jun 16 21:03:15 2011
@@ -350,7 +350,6 @@ url=${svn.entry.url}?pathrev=${svn.entry
dependency groupId=com.google.guava artifactId=guava
version=r08/
dependency groupId=commons-cli artifactId=commons-cli
version=1.1/
dependency groupId=commons-codec artifactId=commons-codec
version=1.2/
- dependency groupId=commons-collections
artifactId=commons-collections version=3.2.1/
dependency groupId=commons-lang artifactId=commons-lang
version=2.4/
dependency groupId=com.googlecode.concurrentlinkedhashmap
artifactId=concurrentlinkedhashmap-lru version=1.2/
dependency groupId=org.antlr artifactId=antlr version=3.2/
@@ -461,7 +460,6 @@ url=${svn.entry.url}?pathrev=${svn.entry
dependency groupId=com.google.guava artifactId=guava/
dependency groupId=commons-cli artifactId=commons-cli/
dependency groupId=commons-codec artifactId=commons-codec/
-dependency groupId=commons-collections
artifactId=commons-collections/
dependency groupId=commons-lang artifactId=commons-lang/
dependency groupId=com.googlecode.concurrentlinkedhashmap
artifactId=concurrentlinkedhashmap-lru/
dependency groupId=org.antlr artifactId=antlr/
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1136679r1=1136678r2=1136679view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Thu Jun 16 21:03:15 2011
@@ -30,8 +30,8 @@ import java.util.concurrent.locks.Reentr
import javax.management.MBeanServer;
import javax.management.ObjectName;
-import org.apache.commons.collections.PredicateUtils;
-import org.apache.commons.collections.iterators.FilterIterator;
+import com.google.common.base.Predicates;
+import com.google.common.collect.Iterators;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -773,7 +773,7 @@ public class CompactionManager implement
executor.beginCompaction(ci);
try
{
-IteratorAbstractCompactedRow nni = new FilterIterator(ci,
PredicateUtils.notNullPredicate());
+IteratorAbstractCompactedRow nni = Iterators.filter(ci,
Predicates.notNull());
// validate the CF as we iterate over it
validator.prepare(cfs);
Modified:
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java?rev=1136679r1=1136678r2=1136679view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
Thu Jun 16 21:03:15 2011
@@ -28,10 +28,10 @@ import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
+import com.google.common.base.Predicates;
+import com.google.common.collect.Iterators;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.commons.collections.PredicateUtils;
-import org.apache.commons.collections.iterators.FilterIterator;
import org.apache.commons.lang.StringUtils;
import org.apache.cassandra.config.DatabaseDescriptor;
@@ -126,7 +126,7 @@ public class CompactionTask extends Abst
SSTableWriter writer;
CompactionIterator ci = new CompactionIterator(type, sstables,
controller); // retain a handle so we can call close()
-IteratorAbstractCompactedRow nni = new FilterIterator(ci,
PredicateUtils.notNullPredicate());
+IteratorAbstractCompactedRow nni = Iterators.filter(ci,
Predicates.notNull());
MapDecoratedKey, Long cachedKeys = new HashMapDecoratedKey, Long();
if (collector != null)
Modified:
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050773#comment-13050773 ] Terje Marthinussen commented on CASSANDRA-2521: --- We stopped using mmap a while back. Benchmarks was highly inconclusive about any performance benefits and the operational benefits of seeing how much memory the darned thing actually uses and general improved stabilty has so far easily outweighed any teorethical lower singel digit performance benefit we could find in static benchmarks and on more realistic use we so far don't really see any performance issues. Personally I have no problems with a patch with that limitation. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2771) Remove commitlog_rotation_threshold_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050796#comment-13050796 ] Hudson commented on CASSANDRA-2771: --- Integrated in Cassandra #930 (See [https://builds.apache.org/job/Cassandra/930/]) Remove commitlog_rotation_threshold_in_mb -- Key: CASSANDRA-2771 URL: https://issues.apache.org/jira/browse/CASSANDRA-2771 Project: Cassandra Issue Type: Improvement Reporter: Patricio Echague Assignee: Patricio Echague Priority: Minor Labels: commitlog Fix For: 1.0 Attachments: CASSANDRA-2771-2-trunk.txt, CASSANDRA-2771-3-trunk.txt Remove the commitlog segment size config setting, nobody has ever changed it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2427) Heuristic or hard cap to prevent fragmented commit logs from bringing down the server
[ https://issues.apache.org/jira/browse/CASSANDRA-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050797#comment-13050797 ] Hudson commented on CASSANDRA-2427: --- Integrated in Cassandra #930 (See [https://builds.apache.org/job/Cassandra/930/]) Heuristic or hard cap to prevent fragmented commit logs from bringing down the server - Key: CASSANDRA-2427 URL: https://issues.apache.org/jira/browse/CASSANDRA-2427 Project: Cassandra Issue Type: Improvement Reporter: Benjamin Coverston Assignee: Patricio Echague Labels: commitlog, hardening Fix For: 1.0 Attachments: CASSANDRA-2427-4-trunk.txt, CASSANDRA-2427-5-trunk.txt, CASSANDRA-2427-6-trunk-explainPropBetter.txt, CASSANDRA-2427-Jconsole-snapshot.tiff, CASSANDRA-2427-trunk.txt Widely divergent write rates on column families can cause the commit log segments to fragment. In some cases we have seen the commit log partition overrun. One solution here would be to create a heuristic for segment fragmentation to trigger a flush (commit log segments/memtable) or simply track the free disk space and force a global flush when the disk gets to 80% capacity. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050809#comment-13050809
]
Jonathan Ellis commented on CASSANDRA-2780:
---
Tatu's feedback:
{quote}
The best way to handle escaping/quoting is to use tools that deal with the
format -- this patch only handles escaping of double-quotes, but similar
problems would occur with linefeeds, or backslashes.
So using a JSON generator / writer would make sense, if project uses one
already? If not, I would recommend adding handling of backslashes and white
space other than space (char codes below 32), as those must be escaped in
String values and keys.
Also: if unquoted keys were required (not valid JSON, but there's lots of data
like that...), Jackson can be configured to accept unquoted field names. That
has its own potential issues (whether escaping is allowed, which characters are
legal in unquotes names), but appears to work well enough that users haven't
complained.
{quote}
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
Attachments: CASSANDRA-2780.patch
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050811#comment-13050811
]
Brandon Williams commented on CASSANDRA-2780:
-
{quote}
So using a JSON generator / writer would make sense, if project uses one
already? If not, I would recommend adding handling of backslashes and white
space other than space (char codes below 32), as those must be escaped in
String values and keys.
{quote}
We don't use one. I believe the historical reason is that one couldn't be
found that would handle things incrementally, but I think that may have changed
by now (sst2j is pretty long in the tooth) and we should probably switch to one
if possible.
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
Attachments: CASSANDRA-2780.patch
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2785) should export JAVA variable in the bin/cassandra and use that in the cassandra-env.sh when check for the java version
should export JAVA variable in the bin/cassandra and use that in the cassandra-env.sh when check for the java version - Key: CASSANDRA-2785 URL: https://issues.apache.org/jira/browse/CASSANDRA-2785 Project: Cassandra Issue Type: Bug Reporter: Jackson Chung I forgot which jira we add this java -version check in the cassandra-env.sh (for adding jamm to the javaagent), but we should probably use the variable JAVA set in bin/cassandra (will need export) and use $JAVA instead of java in the cassandra-env.sh In a situation where JAVA_HOME may have been properly set as the Sun's java but the PATH still have the OpenJDK's java in front, the check will fail to add the jamm.jar, even though the cassandra jvm is properly started via the Sun's java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2497) CLI: issue with keys being interpreted as hex and causing SET statement to fail
[
https://issues.apache.org/jira/browse/CASSANDRA-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
donghwan Ahn updated CASSANDRA-2497:
Comment: was deleted
(was: if when you 'create column family User'
you need all class type definition.
create column family Users with comparator=UTF8Type and
default_validation_class=UTF8Type
and key_validation_class=UTF8Type;
Add key_validation_class=UTF8Type;)
CLI: issue with keys being interpreted as hex and causing SET statement to
fail
---
Key: CASSANDRA-2497
URL: https://issues.apache.org/jira/browse/CASSANDRA-2497
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8 beta 1
Environment: * Single Node instance on MacOSX
* Nightly Compiled Build from 4/18/2011
* Installed from:
https://builds.apache.org/hudson/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/apache-cassandra-2011-04-18_11-02-29-bin.tar.gz
Reporter: Cathy Daw
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.0 beta 2
Attachments: CASSANDRA-2497.patch
*Original Summary*: Issues with Update Column Family and adding a
key_validation_class
_Changed summary because the issue repros on drop/create. see comment._
*Reproduction Steps*
{code}
create column family users with comparator = UTF8Type
and column_metadata = [{column_name: password, validation_class: UTF8Type}];
update column family users with key_validation_class=UTF8Type;
set users['jsmith']['password']='ch@ngem3';
{code}
*EXPECTED RESULT:* After the UPDATE statement, the SET statement should go
through successfully.
*ACTUAL RESULT:* The SET statement gives the same error message, regardless
of the UPDATE statement:
{code}
org.apache.cassandra.db.marshal.MarshalException: cannot parse 'jsmith' as
hex bytes
{code}
*Output from describe keyspace*
{code}
ColumnFamily: users
Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.290624997/62/1440 (millions of
ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
Column Metadata:
Column Name: password
Validation Class: org.apache.cassandra.db.marshal.UTF8Type
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2497) CLI: issue with keys being interpreted as hex and causing SET statement to fail
[
https://issues.apache.org/jira/browse/CASSANDRA-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050837#comment-13050837
]
donghwan Ahn commented on CASSANDRA-2497:
-
if when you 'create column family User'
you need all class type definition.
create column family Users with comparator=UTF8Type and
default_validation_class=UTF8Type
and key_validation_class=UTF8Type;
Add key_validation_class=UTF8Type;
CLI: issue with keys being interpreted as hex and causing SET statement to
fail
---
Key: CASSANDRA-2497
URL: https://issues.apache.org/jira/browse/CASSANDRA-2497
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8 beta 1
Environment: * Single Node instance on MacOSX
* Nightly Compiled Build from 4/18/2011
* Installed from:
https://builds.apache.org/hudson/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/apache-cassandra-2011-04-18_11-02-29-bin.tar.gz
Reporter: Cathy Daw
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.0 beta 2
Attachments: CASSANDRA-2497.patch
*Original Summary*: Issues with Update Column Family and adding a
key_validation_class
_Changed summary because the issue repros on drop/create. see comment._
*Reproduction Steps*
{code}
create column family users with comparator = UTF8Type
and column_metadata = [{column_name: password, validation_class: UTF8Type}];
update column family users with key_validation_class=UTF8Type;
set users['jsmith']['password']='ch@ngem3';
{code}
*EXPECTED RESULT:* After the UPDATE statement, the SET statement should go
through successfully.
*ACTUAL RESULT:* The SET statement gives the same error message, regardless
of the UPDATE statement:
{code}
org.apache.cassandra.db.marshal.MarshalException: cannot parse 'jsmith' as
hex bytes
{code}
*Output from describe keyspace*
{code}
ColumnFamily: users
Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.290624997/62/1440 (millions of
ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
Column Metadata:
Column Name: password
Validation Class: org.apache.cassandra.db.marshal.UTF8Type
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2497) CLI: issue with keys being interpreted as hex and causing SET statement to fail
[
https://issues.apache.org/jira/browse/CASSANDRA-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050838#comment-13050838
]
donghwan Ahn commented on CASSANDRA-2497:
-
Thanks !
I Have been very helpful
CLI: issue with keys being interpreted as hex and causing SET statement to
fail
---
Key: CASSANDRA-2497
URL: https://issues.apache.org/jira/browse/CASSANDRA-2497
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8 beta 1
Environment: * Single Node instance on MacOSX
* Nightly Compiled Build from 4/18/2011
* Installed from:
https://builds.apache.org/hudson/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/apache-cassandra-2011-04-18_11-02-29-bin.tar.gz
Reporter: Cathy Daw
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.0 beta 2
Attachments: CASSANDRA-2497.patch
*Original Summary*: Issues with Update Column Family and adding a
key_validation_class
_Changed summary because the issue repros on drop/create. see comment._
*Reproduction Steps*
{code}
create column family users with comparator = UTF8Type
and column_metadata = [{column_name: password, validation_class: UTF8Type}];
update column family users with key_validation_class=UTF8Type;
set users['jsmith']['password']='ch@ngem3';
{code}
*EXPECTED RESULT:* After the UPDATE statement, the SET statement should go
through successfully.
*ACTUAL RESULT:* The SET statement gives the same error message, regardless
of the UPDATE statement:
{code}
org.apache.cassandra.db.marshal.MarshalException: cannot parse 'jsmith' as
hex bytes
{code}
*Output from describe keyspace*
{code}
ColumnFamily: users
Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.290624997/62/1440 (millions of
ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
Column Metadata:
Column Name: password
Validation Class: org.apache.cassandra.db.marshal.UTF8Type
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of GettingStarted by TylerHobbs
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The GettingStarted page has been changed by TylerHobbs: http://wiki.apache.org/cassandra/GettingStarted?action=diffrev1=54rev2=55 Comment: Update twissandra link to github.com/twissandra/twissandra Cassandra's main API/RPC/Thrift port is 9160. It is a common mistake for API clients to connect to the JMX port instead. - Checking out a demo application like [[http://github.com/ericflo/twissandra|Twissandra]] (Python + Django) will also be useful. + Checking out a demo application like [[http://github.com/twissandra/twissandra|Twissandra]] (Python + Django) will also be useful. Anchor(if_something_goes_wrong)
[jira] [Commented] (CASSANDRA-2780) sstable2json needs to escape quotes
[
https://issues.apache.org/jira/browse/CASSANDRA-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050863#comment-13050863
]
Hudson commented on CASSANDRA-2780:
---
Integrated in Cassandra-0.8 #173 (See
[https://builds.apache.org/job/Cassandra-0.8/173/])
sstable2json escapes quotes.
Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-2780
brandonwilliams :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1136637
Files :
*
/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java
*
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/SSTableExport.java
*
/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
sstable2json needs to escape quotes
---
Key: CASSANDRA-2780
URL: https://issues.apache.org/jira/browse/CASSANDRA-2780
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Timo Nentwig
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 0.8.2
Attachments: CASSANDRA-2780.patch
[default@foo] set transactions[test][data]='{foo:bar}';
$ cat /tmp/json
{
74657374: [[data, {foo:bar}, 1308209845388000]]
}
$ ./json2sstable -s -c transactions -K foo /tmp/json /tmp/ss-g-1-Data.db
Counting keys to import, please wait... (NOTE: to skip this use -n num_keys)
org.codehaus.jackson.JsonParseException: Unexpected character ('f' (code
102)): was expecting comma to separate ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
at
org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
at
org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
at
org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:128)
at
org.codehaus.jackson.impl.JsonParserBase.skipChildren(JsonParserBase.java:263)
at
org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:328)
at
org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:252)
at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:476)
ERROR: Unexpected character ('f' (code 102)): was expecting comma to separate
ARRAY entries
at [Source: /tmp/json2; line: 2, column: 27]
http://www.mail-archive.com/user@cassandra.apache.org/msg14257.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate Compaction Marker
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050864#comment-13050864 ] Hudson commented on CASSANDRA-2769: --- Integrated in Cassandra-0.8 #173 (See [https://builds.apache.org/job/Cassandra-0.8/173/]) Cannot Create Duplicate Compaction Marker - Key: CASSANDRA-2769 URL: https://issues.apache.org/jira/browse/CASSANDRA-2769 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.8.2 Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully. java.io.IOError: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638) at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321) at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294) at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255) at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634) ... 12 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2758) nodetool repair never finishes. Loops forever through merkle trees?
[
https://issues.apache.org/jira/browse/CASSANDRA-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050862#comment-13050862
]
Hudson commented on CASSANDRA-2758:
---
Integrated in Cassandra-0.8 #173 (See
[https://builds.apache.org/job/Cassandra-0.8/173/])
nodetool repair never finishes. Loops forever through merkle trees?
---
Key: CASSANDRA-2758
URL: https://issues.apache.org/jira/browse/CASSANDRA-2758
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.0
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 0.8.1
Attachments:
0001-Fix-MerkleTree.init-to-not-create-non-sensical-trees.patch
I am not sure all steps here is needed, but as part of testing something
else, I set up
node1: initial_token: 1
node2: initial_token: 5
Then:
{noformat}
create keyspace myks
with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy'
with strategy_options = [{ replication_factor:2 }];
use myks;
create column family test with comparator = AsciiType and column_metadata=[
{column_name: 'up_', validation_class: LongType, index_type: 0},
{column_name: 'del_', validation_class: LongType, index_type: 0} ]
and keys_cached = 10 and rows_cached = 1 and
min_compaction_threshold = 2;
quit;
{noformat}
Doing nodetool repair after this gets both nodes busy looping forever.
A quick look at one node in eclipse makes me guess its having fun spinning
through merkle trees, but I have to admit I have not look at it for a long
time.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2781) regression: exposing cache size through MBean
[ https://issues.apache.org/jira/browse/CASSANDRA-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050865#comment-13050865 ] Hudson commented on CASSANDRA-2781: --- Integrated in Cassandra-0.8 #173 (See [https://builds.apache.org/job/Cassandra-0.8/173/]) fix cache mbean getSize patch by Chris Burroughs; reviewed by Ryan King for CASSANDRA-2781 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1136529 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCacheMBean.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/InstrumentingCache.java regression: exposing cache size through MBean - Key: CASSANDRA-2781 URL: https://issues.apache.org/jira/browse/CASSANDRA-2781 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 beta 1 Reporter: Chris Burroughs Assignee: Chris Burroughs Priority: Minor Fix For: 0.8.2 Attachments: 2781-v1.txt Looks like it was part of CASSANDRA-1969. A method called size, as opposed to getSize, won't be exposed through jmx. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2589) row deletes do not remove columns
[ https://issues.apache.org/jira/browse/CASSANDRA-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050872#comment-13050872 ] Aaron Morton commented on CASSANDRA-2589: - I'm passing Integer.MIN_VALUE for the gcBefore so I thought it would only remove colums if they were under a CF tombstone. One of the issues I ran into is that while it's seems technically correct to purge a tombstone after GCGraceSeconds, if it is not written into an SSTable it's lost. row deletes do not remove columns - Key: CASSANDRA-2589 URL: https://issues.apache.org/jira/browse/CASSANDRA-2589 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Aaron Morton Assignee: Aaron Morton Priority: Minor Fix For: 0.8.2 Attachments: 0001-remove-deleted-columns-before-flushing-memtable-v07.patch, 0001-remove-deleted-columns-before-flushing-memtable-v08.patch When a row delete is issued CF.delete() sets the localDeletetionTime and markedForDeleteAt values but does not remove columns which have a lower time stamp. As a result: # Memory which could be freed is held on to (prob not too bad as it's already counted) # The deleted columns are serialised to disk, along with the CF info to say they are no longer valid. # NamesQueryFilter and SliceQueryFilter have to do more work as they filter out the irrelevant columns using QueryFilter.isRelevant() # Also columns written with a lower time stamp after the deletion are added to the CF without checking markedForDeletionAt. This can cause RR to fail, will create another ticket for that and link. This ticket is for a fix to removing the columns. Two options I could think of: # Check for deletion when serialising to SSTable and ignore columns if the have a lower timestamp. Otherwise leave as is so dead columns stay in memory. # Ensure at all times if the CF is deleted all columns it contains have a higher timestamp. ## I *think* this would include all column types (DeletedColumn as well) as the CF deletion has the same effect. But not sure. ## Deleting (potentially) all columns in delete() will take time. Could track the highest timestamp in the CF so the normal case of deleting all cols does not need to iterate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2761) JDBC driver does not build
[
https://issues.apache.org/jira/browse/CASSANDRA-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050884#comment-13050884
]
Rick Shaw commented on CASSANDRA-2761:
--
{quote}
What are these 50 other class dependencies? Where are they being drug in?
{quote}
- o.a.c.db.marshall.*
-- 23 various classes
- o.a.c.utils.ByteBufferUtil
-- org.apache.cassandra.io.util.FileDataInput
-- org.apache.cassandra.io.util.FileUtils
- o.a.c.config.ColumnDefinition
- o.a.c.config.CFMetaData
-- org.apache.cassandra.cache.IRowCacheProvider;
-- org.apache.cassandra.db.migration.avro.ColumnDef
-- org.apache.cassandra.db.ColumnFamilyType;
--- org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor
--- org.apache.cassandra.config.DatabaseDescriptor
org.apache.cassandra.auth.AllowAllAuthenticator
org.apache.cassandra.auth.AllowAllAuthority
org.apache.cassandra.auth.IAuthenticator
org.apache.cassandra.auth.IAuthority
org.apache.cassandra.config.Config.RequestSchedulerId
org.apache.cassandra.db.ColumnFamilyStore
- 13 new items
org.apache.cassandra.db.ColumnFamilyType
org.apache.cassandra.db.DefsTable
- org.apache.cassandra.config.DatabaseDescriptor;
- org.apache.cassandra.config.KSMetaData;
- org.apache.cassandra.db.filter.QueryFilter;
- org.apache.cassandra.db.filter.QueryPath;
- org.apache.cassandra.db.migration.Migration;
- org.apache.cassandra.io.SerDeUtils;
- org.apache.cassandra.service.StorageService;
- org.apache.cassandra.utils.ByteBufferUtil;
- org.apache.cassandra.utils.UUIDGen;
org.apache.cassandra.db.migration.Migration
org.apache.cassandra.dht.IPartitioner
org.apache.cassandra.io.sstable.Descriptor
org.apache.cassandra.io.util.FileUtils
org.apache.cassandra.locator.*
org.apache.cassandra.scheduler.IRequestSchedule;
org.apache.cassandra.scheduler.NoScheduler
--- org.apache.cassandra.db.compaction.CompactionManager
--- org.apache.cassandra.db.filter.QueryFilter
--- org.apache.cassandra.db.filter.QueryPath
--- org.apache.cassandra.dht.IPartitioner
--- org.apache.cassandra.dht.Range
--- org.apache.cassandra.gms.FailureDetector
--- org.apache.cassandra.gms.Gossiper
--- org.apache.cassandra.gms.ApplicationState
--- org.apache.cassandra.net.MessagingService
10 new classes
--- org.apache.cassandra.service.*
--- org.apache.cassandra.utils.WrappedRunnable
-- org.apache.cassandra.db.HintedHandOffManager;
-- org.apache.cassandra.db.SystemTable;
-- org.apache.cassandra.db.Table;
-- org.apache.cassandra.db.ColumnFamilyStore;
-- org.apache.cassandra.db.migration.Migration;
-- org.apache.cassandra.db.compaction.AbstractCompactionStrategy;
-- org.apache.cassandra.io.SerDeUtils;
-- org.apache.cassandra.utils.Pair;
- o.a.c.config.ConfigurationException
The point is there are lots and they are scattered all over the various
packages; It will be very difficult to manage when they change from the driver
package (client side), which is supposed to be able to change independent of
the server code. If a subset of the server code is to be a dependency then that
subset (jar/s) must be managed in the main build not the driver build.
{quote}
What would you call such a jar? When I looked at this, there didn't seem to be
any delineation that made sense.
{quote}
I agree it is not any clear set of packages. They are scattered all over.
As to a name for the jar... I'm not a good namer in the best of circumstances
but I think the intent is to pick those files that are used in common between
client and server. I guess I'd use that as the basis for the name.
JDBC driver does not build
--
Key: CASSANDRA-2761
URL: https://issues.apache.org/jira/browse/CASSANDRA-2761
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Rick Shaw
Fix For: 1.0
Attachments: jdbc-driver-build-v1.txt
Need a way to build (and run tests for) the Java driver.
Also: still some vestigal references to drivers/ in trunk build.xml.
Should we remove drivers/ from the 0.8 branch as well?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2221) 'show create' commands on the CLI to export schema
[ https://issues.apache.org/jira/browse/CASSANDRA-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Morton updated CASSANDRA-2221: Attachment: 0001-add-show-schema-statement-v08-2.patch rebased for v0.8 as 0001-add-show-schema-statement-v08-2.patch Let me know if you want it for 0.7 'show create' commands on the CLI to export schema -- Key: CASSANDRA-2221 URL: https://issues.apache.org/jira/browse/CASSANDRA-2221 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jeremy Hanna Assignee: Aaron Morton Priority: Minor Labels: cli Fix For: 0.8.2 Attachments: 0001-add-show-schema-statement-8.patch, 0001-add-show-schema-statement-v08-2.patch, 0001-add-show-schema-statement.patch It would be nice to have 'show create' type of commands on the command-line so that it would generate the DDL for the schema. A scenario that would make this useful is where a team works out a data model over time with a dev cluster. They want to use parts of that schema for new clusters that they create, like a staging/prod cluster. It would be very handy in this scenario to have some sort of export mechanism. Another use case is for testing purposes - you want to replicate a problem. We currently have schematool for import/export but that is deprecated and it exports into yaml. This new feature would just be able to 'show' - or export if they want the entire keyspace - into a script or commands that could be used in a cli script. It would need to be able to regenerate everything about the keyspace including indexes and metadata. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2769) Cannot Create Duplicate Compaction Marker
[ https://issues.apache.org/jira/browse/CASSANDRA-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050890#comment-13050890 ] Benjamin Coverston commented on CASSANDRA-2769: --- I think Alan has a good point. I don't think it's an appropriate role of the data tracker to modify the set of sstables to be compacted in a task. It's been a bit of a struggle to hack around the behavior of the DataTracker to ensure my compactions happen in the order I want with the SSTables that I want. That should probably be the exclusive domain compaction strategy. Cannot Create Duplicate Compaction Marker - Key: CASSANDRA-2769 URL: https://issues.apache.org/jira/browse/CASSANDRA-2769 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Benjamin Coverston Assignee: Sylvain Lebresne Fix For: 0.8.2 Attachments: 0001-0.8.0-Remove-useless-unmarkCompacting-in-doCleanup.patch, 0001-Do-compact-only-smallerSSTables.patch, 0002-Only-compact-what-has-been-succesfully-marked-as-com.patch Concurrent compaction can trigger the following exception when two threads compact the same sstable. DataTracker attempts to prevent this but apparently not successfully. java.io.IOError: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:638) at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:321) at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:294) at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:255) at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:932) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:173) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:119) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:102) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.io.IOException: Unable to create compaction marker at org.apache.cassandra.io.sstable.SSTableReader.markCompacted(SSTableReader.java:634) ... 12 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
