[jira] Commented: (CASSANDRA-1311) Support (asynchronous) triggers
[ https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998226#comment-12998226 ] Stu Hood commented on CASSANDRA-1311: - I believe that the entity group functionality can actually be implemented as sugar on top of our existing secondary indexes and column nesting (I'll describe what I'm thinking of on that ticket). This ticket, on the other hand, provides novel functionality, so my vote is strongly in favor of getting it in mostly-as-is, with slight adjustments: * ITriggers should be configured to match a filter, as described on CASSANDRA-1601. Rather than triggering for any change to a row, it should trigger for changes to a set of names, or a slice of names. For example: setting a trigger for columns 'age' and 'state' would only fire the trigger if either of those columns changed * The ITrigger contract should not guarantee to give the user all changed columns: only the columns matching the trigger configuration. I'm hoping that in future tickets we can unify these ''at-least-once'' distributed triggers with our ''exactly-once'' local indexes (with UDFs) around common configuration: see CASSANDRA-1601. Support (asynchronous) triggers --- Key: CASSANDRA-1311 URL: https://issues.apache.org/jira/browse/CASSANDRA-1311 Project: Cassandra Issue Type: New Feature Components: Contrib Reporter: Maxim Grinev Fix For: 0.8 Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt, HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf, ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt, trunk-984391-update2.txt Asynchronous triggers is a basic mechanism to implement various use cases of asynchronous execution of application code at database side. For example to support indexes and materialized views, online analytics, push-based data propagation. Please find the motivation, triggers description and list of applications: http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/ An example of using triggers for indexing: http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/ Implementation details are attached. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muga Nishizawa updated CASSANDRA-2225:
--
Attachment: create_table.cli
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at org.apache.cassandra.db.Table.getRow(Table.java:384)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:473)
at
[jira] Created: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified row
keys are stored on Cassandra in advance. Cassandra outputs following exception
to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem occurs
with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns associated
to row key 030yy from sstable generated by json2sstable. null will
be returned as result. Cassandra will output exception to system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and super
columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column
name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at org.apache.cassandra.db.Table.getRow(Table.java:384)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:473)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException:
[jira] Updated: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muga Nishizawa updated CASSANDRA-2225:
--
Attachment: cassandra_sample_insert.py
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at org.apache.cassandra.db.Table.getRow(Table.java:384)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:473)
at
[jira] Updated: (CASSANDRA-2216) Compaction can echo data which breaks upon sstable format changes
[ https://issues.apache.org/jira/browse/CASSANDRA-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2216: -- Affects Version/s: 0.7.1 Fix Version/s: (was: 0.7.1) 0.7.3 Compaction can echo data which breaks upon sstable format changes - Key: CASSANDRA-2216 URL: https://issues.apache.org/jira/browse/CASSANDRA-2216 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Critical Labels: compaction Fix For: 0.7.3 Attachments: 0001-Don-t-echo-data-during-compaction.patch, 2216_v2.patch Original Estimate: 1h Remaining Estimate: 1h While compaction, if for a row we have only 1 sstable holding data, we echo this data. This breaks when we change the data format, creating mixed (corrupted) sstable. (I suspect this is the cause of CASSANDRA-2195, but opening a new ticket until we can confirm that hunch) -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (CASSANDRA-2224) Warning error when a cache is empty
[ https://issues.apache.org/jira/browse/CASSANDRA-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2224. --- Resolution: Duplicate Fix Version/s: (was: 0.8) Assignee: (was: Matthew F. Dennis) and/or CASSANDRA-2172 Warning error when a cache is empty --- Key: CASSANDRA-2224 URL: https://issues.apache.org/jira/browse/CASSANDRA-2224 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Reporter: Joaquin Casares Priority: Minor I get this error after repair stalled and I ran 'sudo killall java' and started up Cassandra again. Most of my caches come with the same error and most of the files they are referencing are empty. INFO 00:04:37,137 reading saved cache /var/lib/cassandra/saved_caches/system-IndexInfo-KeyCache WARN 00:04:37,139 error reading saved cache /var/lib/cassandra/saved_caches/system-IndexInfo-KeyCache java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2297) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2766) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:797) at java.io.ObjectInputStream.init(ObjectInputStream.java:297) at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:255) at org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:198) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:451) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:432) at org.apache.cassandra.db.Table.initCf(Table.java:360) at org.apache.cassandra.db.Table.init(Table.java:290) at org.apache.cassandra.db.Table.open(Table.java:107) at org.apache.cassandra.db.SystemTable.checkHealth(SystemTable.java:207) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:129) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79) === LocationInfo has this in the file: ^@^@^@^AL^@^@^@^AL And reports this error: WARN 00:04:37,179 error reading saved cache /var/lib/cassandra/saved_caches/system-LocationInfo-KeyCache java.io.StreamCorruptedException: invalid stream header: 0001 at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:800) at java.io.ObjectInputStream.init(ObjectInputStream.java:297) at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:255) at org.apache.cassandra.db.ColumnFamilyStore.init(ColumnFamilyStore.java:198) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:451) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:432) at org.apache.cassandra.db.Table.initCf(Table.java:360) at org.apache.cassandra.db.Table.init(Table.java:290) at org.apache.cassandra.db.Table.open(Table.java:107) at org.apache.cassandra.db.SystemTable.checkHealth(SystemTable.java:207) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:129) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79) -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073750 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/io/sstable/CacheWriter.java
Author: jbellis
Date: Wed Feb 23 14:10:50 2011
New Revision: 1073750
URL: http://svn.apache.org/viewvc?rev=1073750view=rev
Log:
fix cache saving on Windows
patch by jbellis; reviewed by mdennis for CASSANDRA-2207
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1073750r1=1073749r2=1073750view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Feb 23 14:10:50 2011
@@ -20,6 +20,7 @@
* fix EOFing on requests for the last bytes in a file (CASSANDRA-2213)
* fix BRAF performance when seeking to EOF (CASSANDRA-2218)
* check for memtable flush_after_mins exceeded every 10s (CASSANDRA-2183)
+ * fix cache saving on Windows (CASSANDRA-2207)
0.7.2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/CacheWriter.java?rev=1073750r1=1073749r2=1073750view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
Wed Feb 23 14:10:50 2011
@@ -89,8 +89,10 @@ public class CacheWriterK, V implement
{
out.close();
}
+
+path.delete(); // ignore error if it didn't exist
if (!tmpFile.renameTo(path))
-throw new IOException(Unable to rename cache to + path);
+throw new IOException(Unable to rename + tmpFile + to +
path);
logger.info(String.format(Saved %s (%d items) in %d ms,
path.getName(), keys.size(),
(System.currentTimeMillis() - start)));
}
[jira] Commented: (CASSANDRA-1311) Support (asynchronous) triggers
[ https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998378#comment-12998378 ] Jonathan Ellis commented on CASSANDRA-1311: --- To be clear: I am -1 on coordinator-based triggers. Support (asynchronous) triggers --- Key: CASSANDRA-1311 URL: https://issues.apache.org/jira/browse/CASSANDRA-1311 Project: Cassandra Issue Type: New Feature Components: Contrib Reporter: Maxim Grinev Fix For: 0.8 Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt, HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf, ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt, trunk-984391-update2.txt Asynchronous triggers is a basic mechanism to implement various use cases of asynchronous execution of application code at database side. For example to support indexes and materialized views, online analytics, push-based data propagation. Please find the motivation, triggers description and list of applications: http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/ An example of using triggers for indexing: http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/ Implementation details are attached. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998379#comment-12998379
]
Jonathan Ellis commented on CASSANDRA-2225:
---
To be clear: the queries work fine on the original sstable, before
round-tripping through json?
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at org.apache.cassandra.db.Table.getRow(Table.java:384)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63)
at
[jira] Commented: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998386#comment-12998386
]
Daniel Lundin commented on CASSANDRA-2104:
--
Nope, never managed to trigger it again.
Closing this issue for now.
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0, 0.7.1
Reporter: Daniel Lundin
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is
[jira] Resolved: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Lundin resolved CASSANDRA-2104.
--
Resolution: Cannot Reproduce
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0, 0.7.1
Reporter: Daniel Lundin
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see:
[jira] Commented: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998390#comment-12998390
]
Pavel Yaskevich commented on CASSANDRA-2225:
Looks like you messed something up with your sstables, what I did is following
(branch cassandra-0.7):
1). Cleaned all data directories and started Cassandra
2). `./bin/cassandra-cli --host localhost create_table.cli`
3). `python cassandra_sample_insert.py` (which took a while to finish)
4). `./bin/nodetool -h localhost compact SampleKS SampleCF` to get all keys in
one compacted sstable (because there were 2 sstables)
5). verified key count in compacted sstable - 50 keys.
6). Stopped Cassandra
7). `./bin/sstable2json /var/lib/cassandra/data/SampleKS/SampleCF-f-3-Data.db
s.json` 88M resulting file
8). cleaned data directory for SampleKS `rm /var/lib/cassandra/data/SampleKS/*`
9). generated new sstable using `./bin/json2sstable -s -K SampleKS -c SampleCF
s.json /var/lib/cassandra/data/SampleKS/SampleCF-f-1-Data.db`
10). Started Cassandra
11). Run `./bin/cassandra-cli --host localhost --keyspace SampleKS`
12). in CLI `get SampleCF['030yy'];` which gave me all columns
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Assignee: Pavel Yaskevich
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
[jira] Updated: (CASSANDRA-2212) Cannot get range slice of super columns in reversed order
[ https://issues.apache.org/jira/browse/CASSANDRA-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2212: Attachment: 0001-Fix-IndexHelp.indexFor-for-reverse-query.patch Thanks a lot Muga for the test script, and you are right, this a bug in getting the index block during reverse queries. Patch attached with unit tests for IndexHelper. Cannot get range slice of super columns in reversed order - Key: CASSANDRA-2212 URL: https://issues.apache.org/jira/browse/CASSANDRA-2212 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.2 Environment: Fedore 11, Intel Core i5 Reporter: Muga Nishizawa Assignee: Sylvain Lebresne Fix For: 0.7.3 Attachments: 0001-Fix-IndexHelp.indexFor-for-reverse-query.patch, cassandra_sample_insert.py, cassandra_sample_rangeslice.py, create_table.cli I cannot get range slice of super columns in reversed order. These data are stored in Cassandra in advance. On the other hand, range slice of these data in normal order can be acquired. You can reproduce the bug by executing attached programs. - 1. Start Cassandra daemon on localhost (number of thrift port is 9160) - 2. Create keyspace and column family, according to create_table.cli, - 3. Execute cassandra_sample_insert.py, storing pairs of row keys and super columns - 4. Execute cassandra_sample_rangeslice.py and get range slice of stored super columns cassandra_sample_insert.py and cassandra_sample_rangeslice.py require pycassa. You will need to execute 4.cassandra_sample_rangeslice.py with following options so that you get range slice of super columns in reversed order. % python cassandra_sample_rangeslice.py -r 00082 00083 On the other hand, to get range slice in normal order, you will need to use following options. % python cassandra_sample_rangeslice.py -f 00082 00083 00082 and 00083 are the specified key range. Range slice can be acquired in normal order but, I cannot get it in reversed order. I assume that there may be a bug within the code for acquiring the index block of specified range. In fact, 00083 is included in gap between lastName of index block and firstName of next index block. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998395#comment-12998395
]
Jonas Borgström commented on CASSANDRA-2104:
Hi Jonathan,
I now have a python script that seems to be able to reproduce this after a few
runs (it does not seem to always happen)
http://jonas.borgstrom.se/cassandra/CASSANDRA-2104.txt
I've tested this on an empty single node 0.7.2 cluster and default
cassandra.yaml, except for in_memory_compaction_limit_in_mb=32 hoping that
would make it easier to reproduce.
A complete copy of the data directory (and config) after running the script a
couple of times can be downloaded here:
http://jonas.borgstrom.se/cassandra/CASSANDRA-2104.tar.gz
And a copy of system.log from booting cassandra using this data directory:
http://jonas.borgstrom.se/cassandra/CASSANDRA-2104-system.log
Let me know if you need any more details.
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0, 0.7.1
Reporter: Daniel Lundin
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
svn commit: r1073760 - in /cassandra/branches/cassandra-0.7: src/java/org/apache/cassandra/cli/CliClient.java test/unit/org/apache/cassandra/cli/CliTest.java
Author: jbellis
Date: Wed Feb 23 14:58:19 2011
New Revision: 1073760
URL: http://svn.apache.org/viewvc?rev=1073760view=rev
Log:
make CliTest less picky about schema update output
patch by jbellis
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java?rev=1073760r1=1073759r2=1073760view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
Wed Feb 23 14:58:19 2011
@@ -752,8 +752,8 @@ public class CliClient extends CliUserHe
KsDef updatedKsDef = updateKsDefAttributes(statement,
currentKsDef);
String mySchemaVersion =
thriftClient.system_update_keyspace(updatedKsDef);
-validateSchemaIsSettled(mySchemaVersion);
sessionState.out.println(mySchemaVersion);
+validateSchemaIsSettled(mySchemaVersion);
keyspacesMap.put(keyspaceName,
thriftClient.describe_keyspace(keyspaceName));
}
catch (InvalidRequestException e)
Modified:
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java?rev=1073760r1=1073759r2=1073760view=diff
==
---
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java
(original)
+++
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java
Wed Feb 23 14:58:19 2011
@@ -27,6 +27,7 @@ import org.junit.Test;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.PrintStream;
+import java.util.regex.Pattern;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
@@ -176,7 +177,7 @@ public class CliTest extends CleanupHelp
assertEquals(errStream.toString() + processing + statement,
, errStream.toString());
if (statement.startsWith(drop ) || statement.startsWith(create
) || statement.startsWith(update ))
{
-
assertTrue(result.matches((.{8})-(.{4})-(.{4})-(.{4})-(.{12})\n));
+assert
Pattern.compile((.{8})-(.{4})-(.{4})-(.{4})-(.{12}).*,
Pattern.DOTALL).matcher(result).matches() : result;
}
else if (statement.startsWith(set ))
{
[jira] Commented: (CASSANDRA-2208) ClientOnly mode is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998398#comment-12998398 ] T Jake Luciani commented on CASSANDRA-2208: --- Minor issue but I see this on the server side, looks like the endpoint doesn't have schema associated with it. ERROR 09:58:10,811 Error in ThreadPoolExecutor java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.waitForSchemaAgreement(HintedHandOffManager.java:259) at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:276) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:400) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ClientOnly mode is broken - Key: CASSANDRA-2208 URL: https://issues.apache.org/jira/browse/CASSANDRA-2208 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: T Jake Luciani Assignee: Gary Dusbabek Fix For: 0.7.3 The new migrations code has left the client mode in a unusable state. client only nodes can't create any RowMutations since they never receive the schema from ring nodes. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2227) add cache loading to row/key cache tests
add cache loading to row/key cache tests Key: CASSANDRA-2227 URL: https://issues.apache.org/jira/browse/CASSANDRA-2227 Project: Cassandra Issue Type: Test Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.3 -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073768 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/service/StorageProxy.java src/java/org/apache/cassandra/thrift/CassandraServer.java
Author: jbellis
Date: Wed Feb 23 15:09:28 2011
New Revision: 1073768
URL: http://svn.apache.org/viewvc?rev=1073768view=rev
Log:
add validateSchemaAgreement call + synchronization to schema modification calls
patch by jbellis; reviewed by gdusbabek for CASSANDRA-
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1073768r1=1073767r2=1073768view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Feb 23 15:09:28 2011
@@ -21,6 +21,8 @@
* fix BRAF performance when seeking to EOF (CASSANDRA-2218)
* check for memtable flush_after_mins exceeded every 10s (CASSANDRA-2183)
* fix cache saving on Windows (CASSANDRA-2207)
+ * add validateSchemaAgreement call + synchronization to schema
+ modification operations (CASSANDRA-)
0.7.2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1073768r1=1073767r2=1073768view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageProxy.java
Wed Feb 23 15:09:28 2011
@@ -616,17 +616,18 @@ public class StorageProxy implements Sto
}
hosts.add(host.getHostAddress());
}
+
+// we're done: the results map is ready to return to the client. the
rest is just debug logging:
if (results.get(UNREACHABLE) != null)
logger.debug(Hosts not in agreement. Didn't get a response from
everybody: + StringUtils.join(results.get(UNREACHABLE), ,));
-// check for version disagreement. log the hosts that don't agree.
for (Map.EntryString, ListString entry : results.entrySet())
{
+// check for version disagreement. log the hosts that don't agree.
if (entry.getKey().equals(UNREACHABLE) ||
entry.getKey().equals(myVersion))
continue;
for (String host : entry.getValue())
logger.debug(%s disagrees (%s), host, entry.getKey());
}
-
if (results.size() == 1)
logger.debug(Schemas are in agreement.);
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java?rev=1073768r1=1073767r2=1073768view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CassandraServer.java
Wed Feb 23 15:09:28 2011
@@ -26,6 +26,8 @@ import java.util.concurrent.ExecutionExc
import java.util.concurrent.Future;
import java.util.concurrent.TimeoutException;
+import com.google.common.base.Predicates;
+import com.google.common.collect.Maps;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -661,11 +663,13 @@ public class CassandraServer implements
}
}
-public String system_add_column_family(CfDef cf_def) throws
InvalidRequestException, TException
+public synchronized String system_add_column_family(CfDef cf_def) throws
InvalidRequestException, TException
{
logger.debug(add_column_family);
state().hasColumnFamilyListAccess(Permission.WRITE);
ThriftValidation.validateCfDef(cf_def);
+validateSchemaAgreement();
+
try
{
applyMigrationOnStage(new
AddColumnFamily(convertToCFMetaData(cf_def)));
@@ -685,10 +689,11 @@ public class CassandraServer implements
}
}
-public String system_drop_column_family(String column_family) throws
InvalidRequestException, TException
+public synchronized String system_drop_column_family(String column_family)
throws InvalidRequestException, TException
{
logger.debug(drop_column_family);
state().hasColumnFamilyListAccess(Permission.WRITE);
+validateSchemaAgreement();
try
{
@@ -709,10 +714,11 @@ public class CassandraServer implements
}
}
-public String system_add_keyspace(KsDef ks_def) throws
InvalidRequestException, TException
+
svn commit: r1073771 - /cassandra/branches/cassandra-0.7/NEWS.txt
Author: jbellis Date: Wed Feb 23 15:12:54 2011 New Revision: 1073771 URL: http://svn.apache.org/viewvc?rev=1073771view=rev Log: add nodetool scrub to NEWS Modified: cassandra/branches/cassandra-0.7/NEWS.txt Modified: cassandra/branches/cassandra-0.7/NEWS.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/NEWS.txt?rev=1073771r1=1073770r2=1073771view=diff == --- cassandra/branches/cassandra-0.7/NEWS.txt (original) +++ cassandra/branches/cassandra-0.7/NEWS.txt Wed Feb 23 15:12:54 2011 @@ -1,3 +1,18 @@ +0.7.3 += + +Upgrading +- +- 0.7.1 and 0.7.2 shipped with a bug that caused incorrect row-level + bloom filters to be generated when compacting sstables generated + with earlier versions. This would manifest in IOExceptions during + column name-based queries. 0.7.3 provides nodetool scrub to + rebuild sstables with correct bloom filters, with no data lost. + (If your cluster was never on 0.7.0 or earlier, you don't have to + worry about this.) Note that nodetool scrub will snapshot your + data files before rebuilding, just in case. + + 0.7.1 =
[jira] Updated: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2104:
--
Affects Version/s: (was: 0.7.1)
Fix Version/s: 0.7.3
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Daniel Lundin
Fix For: 0.7.3
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
-
For
[jira] Reopened: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reopened CASSANDRA-2104:
---
Jonas, are you also seeing the error only on rows larger than
in_memory_compaction_limit then?
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Daniel Lundin
Fix For: 0.7.3
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is
[jira] Commented: (CASSANDRA-2187) Cassandra Cli hangs forever if schema does not settle within timeout window
[ https://issues.apache.org/jira/browse/CASSANDRA-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998410#comment-12998410 ] Hudson commented on CASSANDRA-2187: --- Integrated in Cassandra-0.7 #309 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/309/]) Cassandra Cli hangs forever if schema does not settle within timeout window --- Key: CASSANDRA-2187 URL: https://issues.apache.org/jira/browse/CASSANDRA-2187 Project: Cassandra Issue Type: Bug Reporter: Chris Goffinet Assignee: Chris Goffinet Priority: Minor Attachments: 0001-Fix-Cassandra-cli-to-respect-timeout-if-schema-does-.patch validateSchemaIsSettled will hang in the while loop since we never update start if migrations never settle. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2183) memtable_flush_after_mins setting not working
[
https://issues.apache.org/jira/browse/CASSANDRA-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998409#comment-12998409
]
Hudson commented on CASSANDRA-2183:
---
Integrated in Cassandra-0.7 #309 (See
[https://hudson.apache.org/hudson/job/Cassandra-0.7/309/])
memtable_flush_after_mins setting not working
-
Key: CASSANDRA-2183
URL: https://issues.apache.org/jira/browse/CASSANDRA-2183
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Ching-cheng
Assignee: Jonathan Ellis
Priority: Minor
Fix For: 0.7.3
Attachments: 2183.txt
Original Estimate: 1h
Remaining Estimate: 1h
We have observed the behavior that memtable_flush_after_mins setting not
working occasionally. After some testing and code digging, we finally
figured out what going on.
The memtable_flush_after_mins won't work on certain condition with current
implementation in Cassandra.
In org.apache.cassandra.db.Table, the scheduled flush task is setup by the
following code during construction.
--
int minCheckMs = Integer.MAX_VALUE;
for (ColumnFamilyStore cfs : columnFamilyStores.values())
{
minCheckMs = Math.min(minCheckMs, cfs.getMemtableFlushAfterMins() * 60 *
1000);
}
Runnable runnable = new Runnable()
{
public void run()
{
for (ColumnFamilyStore cfs : columnFamilyStores.values())
{
cfs.forceFlushIfExpired();
}
}
};
flushTask = StorageService.scheduledTasks.scheduleWithFixedDelay(runnable,
minCheckMs, minCheckMs, TimeUnit.MILLISECONDS);
--
Now for our application, we will create a keyspacewithout without any
columnfamily first. And only add needed columnfamily later depends on
request.
However, when keyspacegot created (without any columnfamily ), the above code
will actually schedule a fixed delay flush check task with Integer.MAX_VALUE
ms
since there is no columnfamily yet.
Later when you add columnfamily to this empty keyspace, the initCf() method
in Table.java doesn't check whether the scheduled flush check task interval
need
to be updated or not. To fix this, we'd need to restart the Cassandra after
columnfamily added into the keyspace.
I would suggest that add additional logic in initCf() method to recreate a
scheduled flush check task if needed.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2207) IOException in CompactionExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998411#comment-12998411 ] Hudson commented on CASSANDRA-2207: --- Integrated in Cassandra-0.7 #309 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/309/]) IOException in CompactionExecutor - Key: CASSANDRA-2207 URL: https://issues.apache.org/jira/browse/CASSANDRA-2207 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Environment: WindowsXP(SP3) 32 bit Reporter: ruslan.usifov Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.3 Attachments: 2207.txt Original Estimate: 1h Remaining Estimate: 1h I launch clean cassandra 7.2 instalation, and after few days i look at system.log follow error (more then 10 times): ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.RuntimeException: java.io.IOException: Unable to rename cache to F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Unable to rename cache to F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache at org.apache.cassandra.io.sstable.CacheWriter.saveCache(CacheWriter.java:85) at org.apache.cassandra.db.CompactionManager$9.runMayThrow(CompactionManager.java:746) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 6 more -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2222) Make it less easy for user to aim the schema change gun at his foot
[ https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998413#comment-12998413 ] Hudson commented on CASSANDRA-: --- Integrated in Cassandra-0.7 #310 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/310/]) add validateSchemaAgreement call + synchronization to schema modification calls patch by jbellis; reviewed by gdusbabek for CASSANDRA- Make it less easy for user to aim the schema change gun at his foot --- Key: CASSANDRA- URL: https://issues.apache.org/jira/browse/CASSANDRA- Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 0.7.3 Attachments: .txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
Race conditions when reinitialisating nodes (OOM + Nullpointer)
---
Key: CASSANDRA-2228
URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Fix For: 0.7.3
I had a corrupt system table which wouldn't compact anymore and I deleted the
files and restarted cassandra and let it take the same token/ip address.
I experienced the same errors when I'm adding a newly installed node under the
same token/ip address before calling repair.
1)
After a few seconds/minutes, I get a OOM error:
INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
table_userentriesrep factor:3rep
strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Migrations at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
operations)
INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157) Writing
Memtable-Migrations@2121008793(12529 bytes, 1 operations)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Schema at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
table_sourcedetection.table_sourcedetection 0,0
0/00/20
ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
(line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at
org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
at
org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2) If I restart then, I'm getting an Nullpointer exception. The OOM error will
only appear once.
ERROR [main] 2011-02-23 16:42:32,782 AbstractCassandraDaemon.java (line 333)
Exception encountered during startup.
java.lang.NullPointerException
at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at
org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:925)
at
org.apache.cassandra.service.MigrationManager.passiveAnnounce(MigrationManager.java:105)
at
[jira] Commented: (CASSANDRA-2223) ClientOnly mode is creating directories
[ https://issues.apache.org/jira/browse/CASSANDRA-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998421#comment-12998421 ] T Jake Luciani commented on CASSANDRA-2223: --- The client_only example no longer works with this patch: jake@Jake-Lucianis-MacBook-Pro(client_only)$ ./bin/client_only write 11/02/23 10:53:01 INFO config.DatabaseDescriptor: Loading settings from jar:file:/Users/jake/workspace/cassandra-git/contrib/client_only/build/client_only.jar!/cassandra.yaml 11/02/23 10:53:01 INFO config.DatabaseDescriptor: DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap 11/02/23 10:53:01 INFO service.StorageService: Starting up client gossip 11/02/23 10:53:01 INFO gms.Gossiper: Node /127.0.0.1 has restarted, now UP again 11/02/23 10:53:02 INFO gms.Gossiper: InetAddress /127.0.0.1 is now dead. 11/02/23 10:53:02 INFO gms.Gossiper: InetAddress /127.0.0.1 is now UP Exception in thread main java.lang.AssertionError at org.apache.cassandra.db.ColumnFamily.create(ColumnFamily.java:66) at org.apache.cassandra.db.ColumnFamily.create(ColumnFamily.java:61) at org.apache.cassandra.db.RowMutation.add(RowMutation.java:144) at org.apache.cassandra.db.RowMutation.add(RowMutation.java:152) at ClientOnlyExample.testWriting(ClientOnlyExample.java:68) at ClientOnlyExample.main(ClientOnlyExample.java:127) ClientOnly mode is creating directories --- Key: CASSANDRA-2223 URL: https://issues.apache.org/jira/browse/CASSANDRA-2223 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.3 Attachments: v1-0001-client-only-mode-shouldn-t-create-any-local-data.txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998422#comment-12998422
]
Thibaut commented on CASSANDRA-2228:
The normal heap usage of the node is about half of the allocated heap:
Gossip active: true
Load : 30.99 GB
Generation No: 1298476238
Uptime (seconds) : 308
Heap Memory (MB) : 1544.41 / 2493.38
Race conditions when reinitialisating nodes (OOM + Nullpointer)
---
Key: CASSANDRA-2228
URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Fix For: 0.7.3
I had a corrupt system table which wouldn't compact anymore and I deleted the
files and restarted cassandra and let it take the same token/ip address.
I experienced the same errors when I'm adding a newly installed node under
the same token/ip address before calling repair.
1)
After a few seconds/minutes, I get a OOM error:
INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
table_userentriesrep factor:3rep
strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Migrations at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
operations)
INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Schema at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
table_sourcedetection.table_sourcedetection 0,0
0/00/20
ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
(line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at
org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
at
org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2) If I restart then, I'm getting an Nullpointer
[jira] Commented: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998430#comment-12998430
]
Jonas Borgström commented on CASSANDRA-2104:
Jonathan, yes that seems to be the case. For me key 3139 (19) seems to be the
one triggering this. So increasing in_memory_compaction_limit enough to cover
that key seems to do the trick. Even if some other keys are compacted
incrementally.
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Daniel Lundin
Fix For: 0.7.3
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
[jira] Updated: (CASSANDRA-2223) ClientOnly mode is creating directories
[ https://issues.apache.org/jira/browse/CASSANDRA-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2223: - Attachment: v2-0004-update-contrib_client-only-meta.txt v2-0003-make-sure-fat-clients-gossip-their-schema.txt v2-0002-avoid-creating-local-files-when-in-fat-client-mode.txt v2-0001-ClientOnlyExample-waits-for-schema-to-arrive-tries-to-.txt ClientOnly mode is creating directories --- Key: CASSANDRA-2223 URL: https://issues.apache.org/jira/browse/CASSANDRA-2223 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.3 Attachments: v1-0001-client-only-mode-shouldn-t-create-any-local-data.txt, v2-0001-ClientOnlyExample-waits-for-schema-to-arrive-tries-to-.txt, v2-0002-avoid-creating-local-files-when-in-fat-client-mode.txt, v2-0003-make-sure-fat-clients-gossip-their-schema.txt, v2-0004-update-contrib_client-only-meta.txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-2228:
-
Assignee: Gary Dusbabek
The OOM is because flushing allocates a buffer the size of
in_memory_compaction_limit. Sounds like you need to lower that.
The NPE looks like a bug.
Race conditions when reinitialisating nodes (OOM + Nullpointer)
---
Key: CASSANDRA-2228
URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Reporter: Thibaut
Assignee: Gary Dusbabek
Fix For: 0.7.3
I had a corrupt system table which wouldn't compact anymore and I deleted the
files and restarted cassandra and let it take the same token/ip address.
I experienced the same errors when I'm adding a newly installed node under
the same token/ip address before calling repair.
1)
After a few seconds/minutes, I get a OOM error:
INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
table_userentriesrep factor:3rep
strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=null,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Migrations at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
operations)
INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
666) switching in a fresh Memtable for Schema at
CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
position=226075)
INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
table_sourcedetection.table_sourcedetection 0,0
0/00/20
ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
(line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.init(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at
org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:126)
at
org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:75)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2) If I restart then, I'm getting an Nullpointer exception. The OOM error
will
[jira] Assigned: (CASSANDRA-2104) IndexOutOfBoundsException during lazy row compaction (using TimeUUID comparator)
[
https://issues.apache.org/jira/browse/CASSANDRA-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-2104:
-
Assignee: Sylvain Lebresne
IndexOutOfBoundsException during lazy row compaction (using TimeUUID
comparator)
Key: CASSANDRA-2104
URL: https://issues.apache.org/jira/browse/CASSANDRA-2104
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0
Reporter: Daniel Lundin
Assignee: Sylvain Lebresne
Fix For: 0.7.3
I ran into an exception when lazily compacting wide rows of TimeUUID columns.
It seems to trigger when a row is larger than
{{in_memory_compaction_limit_in_mb}}.
Traceback:
{noformat}
INFO [CompactionExecutor:1] 2011-02-03 10:59:59,262 CompactionIterator.java
(line 135) Compacting large row X (76999384 bytes) incrementally
ERROR [CompactionExecutor:1] 2011-02-03 10:59:59,266
AbstractCassandraDaemon.java (line 114) Fatal exception in thread T
hread[CompactionExecutor:1,1,main]
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:121)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compareTimestampBytes(TimeUUIDType.java:56)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:45)
at
org.apache.cassandra.db.marshal.TimeUUIDType.compare(TimeUUIDType.java:29)
at
java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606
)
at
java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:864)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1893)
at
org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:170)
at
org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:195)
at
org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:221)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:204)
at
org.apache.cassandra.io.LazilyCompactedRow$LazyColumnIterator.reduce(LazilyCompactedRow.java:185)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:62)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at
org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at
org.apache.cassandra.io.LazilyCompactedRow.init(LazilyCompactedRow.java:88)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:426)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
[jira] Commented: (CASSANDRA-2223) ClientOnly mode is creating directories
[ https://issues.apache.org/jira/browse/CASSANDRA-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998453#comment-12998453 ] T Jake Luciani commented on CASSANDRA-2223: --- v2 works great. verified no dirs are created. +1 Only thing I've been seeing with client mode in general is this on startup (randomly) 11/02/23 12:03:46 INFO service.StorageService: Starting up client gossip Exception in thread Thread-2 java.io.IOError: java.io.EOFException at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:73) Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:375) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:61) Seems to cause no issues though... ClientOnly mode is creating directories --- Key: CASSANDRA-2223 URL: https://issues.apache.org/jira/browse/CASSANDRA-2223 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.3 Attachments: v1-0001-client-only-mode-shouldn-t-create-any-local-data.txt, v2-0001-ClientOnlyExample-waits-for-schema-to-arrive-tries-to-.txt, v2-0002-avoid-creating-local-files-when-in-fat-client-mode.txt, v2-0003-make-sure-fat-clients-gossip-their-schema.txt, v2-0004-update-contrib_client-only-meta.txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2212) Cannot get range slice of super columns in reversed order
[ https://issues.apache.org/jira/browse/CASSANDRA-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998456#comment-12998456 ] Jonathan Ellis commented on CASSANDRA-2212: --- Sylvain, can you summarize the bug and how this patch fixes it? Cannot get range slice of super columns in reversed order - Key: CASSANDRA-2212 URL: https://issues.apache.org/jira/browse/CASSANDRA-2212 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.2 Environment: Fedore 11, Intel Core i5 Reporter: Muga Nishizawa Assignee: Sylvain Lebresne Fix For: 0.7.3 Attachments: 0001-Fix-IndexHelp.indexFor-for-reverse-query.patch, cassandra_sample_insert.py, cassandra_sample_rangeslice.py, create_table.cli I cannot get range slice of super columns in reversed order. These data are stored in Cassandra in advance. On the other hand, range slice of these data in normal order can be acquired. You can reproduce the bug by executing attached programs. - 1. Start Cassandra daemon on localhost (number of thrift port is 9160) - 2. Create keyspace and column family, according to create_table.cli, - 3. Execute cassandra_sample_insert.py, storing pairs of row keys and super columns - 4. Execute cassandra_sample_rangeslice.py and get range slice of stored super columns cassandra_sample_insert.py and cassandra_sample_rangeslice.py require pycassa. You will need to execute 4.cassandra_sample_rangeslice.py with following options so that you get range slice of super columns in reversed order. % python cassandra_sample_rangeslice.py -r 00082 00083 On the other hand, to get range slice in normal order, you will need to use following options. % python cassandra_sample_rangeslice.py -f 00082 00083 00082 and 00083 are the specified key range. Range slice can be acquired in normal order but, I cannot get it in reversed order. I assume that there may be a bug within the code for acquiring the index block of specified range. In fact, 00083 is included in gap between lastName of index block and firstName of next index block. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2212) Cannot get range slice of super columns in reversed order
[ https://issues.apache.org/jira/browse/CASSANDRA-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998462#comment-12998462 ] Sylvain Lebresne commented on CASSANDRA-2212: - Sure. The problem is that we were not picking the right index slot for reverse query. Let's take the example from the unit test, and say your index look like this: [0..5][10..15][20..25] And say you look for the slice [13..17]. When doing forward slice, we we doing a binary search comparing 13 (the start of the query) to the lastName part of the index slot, which is fine. You'll end up with the first slot, going from left to right, that may contain the start. When doing a reverse query, we were doing the same thing, only using as a start column the end of the query, aka 17 in my example. However, comparing 17 with the lastName of each index slot, you end up selecting the last slot, which is wrong (the slice exit early since 17 is not in the range). What you want to do is pick the first slot, but now going from right to left, that may contain start. So you want to find the slot where firstName start and take the slot just before. I hope I'm clear. Anyway, that's what the patch does. Cannot get range slice of super columns in reversed order - Key: CASSANDRA-2212 URL: https://issues.apache.org/jira/browse/CASSANDRA-2212 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.2 Environment: Fedore 11, Intel Core i5 Reporter: Muga Nishizawa Assignee: Sylvain Lebresne Fix For: 0.7.3 Attachments: 0001-Fix-IndexHelp.indexFor-for-reverse-query.patch, cassandra_sample_insert.py, cassandra_sample_rangeslice.py, create_table.cli I cannot get range slice of super columns in reversed order. These data are stored in Cassandra in advance. On the other hand, range slice of these data in normal order can be acquired. You can reproduce the bug by executing attached programs. - 1. Start Cassandra daemon on localhost (number of thrift port is 9160) - 2. Create keyspace and column family, according to create_table.cli, - 3. Execute cassandra_sample_insert.py, storing pairs of row keys and super columns - 4. Execute cassandra_sample_rangeslice.py and get range slice of stored super columns cassandra_sample_insert.py and cassandra_sample_rangeslice.py require pycassa. You will need to execute 4.cassandra_sample_rangeslice.py with following options so that you get range slice of super columns in reversed order. % python cassandra_sample_rangeslice.py -r 00082 00083 On the other hand, to get range slice in normal order, you will need to use following options. % python cassandra_sample_rangeslice.py -f 00082 00083 00082 and 00083 are the specified key range. Range slice can be acquired in normal order but, I cannot get it in reversed order. I assume that there may be a bug within the code for acquiring the index block of specified range. In fact, 00083 is included in gap between lastName of index block and firstName of next index block. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073847 - in /cassandra/branches/cassandra-0.7: ./ src/java/org/apache/cassandra/db/columniterator/ src/java/org/apache/cassandra/io/sstable/
Author: jbellis
Date: Wed Feb 23 17:54:43 2011
New Revision: 1073847
URL: http://svn.apache.org/viewvc?rev=1073847view=rev
Log:
fix reversed slice queries on large rows
patch by slebresne; reviewed by jbellis for CASSANDRA-2212
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1073847r1=1073846r2=1073847view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Feb 23 17:54:43 2011
@@ -23,6 +23,7 @@
* fix cache saving on Windows (CASSANDRA-2207)
* add validateSchemaAgreement call + synchronization to schema
modification operations (CASSANDRA-)
+ * fix for reversed slice queries on large rows (CASSANDRA-2212)
0.7.2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java?rev=1073847r1=1073846r2=1073847view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
Wed Feb 23 17:54:43 2011
@@ -146,8 +146,6 @@ class IndexedSliceReader extends Abstrac
file.readInt(); // column count
this.mark = file.mark();
curRangeIndex = IndexHelper.indexFor(startColumn, indexes,
comparator, reversed);
-if (reversed curRangeIndex == indexes.size())
-curRangeIndex--;
}
public boolean getNextBlock() throws IOException
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java?rev=1073847r1=1073846r2=1073847view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
Wed Feb 23 17:54:43 2011
@@ -160,7 +160,7 @@ public class SSTableNamesIterator extend
/* get the various column ranges we have to read */
AbstractType comparator = metadata.comparator;
-SortedSetIndexHelper.IndexInfo ranges = new
TreeSetIndexHelper.IndexInfo(IndexHelper.getComparator(comparator));
+SortedSetIndexHelper.IndexInfo ranges = new
TreeSetIndexHelper.IndexInfo(IndexHelper.getComparator(comparator, false));
for (ByteBuffer name : filteredColumnNames)
{
int index = IndexHelper.indexFor(name, indexList, comparator,
false);
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/IndexHelper.java?rev=1073847r1=1073846r2=1073847view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
Wed Feb 23 17:54:43 2011
@@ -111,8 +111,7 @@ public class IndexHelper
}
/**
- * the index of the IndexInfo in which @name will be found.
- * If the index is @indexList.size(), the @name appears nowhere.
+ * The index of the IndexInfo in which a scan starting with @name should
begin.
*
* @param name
* name of the index
@@ -133,19 +132,40 @@ public class IndexHelper
if (name.remaining() == 0 reversed)
return indexList.size() - 1;
IndexInfo target = new IndexInfo(name, name, 0, 0);
-int index = Collections.binarySearch(indexList, target,
getComparator(comparator));
-return index 0 ? -1 * (index + 1) : index;
+/*
+Take the example from the unit test, and say your index looks like
this:
+[0..5][10..15][20..25]
+and you look for the slice [13..17].
+
+When doing forward slice, we we
[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998485#comment-12998485 ] Vivek Mishra commented on CASSANDRA-2124: - Hi Eric, Looked into python decoder as a newbie to python. But it looks to me as : 1) Decode column value using column family validator and comparator. Same is with java decoder Logic is : 1) On invocation of CassandraStatement execution, make static call to ColumnDecoder with query and keyspace. 2) Implicitly retrieve column family using pattern matching. 3) CassandraResultSet.next() method is changed to pick columnValue from ColumnDecoder.getColumnValue(col.getValue()). This automatically decodes(cast) object into required format. I can see that it looks to me both (py and java) more or less on same path. Having something like describe_column_family later can be a big thing. JDBC driver for CQL --- Key: CASSANDRA-2124 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124 Project: Cassandra Issue Type: New Feature Components: API Reporter: Eric Evans Assignee: Vivek Mishra Priority: Minor Labels: cql Attachments: Cassandra-2124_v1.0, cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, cassandra_generic_decoder.patch A simple connection class and corresponding pool was created for CQL as a part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a replacement for) would also be interesting. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998487#comment-12998487 ] Vivek Mishra commented on CASSANDRA-2124: - plaaning to add support for getBigDecimal(), getBytes etc in CassandraResultSet using above mentioned logic. JDBC driver for CQL --- Key: CASSANDRA-2124 URL: https://issues.apache.org/jira/browse/CASSANDRA-2124 Project: Cassandra Issue Type: New Feature Components: API Reporter: Eric Evans Assignee: Vivek Mishra Priority: Minor Labels: cql Attachments: Cassandra-2124_v1.0, cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, cassandra_generic_decoder.patch A simple connection class and corresponding pool was created for CQL as a part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a replacement for) would also be interesting. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2229) Back off compaction after failure
Back off compaction after failure - Key: CASSANDRA-2229 URL: https://issues.apache.org/jira/browse/CASSANDRA-2229 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.2 Reporter: Nick Bailey Priority: Minor Fix For: 0.8 When compaction fails (for one of the multitude of reasons it can fail, generally some sort of 'corruption'), we should back off on attempting to compact that column family. Continuously trying to compact it will just waste resources. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073884 - in /cassandra/branches/cassandra-0.7: ./ contrib/client_only/conf/ contrib/client_only/src/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/ src/java/org/
Author: gdusbabek Date: Wed Feb 23 19:03:57 2011 New Revision: 1073884 URL: http://svn.apache.org/viewvc?rev=1073884view=rev Log: fat clients were creating local data. patch by gdusbabek, reviewed by tjake. CASSANDRA-2223 Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/contrib/client_only/conf/cassandra.yaml cassandra/branches/cassandra-0.7/contrib/client_only/src/ClientOnlyExample.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/commitlog/CommitLog.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/MigrationManager.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1073884r1=1073883r2=1073884view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Feb 23 19:03:57 2011 @@ -24,6 +24,7 @@ * add validateSchemaAgreement call + synchronization to schema modification operations (CASSANDRA-) * fix for reversed slice queries on large rows (CASSANDRA-2212) + * fat clients were writing local data (CASSANDRA-2223) 0.7.2 Modified: cassandra/branches/cassandra-0.7/contrib/client_only/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/client_only/conf/cassandra.yaml?rev=1073884r1=1073883r2=1073884view=diff == --- cassandra/branches/cassandra-0.7/contrib/client_only/conf/cassandra.yaml (original) +++ cassandra/branches/cassandra-0.7/contrib/client_only/conf/cassandra.yaml Wed Feb 23 19:03:57 2011 @@ -34,6 +34,8 @@ hinted_handoff_enabled: true # this defines the maximum amount of time a dead host will have hints # generated. After it has been dead this long, hints will be dropped. max_hint_window_in_ms: 360 # one hour +# Sleep this long after delivering each row or row fragment +hinted_handoff_throttle_delay_in_ms: 50 # authentication backend, implementing IAuthenticator; used to identify users authenticator: org.apache.cassandra.auth.AllowAllAuthenticator @@ -90,6 +92,31 @@ commitlog_sync: periodic # milliseconds. commitlog_sync_period_in_ms: 1 +# emergency pressure valve: each time heap usage after a full (CMS) +# garbage collection is above this fraction of the max, Cassandra will +# flush the largest memtables. +# +# Set to 1.0 to disable. Setting this lower than +# CMSInitiatingOccupancyFraction is not likely to be useful. +# +# RELYING ON THIS AS YOUR PRIMARY TUNING MECHANISM WILL WORK POORLY: +# it is most effective under light to moderate load, or read-heavy +# workloads; under truly massive write load, it will often be too +# little, too late. +flush_largest_memtables_at: 0.75 + +# emergency pressure valve #2: the first time heap usage after a full +# (CMS) garbage collection is above this fraction of the max, +# Cassandra will reduce cache maximum _capacity_ to the given fraction +# of the current _size_. Should usually be set substantially above +# flush_largest_memtables_at, since that will have less long-term +# impact on the system. +# +# Set to 1.0 to disable. Setting this lower than +# CMSInitiatingOccupancyFraction is not likely to be useful. +reduce_cache_sizes_at: 0.85 +reduce_cache_capacity_to: 0.6 + # Addresses of hosts that are deemed contact points. # Cassandra nodes use this list of hosts to find each other and learn # the topology of the ring. You must change this if you are running @@ -199,6 +226,11 @@ column_index_size_in_kb: 64 # will be logged specifying the row key. in_memory_compaction_limit_in_mb: 64 +# Track cached row keys during compaction, and re-cache their new +# positions in the compacted sstable. Disable if you use really large +# key caches. +compaction_preheat_key_cache: true + # Time to wait for a reply from other nodes before failing the command rpc_timeout_in_ms: 1 Modified: cassandra/branches/cassandra-0.7/contrib/client_only/src/ClientOnlyExample.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/client_only/src/ClientOnlyExample.java?rev=1073884r1=1073883r2=1073884view=diff == --- cassandra/branches/cassandra-0.7/contrib/client_only/src/ClientOnlyExample.java (original) +++
svn commit: r1073886 - /cassandra/trunk/drivers/py/cql/marshal.py
Author: eevans Date: Wed Feb 23 19:09:17 2011 New Revision: 1073886 URL: http://svn.apache.org/viewvc?rev=1073886view=rev Log: fixup typo in CQL Python driver Modified: cassandra/trunk/drivers/py/cql/marshal.py Modified: cassandra/trunk/drivers/py/cql/marshal.py URL: http://svn.apache.org/viewvc/cassandra/trunk/drivers/py/cql/marshal.py?rev=1073886r1=1073885r2=1073886view=diff == --- cassandra/trunk/drivers/py/cql/marshal.py (original) +++ cassandra/trunk/drivers/py/cql/marshal.py Wed Feb 23 19:09:17 2011 @@ -76,7 +76,7 @@ def unmarshal(bytestr, typestr): elif typestr == org.apache.cassandra.db.marshal.LexicalUUIDType: return UUID(bytes=bytestr) elif typestr == org.apache.cassandra.db.marshal.TimeUUIDType: -return UUID(bytes=bytetr) +return UUID(bytes=bytestr) else: return bytestr
svn commit: r1073888 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra: db/BinaryMemtable.java db/ColumnFamilyStore.java db/CompactionManager.java db/Memtable.java io/sstable/SSTabl
Author: jbellis
Date: Wed Feb 23 19:13:01 2011
New Revision: 1073888
URL: http://svn.apache.org/viewvc?rev=1073888view=rev
Log:
simplify SSTableWriter constructors and add
CFS.createFlushWriter/createCompactionWriter
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java?rev=1073888r1=1073887r2=1073888view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
Wed Feb 23 19:13:01 2011
@@ -125,8 +125,7 @@ public class BinaryMemtable implements I
private SSTableReader writeSortedContents(ListDecoratedKey sortedKeys)
throws IOException
{
logger.info(Writing + this);
-String path = cfs.getFlushPath();
-SSTableWriter writer = new SSTableWriter(path, sortedKeys.size(),
cfs.metadata, cfs.partitioner);
+SSTableWriter writer = cfs.createFlushWriter(sortedKeys.size());
for (DecoratedKey key : sortedKeys)
{
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1073888r1=1073887r2=1073888view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Wed Feb 23 19:13:01 2011
@@ -2063,4 +2063,14 @@ public class ColumnFamilyStore implement
ssTables.getKeyCache().setCapacity(newCapacity);
}
}
+
+public SSTableWriter createFlushWriter(long estimatedRows) throws
IOException
+{
+return new SSTableWriter(getFlushPath(), estimatedRows, metadata,
partitioner);
+}
+
+public SSTableWriter createCompactionWriter(long estimatedRows, String
location) throws IOException
+{
+return new SSTableWriter(getTempSSTablePath(location), estimatedRows,
metadata, partitioner);
+}
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java?rev=1073888r1=1073887r2=1073888view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
Wed Feb 23 19:13:01 2011
@@ -444,8 +444,7 @@ public class CompactionManager implement
return 0;
}
-String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
-writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
+writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
while (nni.hasNext())
{
AbstractCompactedRow row = nni.next();
@@ -706,8 +705,7 @@ public class CompactionManager implement
if (writer == null)
{
FileUtils.createDirectory(compactionFileLocation);
-String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
-writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
+writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
}
return writer;
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java?rev=1073888r1=1073887r2=1073888view=diff
==
---
svn commit: r1073889 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra: db/BinaryMemtable.java db/ColumnFamilyStore.java db/CompactionManager.java db/Memtable.java io/sstable/SSTabl
Author: jbellis
Date: Wed Feb 23 19:14:15 2011
New Revision: 1073889
URL: http://svn.apache.org/viewvc?rev=1073889view=rev
Log:
revert last until tests are fixed
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java?rev=1073889r1=1073888r2=1073889view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
Wed Feb 23 19:14:15 2011
@@ -125,7 +125,8 @@ public class BinaryMemtable implements I
private SSTableReader writeSortedContents(ListDecoratedKey sortedKeys)
throws IOException
{
logger.info(Writing + this);
-SSTableWriter writer = cfs.createFlushWriter(sortedKeys.size());
+String path = cfs.getFlushPath();
+SSTableWriter writer = new SSTableWriter(path, sortedKeys.size(),
cfs.metadata, cfs.partitioner);
for (DecoratedKey key : sortedKeys)
{
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1073889r1=1073888r2=1073889view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Wed Feb 23 19:14:15 2011
@@ -2063,14 +2063,4 @@ public class ColumnFamilyStore implement
ssTables.getKeyCache().setCapacity(newCapacity);
}
}
-
-public SSTableWriter createFlushWriter(long estimatedRows) throws
IOException
-{
-return new SSTableWriter(getFlushPath(), estimatedRows, metadata,
partitioner);
-}
-
-public SSTableWriter createCompactionWriter(long estimatedRows, String
location) throws IOException
-{
-return new SSTableWriter(getTempSSTablePath(location), estimatedRows,
metadata, partitioner);
-}
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java?rev=1073889r1=1073888r2=1073889view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
Wed Feb 23 19:14:15 2011
@@ -444,7 +444,8 @@ public class CompactionManager implement
return 0;
}
-writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
+String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
+writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
while (nni.hasNext())
{
AbstractCompactedRow row = nni.next();
@@ -705,7 +706,8 @@ public class CompactionManager implement
if (writer == null)
{
FileUtils.createDirectory(compactionFileLocation);
-writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
+String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
+writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
}
return writer;
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java?rev=1073889r1=1073888r2=1073889view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
(original)
+++
[jira] Created: (CASSANDRA-2230) Allow to specify ConsistencyLevel in cassandra-cli
Allow to specify ConsistencyLevel in cassandra-cli -- Key: CASSANDRA-2230 URL: https://issues.apache.org/jira/browse/CASSANDRA-2230 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Norman Maurer Assignee: Norman Maurer Priority: Minor At the moment when using cassandra-cli all operations are done with ConsistencyLevel.ONE. This works well but I think it would be overall useful to allow to override this ConsistencyLevel on per command basis. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2230) Allow to specify ConsistencyLevel in cassandra-cli
[ https://issues.apache.org/jira/browse/CASSANDRA-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998510#comment-12998510 ] Norman Maurer commented on CASSANDRA-2230: -- I will work on a patch for this... Allow to specify ConsistencyLevel in cassandra-cli -- Key: CASSANDRA-2230 URL: https://issues.apache.org/jira/browse/CASSANDRA-2230 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Norman Maurer Assignee: Norman Maurer Priority: Minor At the moment when using cassandra-cli all operations are done with ConsistencyLevel.ONE. This works well but I think it would be overall useful to allow to override this ConsistencyLevel on per command basis. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2229) Back off compaction after failure
[ https://issues.apache.org/jira/browse/CASSANDRA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998511#comment-12998511 ] Ryan King commented on CASSANDRA-2229: -- It might be worth keeping track of the specific SSTables involved in the failed compaction and skipping those. Its possible we could make some progress on compaction in scenarios where a single sstable is corrupt. Back off compaction after failure - Key: CASSANDRA-2229 URL: https://issues.apache.org/jira/browse/CASSANDRA-2229 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.2 Reporter: Nick Bailey Priority: Minor Fix For: 0.8 When compaction fails (for one of the multitude of reasons it can fail, generally some sort of 'corruption'), we should back off on attempting to compact that column family. Continuously trying to compact it will just waste resources. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073894 - in /cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress: ./ operations/ util/
Author: brandonwilliams
Date: Wed Feb 23 19:25:40 2011
New Revision: 1073894
URL: http://svn.apache.org/viewvc?rev=1073894view=rev
Log:
Switch stress.java to a producer/consumer model for better performance.
Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-2020
Removed:
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/util/OperationThread.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/util/Range.java
Modified:
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Session.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Stress.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/operations/IndexedRangeSlicer.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/operations/Inserter.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/operations/MultiGetter.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/operations/RangeSlicer.java
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/operations/Reader.java
Modified:
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Session.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Session.java?rev=1073894r1=1073893r2=1073894view=diff
==
---
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Session.java
(original)
+++
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Session.java
Wed Feb 23 19:25:40 2011
@@ -20,8 +20,8 @@ package org.apache.cassandra.contrib.str
import java.io.*;
import java.nio.ByteBuffer;
import java.util.*;
-import java.util.concurrent.atomic.AtomicIntegerArray;
-import java.util.concurrent.atomic.AtomicLongArray;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
import org.apache.commons.cli.*;
@@ -38,9 +38,9 @@ public class Session
// command line options
public static final Options availableOptions = new Options();
-public final AtomicIntegerArray operationCount;
-public final AtomicIntegerArray keyCount;
-public final AtomicLongArray latencies;
+public final AtomicInteger operations;
+public final AtomicInteger keys;
+public final AtomicLonglatency;
static
{
@@ -93,7 +93,7 @@ public class Session
private PrintStream out = System.out;
private IndexType indexType = null;
-private Stress.Operation operation = Stress.Operation.INSERT;
+private Stress.Operations operation = Stress.Operations.INSERT;
private ColumnFamilyType columnFamilyType = ColumnFamilyType.Standard;
private ConsistencyLevel consistencyLevel = ConsistencyLevel.ONE;
private String replicationStrategy =
org.apache.cassandra.locator.SimpleStrategy;
@@ -183,7 +183,7 @@ public class Session
unframed = Boolean.parseBoolean(cmd.getOptionValue(m));
if (cmd.hasOption(o))
-operation =
Stress.Operation.valueOf(cmd.getOptionValue(o).toUpperCase());
+operation =
Stress.Operations.valueOf(cmd.getOptionValue(o).toUpperCase());
if (cmd.hasOption(u))
superColumns = Integer.parseInt(cmd.getOptionValue(u));
@@ -248,9 +248,9 @@ public class Session
mean = numKeys / 2;
sigma = numKeys * STDev;
-operationCount = new AtomicIntegerArray(threads);
-keyCount = new AtomicIntegerArray(threads);
-latencies = new AtomicLongArray(threads);
+operations = new AtomicInteger();
+keys = new AtomicInteger();
+latency = new AtomicLong();
}
public int getCardinality()
@@ -323,7 +323,7 @@ public class Session
return ignoreErrors;
}
-public Stress.Operation getOperation()
+public Stress.Operations getOperation()
{
return operation;
}
Modified:
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Stress.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Stress.java?rev=1073894r1=1073893r2=1073894view=diff
==
---
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Stress.java
(original)
+++
cassandra/branches/cassandra-0.7/contrib/stress/src/org/apache/cassandra/contrib/stress/Stress.java
Wed Feb 23 19:25:40 2011
@@ -18,15 +18,18 @@
package
svn commit: r1073896 [2/2] - in /cassandra/trunk: ./ contrib/ contrib/client_only/conf/ contrib/client_only/src/ contrib/stress/src/org/apache/cassandra/contrib/stress/ contrib/stress/src/org/apache/c
Modified: cassandra/trunk/src/java/org/apache/cassandra/io/PrecompactedRow.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/PrecompactedRow.java?rev=1073896r1=1073895r2=1073896view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/PrecompactedRow.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/PrecompactedRow.java Wed
Feb 23 19:32:42 2011
@@ -58,7 +58,7 @@ public class PrecompactedRow extends Abs
this.headerBuffer = new DataOutputBuffer();
}
-public PrecompactedRow(ColumnFamilyStore cfStore,
ListSSTableIdentityIterator rows, boolean major, int gcBefore)
+public PrecompactedRow(ColumnFamilyStore cfStore,
ListSSTableIdentityIterator rows, boolean major, int gcBefore, boolean
forceDeserialize)
{
super(rows.get(0).getKey());
buffer = new DataOutputBuffer();
@@ -71,7 +71,7 @@ public class PrecompactedRow extends Abs
}
boolean shouldPurge = major || !cfStore.isKeyInRemainingSSTables(key,
sstables);
-if (rows.size() 1 || shouldPurge)
+if (rows.size() 1 || shouldPurge ||
!rows.get(0).sstable.descriptor.isLatestVersion || forceDeserialize)
{
ColumnFamily cf = null;
for (SSTableIdentityIterator row : rows)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/sstable/CacheWriter.java?rev=1073896r1=1073895r2=1073896view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/sstable/CacheWriter.java
Wed Feb 23 19:32:42 2011
@@ -89,8 +89,10 @@ public class CacheWriterK, V implement
{
out.close();
}
+
+path.delete(); // ignore error if it didn't exist
if (!tmpFile.renameTo(path))
-throw new IOException(Unable to rename cache to + path);
+throw new IOException(Unable to rename + tmpFile + to +
path);
logger.info(String.format(Saved %s (%d items) in %d ms,
path.getName(), keys.size(),
(System.currentTimeMillis() - start)));
}
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexHelper.java?rev=1073896r1=1073895r2=1073896view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/sstable/IndexHelper.java
Wed Feb 23 19:32:42 2011
@@ -111,8 +111,7 @@ public class IndexHelper
}
/**
- * the index of the IndexInfo in which @name will be found.
- * If the index is @indexList.size(), the @name appears nowhere.
+ * The index of the IndexInfo in which a scan starting with @name should
begin.
*
* @param name
* name of the index
@@ -133,19 +132,40 @@ public class IndexHelper
if (name.remaining() == 0 reversed)
return indexList.size() - 1;
IndexInfo target = new IndexInfo(name, name, 0, 0);
-int index = Collections.binarySearch(indexList, target,
getComparator(comparator));
-return index 0 ? -1 * (index + 1) : index;
+/*
+Take the example from the unit test, and say your index looks like
this:
+[0..5][10..15][20..25]
+and you look for the slice [13..17].
+
+When doing forward slice, we we doing a binary search comparing 13
(the start of the query)
+to the lastName part of the index slot. You'll end up with the first
slot, going from left to right,
+that may contain the start.
+
+When doing a reverse slice, we do the same thing, only using as a
start column the end of the query,
+i.e. 17 in this example, compared to the firstName part of the index
slots. bsearch will give us the
+first slot where firstName start ([20..25] here), so we subtract an
extra one to get the slot just before.
+*/
+int index = Collections.binarySearch(indexList, target,
getComparator(comparator, reversed));
+return index 0 ? -index - (reversed ? 2 : 1) : index;
}
-public static ComparatorIndexInfo getComparator(final AbstractType
nameComparator)
+public static ComparatorIndexInfo getComparator(final AbstractType
nameComparator, boolean reversed)
{
-return new ComparatorIndexInfo()
-{
-public int compare(IndexInfo o1, IndexInfo o2)
-{
-return nameComparator.compare(o1.lastName, o2.lastName);
-
[jira] Resolved: (CASSANDRA-2200) stress.java doesn't insert the correct amount of rows
[ https://issues.apache.org/jira/browse/CASSANDRA-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-2200. - Resolution: Fixed Solved by CASSANDRA-2020 stress.java doesn't insert the correct amount of rows - Key: CASSANDRA-2200 URL: https://issues.apache.org/jira/browse/CASSANDRA-2200 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.1 Reporter: Brandon Williams Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.3 Original Estimate: 8h Remaining Estimate: 8h For example, if you pass -n 200 you only get 1999800 (with 300 threads at least, didn't check if it was related) -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073900 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db: BinaryMemtable.java ColumnFamilyStore.java CompactionManager.java Memtable.java
Author: jbellis
Date: Wed Feb 23 19:38:44 2011
New Revision: 1073900
URL: http://svn.apache.org/viewvc?rev=1073900view=rev
Log:
add CFS.createFlushWriter/createCompactionWriter
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java?rev=1073900r1=1073899r2=1073900view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/BinaryMemtable.java
Wed Feb 23 19:38:44 2011
@@ -125,8 +125,7 @@ public class BinaryMemtable implements I
private SSTableReader writeSortedContents(ListDecoratedKey sortedKeys)
throws IOException
{
logger.info(Writing + this);
-String path = cfs.getFlushPath();
-SSTableWriter writer = new SSTableWriter(path, sortedKeys.size(),
cfs.metadata, cfs.partitioner);
+SSTableWriter writer = cfs.createFlushWriter(sortedKeys.size());
for (DecoratedKey key : sortedKeys)
{
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1073900r1=1073899r2=1073900view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Wed Feb 23 19:38:44 2011
@@ -2063,4 +2063,14 @@ public class ColumnFamilyStore implement
ssTables.getKeyCache().setCapacity(newCapacity);
}
}
+
+public SSTableWriter createFlushWriter(long estimatedRows) throws
IOException
+{
+return new SSTableWriter(getFlushPath(), estimatedRows, metadata,
partitioner);
+}
+
+public SSTableWriter createCompactionWriter(long estimatedRows, String
location) throws IOException
+{
+return new SSTableWriter(getTempSSTablePath(location), estimatedRows,
metadata, partitioner);
+}
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java?rev=1073900r1=1073899r2=1073900view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/CompactionManager.java
Wed Feb 23 19:38:44 2011
@@ -444,8 +444,7 @@ public class CompactionManager implement
return 0;
}
-String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
-writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
+writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
while (nni.hasNext())
{
AbstractCompactedRow row = nni.next();
@@ -706,8 +705,7 @@ public class CompactionManager implement
if (writer == null)
{
FileUtils.createDirectory(compactionFileLocation);
-String newFilename = new
File(cfs.getTempSSTablePath(compactionFileLocation)).getAbsolutePath();
-writer = new SSTableWriter(newFilename, expectedBloomFilterSize,
cfs.metadata, cfs.partitioner);
+writer = cfs.createCompactionWriter(expectedBloomFilterSize,
compactionFileLocation);
}
return writer;
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java?rev=1073900r1=1073899r2=1073900view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Memtable.java
Wed Feb 23 19:38:44 2011
@@ -155,7 +155,7 @@ public class Memtable implements Compara
private SSTableReader
[jira] Created: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal --- Key: CASSANDRA-2231 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.3 Reporter: Ed Anuff Priority: Minor CompositeType is a custom comparer that makes it possible to create comparable composite values out of the basic types that Cassandra currently supports, such as Long, UUID, etc. This is very useful in both the creation of custom inverted indexes using columns in a skinny row, where each column name is a composite value, and also when using Cassandra's built-in secondary index support, where it can be used to encode the values in the columns that Cassandra indexes. One scenario for the usage of these is documented here: http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for contribution is attached and has been previously maintained on github here: https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2223) ClientOnly mode is creating directories
[ https://issues.apache.org/jira/browse/CASSANDRA-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998532#comment-12998532 ] Hudson commented on CASSANDRA-2223: --- Integrated in Cassandra-0.7 #312 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/312/]) fat clients were creating local data. patch by gdusbabek, reviewed by tjake. CASSANDRA-2223 ClientOnly mode is creating directories --- Key: CASSANDRA-2223 URL: https://issues.apache.org/jira/browse/CASSANDRA-2223 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.2 Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.7.3 Attachments: v1-0001-client-only-mode-shouldn-t-create-any-local-data.txt, v2-0001-ClientOnlyExample-waits-for-schema-to-arrive-tries-to-.txt, v2-0002-avoid-creating-local-files-when-in-fat-client-mode.txt, v2-0003-make-sure-fat-clients-gossip-their-schema.txt, v2-0004-update-contrib_client-only-meta.txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2020) stress.java performance falls off heavily towards the end
[ https://issues.apache.org/jira/browse/CASSANDRA-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998531#comment-12998531 ] Hudson commented on CASSANDRA-2020: --- Integrated in Cassandra-0.7 #312 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/312/]) Switch stress.java to a producer/consumer model for better performance. Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-2020 stress.java performance falls off heavily towards the end - Key: CASSANDRA-2020 URL: https://issues.apache.org/jira/browse/CASSANDRA-2020 Project: Cassandra Issue Type: Improvement Components: Contrib Reporter: Brandon Williams Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.3 Attachments: CASSANDRA-2020.patch Original Estimate: 8h Remaining Estimate: 8h This is due to threads completing towards the end, such that there aren't enough to fully stress the cluster. The main problem here is that stress.java is a straight port of stress.py, where each thread runs through some range until it's done, and the threads finish at different times (probably offset by jvm warmup time.) Instead, a producer/consumer model would work better. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Anuff updated CASSANDRA-2231: Attachment: edanuff-CassandraCompositeType-1e253c4.zip https://github.com/edanuff/CassandraCompositeType Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal --- Key: CASSANDRA-2231 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.3 Reporter: Ed Anuff Priority: Minor Attachments: edanuff-CassandraCompositeType-1e253c4.zip CompositeType is a custom comparer that makes it possible to create comparable composite values out of the basic types that Cassandra currently supports, such as Long, UUID, etc. This is very useful in both the creation of custom inverted indexes using columns in a skinny row, where each column name is a composite value, and also when using Cassandra's built-in secondary index support, where it can be used to encode the values in the columns that Cassandra indexes. One scenario for the usage of these is documented here: http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for contribution is attached and has been previously maintained on github here: https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2231: -- Reviewer: slebresne Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal --- Key: CASSANDRA-2231 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.3 Reporter: Ed Anuff Priority: Minor Attachments: edanuff-CassandraCompositeType-1e253c4.zip CompositeType is a custom comparer that makes it possible to create comparable composite values out of the basic types that Cassandra currently supports, such as Long, UUID, etc. This is very useful in both the creation of custom inverted indexes using columns in a skinny row, where each column name is a composite value, and also when using Cassandra's built-in secondary index support, where it can be used to encode the values in the columns that Cassandra indexes. One scenario for the usage of these is documented here: http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for contribution is attached and has been previously maintained on github here: https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2232) Clean up and document EstimatedHistogram
Clean up and document EstimatedHistogram Key: CASSANDRA-2232 URL: https://issues.apache.org/jira/browse/CASSANDRA-2232 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.3 EstimatedHistogram treats adding value n as adding a value infinitesimally greater than n. This barely made sense for the original goal of latency tracking but is clearly broken for inherently integral data like sstables-per-read. Also, median() is broken, but even a non-broken median() would not be correct to use in mean row size reporting which is its only caller. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2232) Clean up and document EstimatedHistogram
[ https://issues.apache.org/jira/browse/CASSANDRA-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2232: -- Attachment: 2232.txt fixes problems and adds documentation Clean up and document EstimatedHistogram Key: CASSANDRA-2232 URL: https://issues.apache.org/jira/browse/CASSANDRA-2232 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.3 Attachments: 2232.txt EstimatedHistogram treats adding value n as adding a value infinitesimally greater than n. This barely made sense for the original goal of latency tracking but is clearly broken for inherently integral data like sstables-per-read. Also, median() is broken, but even a non-broken median() would not be correct to use in mean row size reporting which is its only caller. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1073986 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/tools/SSTableImport.java
Author: jbellis
Date: Wed Feb 23 23:28:02 2011
New Revision: 1073986
URL: http://svn.apache.org/viewvc?rev=1073986view=rev
Log:
turnoff string interning in json2sstable
patch by jbellis for CASSANDRA-2189
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1073986r1=1073985r2=1073986view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Wed Feb 23 23:28:02 2011
@@ -25,6 +25,7 @@
modification operations (CASSANDRA-)
* fix for reversed slice queries on large rows (CASSANDRA-2212)
* fat clients were writing local data (CASSANDRA-2223)
+ * turn off string interning in json2sstable (CASSANDRA-2189)
0.7.2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java?rev=1073986r1=1073985r2=1073986view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/tools/SSTableImport.java
Wed Feb 23 23:28:02 2011
@@ -360,7 +360,7 @@ public class SSTableImport
*/
private static JsonParser getParser(String fileName) throws IOException
{
-return factory.createJsonParser(new File(fileName));
+return factory.createJsonParser(new
File(fileName)).configure(JsonParser.Feature.INTERN_FIELD_NAMES, false);
}
/**
[jira] Commented: (CASSANDRA-2008) CLI help incorrect in places
[ https://issues.apache.org/jira/browse/CASSANDRA-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998610#comment-12998610 ] Jonathan Ellis commented on CASSANDRA-2008: --- How is this coming? CLI help incorrect in places Key: CASSANDRA-2008 URL: https://issues.apache.org/jira/browse/CASSANDRA-2008 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Aaron Morton Assignee: Aaron Morton Priority: Trivial Fix For: 0.7.3 Found some errors in the CLI help, such as these for create column family. - memtable_operations: Flush memtables after this many operations - memtable_throughput: ... or after this many bytes have been written - memtable_flush_after: ... or after this many seconds Should be millions of ops, MB's written and minutes not seconds. Have confirmed thats how the values are used. Will check all the help. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra trying to delete nonexisting file
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2206: -- Attachment: 2206-v2.txt v2 also fixes a bug where tmp and non-tmp Descriptors were considered equal, which is what confused the file scanning for scrub in your example Startup fails due to cassandra trying to delete nonexisting file Key: CASSANDRA-2206 URL: https://issues.apache.org/jira/browse/CASSANDRA-2206 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.2 Environment: linux Reporter: Thibaut Assignee: Jonathan Ellis Fix For: 0.7.3 Attachments: 2206-v2.txt, 2206.txt Original Estimate: 1h Remaining Estimate: 1h Hi, On one of our nodes, startup fails due to cassandra trying to delete a nonexistant Data file (see below). Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured. Directory listing: total 109M -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db Cassandra log: /software/cassandra/bin/cassandra rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized INFO 12:47:29,030 Heap size: 2614493184/2614493184 INFO 12:47:29,031 JNA not found. Native methods will be disabled. INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244 INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244 INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137 INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352 INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353 INFO
[jira] Commented: (CASSANDRA-2189) json2sstable fails due to OutOfMemory
[ https://issues.apache.org/jira/browse/CASSANDRA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998616#comment-12998616 ] Hudson commented on CASSANDRA-2189: --- Integrated in Cassandra-0.7 #313 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/313/]) turnoff string interning in json2sstable patch by jbellis for CASSANDRA-2189 json2sstable fails due to OutOfMemory - Key: CASSANDRA-2189 URL: https://issues.apache.org/jira/browse/CASSANDRA-2189 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.0 Environment: linux Reporter: Shotaro Kamio Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.3 Attachments: 2189.txt Original Estimate: 1h Remaining Estimate: 1h I have a json file created with sstable2json for a column family of super column type. Its size is about 1.9GB. (It's a dump of all keys because I cannot find out how to specify keys to dump in sstable2json.) When I tried to create sstable from the json file, it failed with OutOfMemoryError as follows. WARN 00:31:58,595 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored. Exception in thread main java.lang.OutOfMemoryError: PermGen space at java.lang.String.intern(Native Method) at org.codehaus.jackson.util.InternCache.intern(InternCache.java:40) at org.codehaus.jackson.sym.BytesToNameCanonicalizer.addName(BytesToNameCanonicalizer.java:471) at org.codehaus.jackson.impl.Utf8StreamParser.addName(Utf8StreamParser.java:893) at org.codehaus.jackson.impl.Utf8StreamParser.findName(Utf8StreamParser.java:773) at org.codehaus.jackson.impl.Utf8StreamParser.parseLongFieldName(Utf8StreamParser.java:379) at org.codehaus.jackson.impl.Utf8StreamParser.parseMediumFieldName(Utf8StreamParser.java:347) at org.codehaus.jackson.impl.Utf8StreamParser._parseFieldName(Utf8StreamParser.java:304) at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:140) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.mapObject(UntypedObjectDeserializer.java:93) at org.codehaus.jackson.map.deser.UntypedObjectDeserializer.deserialize(UntypedObjectDeserializer.java:65) at org.codehaus.jackson.map.deser.MapDeserializer._readAndBind(MapDeserializer.java:197) at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:145) at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:23) at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:1261) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:517) at org.codehaus.jackson.JsonParser.readValueAs(JsonParser.java:897) at org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:208) at org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:197) at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:421) So, what I had to is that split the json file with split command and modify them to be correct json file. Create sstable for each small json files. Could you change json2sstable to avoid OutOfMemory? -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of DebianPackaging by DavidStrauss
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The DebianPackaging page has been changed by DavidStrauss.
http://wiki.apache.org/cassandra/DebianPackaging?action=diffrev1=16rev2=17
--
== Official Debian Package ==
+
+ Before installing, you will need a compatible JVM. If you're on a Debian or
Ubuntu system, install OpenJDK:
+ {{{
+ sudo apt-get install default-jdk
+ }}}
To install on Debian or Debian derivatives, use the following sources:
[Cassandra Wiki] Update of DebianPackaging by DavidStrauss
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The DebianPackaging page has been changed by DavidStrauss.
http://wiki.apache.org/cassandra/DebianPackaging?action=diffrev1=17rev2=18
--
== Official Debian Package ==
- Before installing, you will need a compatible JVM. If you're on a Debian or
Ubuntu system, install OpenJDK:
+ To run Cassandra, you will need a compatible JVM. If you're on a Debian or
Ubuntu system, you should probably install OpenJDK:
{{{
sudo apt-get install default-jdk
}}}
[Cassandra Wiki] Update of DebianPackaging by DavidStrauss
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The DebianPackaging page has been changed by DavidStrauss.
The comment on this change is: Correct spelling, drop instructions to install
default-jdk because the Cassandra package should handle that..
http://wiki.apache.org/cassandra/DebianPackaging?action=diffrev1=18rev2=19
--
== Official Debian Package ==
-
- To run Cassandra, you will need a compatible JVM. If you're on a Debian or
Ubuntu system, you should probably install OpenJDK:
- {{{
- sudo apt-get install default-jdk
- }}}
To install on Debian or Debian derivatives, use the following sources:
@@ -65, +60 @@
Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing)
}}}
- By default, installing the Casasndra Debian package or its build dependencies
will pull in OpenJDK. For runtime purposes this will work fine, but due to an
issue with the packaging of OpenJDK in Lenny (see
http://bugs.debian.org/501487), building the package from source will fail. If
you need to (re)build the package on Lenny, install `sun-java6-jdk` before-hand
(`sun-java6-jdk` provides `java6-sdk` which satisfies the dependency), or use
the alternatives system to change the default compiler afterward.
+ By default, installing the Cassandra Debian package or its build dependencies
will pull in OpenJDK. For runtime purposes this will work fine, but due to an
issue with the packaging of OpenJDK in Lenny (see
http://bugs.debian.org/501487), building the package from source will fail. If
you need to (re)build the package on Lenny, install `sun-java6-jdk` before-hand
(`sun-java6-jdk` provides `java6-sdk` which satisfies the dependency), or use
the alternatives system to change the default compiler afterward.
{{{
sudo update-alternatives --config javac
[jira] Commented: (CASSANDRA-2100) Restart required to change cache_save_period
[ https://issues.apache.org/jira/browse/CASSANDRA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998621#comment-12998621 ] Jon Hermes commented on CASSANDRA-2100: --- I thought that UpdateColumnFamily.applyModels() called CFS.reload() in all cases, but that only happens when the server is not in client mode. CFMetaData.apply(avro CfDef) will need to call the update logic. Patch forthcoming. Restart required to change cache_save_period Key: CASSANDRA-2100 URL: https://issues.apache.org/jira/browse/CASSANDRA-2100 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Nick Bailey Assignee: Jon Hermes Priority: Minor Fix For: 0.7.3 Attachments: 2100-2.txt, 2100.txt Original Estimate: 2h Remaining Estimate: 2h The cache_save_period is set in the schema for each column family. However this value is only checked when a node starts up so changing this value isn't really dynamic. We should actually change this when the schema changes instead of having to restart. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2025) generalized way of expressing hierarchical values
[
https://issues.apache.org/jira/browse/CASSANDRA-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Evans updated CASSANDRA-2025:
--
Attachment:
v1-0001-CASSANDRA-2025-updated-consistency-level-spec-removed.txt
generalized way of expressing hierarchical values
-
Key: CASSANDRA-2025
URL: https://issues.apache.org/jira/browse/CASSANDRA-2025
Project: Cassandra
Issue Type: Sub-task
Components: API
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
Labels: cql
Fix For: 0.8
Attachments:
v1-0001-CASSANDRA-2025-updated-consistency-level-spec-removed.txt
Original Estimate: 0h
Remaining Estimate: 0h
While hashing out {{CREATE KEYSPACE}}, it became obvious that we needed a
syntax for expressing hierarchical values. Properties like
{{replication_factor}} can be expressed simply using keyword arguments like
({{replication_factor = 3}}), but {{strategy_options}} is a map of strings.
The solution I took in CASSANDRA-1709 was to dot-delimit map name and
option key, so for example:
{code:style=SQL}
CREATE KEYSPACE keyspace WITH ... AND strategy_options.DC1 = 1 ...
{code}
This led me to wonder if this was a general enough approach for any future
cases that might come up. One example might be compound/composite column
names. Dot-delimiting is a bad choice here since it rules out ever
introducing a float literal.
One suggestion would be to colon-delimit, so for example:
{code:style=SQL}
CREATE KEYSPACE keyspace WITH ... AND strategy_options:DC1 = 1 ...
{code}
Or in the case of composite column names:
{code:style=SQL}
SELECT columnA:columnB,column1:column2 FROM Standard2 USING
CONSISTENCY.QUORUM WHERE KEY = key;
UPDATE Standard2 SET columnA:columnB = valueC, column1:column2 = value3 WHERE
KEY = key;
{code}
As an aside, this also led me to the conclusion that {{CONSISTENCY.LEVEL}}
is probably a bad choice for consistency level specification. It mirrors the
underlying enum for no good reason and should probably be changed to
{{CONSISTENCY LEVEL}} (i.e. omitting the separator). For example:
{code:style=SQL}
SELECT column FROM Standard2 USING CONSISTENCY QUORUM WHERE KEY = key;
{code}
Thoughts?
*Edit: improved final example*
*Edit: restore final example, create new one (gah).*
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of CassandraCli by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The CassandraCli page has been changed by jeremyhanna.
The comment on this change is: Added some more example information..
http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=27rev2=28
--
Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit.
[default@unknown] connect localhost/9160;
Connected to: Test Cluster on localhost/9160
- [default@unknown] create keyspace Keyspace1;
+ [default@unknown] create keyspace Twissandra;
d105c4f1-3c93-11e0-9fb5-e700f669bcfc
- [default@unknown] use Keyspace1;
+ [default@unknown] use Twissandra;
- Authenticated to keyspace: Keyspace1
+ Authenticated to keyspace: Twissandra
- [default@Keyspace1] create column family Standard2;
+ [default@Twissandra] create column family User with comparator = UTF8Type;
00389812-3c94-11e0-9fb5-e700f669bcfc
- [default@Keyspace1] quit;
+ [default@Twissandra] quit;
evans@achilles:~/cassandra$
}}}
- In the above example we started the cli with no options. You can specify
things like `-host`, `-port`, `-keyspace`, `-username`, `-password`, etc. Use
`bin/cassandra-cli -?` for a full set of options. We went on to connect to our
local Cassandra node. We created keyspace `Keyspace1` and column family
`Standard2` with the default options. Then we exited from our cli shell.
+ In the above example we started the cli with no options. You can specify
things like `-host`, `-port`, `-keyspace`, `-username`, `-password`, etc. Use
`bin/cassandra-cli -?` for a full set of options.
+
+ We went on to connect to our local Cassandra node. We created keyspace
`Twissandra` and column family `User` with the default options. Note that with
the column family, we used a UTF8Type comparator. That means that the columns
will be sorted based on UTF8Type sorting. It also means that when the column
names are displayed on the command-line, they will be displayed as UTF8Type
(readable) text. For more information and options for creating column families
type `help create column family;` on the command line. Finally, we exited from
our cli shell.
Let's get back into the shell with some options specified and create some
data.
@@ -30, +32 @@
Welcome to cassandra CLI.
Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit.
- [default@unknown] use Keyspace1;
+ [default@unknown] use Twissandra;
- Authenticated to keyspace: Keyspace1
+ Authenticated to keyspace: Twissandra
- [default@Keyspace1] set Standard2['jsmith']['first'] = 'John';
+ [default@Twissandra] set User['jsmith']['first'] = 'John';
Value inserted.
- [default@Keyspace1] set Standard2['jsmith']['last'] = 'Smith';
+ [default@Twissandra] set User['jsmith']['last'] = 'Smith';
Value inserted.
- [default@Keyspace1] set Standard2['jsmith']['age'] = '42';
+ [default@Twissandra] set User['jsmith']['age'] = '39';
Value inserted.
}}}
- Note that before we can start adding data, we have to `use Keyspace1` to set
our context. We created a record in the `Standard2` column family using the key
`jsmith`. This record has three columns, `first`, `last`, and `age`. Each of
these commands is the equivalent to an `insert()` using the [[API|Thrift API]].
+ Note that before we can start adding data, we have to `use Twissandra` to set
our context. We created a record in the `User` column family using the key
`jsmith`. This record has three columns, `first`, `last`, and `age`. Each of
these commands is the equivalent to an `insert()` using the [[API|Thrift API]].
Now let's read back the `jsmith` row to see what it contains:
{{{
- [default@Keyspace1] get Standard2['jsmith'];
- = (column=616765, value=3432, timestamp=1298168118014000)
+ [default@Twissandra] get User['jsmith'];
+ = (column=age, value=3339, timestamp=1298504259386000)
- = (column=6669727374, value=4a6f686e, timestamp=1298166059047000)
+ = (column=first, value=4a6f686e, timestamp=1298504239938000)
- = (column=6c617374, value=536d697468, timestamp=1298168112533000)
+ = (column=last, value=536d697468, timestamp=129850424857)
Returned 3 results.
}}}
Note: Using the `get` command in this form is the equivalent to a
`get_slice()` using the [[API|Thrift API]].
+ Why are the values all hex? It's because the default validation class is
BytesType, which displays in hex in the output. Let's update the column
metadata of the column family to not only make them output in a readable
format, but also add a secondary index on age. We'll also add another record so
that we can see the secondary index work.
+ {{{
+ [default@Twissandra] update column family User with
+ ... column_metadata =
+ ... [
+ ... {column_name: first, validation_class: UTF8Type},
+ ... {column_name: last, validation_class: UTF8Type},
+ ... {column_name:
[jira] Updated: (CASSANDRA-2100) Restart required to change cache_save_period
[ https://issues.apache.org/jira/browse/CASSANDRA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Hermes updated CASSANDRA-2100: -- Attachment: 2100-3.txt Slight surprise in testing due to 2172 changes, but tests correctly (save period is updated at runtime for both JMX and schema migrations (both client and non-client mode)). Restart required to change cache_save_period Key: CASSANDRA-2100 URL: https://issues.apache.org/jira/browse/CASSANDRA-2100 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Nick Bailey Assignee: Jon Hermes Priority: Minor Fix For: 0.7.3 Attachments: 2100-2.txt, 2100-3.txt, 2100.txt Original Estimate: 2h Remaining Estimate: 2h The cache_save_period is set in the schema for each column family. However this value is only checked when a node starts up so changing this value isn't really dynamic. We should actually change this when the schema changes instead of having to restart. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2025) generalized way of expressing hierarchical values
[
https://issues.apache.org/jira/browse/CASSANDRA-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998627#comment-12998627
]
Eric Evans commented on CASSANDRA-2025:
---
Patch attached to remove the '.' from consistency level specifications.
generalized way of expressing hierarchical values
-
Key: CASSANDRA-2025
URL: https://issues.apache.org/jira/browse/CASSANDRA-2025
Project: Cassandra
Issue Type: Sub-task
Components: API
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
Labels: cql
Fix For: 0.8
Attachments:
v1-0001-CASSANDRA-2025-updated-consistency-level-spec-removed.txt
Original Estimate: 0h
Remaining Estimate: 0h
While hashing out {{CREATE KEYSPACE}}, it became obvious that we needed a
syntax for expressing hierarchical values. Properties like
{{replication_factor}} can be expressed simply using keyword arguments like
({{replication_factor = 3}}), but {{strategy_options}} is a map of strings.
The solution I took in CASSANDRA-1709 was to dot-delimit map name and
option key, so for example:
{code:style=SQL}
CREATE KEYSPACE keyspace WITH ... AND strategy_options.DC1 = 1 ...
{code}
This led me to wonder if this was a general enough approach for any future
cases that might come up. One example might be compound/composite column
names. Dot-delimiting is a bad choice here since it rules out ever
introducing a float literal.
One suggestion would be to colon-delimit, so for example:
{code:style=SQL}
CREATE KEYSPACE keyspace WITH ... AND strategy_options:DC1 = 1 ...
{code}
Or in the case of composite column names:
{code:style=SQL}
SELECT columnA:columnB,column1:column2 FROM Standard2 USING
CONSISTENCY.QUORUM WHERE KEY = key;
UPDATE Standard2 SET columnA:columnB = valueC, column1:column2 = value3 WHERE
KEY = key;
{code}
As an aside, this also led me to the conclusion that {{CONSISTENCY.LEVEL}}
is probably a bad choice for consistency level specification. It mirrors the
underlying enum for no good reason and should probably be changed to
{{CONSISTENCY LEVEL}} (i.e. omitting the separator). For example:
{code:style=SQL}
SELECT column FROM Standard2 USING CONSISTENCY QUORUM WHERE KEY = key;
{code}
Thoughts?
*Edit: improved final example*
*Edit: restore final example, create new one (gah).*
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of CassandraCli by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The CassandraCli page has been changed by jeremyhanna.
http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=28rev2=29
--
Returned 3 results.
}}}
Note: Using the `get` command in this form is the equivalent to a
`get_slice()` using the [[API|Thrift API]].
+
Why are the values all hex? It's because the default validation class is
BytesType, which displays in hex in the output. Let's update the column
metadata of the column family to not only make them output in a readable
format, but also add a secondary index on age. We'll also add another record so
that we can see the secondary index work.
{{{
[Cassandra Wiki] Trivial Update of CassandraCli by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The CassandraCli page has been changed by jeremyhanna.
http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=29rev2=30
--
}}}
Note: Using the `get` command in this form is the equivalent to a
`get_slice()` using the [[API|Thrift API]].
- Why are the values all hex? It's because the default validation class is
BytesType, which displays in hex in the output. Let's update the column
metadata of the column family to not only make them output in a readable
format, but also add a secondary index on age. We'll also add another record so
that we can see the secondary index work.
+ Why are the values all hex? It's because the default validation class is
!BytesType, which displays in hex in the output. Let's update the column
metadata of the column family to not only make them output in a readable
format, but also add a secondary index on age. We'll also add another record so
that we can see the secondary index work.
{{{
[default@Twissandra] update column family User with
[Cassandra Wiki] Update of CassandraCli by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The CassandraCli page has been changed by jeremyhanna.
The comment on this change is: Added an example with scripting schema creation..
http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=30rev2=31
--
}}}
In the above example, you can see that we can span commands over multiple
lines. We add column metadata that validates the column data as well as display
value unencoded in the cli output. We also add an index on age. We add one more
row with an age of '42' and finally query the column family for rows that with
an age of 42.
+ One final thing that is very handy about the cassandra-cli, you can script
your schema creation in a file and run it through the cli. You just create a
text file with any number of creation commands and run the cli with the `-f`
option:
+
+ {{{
+ tblose@quasar:~/dev/workspaces/cassandra$ bin/cassandra-cli -host localhost
-port 9160 -f ~/cassandra-schema.txt
+ Connected to: Test Cluster on localhost/9160
+ 1eafa8f4-3faf-11e0-a627-e700f669bcfc
+ Authenticated to keyspace: Twissandra
+ 1f09fdf5-3faf-11e0-a627-e700f669bcfc
+ }}}
+
+ with `cassandra-schema.txt`:
+
+ {{{
+ create keyspace Twissandra;
+ use Twissandra;
+
+ create column family User with
+ comparator = UTF8Type and
+ column_metadata =
+ [
+ {column_name: first, validation_class: UTF8Type},
+ {column_name: last, validation_class: UTF8Type},
+ {column_name: age, validation_class: UTF8Type, index_type: KEYS}
+ ];
+ }}}
+
This has just been a brief introduction with a couple of examples. For more
information on how things work, type `help;` on the cli by itself or with any
of the commands you're interested in.
[jira] Commented: (CASSANDRA-2008) CLI help incorrect in places
[
https://issues.apache.org/jira/browse/CASSANDRA-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998635#comment-12998635
]
Jon Hermes commented on CASSANDRA-2008:
---
bq. I am not a fan either of the cli somehow extracting yaml comments
Not what I was suggesting, but Aaron nailed it with the `commands: {name,help}`
blocks.
Hadn't seen #2007 yet, but that looks very good. Separating node.yaml and
schema.yaml is good, but separating and throwing away schema.yaml (for a script
solution) is even better.
I would rename `o.a.c.cli.CliHelp.yaml` to something more generic and move it
out of cli(conf/config_help.yaml). The cli can read it from there, it's an
intuitive place to find it for a new user, and it can be mirrored onto the wiki
and anywhere else it's referenced.
If there were a way to couple it with or generate it from
interface/cassandra.thrift:{Ks,Cf}Def, o.a.c.config.Raw{Keyspace,ColumnFamily}
and o.a.c.config.{KS,CF}MetaData, it would be even better-er. Currently all
three are kept in sync by force of will which has introduced multiple bugs
already. (Note: this is not in the scope of this ticket, but would be worth
keeping in mind.)
CLI help incorrect in places
Key: CASSANDRA-2008
URL: https://issues.apache.org/jira/browse/CASSANDRA-2008
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 0.7.0
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Trivial
Fix For: 0.7.3
Found some errors in the CLI help, such as these for create column family.
- memtable_operations: Flush memtables after this many operations
- memtable_throughput: ... or after this many bytes have been written
- memtable_flush_after: ... or after this many seconds
Should be millions of ops, MB's written and minutes not seconds. Have
confirmed thats how the values are used. Will check all the help.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of CassandraCli by jeremyhanna
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The CassandraCli page has been changed by jeremyhanna. http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=32rev2=33 -- [default@Twissandra] set User['jsmith']['age'] = '39'; Value inserted. }}} - Note that before we can start adding data, we have to `use Twissandra` to set our context. We created a record in the `User` column family using the key `jsmith`. This record has three columns, `first`, `last`, and `age`. Each of these commands is the equivalent to an `insert()` using the [[API|Thrift API]]. + Note that before we can start adding data, we have to `use Twissandra;` to set our context. We created a record in the `User` column family using the key `jsmith`. This record has three columns, `first`, `last`, and `age`. Each of these commands is the equivalent to an `insert()` using the [[API|Thrift API]]. Now let's read back the `jsmith` row to see what it contains:
[Cassandra Wiki] Trivial Update of CassandraCli by jeremyhanna
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The CassandraCli page has been changed by jeremyhanna. http://wiki.apache.org/cassandra/CassandraCli?action=diffrev1=33rev2=34 -- 1 Row Returned. }}} - In the above example, you can see that we can span commands over multiple lines. We add column metadata that validates the column data as well as display value unencoded in the cli output. We also add an index on age. We add one more row with an age of '42' and finally query the column family for rows that with an age of 42. + In the above example, you can see that we can span commands over multiple lines. We add column metadata that validates the column data as well as display value unencoded in the cli output. We also add an index on age. The `KEYS` index type means that we can only perform equality operations over it. We add one more row with an age of '42' and finally query the column family for rows with an age of 42. One final thing that is very handy about the cassandra-cli, you can script your schema creation in a file and run it through the cli. You just create a text file with any number of creation commands and run the cli with the `-f` option:
[jira] Commented: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998649#comment-12998649 ] T Jake Luciani commented on CASSANDRA-2176: --- bq. What's going on w/ the diff to CustomTPS, is that your IDE going crazy w/ whitespace changes? The current file is totally unformatted. The core of the changes is in the serve() where it makes sure activeClient counts is = max. The activeClients is decremented when a client hangs up see WorkerProcess.run bq. is there any reason not to make these cassandra.conf options instead of -D flags? No, I just figures most people don't care, would you like me to add this there? Add configuration setting to cap the number of Thrift connections - Key: CASSANDRA-2176 URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.3 Attachments: 0001-CASSANDRA-2176-limit-number-of-connected-clients.txt Original Estimate: 4h Remaining Estimate: 4h At least until CASSANDRA-1405 is done, it's useful to have a connection cap to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1074010 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
Author: jbellis
Date: Thu Feb 24 01:41:29 2011
New Revision: 1074010
URL: http://svn.apache.org/viewvc?rev=1074010view=rev
Log:
reformat
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java?rev=1074010r1=1074009r2=1074010view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
Thu Feb 24 01:41:29 2011
@@ -39,151 +39,180 @@ import org.apache.thrift.transport.TTran
/**
* Slightly modified version of the Apache Thrift TThreadPoolServer.
- *
+ * p/
* This allows passing an executor so you have more control over the actual
* behaviour of the tasks being run.
- *
+ * p/
* Newer version of Thrift should make this obsolete.
*/
-public class CustomTThreadPoolServer extends TServer {
+public class CustomTThreadPoolServer extends TServer
+{
-private static final Logger LOGGER =
LoggerFactory.getLogger(CustomTThreadPoolServer.class.getName());
+private static final Logger LOGGER =
LoggerFactory.getLogger(CustomTThreadPoolServer.class.getName());
-// Executor service for handling client connections
-private ExecutorService executorService_;
+// Executor service for handling client connections
+private ExecutorService executorService_;
-// Flag for stopping the server
-private volatile boolean stopped_;
-
-// Server options
-private Options options_;
-
-// Customizable server options
-public static class Options {
- public int minWorkerThreads = 5;
- public int maxWorkerThreads = Integer.MAX_VALUE;
- public int stopTimeoutVal = 60;
- public TimeUnit stopTimeoutUnit = TimeUnit.SECONDS;
-}
-
-
-public CustomTThreadPoolServer(TProcessorFactory tProcessorFactory,
-TServerSocket tServerSocket,
-TTransportFactory inTransportFactory,
-TTransportFactory outTransportFactory,
-TProtocolFactory tProtocolFactory,
-TProtocolFactory tProtocolFactory2,
-Options options,
-ExecutorService executorService) {
-
-super(tProcessorFactory, tServerSocket, inTransportFactory,
outTransportFactory,
-tProtocolFactory, tProtocolFactory2);
-options_ = options;
-executorService_ = executorService;
-}
-
-
-public void serve() {
- try {
- serverTransport_.listen();
- } catch (TTransportException ttx) {
- LOGGER.error(Error occurred during listening., ttx);
- return;
- }
-
- stopped_ = false;
- while (!stopped_) {
- int failureCount = 0;
- try {
- TTransport client = serverTransport_.accept();
- WorkerProcess wp = new WorkerProcess(client);
- executorService_.execute(wp);
- } catch (TTransportException ttx) {
- if (!stopped_) {
- ++failureCount;
- LOGGER.warn(Transport error occurred during acceptance of
message., ttx);
- }
- }
- }
-
- executorService_.shutdown();
-
- // Loop until awaitTermination finally does return without a interrupted
- // exception. If we don't do this, then we'll shut down prematurely. We
want
- // to let the executorService clear it's task queue, closing client
sockets
- // appropriately.
- long timeoutMS =
options_.stopTimeoutUnit.toMillis(options_.stopTimeoutVal);
- long now = System.currentTimeMillis();
- while (timeoutMS = 0) {
- try {
- executorService_.awaitTermination(timeoutMS,
TimeUnit.MILLISECONDS);
- break;
- } catch (InterruptedException ix) {
- long newnow = System.currentTimeMillis();
- timeoutMS -= (newnow - now);
- now = newnow;
- }
- }
-}
-
-public void stop() {
- stopped_ = true;
- serverTransport_.interrupt();
-}
-
-private class WorkerProcess implements Runnable {
-
- /**
-* Client that this services.
-*/
- private TTransport client_;
-
- /**
-* Default constructor.
-*
-* @param client Transport to process
-*/
- private WorkerProcess(TTransport client) {
- client_ = client;
- }
-
- /**
-* Loops on processing a client forever
-*/
- public void run() {
- TProcessor processor = null;
- TTransport inputTransport = null;
- TTransport outputTransport = null;
- TProtocol inputProtocol = null;
- TProtocol outputProtocol = null;
- try {
-
[jira] Commented: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998657#comment-12998657 ] Jonathan Ellis commented on CASSANDRA-2176: --- bq. The current file is totally unformatted Committed a reformat so we don't have to mix them together. bq. would you like me to add this there? Yes, let's do that. Add configuration setting to cap the number of Thrift connections - Key: CASSANDRA-2176 URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.3 Attachments: 0001-CASSANDRA-2176-limit-number-of-connected-clients.txt Original Estimate: 4h Remaining Estimate: 4h At least until CASSANDRA-1405 is done, it's useful to have a connection cap to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1074018 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/config/CFMetaData.java src/java/org/apache/cassandra/io/sstable/Descriptor.java src/java/org/apac
Author: jbellis
Date: Thu Feb 24 02:17:16 2011
New Revision: 1074018
URL: http://svn.apache.org/viewvc?rev=1074018view=rev
Log:
DEFAULT_MEMTABLE_LIFETIME_IN_MINS to 24h
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1074018r1=1074017r2=1074018view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Feb 24 02:17:16 2011
@@ -26,6 +26,7 @@
* fix for reversed slice queries on large rows (CASSANDRA-2212)
* fat clients were writing local data (CASSANDRA-2223)
* turn off string interning in json2sstable (CASSANDRA-2189)
+ * set DEFAULT_MEMTABLE_LIFETIME_IN_MINS to 24h
0.7.2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java?rev=1074018r1=1074017r2=1074018view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java
Thu Feb 24 02:17:16 2011
@@ -56,7 +56,7 @@ public final class CFMetaData
public final static int DEFAULT_GC_GRACE_SECONDS = 864000;
public final static int DEFAULT_MIN_COMPACTION_THRESHOLD = 4;
public final static int DEFAULT_MAX_COMPACTION_THRESHOLD = 32;
-public final static int DEFAULT_MEMTABLE_LIFETIME_IN_MINS = 60;
+public final static int DEFAULT_MEMTABLE_LIFETIME_IN_MINS = 60 * 24;
public final static int DEFAULT_MEMTABLE_THROUGHPUT_IN_MB =
sizeMemtableThroughput();
public final static double DEFAULT_MEMTABLE_OPERATIONS_IN_MILLIONS =
sizeMemtableOperations(DEFAULT_MEMTABLE_THROUGHPUT_IN_MB);
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java?rev=1074018r1=1074017r2=1074018view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
Thu Feb 24 02:17:16 2011
@@ -199,7 +199,11 @@ public class Descriptor
if (!(o instanceof Descriptor))
return false;
Descriptor that = (Descriptor)o;
-return that.directory.equals(this.directory) that.generation ==
this.generation that.ksname.equals(this.ksname)
that.cfname.equals(this.cfname);
+return that.directory.equals(this.directory)
+that.generation == this.generation
+that.ksname.equals(this.ksname)
+that.cfname.equals(this.cfname)
+that.temporary == this.temporary;
}
@Override
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1074018r1=1074017r2=1074018view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Thu Feb 24 02:17:16 2011
@@ -22,9 +22,12 @@ package org.apache.cassandra.io.sstable;
import java.io.*;
import java.nio.ByteBuffer;
import java.util.Arrays;
+import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
+import com.google.common.collect.Sets;
+
import org.apache.cassandra.utils.ByteBufferUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -204,8 +207,10 @@ public class SSTableWriter extends SSTab
Descriptor newdesc = tmpdesc.asTemporary(false);
try
{
-for (Component component : components)
+// do -Data last because -Data present should mean the sstable was
completely renamed before crash
+for (Component component : Sets.difference(components,
Collections.singleton(Component.DATA)))
FBUtilities.renameWithConfirm(tmpdesc.filenameFor(component),
svn commit: r1074019 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/config/CFMetaData.java
Author: jbellis Date: Thu Feb 24 02:18:34 2011 New Revision: 1074019 URL: http://svn.apache.org/viewvc?rev=1074019view=rev Log: revert unreviewed changes to SSTW/Descriptor Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1074019r1=1074018r2=1074019view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Feb 24 02:18:34 2011 @@ -26,7 +26,6 @@ * fix for reversed slice queries on large rows (CASSANDRA-2212) * fat clients were writing local data (CASSANDRA-2223) * turn off string interning in json2sstable (CASSANDRA-2189) - * set DEFAULT_MEMTABLE_LIFETIME_IN_MINS to 24h 0.7.2 Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java?rev=1074019r1=1074018r2=1074019view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/CFMetaData.java Thu Feb 24 02:18:34 2011 @@ -56,7 +56,7 @@ public final class CFMetaData public final static int DEFAULT_GC_GRACE_SECONDS = 864000; public final static int DEFAULT_MIN_COMPACTION_THRESHOLD = 4; public final static int DEFAULT_MAX_COMPACTION_THRESHOLD = 32; -public final static int DEFAULT_MEMTABLE_LIFETIME_IN_MINS = 60 * 24; +public final static int DEFAULT_MEMTABLE_LIFETIME_IN_MINS = 60; public final static int DEFAULT_MEMTABLE_THROUGHPUT_IN_MB = sizeMemtableThroughput(); public final static double DEFAULT_MEMTABLE_OPERATIONS_IN_MILLIONS = sizeMemtableOperations(DEFAULT_MEMTABLE_THROUGHPUT_IN_MB);
svn commit: r1074021 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable: Descriptor.java SSTableWriter.java
Author: jbellis
Date: Thu Feb 24 02:20:05 2011
New Revision: 1074021
URL: http://svn.apache.org/viewvc?rev=1074021view=rev
Log:
revert unreviewed changes to SSTW/Descriptor, take 2
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java?rev=1074021r1=1074020r2=1074021view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
Thu Feb 24 02:20:05 2011
@@ -199,11 +199,7 @@ public class Descriptor
if (!(o instanceof Descriptor))
return false;
Descriptor that = (Descriptor)o;
-return that.directory.equals(this.directory)
-that.generation == this.generation
-that.ksname.equals(this.ksname)
-that.cfname.equals(this.cfname)
-that.temporary == this.temporary;
+return that.directory.equals(this.directory) that.generation ==
this.generation that.ksname.equals(this.ksname)
that.cfname.equals(this.cfname);
}
@Override
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1074021r1=1074020r2=1074021view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Thu Feb 24 02:20:05 2011
@@ -22,12 +22,9 @@ package org.apache.cassandra.io.sstable;
import java.io.*;
import java.nio.ByteBuffer;
import java.util.Arrays;
-import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
-import com.google.common.collect.Sets;
-
import org.apache.cassandra.utils.ByteBufferUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -207,10 +204,8 @@ public class SSTableWriter extends SSTab
Descriptor newdesc = tmpdesc.asTemporary(false);
try
{
-// do -Data last because -Data present should mean the sstable was
completely renamed before crash
-for (Component component : Sets.difference(components,
Collections.singleton(Component.DATA)))
+for (Component component : components)
FBUtilities.renameWithConfirm(tmpdesc.filenameFor(component),
newdesc.filenameFor(component));
-FBUtilities.renameWithConfirm(tmpdesc.filenameFor(Component.DATA),
newdesc.filenameFor(Component.DATA));
}
catch (IOException e)
{
[jira] Commented: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998672#comment-12998672
]
Muga Nishizawa commented on CASSANDRA-2225:
---
I'm so sorry, this was not a bug and my operation mistake. I was able to
acquire data on sstable generated by json2sstable. I had forgotten compaction
of multiple sstables before creating JSON data with sstable2json.
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Assignee: Pavel Yaskevich
Fix For: 0.7.3
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at
[jira] Updated: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-2176: -- Attachment: 0001-CASSANDRA-2176-limit-connected-clients-and-config.txt Add configuration setting to cap the number of Thrift connections - Key: CASSANDRA-2176 URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.3 Attachments: 0001-CASSANDRA-2176-limit-connected-clients-and-config.txt, 0001-CASSANDRA-2176-limit-number-of-connected-clients.txt Original Estimate: 4h Remaining Estimate: 4h At least until CASSANDRA-1405 is done, it's useful to have a connection cap to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-2176: -- Attachment: (was: 0001-CASSANDRA-2176-limit-number-of-connected-clients.txt) Add configuration setting to cap the number of Thrift connections - Key: CASSANDRA-2176 URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.3 Attachments: 0001-CASSANDRA-2176-limit-connected-clients-and-config.txt Original Estimate: 4h Remaining Estimate: 4h At least until CASSANDRA-1405 is done, it's useful to have a connection cap to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998679#comment-12998679 ] T Jake Luciani commented on CASSANDRA-2176: --- k, changes attached Add configuration setting to cap the number of Thrift connections - Key: CASSANDRA-2176 URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.3 Attachments: 0001-CASSANDRA-2176-limit-connected-clients-and-config.txt Original Estimate: 4h Remaining Estimate: 4h At least until CASSANDRA-1405 is done, it's useful to have a connection cap to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (CASSANDRA-2225) Cannot get columns from sstable generated by json2sstable
[
https://issues.apache.org/jira/browse/CASSANDRA-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2225.
---
Resolution: Invalid
Fix Version/s: (was: 0.7.3)
Assignee: (was: Pavel Yaskevich)
Cannot get columns from sstable generated by json2sstable
-
Key: CASSANDRA-2225
URL: https://issues.apache.org/jira/browse/CASSANDRA-2225
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: Fedora 11, Intel Core i5, JDK 1.6.0_20
Reporter: Muga Nishizawa
Attachments: cassandra_sample_insert.py, create_table.cli
I cannot get columns on Cassandra that has sstable generated by json2sstable.
It returns null as its result. Columns that are associated to specified
row keys are stored on Cassandra in advance. Cassandra outputs following
exception to system.log.
This Cassandra has sstable that was generated by json2sstable. I stored data
in Cassandra, shut it down, then create JSON data from its sstable with
sstable2json and I generate sstable from JSON data with json2sstable in
advance. I could check that columns are included in JSON data file. But
columns could not be acquired from the generated sstable. This problem
occurs with or without using Pavel's patch on CASSANDRA-2188.
I attached programs so that you can know detail of data stored in Cassandra.
You will be able to reproduce this problem by executing attached programs,
sstable2json and json2sstable. For example, I could not get columns
associated to row key 030yy from sstable generated by json2sstable.
null will be returned as result. Cassandra will output exception to
system.log.
- 1. Start Cassandra daemon on localhost (number of thrift port is 9160)
- 2. Create keyspace and column family, according to create_table.cli
- 3. Execute cassandra_sample_insert.py, storing pairs of row keys and
super columns
cassandra_sample_insert.py requires pycassa
- 4. Shutdown Cassandra daemon
- 5. Execute sstable2json and create JSON data
- 6. Execute json2sstable and generate sstable from JSON data
- 7. Start Cassandra daemon again
- 8. Get columns related to row key 030yy (but, I could not get)
{quote}
ERROR 15:45:18,228 Fatal exception in thread Thread[ReadStage:2,5,main]
java.io.IOError:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0
at
org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at
org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:1)
at
java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1493)
at
java.util.concurrent.ConcurrentSkipListMap.init(ConcurrentSkipListMap.java:1443)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:1)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:1)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
at
org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(CollatingIterator.java:364)
at
org.apache.commons.collections.iterators.CollatingIterator.hasNext(CollatingIterator.java:217)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:55)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:118)
at
org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(QueryFilter.java:142)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1290)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1167)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1095)
at org.apache.cassandra.db.Table.getRow(Table.java:384)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:63)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:473)
[jira] Created: (CASSANDRA-2233) Add unified UUIDType
Add unified UUIDType Key: CASSANDRA-2233 URL: https://issues.apache.org/jira/browse/CASSANDRA-2233 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.3 Reporter: Ed Anuff Priority: Minor Unified UUIDType comparator, compares as time-based if both UUIDs are time-based, otherwise uses byte comparison. Based on code from the current LexicalUUIDType and TimeUUIDType comparers, so performance and behavior should be consistent and compatible. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2233) Add unified UUIDType
[ https://issues.apache.org/jira/browse/CASSANDRA-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Anuff updated CASSANDRA-2233: Attachment: UUIDType.java Add unified UUIDType Key: CASSANDRA-2233 URL: https://issues.apache.org/jira/browse/CASSANDRA-2233 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.3 Reporter: Ed Anuff Priority: Minor Attachments: UUIDType.java Unified UUIDType comparator, compares as time-based if both UUIDs are time-based, otherwise uses byte comparison. Based on code from the current LexicalUUIDType and TimeUUIDType comparers, so performance and behavior should be consistent and compatible. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1684) Entity groups
[ https://issues.apache.org/jira/browse/CASSANDRA-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998732#comment-12998732 ] Ed Anuff commented on CASSANDRA-1684: - This is something I've been thinking about while consolidating the number of column families within an application so that I ended up with row keys that were constructed from concatenating an entity id with various other strings (eg. 9081bd70-3fe4-11e0-9207-0800200c9a66:something ). Is it feasible to have a partitioner that hashed on just the first x bytes in a key? Do tokens have to be one-to-one unique with keys, or could you have multiple keys share the same token? (apparently that's currently possible, although an extreme edge case, with the RandomPartitioner) Entity groups - Key: CASSANDRA-1684 URL: https://issues.apache.org/jira/browse/CASSANDRA-1684 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Fix For: 0.8 Original Estimate: 80h Remaining Estimate: 80h Supporting entity groups similar to App Engine's (that is, allow rows to be part of a parent entity group, whose key is used for routing instead of the row itself) allows several improvements: - batches within an EG can be atomic across multiple rows - order-by-value queries within an EG only have to touch a single replica even with RandomPartitioner -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
