[jira] [Commented] (CASSANDRA-4750) Add jmx/nodetool methods to enable/disable hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481197#comment-13481197 ] Alexey Zotov commented on CASSANDRA-4750: - I think that would be good to save current methods and add methods for pause/resume hints delivery processes. So we will have separate methods to disable/enable the future hints storing and for pause/resume hints delivery processes. It will be implemented in HHOM as boolean flag that could be changed through JMX and nodetool. Also that flag should be save own state in system.local CF (patch will be creaed only for 1.2 version). What do you think about it? Add jmx/nodetool methods to enable/disable hinted handoff - Key: CASSANDRA-4750 URL: https://issues.apache.org/jira/browse/CASSANDRA-4750 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams Assignee: Alexey Zotov Labels: hintedhandoff, nodetool Fix For: 1.1.7, 1.2.0 beta 2 Attachments: cassandra-1.1-4750-enable_disable_hh.txt, cassandra-1.2-4750-enable_disable_hh.txt Title says it all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4832) AssertionError: keys must not be empty
[ https://issues.apache.org/jira/browse/CASSANDRA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481209#comment-13481209 ] Joe Fox commented on CASSANDRA-4832: I've run into the same symptoms as Chris, but while attempting to migrate data between a 1.1.1 cluster and 1.1.6. Whilst using sstableloader to import the data, nodes would hit certain column indexes and stop processing further requests once the assertion was thrown - restarting the node would almost immediately throw the assertion again, and the node would just fail to rejoin the ring. tpstats would show active flushwriter tasks but no further node activity. We had to work around the issue by: 1. Removing all indexes from our target cluster schema 2. Importing the data via sstableloader 3. Scanning through relevant column families and inserting data into any empty indexed columns 4. Re-applying the indexes to the target cluster schema Only then was the migration successful. I'll also note that attempting to apply an index to a column which has null data will also throw the cluster out of sync as the nodes which throw the assertion fail to migrate their schemas properly INFO [Creating index: Transactions.TransactionsCountryCode] 2012-10-21 07:45:25,978 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsCountryCode@1802367190(38862/169512 serialized/live bytes, 762 ops) INFO [Creating index: Transactions.TransactionsStatus] 2012-10-21 07:45:25,980 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsStatus@673679943(38862/125966 serialized/live bytes, 762 ops) INFO [FlushWriter:1] 2012-10-21 07:45:25,987 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsCountryCode@1802367190(38862/169512 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,004 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsStatus@673679943(38862/125966 serialized/live bytes, 762 ops) ERROR [FlushWriter:1] 2012-10-21 07:45:26,004 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[FlushWriter:1,5,main] java.lang.AssertionError: Keys must not be empty at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:133) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:176) at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:295) at org.apache.cassandra.db.Memtable.access$600(Memtable.java:48) at org.apache.cassandra.db.Memtable$5.runMayThrow(Memtable.java:316) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) INFO [FlushWriter:2] 2012-10-21 07:45:26,049 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/***/Transactions/***-Transactions.TransactionsStatus-hf-2-Data.db (35259 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) INFO [Creating index: Transactions.TransactionsLastUpdateDate] 2012-10-21 07:45:26,313 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsLastUpdateDate@1912098049(38862/240277 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,314 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsLastUpdateDate@1912098049(38862/240277 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,743 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/***/Transactions/***-Transactions.TransactionsLastUpdateDate-hf-2-Data.db (37024 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) INFO [main] 2012-10-21 07:45:27,052 CommitLogReplayer.java (line 272) Finished reading /var/lib/cassandra/commitlog/CommitLog-1350768744805.log INFO [main] 2012-10-21 07:45:27,054 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Versions@1851630436(83/103 serialized/live bytes, 3 ops) INFO [FlushWriter:2] 2012-10-21 07:45:27,054 Memtable.java (line 264) Writing Memtable-Versions@1851630436(83/103 serialized/live bytes, 3 ops) INFO [FlushWriter:2] 2012-10-21 07:45:27,061 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/system/Versions/system-Versions-hf-1-Data.db (247 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) AssertionError: keys must not be empty -- Key: CASSANDRA-4832 URL: https://issues.apache.org/jira/browse/CASSANDRA-4832 Project: Cassandra Issue Type: Bug
git commit: Fix Subcolumn slice ends not respected
Updated Branches:
refs/heads/trunk 81209f1c8 - f02928f88
Fix Subcolumn slice ends not respected
patch by vijay; reviewed by thobbs slebresne for CASSANDRA-4826
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f02928f8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f02928f8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f02928f8
Branch: refs/heads/trunk
Commit: f02928f887ce58e317b87341a837c61068510bae
Parents: 81209f1
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Mon Oct 22 09:19:06 2012 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Mon Oct 22 09:19:06 2012 +0200
--
CHANGES.txt|1 +
.../cassandra/db/filter/SliceQueryFilter.java | 30 +-
2 files changed, 20 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f02928f8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9fbccb0..e14ffa7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -31,6 +31,7 @@
* Fix potential NPE during CFS reload (CASSANDRA-4786)
* Composite indexes may miss results (CASSANDRA-4796)
* Move consistency level to the protocol level (CASSANDRA-4734, 4824)
+ * Fix Subcolumn slice ends not respected (CASSANDRA-4826)
Merged from 1.1:
* fix indexing empty column values (CASSANDRA-4832)
* allow JdbcDate to compose null Date objects (CASSANDRA-4830)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f02928f8/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
b/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
index 2de284a..af63fa0 100644
--- a/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
@@ -25,6 +25,7 @@ import java.util.Comparator;
import java.util.Iterator;
import java.util.List;
+import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterators;
import com.google.common.collect.Lists;
import org.slf4j.Logger;
@@ -104,7 +105,7 @@ public class SliceQueryFilter implements IFilter
// we clone shallow, then add, under the theory that generally we're
interested in a relatively small number of subcolumns.
// this may be a poor assumption.
SuperColumn scFiltered = superColumn.cloneMeShallow();
-IteratorIColumn subcolumns;
+final IteratorIColumn subcolumns;
if (reversed)
{
ListIColumn columnsAsList = new
ArrayListIColumn(superColumn.getSubColumns());
@@ -114,20 +115,27 @@ public class SliceQueryFilter implements IFilter
{
subcolumns = superColumn.getSubColumns().iterator();
}
-
-// iterate until we get to the real start column
-ComparatorByteBuffer comparator = reversed ?
superColumn.getComparator().reverseComparator : superColumn.getComparator();
-while (subcolumns.hasNext())
+final ComparatorByteBuffer comparator = reversed ?
superColumn.getComparator().reverseComparator : superColumn.getComparator();
+IteratorIColumn results = new AbstractIteratorIColumn()
{
-IColumn column = subcolumns.next();
-if (comparator.compare(column.name(), start()) = 0)
+protected IColumn computeNext()
{
-subcolumns =
Iterators.concat(Iterators.singletonIterator(column), subcolumns);
-break;
+while (subcolumns.hasNext())
+{
+IColumn subcolumn = subcolumns.next();
+// iterate until we get to the real start column
+if (comparator.compare(subcolumn.name(), start()) 0)
+continue;
+// exit loop when columns are out of the range.
+if (finish().remaining() 0
comparator.compare(subcolumn.name(), finish()) 0)
+break;
+return subcolumn;
+}
+return endOfData();
}
-}
+};
// subcolumns is either empty now, or has been redefined in the loop
above. either is ok.
-collectReducedColumns(scFiltered, subcolumns, gcBefore);
+collectReducedColumns(scFiltered, results, gcBefore);
return scFiltered;
}
[jira] [Updated] (CASSANDRA-4843) When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start
[ https://issues.apache.org/jira/browse/CASSANDRA-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4843: Fix Version/s: 1.2.0 beta 2 When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start - Key: CASSANDRA-4843 URL: https://issues.apache.org/jira/browse/CASSANDRA-4843 Project: Cassandra Issue Type: Bug Reporter: Edward Capriolo Fix For: 1.2.0 beta 2 ERROR 10:17:20,341 Cannot open /home/edward/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hf-1 because partitioner does not match org.apache.cassandra.dht.RandomPartitioner != org.apache.cassandra.dht.Murmur3Partitioner This is because 1.2 has a new default partitioner, why are we changing the default? Is this wise? The current partitioner has been rock solid for years. Should the previously known partition be stored in the schema like the previously know seed nodes, and schema? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4843) When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start
[ https://issues.apache.org/jira/browse/CASSANDRA-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481241#comment-13481241 ] Sylvain Lebresne commented on CASSANDRA-4843: - bq. why are we changing the default? The reason is that md5 is a bit cpu intensive and in some 2ndary index requests (that does a token computation for each key on disk it scans for internal reason that can't be changed easily) this was a bottleneck. The new default is the Murmur3Partitioner that is much cheaper to compute. Besides, for every query we do compute a bunch of token and vnodes will probably not reduce that, so it's a generic improvement. That being said, I fully agree that the current upgrade experience is pretty harsh (even the NEWS file don't clearly explain the action to take to avoid this error). And since we save the partitonner and don't start if the user change it in the yaml, maybe it's time to change the behavior so that if a partitioner is saved in the system table, we use that (and log a warning if it differs from the yaml configured one). When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start - Key: CASSANDRA-4843 URL: https://issues.apache.org/jira/browse/CASSANDRA-4843 Project: Cassandra Issue Type: Bug Reporter: Edward Capriolo Fix For: 1.2.0 beta 2 ERROR 10:17:20,341 Cannot open /home/edward/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hf-1 because partitioner does not match org.apache.cassandra.dht.RandomPartitioner != org.apache.cassandra.dht.Murmur3Partitioner This is because 1.2 has a new default partitioner, why are we changing the default? Is this wise? The current partitioner has been rock solid for years. Should the previously known partition be stored in the schema like the previously know seed nodes, and schema? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4844) CQL3 Create keyspace statement not compatible with older versions.
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne resolved CASSANDRA-4844.
-
Resolution: Won't Fix
Yes, the syntax has changed in a breaking way. In fact, that's not even the
biggest breaking change in that we've also removed the consistency level from
the language (to move it to the protocol level). This is unfortunate *but*
we've bee clear that CQL3 was beta in 1.1 (I admit the datastax doc isn't that
clear and I'll ask them to do something about it, but the official CQL3 doc for
1.1 is: http://cassandra.apache.org/doc/cql3/CQL.html). I note that this is
indicated in the NEWS file.
CQL3 Create keyspace statement not compatible with older versions.
--
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481264#comment-13481264
]
Sylvain Lebresne commented on CASSANDRA-4815:
-
bq. Can I create a schema less table?
Yes. The following as-schemaless-as-can-possibly-be thrift/cli definition:
{noformat}
create column family schemaless
with key_validation_class = BytesType
and comparator = BytesType
and default_validation_class = BytesType
{noformat}
is *equivalent* to the following CQL3 definition
{noformat}
CREATE TABLE schemaless (
key blob,
column blob,
value blob,
PRIMARY KEY (key, column)
) WITH COMPACT STORAGE
{noformat}
And to be clear, when I say equivalent, I mean equivalent. If you create the
first definion above, you can use the column family in CQL3 as if it was
defined by the second definition (as in, you don't have to do the CREATE TABLE
itself), or you can create the table in CQL3 first with the second query and
query it in thrift exactly as if it had been created by the first definition.
The composite primary key is what tells CQL3 that it's a transposed wide CF.
In other words, in CQL3, 'key' will map to the row key, 'column' will map to
the internal column name and 'value' will map to the internal column value. I
note that 'key', 'column' and 'value' are the default names that CQL3 picks for
you when you haven't explicitely defined user friendlier one (in other words,
when you upgrade from thrift). CASSANDRA-4822 is open to allow you to rename
those default names to more user friendly ones if you so wish (and to be clear,
doing so as no impact whatsoever on what is stored, it just declare the new
names as CQL3 metadata).
bq. I guess this is slightly more difficult to express composite slices.
It's possibly nitpicking, but I would talk of a difficulty in poperly
paginating composites. But yes, that's one of the very few things that CQL3 is
not currently very good at. But we'll fix it (and the good thing about having a
query language is that it will be trivial to fix it without a backward
incompatible breaking change). That being said, I do believe that once you
start doing real life example, it's not really a blocker. Most of the time,
when you use composites in real life, you want to slice over one of the
component, which works fine. That's why it's really more a problem for slightly
more complex pagination over composite wide rows. There is also CASSANDRA-4415
that will fix the need for a good part of the manual pagination people do right
now.
bq. If we have an old style schema don't we need to be able to alter a current
table.
As explained above, thrift CF *are* directly accessible from CQL3 (without
any redefinition, and that's why trying to create the table in CQL3 is not
legal). However, you won't nice column names if you do so (but rather the
'key', 'column' and 'value' generic names above). Again, CASSANDRA-4822 will
allow to declare nice names without having to do complex operation (like
trashing your thrift schema so that CQL3 allow the redefinition).
bq. What is going to happen if Cassandra and the CQL language actually adds
true composite row keys?
It does already: CASSANDRA-4179. You just declare
{noformat}
PRIMARY KEY ((id_part1, id_part2), tag_name).
{noformat}
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra
[jira] [Comment Edited] (CASSANDRA-4832) AssertionError: keys must not be empty
[ https://issues.apache.org/jira/browse/CASSANDRA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481209#comment-13481209 ] Joe Fox edited comment on CASSANDRA-4832 at 10/22/12 8:52 AM: -- I've run into the same symptoms as Chris, but while attempting to migrate data between a 1.1.1 cluster and 1.1.6. Whilst using sstableloader to import the data, nodes would hit certain column indexes and stop processing further requests once the assertion was thrown - restarting the node would almost immediately throw the assertion again, and the node would just fail to rejoin the ring. tpstats would show active flushwriter tasks but no further node activity. We had to work around the issue by: 1. Removing all indexes from our target cluster schema 2. Importing the data via sstableloader 3. Scanning through relevant column families and inserting data into any empty indexed columns 4. Re-applying the indexes to the target cluster schema Only then was the migration successful. I'll also note that attempting to apply an index to a column which has null data will also throw the cluster out of sync as the nodes which throw the assertion fail to migrate their schemas properly INFO [Creating index: Transactions.TransactionsCountryCode] 2012-10-21 07:45:25,978 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsCountryCode@1802367190(38862/169512 serialized/live bytes, 762 ops) INFO [Creating index: Transactions.TransactionsStatus] 2012-10-21 07:45:25,980 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsStatus@673679943(38862/125966 serialized/live bytes, 762 ops) INFO [FlushWriter:1] 2012-10-21 07:45:25,987 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsCountryCode@1802367190(38862/169512 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,004 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsStatus@673679943(38862/125966 serialized/live bytes, 762 ops) ERROR [FlushWriter:1] 2012-10-21 07:45:26,004 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[FlushWriter:1,5,main] java.lang.AssertionError: Keys must not be empty at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:133) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:176) at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:295) at org.apache.cassandra.db.Memtable.access$600(Memtable.java:48) at org.apache.cassandra.db.Memtable$5.runMayThrow(Memtable.java:316) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) INFO [FlushWriter:2] 2012-10-21 07:45:26,049 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/xx/Transactions/xx-Transactions.TransactionsStatus-hf-2-Data.db (35259 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) INFO [Creating index: Transactions.TransactionsLastUpdateDate] 2012-10-21 07:45:26,313 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Transactions.TransactionsLastUpdateDate@1912098049(38862/240277 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,314 Memtable.java (line 264) Writing Memtable-Transactions.TransactionsLastUpdateDate@1912098049(38862/240277 serialized/live bytes, 762 ops) INFO [FlushWriter:2] 2012-10-21 07:45:26,743 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/xx/Transactions/xx-Transactions.TransactionsLastUpdateDate-hf-2-Data.db (37024 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) INFO [main] 2012-10-21 07:45:27,052 CommitLogReplayer.java (line 272) Finished reading /var/lib/cassandra/commitlog/CommitLog-1350768744805.log INFO [main] 2012-10-21 07:45:27,054 ColumnFamilyStore.java (line 659) Enqueuing flush of Memtable-Versions@1851630436(83/103 serialized/live bytes, 3 ops) INFO [FlushWriter:2] 2012-10-21 07:45:27,054 Memtable.java (line 264) Writing Memtable-Versions@1851630436(83/103 serialized/live bytes, 3 ops) INFO [FlushWriter:2] 2012-10-21 07:45:27,061 Memtable.java (line 305) Completed flushing /var/lib/cassandra/data/system/Versions/system-Versions-hf-1-Data.db (247 bytes) for commitlog position ReplayPosition(segmentId=1350805525722, position=0) was (Author: e1zorro): I've run into the same symptoms as Chris, but while attempting to migrate data between a 1.1.1 cluster and 1.1.6. Whilst using sstableloader to import the data, nodes would hit
[jira] [Reopened] (CASSANDRA-4844) CQL3 Create keyspace statement not compatible with older versions.
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo reopened CASSANDRA-4844:
CQL3 Create keyspace statement not compatible with older versions.
--
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4844) CQL3 Create keyspace statement not compatible with older versions.
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481381#comment-13481381
]
Edward Capriolo commented on CASSANDRA-4844:
I realize you were clear that 1.1 was beta, but there are DZONE refcards and
blogs all over the internet instructing people on how to create keyspaces.
Cassandra typically approaches these things we changed it, deal with it. Do
not be soo quick to close the issues because the inline documentation in cqlsh
is still wrong.
CQL3 Create keyspace statement not compatible with older versions.
--
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4844) CQL3 Create keyspace statement not compatible with older versions.
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481389#comment-13481389
]
Edward Capriolo commented on CASSANDRA-4844:
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol 19.34.0]
Use HELP for help.
cqlsh help ;
Documented commands (type help topic):
ASSUME CAPTURE COPY DESC DESCRIBE EXIT HELP SELECT SHOW SOURCE USE
Miscellaneous help topics:
==
DROP_INDEX BOOLEAN_INPUTSELECT_LIMIT
ALTER_DROP CREATE DELETE_WHERE
SELECT_EXPRTIMESTAMP_OUTPUT UPDATE_USING
UUID_INPUT CREATE_TABLE_OPTIONS UPDATE_WHERE
DELETE_COLUMNS ALTER_ALTER SELECT_WHERE
DROP_TABLE CREATE_TABLE CONSISTENCYLEVEL
CREATE_TABLE_TYPES SELECT_COLUMNFAMILY CREATE_INDEX
ALTER_WITH SELECT_TABLE CREATE_KEYSPACE
TYPES CREATE_COLUMNFAMILY_OPTIONS ASCII_OUTPUT
APPLY BEGINDROP
DELETE_USING UPDATE_SET TIMESTAMP_INPUT
CREATE_COLUMNFAMILY_TYPES UPDATE_COUNTERS ALTER
DROP_COLUMNFAMILY TRUNCATE CREATE_COLUMNFAMILY
BLOB_INPUT INSERT DELETE
ALTER_ADD TEXT_OUTPUT
DROP_KEYSPACE UPDATE
Undocumented commands:
==
DEBUG IMPORT_INSERT
cqlsh help CREATE_KEYSPACE;
CREATE KEYSPACE ksname WITH strategy_class = 'strategy'
[AND strategy_options:option = val];
The CREATE KEYSPACE statement creates a new top-level namespace (aka
keyspace). Valid names are any string constructed of alphanumeric
characters and underscores. Names which do not work as valid
identifiers or integers should be quoted as string literals. Properties
such as replication strategy and count are specified during creation
using the following accepted keyword arguments:
strategy_class [required]: The name of the replication strategy class
which should be used for the new keyspace. Some often-used classes
are SimpleStrategy and NetworkTopologyStrategy.
strategy_options: Most strategies require additional arguments which
can be supplied by appending the option name to strategy_options,
separated by a colon (:). For example, a strategy option of DC1
with a value of 1 would be specified as strategy_options:DC1 = 1.
The replication factor option for SimpleStrategy could be
strategy_options:replication_factor=3.
cqlsh
{noformat}
CQL3 Create keyspace statement not compatible with older versions.
--
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4049) Add generic way of adding SSTable components required custom compaction strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated CASSANDRA-4049: -- Attachment: 4049-v4.txt [~jbellis] Looks and works good. I added a guard against adding the same component more than once (just a cosmetic thing to avoid duplicate TOC entries, because components is a set anyway). Attached as 4049-v4.txt. You can go on and merge it into 1.1.7. Add generic way of adding SSTable components required custom compaction strategy Key: CASSANDRA-4049 URL: https://issues.apache.org/jira/browse/CASSANDRA-4049 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Piotr Kołaczkowski Assignee: Piotr Kołaczkowski Priority: Minor Labels: compaction Fix For: 1.1.7 Attachments: 4049-v3.txt, 4049-v4.txt, pluggable_custom_components-1.1.5-2.patch, pluggable_custom_components-1.1.5.patch CFS compaction strategy coming up in the next DSE release needs to store some important information in Tombstones.db and RemovedKeys.db files, one per sstable. However, currently Cassandra issues warnings when these files are found in the data directory. Additionally, when switched to SizeTieredCompactionStrategy, the files are left in the data directory after compaction. The attached patch adds new components to the Component class so Cassandra knows about those files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4838) ColumnFamilyRecordWriter does not allow adjustment of Thrift/socket timeout
[ https://issues.apache.org/jira/browse/CASSANDRA-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4838. --- Resolution: Not A Problem I think you're confusing the socket timeout (from client to coordinator) with the RPC timeout (from coordinator to replicas). Increasing the former will not help when you are hitting the latter. Instead, you should lower mapreduce.output.columnfamilyoutputformat.batch.threshold. ColumnFamilyRecordWriter does not allow adjustment of Thrift/socket timeout --- Key: CASSANDRA-4838 URL: https://issues.apache.org/jira/browse/CASSANDRA-4838 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.0.10 Environment: Linux/Ubuntu, DSE 2.1 / Cassandra 1.0.10 Reporter: Evan Chan Attachments: cassandra-thrift-timeout.patch I'm using ColumnFamilyRecordWriter with an M/R job to dump data into a cluster, it was running Cassandra 1.0.10, and is now running DSE 2.1. Either way, I hit Thrift timeout errors sometimes. Looking at the code, it seems that it does not allow for setting of the Thrift timeout, and this code has not been updated in the latest trunk. I have a patch that adds a configuration parameter to allow adjustment of the Thrift timeout in CFRecordWriter. Let me know if you guys would be interested. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4788) streaming can put files in the wrong location
[ https://issues.apache.org/jira/browse/CASSANDRA-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481447#comment-13481447 ] Yuki Morishita commented on CASSANDRA-4788: --- Actually, it does regardless of streaming type. I observed it with sstableloader. SSTables are put in right place when they get compacted, so maybe that's why you only see when bootstrapping. streaming can put files in the wrong location - Key: CASSANDRA-4788 URL: https://issues.apache.org/jira/browse/CASSANDRA-4788 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 beta 1 Reporter: Brandon Williams Assignee: Yuki Morishita Fix For: 1.2.0 beta 2 Attachments: 4788.txt Some, but not all streaming incorrectly puts files in the top level data directory. Easiest way to repro that I've seen is bootstrap where it happens 100% of the time, but other operations like move and repair seem to do the right thing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4835) Appending/Prepending items to list using BATCH
[
https://issues.apache.org/jira/browse/CASSANDRA-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481449#comment-13481449
]
Jonathan Ellis commented on CASSANDRA-4835:
---
You should consider this an implementation quirk (possibly even a bug), not a
guarantee.
Appending/Prepending items to list using BATCH
--
Key: CASSANDRA-4835
URL: https://issues.apache.org/jira/browse/CASSANDRA-4835
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Krzysztof Cieslinski
Priority: Minor
As I know, there is no any guarantee that commands that are inside BATCH
block will execute in same order, as they are stored in the BATCH block.
But...
I have made two tests:
First appends some items to the empty list, and the second one, prepends
items, also to the empty list. Both of them are using UPDATE commands stored
in the BATCH block.
Results of those tests are as follow:
First:
When appending new items to list, USING commands are executed in the
same order as they are stored i BATCH.
Second:
When prepending new items to list, USING commands are executed in
random order.
So, in other words below code:
{code:xml}
BEGIN BATCH
UPDATE... list_name = list_name + [ '1' ]
UPDATE... list_name = list_name + [ '2' ]
UPDATE... list_name = list_name + [ '3' ]
APPLY BATCH;{code}
always results in [ '1', '2', '3' ],
but this code:
{code:xml}
BEGIN BATCH
UPDATE... list_name = [ '1' ] + list_name
UPDATE... list_name = [ '2' ] + list_name
UPDATE... list_name = [ '3' ] + list_name
APPLY BATCH;{code}
results in randomly ordered list, like [ '2', '1', '3' ](expected result
is [ '3', '2', '1' ])
So somehow, when appending items to list, commands from BATCH are executed in
order as they are stored, but when prepending, the order is random.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4840) remnants of removed nodes remain after removal
[ https://issues.apache.org/jira/browse/CASSANDRA-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481451#comment-13481451 ] Jonathan Ellis commented on CASSANDRA-4840: --- Does this happen in 1.1 as well? remnants of removed nodes remain after removal -- Key: CASSANDRA-4840 URL: https://issues.apache.org/jira/browse/CASSANDRA-4840 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.10 Reporter: Jeremy Hanna Assignee: Brandon Williams Priority: Minor After nodes are removed from the ring and no longer appear in any of the nodes' nodetool ring output, some of the dead nodes show up in the o.a.c.net.FailureDetector SimpleStates metadata. Also, some of the JMX stats are updating for the removed nodes (ie RecentTimeoutsPerHost and ResponsePendingTasks). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3017) add a Message size limit
[ https://issues.apache.org/jira/browse/CASSANDRA-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481453#comment-13481453 ] Jonathan Ellis commented on CASSANDRA-3017: --- MessageIn.read is the right place to be defensive now. add a Message size limit Key: CASSANDRA-3017 URL: https://issues.apache.org/jira/browse/CASSANDRA-3017 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Kirk True Priority: Minor Labels: lhf Attachments: 0001-use-the-thrift-max-message-size-for-inter-node-messa.patch We protect the server from allocating huge buffers for malformed message with the Thrift frame size (CASSANDRA-475). But we don't have similar protection for the inter-node Message objects. Adding this would be good to deal with malicious adversaries as well as a malfunctioning cluster participant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4839) Online toggle for node write-only status
[ https://issues.apache.org/jira/browse/CASSANDRA-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481457#comment-13481457 ] Jonathan Ellis commented on CASSANDRA-4839: --- My question here is, how does a node know when it's done receiving hints? Online toggle for node write-only status Key: CASSANDRA-4839 URL: https://issues.apache.org/jira/browse/CASSANDRA-4839 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Rick Branson Priority: Minor It would be really great if users could disable/enable reads on a given node, while still allowing write operations to take place. This would be similar to how we enable/disable thrift and gossip using JMX. The scenario for using this is that often a node needs to be brought down for maintenance for a few minutes, and while the node is catching up from hints, which can take 10-30 minutes depending on write load, it will serve stale data. Do the math for a rolling restart of a large cluster and you have potential windows of hours or days where a large amount of inconsistency is surfacing. Avoiding this large time gap of inconsistency during regular maintenance alleviates concerns about inconsistent data surfaced to users during normal, planned activities. While a read consistency ONE can indeed be used to prevent any inconsistency from the scenario above, it seems ridiculous to always incur the cost to cover the 0.1% case. In addition, it would open up the ability for a node to (optionally) automatically go dark for reads while it's receiving hints after joining the cluster or perhaps during repair. These obviously have their own complications and justify separate tickets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4839) Online toggle for node write-only status
[ https://issues.apache.org/jira/browse/CASSANDRA-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481456#comment-13481456 ] Jonathan Ellis commented on CASSANDRA-4839: --- Thrift is only used for client connections. Online toggle for node write-only status Key: CASSANDRA-4839 URL: https://issues.apache.org/jira/browse/CASSANDRA-4839 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Rick Branson Priority: Minor It would be really great if users could disable/enable reads on a given node, while still allowing write operations to take place. This would be similar to how we enable/disable thrift and gossip using JMX. The scenario for using this is that often a node needs to be brought down for maintenance for a few minutes, and while the node is catching up from hints, which can take 10-30 minutes depending on write load, it will serve stale data. Do the math for a rolling restart of a large cluster and you have potential windows of hours or days where a large amount of inconsistency is surfacing. Avoiding this large time gap of inconsistency during regular maintenance alleviates concerns about inconsistent data surfaced to users during normal, planned activities. While a read consistency ONE can indeed be used to prevent any inconsistency from the scenario above, it seems ridiculous to always incur the cost to cover the 0.1% case. In addition, it would open up the ability for a node to (optionally) automatically go dark for reads while it's receiving hints after joining the cluster or perhaps during repair. These obviously have their own complications and justify separate tickets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4833) get_count with 'count' param between 1024 and ~actual column count fails
[ https://issues.apache.org/jira/browse/CASSANDRA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4833: -- Attachment: 4833-1.1-v2.txt Thanks for review, Tyler. What we want here is to fetch last page with remainder, not whole page size. So we still need requestedCount -= newColumns. Attaching v2 for this. get_count with 'count' param between 1024 and ~actual column count fails Key: CASSANDRA-4833 URL: https://issues.apache.org/jira/browse/CASSANDRA-4833 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6, 1.2.0 beta 1 Reporter: Tyler Hobbs Assignee: Yuki Morishita Attachments: 4833-1.1.txt, 4833-1.1-v2.txt, 4833-get-count-repro.py If you run get_count() with the 'count' param of the SliceRange set to a number between 1024 and (approximately) the actual number of columns in the row, something seems to silently fail internally, resulting in a client side timeout. Using a 'count' param outside of this range (lower or much higher) works just fine. This seems to affect all of 1.1 and 1.2.0-beta1, but not 1.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4784) Create separate sstables for each token range handled by a node
[ https://issues.apache.org/jira/browse/CASSANDRA-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481465#comment-13481465 ] Jonathan Ellis commented on CASSANDRA-4784: --- But if vnodes (which is already complete and ready to use in 1.2) makes bootstrap fast enough to saturate 10 gigE, I don't see much utility in adding further complexity for 1.3. I do still see this as useful for backup/restore in the general case though. Create separate sstables for each token range handled by a node --- Key: CASSANDRA-4784 URL: https://issues.apache.org/jira/browse/CASSANDRA-4784 Project: Cassandra Issue Type: Improvement Components: Core Reporter: sankalp kohli Priority: Minor Labels: perfomance Currently, each sstable has data for all the ranges that node is handling. If we change that and rather have separate sstables for each range that node is handling, it can lead to some improvements. Improvements 1) Node rebuild will be very fast as sstables can be directly copied over to the bootstrapping node. It will minimize any application level logic. We can directly use Linux native methods to transfer sstables without using CPU and putting less pressure on the serving node. I think in theory it will be the fastest way to transfer data. 2) Backup can only transfer sstables for a node which belong to its primary keyrange. 3) ETL process can only copy one replica of data and will be much faster. Changes: We can split the writes into multiple memtables for each range it is handling. The sstables being flushed from these can have details of which range of data it is handling. There will be no change I think for any reads as they work with interleaved data anyway. But may be we can improve there as well? Complexities: The change does not look very complicated. I am not taking into account how it will work when ranges are being changed for nodes. Vnodes might make this work more complicated. We can also have a bit on each sstable which says whether it is primary data or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4844) CQL3 Create keyspace statement not compatible with older versions.
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481466#comment-13481466
]
Sylvain Lebresne commented on CASSANDRA-4844:
-
bq. Cassandra typically approaches these things we changed it, deal with it.
I won't pretend we have been perfect in the past on these things nor will I
pretend that we'll be perfect in the future. But we're trying to be better and
I dare say that we do improve. And I spend way too much of my time (and I'm not
the only one) writing code whose only purpose is to ensure backward
compatibility (and I will continue doing it because I absolutely believe it is
important, but it's rarely fun) to let you say that we don't care. Trust me, we
care.
But we've marked CQL3 beta in 1.1 exactly because we knew we wouldn't get
everything right the first time, and we wanted the freedom to fix those thing
to be confident about CQL3 final (and at least as far as I'm concerned, I do
intend to do everything possible to provide backward compatibility for CQL3
final once it's released, because that's part of the deal). So yes, backward
compatibility is a major concern of mine, but no I disagree that we should keep
backward compatibility on things we've explicitly declared as beta, even if
there is blogs about it.
bq. the inline documentation in cqlsh is still wrong
Good point, we'll need to fix it (that's not the only cql doc that is not up to
date for 1.2 though). Do feel free to reopen with an update title for this.
CQL3 Create keyspace statement not compatible with older versions.
--
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4841) Exception thrown during nodetool repair -pr
[ https://issues.apache.org/jira/browse/CASSANDRA-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481468#comment-13481468 ] Jonathan Ellis commented on CASSANDRA-4841: --- Doesn't this just mean JMX got tired and timed out before the repair completed? Nothing here indicates that repair crashed a node. Exception thrown during nodetool repair -pr --- Key: CASSANDRA-4841 URL: https://issues.apache.org/jira/browse/CASSANDRA-4841 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6 Environment: Ubuntu 12.04 x64, 32GB RAM Reporter: Michael Kjellman Priority: Critical During a nodetool repair -pr an exception was thrown. root@scl-cas04:~# nodetool repair -pr Exception in thread main java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.io.EOFException at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:227) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:160) at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1017) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:305) at $Proxy0.forceTableRepairPrimaryRange(Unknown Source) at org.apache.cassandra.tools.NodeProbe.forceTableRepairPrimaryRange(NodeProbe.java:209) at org.apache.cassandra.tools.NodeCmd.optionalKSandCFs(NodeCmd.java:1044) at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:834) Caused by: java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:213) ... 9 more Logs: WARN 20:53:27,135 Heap is 0.9710037912028483 full. You may need to reduce memtable and/or cache sizes. Cassandr a will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold i n cassandra.yaml if you don't want Cassandra to do this automatically regardless of configuration issues a repair shouldn't crash a node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4794) cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException
[ https://issues.apache.org/jira/browse/CASSANDRA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4794. --- Resolution: Duplicate cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException --- Key: CASSANDRA-4794 URL: https://issues.apache.org/jira/browse/CASSANDRA-4794 Project: Cassandra Issue Type: Bug Components: API Affects Versions: 1.2.0 beta 1 Environment: C++ Reporter: debadatta das Assignee: Aleksey Yeschenko Fix For: 1.2.0 beta 2 Attachments: InsertExample.java.txt, log_java.txt, sample_AtomicBatchMutate.cpp Hi, We have installed cassandra 1.2.0 beta with thrift 0.7.0. We are using cpp interface. When we use batch_mutate API, it works fine. But when we are using the new atomic_batch_mutate API with same parameters as batch_mutate, it fails with org::apache::cassandra::TimedOutException, what(): Default TException. We get the same TException error even after increasing Send/Reciv timeout values of Tsocket to 15 seconds or more. Details: cassandra ring: cassandra ring with single node consistency level paramter to atomic_batch_mutate ConsistencyLevel::ONE Thrift version: same results with thrift 0.5.0 and thrift 0.7.0. thrift 0.8.0 seems unsupported with cassanda 1.2.0. Gives compilation error for cpp interface build. We are calling atomic_batch_mutate() with same parameters as batch_mutate. cassclient.atomic_batch_mutate(outermap1, ConsistencyLevel::ONE); where outmap1 is mapstring, mapstring, vectorMutation outermap1; Please point out if anything is missing while using atomic_batch_mutate or the reason behind the failure. The logs in cassandra system.log we get during atomic_batch_mutate failure are: INFO [ScheduledTasks:1] 2012-10-10 04:47:30,604 MessagingService.java (line 800) 1 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-10-10 04:47:30,606 StatusLogger.java (line 53) Pool Name Active Pending Blocked INFO [ScheduledTasks:1] 2012-10-10 04:47:30,607 StatusLogger.java (line 68) ReadStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) RequestResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReadRepairStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) MutationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReplicateOnWriteStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) GossipStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) AntiEntropyStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MigrationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) StreamStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MemtablePostFlusher 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) FlushWriter 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) MiscStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) commitlog_archiver 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) InternalResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 73) CompactionManager 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 85) MessagingService n/a 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 95) Cache Type Size Capacity KeysToSave Provider INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 96) KeyCache 227 74448896 all INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 102) RowCache 0 0 all org.apache.cassandra.cache.SerializingCacheProvider INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 109) ColumnFamily Memtable ops,data INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) KeyspaceTest.CF_Test 1,71 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.local 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.peers 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.batchlog 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.NodeIdInfo 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.LocationInfo 0,0 INFO [ScheduledTasks:1] 2012-10-10
[jira] [Commented] (CASSANDRA-4833) get_count with 'count' param between 1024 and ~actual column count fails
[ https://issues.apache.org/jira/browse/CASSANDRA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481483#comment-13481483 ] Tyler Hobbs commented on CASSANDRA-4833: The patch needs to be rebased, but I'm +1 on the code changes get_count with 'count' param between 1024 and ~actual column count fails Key: CASSANDRA-4833 URL: https://issues.apache.org/jira/browse/CASSANDRA-4833 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6, 1.2.0 beta 1 Reporter: Tyler Hobbs Assignee: Yuki Morishita Attachments: 4833-1.1.txt, 4833-1.1-v2.txt, 4833-get-count-repro.py If you run get_count() with the 'count' param of the SliceRange set to a number between 1024 and (approximately) the actual number of columns in the row, something seems to silently fail internally, resulting in a client side timeout. Using a 'count' param outside of this range (lower or much higher) works just fine. This seems to affect all of 1.1 and 1.2.0-beta1, but not 1.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4842) DateType in Column MetaData causes server crash
[
https://issues.apache.org/jira/browse/CASSANDRA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4842:
--
Reviewer: jbellis
Priority: Minor (was: Major)
Affects Version/s: (was: 1.1.6)
(was: 1.1.5)
(was: 1.2.0 beta 1)
1.1.0
Fix Version/s: 1.2.0 beta 2
1.1.7
Assignee: Aleksey Yeschenko
verified that this only affects 1.1, not 1.0, probably due to changes in schema
handling.
DateType in Column MetaData causes server crash
---
Key: CASSANDRA-4842
URL: https://issues.apache.org/jira/browse/CASSANDRA-4842
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.0
Environment: All
Reporter: Russell Bradberry
Assignee: Aleksey Yeschenko
Priority: Minor
Fix For: 1.1.7, 1.2.0 beta 2
when creating a column family with column metadata containing a date, there
is a server crash that will prevent startup.
To recreate from the cli:
{code}
create keyspace test;
use test;
create column family foo
with column_type = 'Standard'
and comparator = 'CompositeType(LongType,DateType)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'UTF8Type'
and column_metadata = [
{ column_name : '1234:1350695443433', validation_class : BooleanType}
];
{code}
Produces this error in the logs:
{code}
ERROR 21:11:18,795 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:373)
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:194)
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:141)
at
org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:931)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3410)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3398)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:369)
... 11 more
Caused by: org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.fromString(AbstractCompositeType.java:213)
at
org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:257)
at
org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1318)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1250)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:299)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:434)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:346)
at
org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:217)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.text.ParseException: Unable to parse the date: 2012-10-19 21
at org.apache.commons.lang.time.DateUtils.parseDate(DateUtils.java:285)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:113)
... 14 more
ERROR 21:11:18,795
[jira] [Updated] (CASSANDRA-4842) DateType in Column MetaData causes server crash
[
https://issues.apache.org/jira/browse/CASSANDRA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4842:
--
Assignee: Pavel Yaskevich (was: Aleksey Yeschenko)
DateType in Column MetaData causes server crash
---
Key: CASSANDRA-4842
URL: https://issues.apache.org/jira/browse/CASSANDRA-4842
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.0
Environment: All
Reporter: Russell Bradberry
Assignee: Pavel Yaskevich
Priority: Minor
Fix For: 1.1.7, 1.2.0 beta 2
when creating a column family with column metadata containing a date, there
is a server crash that will prevent startup.
To recreate from the cli:
{code}
create keyspace test;
use test;
create column family foo
with column_type = 'Standard'
and comparator = 'CompositeType(LongType,DateType)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'UTF8Type'
and column_metadata = [
{ column_name : '1234:1350695443433', validation_class : BooleanType}
];
{code}
Produces this error in the logs:
{code}
ERROR 21:11:18,795 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:373)
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:194)
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:141)
at
org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:931)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3410)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3398)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:369)
... 11 more
Caused by: org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.fromString(AbstractCompositeType.java:213)
at
org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:257)
at
org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1318)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1250)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:299)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:434)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:346)
at
org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:217)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.text.ParseException: Unable to parse the date: 2012-10-19 21
at org.apache.commons.lang.time.DateUtils.parseDate(DateUtils.java:285)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:113)
... 14 more
ERROR 21:11:18,795 Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
[jira] [Updated] (CASSANDRA-4764) Clean out STREAM_STAGE vestiges
[ https://issues.apache.org/jira/browse/CASSANDRA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4764: -- Summary: Clean out STREAM_STAGE vestiges (was: Bulk Loading should run in STREAM_STAGE) Clean out STREAM_STAGE vestiges --- Key: CASSANDRA-4764 URL: https://issues.apache.org/jira/browse/CASSANDRA-4764 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Nick Bailey Assignee: Jonathan Ellis Priority: Minor Labels: streaming Fix For: 1.2.0 beta 2 Attachments: 4764.txt Currently it appears as though bulk loading operations don't run in any stage. Seems like they should be running in STREAM_STAGE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4755) Bulk loader won't work with CQL3
[ https://issues.apache.org/jira/browse/CASSANDRA-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4755: -- Reviewer: jbellis (was: yukim) Bulk loader won't work with CQL3 Key: CASSANDRA-4755 URL: https://issues.apache.org/jira/browse/CASSANDRA-4755 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.2.0 beta 1 Reporter: Nick Bailey Assignee: Aleksey Yeschenko Fix For: 1.2.0 Attachments: CASSANDRA-4755.txt Currently the bulk loader uses thrift to get the schema and validate it before bulk loading a cf. Since we stopped returning cql3 cfs through describe_keyspaces, the bulk loader will be unable to load those cfs. If we figure out/add a way to validate the schema over jmx, we could use that for getting token ranges as well and drop thrift completely from the bulk loader. Another option might be querying system tables manually to validate things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: fix BulkLoader recognition of CQL3 columnfamilies patch by Aleksey Yeschenko; reviewed by jbellis for CASSANDRA-4755
Updated Branches:
refs/heads/trunk f02928f88 - a58b87020
fix BulkLoader recognition of CQL3 columnfamilies
patch by Aleksey Yeschenko; reviewed by jbellis for CASSANDRA-4755
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58b8702
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58b8702
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58b8702
Branch: refs/heads/trunk
Commit: a58b87020b4315eab48482666450049fb7cf0674
Parents: f02928f
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Oct 22 11:48:07 2012 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Oct 22 11:48:07 2012 -0500
--
CHANGES.txt|1 +
.../org/apache/cassandra/tools/BulkLoader.java | 32 +++---
2 files changed, 17 insertions(+), 16 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58b8702/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e14ffa7..4026428 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2-beta2
+ * fix BulkLoader recognition of CQL3 columnfamilies (CASSANDRA-4755)
* Sort commitlog segments for replay by id instead of mtime (CASSANDRA-4793)
* Make hint delivery asynchronous (CASSANDRA-4761)
* Pluggable Thrift transport factories for CLI and cqlsh (CASSANDRA-4609,
4610)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58b8702/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index f2f018f..c838cb0 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -23,15 +23,19 @@ import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.*;
+import org.apache.commons.cli.*;
+
import org.apache.cassandra.auth.IAuthenticator;
import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.db.SystemTable;
+import org.apache.cassandra.db.Table;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.io.sstable.SSTableLoader;
import org.apache.cassandra.streaming.PendingFile;
import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.OutputHandler;
-import org.apache.commons.cli.*;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TFramedTransport;
@@ -162,7 +166,7 @@ public class BulkLoader
sb.append([total: ).append(totalSize == 0 ? 100L : totalProgress
* 100L / totalSize).append( - );
sb.append(mbPerSec(deltaProgress, deltaTime)).append(MB/s);
-sb.append( (avg: ).append(mbPerSec(totalProgress, time -
startTime)).append(MB/s)]);;
+sb.append( (avg: ).append(mbPerSec(totalProgress, time -
startTime)).append(MB/s)]);
System.out.print(sb.toString());
return done;
}
@@ -176,7 +180,7 @@ public class BulkLoader
static class ExternalClient extends SSTableLoader.Client
{
-private final MapString, SetString knownCfs = new HashMapString,
SetString();
+private final SetString knownCfs = new HashSetString();
private final SetInetAddress hosts;
private final int rpcPort;
private final String user;
@@ -198,17 +202,14 @@ public class BulkLoader
{
try
{
-
// Query endpoint to ranges map and schemas from thrift
InetAddress host = hostiter.next();
Cassandra.Client client =
createThriftClient(host.getHostAddress(), rpcPort, this.user, this.passwd);
-ListTokenRange tokenRanges =
client.describe_ring(keyspace);
-ListKsDef ksDefs = client.describe_keyspaces();
setPartitioner(client.describe_partitioner());
Token.TokenFactory tkFactory =
getPartitioner().getTokenFactory();
-for (TokenRange tr : tokenRanges)
+for (TokenRange tr : client.describe_ring(keyspace))
{
RangeToken range = new
RangeToken(tkFactory.fromString(tr.start_token),
tkFactory.fromString(tr.end_token));
for (String ep : tr.endpoints)
@@ -217,13 +218,13 @@ public class BulkLoader
}
}
-for (KsDef ksDef : ksDefs)
-
[jira] [Resolved] (CASSANDRA-4832) AssertionError: keys must not be empty
[ https://issues.apache.org/jira/browse/CASSANDRA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4832. --- Resolution: Fixed AssertionError: keys must not be empty -- Key: CASSANDRA-4832 URL: https://issues.apache.org/jira/browse/CASSANDRA-4832 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6 Environment: Debian 6.0.5 Reporter: Tristan Seligmann Assignee: Tristan Seligmann Priority: Minor Labels: indexing Fix For: 1.1.7 Attachments: FlushWriterKeyAssertionBlock.txt I'm getting errors like this logged: INFO 07:08:32,104 Compacting [SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-114-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-113-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-110-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hd-108-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hd-106-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hd-107-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-112-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-109-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Fusion/quoteinfo/Fusion-quoteinfo.quoteinfo_search_value_idx-hf-111-Data.db')] ERROR 07:08:32,108 Exception in thread Thread[CompactionExecutor:5,1,main] java.lang.AssertionError: Keys must not be empty at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:133) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:154) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:159) at org.apache.cassandra.db.compaction.CompactionManager$1.runMayThrow(CompactionManager.java:154) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) I'm not really sure when this started happening; they tend to be logged during a repair but I can't reproduce the error 100% reliably. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4834) Old-style mapred interface only populates row key for first column when using wide rows
[ https://issues.apache.org/jira/browse/CASSANDRA-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4834: -- Reviewer: brandon.williams Assignee: Ben Kempe Old-style mapred interface only populates row key for first column when using wide rows --- Key: CASSANDRA-4834 URL: https://issues.apache.org/jira/browse/CASSANDRA-4834 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Reporter: Ben Kempe Assignee: Ben Kempe Priority: Minor Fix For: 1.1.7 Attachments: TestJob.java, TestJobOldHadoop.java, trunk-CASSANDRA-4834.txt When using the ColumnFamilyRecordReader with the old-style Hadoop interface to iterate over wide row columns, the row key is only populated on the first column. See attached tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4834) Old-style mapred interface only populates row key for first column when using wide rows
[ https://issues.apache.org/jira/browse/CASSANDRA-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4834: -- Affects Version/s: (was: 1.1.5) 1.1.0 Fix Version/s: 1.1.7 Old-style mapred interface only populates row key for first column when using wide rows --- Key: CASSANDRA-4834 URL: https://issues.apache.org/jira/browse/CASSANDRA-4834 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Reporter: Ben Kempe Assignee: Ben Kempe Priority: Minor Fix For: 1.1.7 Attachments: TestJob.java, TestJobOldHadoop.java, trunk-CASSANDRA-4834.txt When using the ColumnFamilyRecordReader with the old-style Hadoop interface to iterate over wide row columns, the row key is only populated on the first column. See attached tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-4208) ColumnFamilyOutputFormat should support writing to multiple column families
[ https://issues.apache.org/jira/browse/CASSANDRA-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reopened CASSANDRA-4208: --- ColumnFamilyOutputFormat should support writing to multiple column families --- Key: CASSANDRA-4208 URL: https://issues.apache.org/jira/browse/CASSANDRA-4208 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 1.1.0 Reporter: Robbie Strickland Assignee: Robbie Strickland Fix For: 1.2.0 beta 2 Attachments: cassandra-1.1-4208.txt, cassandra-1.1-4208-v2.txt, cassandra-1.1-4208-v3.txt, cassandra-1.1-4208-v4.txt, trunk-4208.txt, trunk-4208-v2.txt, trunk-4208-v3.txt It is not currently possible to output records to more than one column family in a single reducer. Considering that writing values to Cassandra often involves multiple column families (i.e. updating your index when you insert a new value), this seems overly restrictive. I am submitting a patch that moves the specification of column family from the job configuration to the write() call in ColumnFamilyRecordWriter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4844) cqlsh help for CQL3 Create (and probably Alter) statements needs to be updated for Map syntax change
[
https://issues.apache.org/jira/browse/CASSANDRA-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4844:
--
Reviewer: brandon.williams
Priority: Minor (was: Major)
Assignee: Aleksey Yeschenko
Summary: cqlsh help for CQL3 Create (and probably Alter) statements needs
to be updated for Map syntax change (was: CQL3 Create keyspace statement not
compatible with older versions.)
Updated. See also CASSANDRA-4811.
cqlsh help for CQL3 Create (and probably Alter) statements needs to be
updated for Map syntax change
Key: CASSANDRA-4844
URL: https://issues.apache.org/jira/browse/CASSANDRA-4844
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Edward Capriolo
Assignee: Aleksey Yeschenko
Priority: Minor
Following the advice here.
http://www.datastax.com/docs/1.1/dml/using_cql
{noformat}
[edward@tablitha apache-cassandra-1.2.0-beta1]$ bin/cqlsh -3
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0-beta1 | CQL spec 3.0.0 | Thrift protocol
19.34.0]
Use HELP for help.
cqlsh CREATE KEYSPACE demodb WITH strategy_class =
'org.apache.cassandra.locator.SimpleStrategy' AND
strategy_options:replication_factor='1' ;
Bad Request: line 1:129 mismatched input ':' expecting '='
Perhaps you meant to use CQL 2? Try using the -2 option when starting cqlsh.
{noformat}
http://www.datastax.com/docs/1.1/references/cql/CREATE_KEYSPACE
{noformat}
cqlsh CREATE KEYSPACE Excelsior WITH strategy_class = 'SimpleStrategy'
... AND strategy_options:replication_factor = 1;
Bad Request: line 2:22 mismatched input ':' expecting '='
{noformat}
It seems that in Cassandra 1.2 I am no longer able to create a keyspace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4208) ColumnFamilyOutputFormat should support writing to multiple column families
[ https://issues.apache.org/jira/browse/CASSANDRA-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481501#comment-13481501 ] Robbie Strickland commented on CASSANDRA-4208: -- [~mkjellman] - I think the BOF support should be in a separate issue, since CFOF and BOF don't depend on each other for the MultipleOutputs functionality--and because this issue specifically addresses CFOF. ColumnFamilyOutputFormat should support writing to multiple column families --- Key: CASSANDRA-4208 URL: https://issues.apache.org/jira/browse/CASSANDRA-4208 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 1.1.0 Reporter: Robbie Strickland Assignee: Robbie Strickland Fix For: 1.2.0 beta 2 Attachments: cassandra-1.1-4208.txt, cassandra-1.1-4208-v2.txt, cassandra-1.1-4208-v3.txt, cassandra-1.1-4208-v4.txt, trunk-4208.txt, trunk-4208-v2.txt, trunk-4208-v3.txt It is not currently possible to output records to more than one column family in a single reducer. Considering that writing values to Cassandra often involves multiple column families (i.e. updating your index when you insert a new value), this seems overly restrictive. I am submitting a patch that moves the specification of column family from the job configuration to the write() call in ColumnFamilyRecordWriter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4794) cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException
[ https://issues.apache.org/jira/browse/CASSANDRA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481502#comment-13481502 ] debadatta das commented on CASSANDRA-4794: -- Hi Aleksey, I am using the following package available on http://cassandra.apache.org/download/ link Development Cassandra Server Releases (not production ready) The latest development release is 1.2.0-beta1 (released on 2012-09-24). •apache-cassandra-1.2.0-beta1-bin.tar.gz [PGP] [MD5] [SHA1] When u say Please get the latest possible Cassandra (trunk), I am not sure what you meant. Do you want me to use 1.2.0 beta2 version. If yes can you let me know where could I get the package. On the download link I mentioned above, I can't find either binary or source package for 1.2.0 beta2. Please let me know where can I get the latest cassandra package and whether it is binary package or source package. For your information, I am using thrift 0.7.0. Please let me know whether I should use later thrift version. When I tried to compile CPP program with thrift 0.8.0, it failed. Regards, Debadatta cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException --- Key: CASSANDRA-4794 URL: https://issues.apache.org/jira/browse/CASSANDRA-4794 Project: Cassandra Issue Type: Bug Components: API Affects Versions: 1.2.0 beta 1 Environment: C++ Reporter: debadatta das Assignee: Aleksey Yeschenko Fix For: 1.2.0 beta 2 Attachments: InsertExample.java.txt, log_java.txt, sample_AtomicBatchMutate.cpp Hi, We have installed cassandra 1.2.0 beta with thrift 0.7.0. We are using cpp interface. When we use batch_mutate API, it works fine. But when we are using the new atomic_batch_mutate API with same parameters as batch_mutate, it fails with org::apache::cassandra::TimedOutException, what(): Default TException. We get the same TException error even after increasing Send/Reciv timeout values of Tsocket to 15 seconds or more. Details: cassandra ring: cassandra ring with single node consistency level paramter to atomic_batch_mutate ConsistencyLevel::ONE Thrift version: same results with thrift 0.5.0 and thrift 0.7.0. thrift 0.8.0 seems unsupported with cassanda 1.2.0. Gives compilation error for cpp interface build. We are calling atomic_batch_mutate() with same parameters as batch_mutate. cassclient.atomic_batch_mutate(outermap1, ConsistencyLevel::ONE); where outmap1 is mapstring, mapstring, vectorMutation outermap1; Please point out if anything is missing while using atomic_batch_mutate or the reason behind the failure. The logs in cassandra system.log we get during atomic_batch_mutate failure are: INFO [ScheduledTasks:1] 2012-10-10 04:47:30,604 MessagingService.java (line 800) 1 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-10-10 04:47:30,606 StatusLogger.java (line 53) Pool Name Active Pending Blocked INFO [ScheduledTasks:1] 2012-10-10 04:47:30,607 StatusLogger.java (line 68) ReadStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) RequestResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReadRepairStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) MutationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReplicateOnWriteStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) GossipStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) AntiEntropyStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MigrationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) StreamStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MemtablePostFlusher 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) FlushWriter 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) MiscStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) commitlog_archiver 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) InternalResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 73) CompactionManager 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 85) MessagingService n/a 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 95) Cache Type Size Capacity KeysToSave Provider INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java
[jira] [Resolved] (CASSANDRA-4845) Unclear that cqlsh help requires quoating
[
https://issues.apache.org/jira/browse/CASSANDRA-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-4845.
---
Resolution: Duplicate
Fixing in CASSANDRA-4811
Unclear that cqlsh help requires quoating
-
Key: CASSANDRA-4845
URL: https://issues.apache.org/jira/browse/CASSANDRA-4845
Project: Cassandra
Issue Type: Improvement
Reporter: Edward Capriolo
For about two weeks I have been under the impression that most of the CQL
help topics are unimplemented. Maybe I am dense, but I never would have
guessed I had to quote certain help topics to get help and not others.
{noformat}
cqlsh:testks HELP INSERT;
Improper HELP command.
cqlsh:testks HELP 'INSERT';
INSERT INTO [keyspace.]tablename
( colname1, colname2 [, colname3 [, ...]] )
VALUES ( colval1, colval2 [, colval3 [, ...]] )
{noformat}
Either we should fix cqlsh so we do not need to quote these things, or we
should actually put the quotes in the help menu which might indicate this to
someone.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481506#comment-13481506
]
Jonathan Ellis commented on CASSANDRA-4815:
---
These are good questions and I wanted to reach a wider audience than the Jira
followers, so I wrote a blog post to address the questions here:
http://www.datastax.com/dev/blog/cql3-for-cassandra-experts
Please let me know if that clarifies things.
(Note that all the examples there work in 1.1 as well as 1.2, with the
exception of the cql3 CREATE and ALTER for song_tags. Those require 1.2. All
the pasted output is in fact from 1.1.)
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra Database, not looking for a seQL']['likesToday']=34;
set ['Cassandra Database, not looking for a
seQL']['credits-dir']='director:asf';
set ['Cassandra Database, not looking for a
seQL']['credits-jir]='jiraguy:bob';
set ['Cassandra Database, not looking for a seQL']['tags-action']='';
set ['Cassandra Database, not looking for a seQL']['tags-adventure']='';
set ['Cassandra Database, not looking for a seQL']['tags-romance']='';
set ['Cassandra Database, not looking for a seQL']['tags-programming']='';
This is the correct way to do it. 1 seek to find all the information related
to a movie. As long as this row does
not get large there is no reason to optimize by breaking data into other
column families. (Notice you can not transpose this
because movies is two 1-to-many relationships of potentially different types)
Lets look at the CQL3 way to do this design:
First, contrary to the original design of cassandra CQL does not like wide
rows. It also does not have a good way to dealing with dynamic rows together
with static rows either.
You have two options:
Option 1: lose all schema
create table movies ( name string, column blob, value blob, primary
key(name)) with compact storage.
This method is not so hot we have not lost all our validators, and by the way
you have to physically shutdown everything and rename files and recreate your
schema if you want to inform cassandra that a current table should be
compact. This could at very least be just a metadata change. Also you can not
add column schema either.
Option 2 Normalize (is even worse)
create table movie (name String, description string, likestoday int,
blacklisted int);
create table movecredits( name string, role string, personname string,
primary key(name,role) );
create table movetags( name string, tag string, primary key (name,tag) );
This is a terrible design, of the 4 key characteristics how cassandra data
should be designed it fails 3:
It does not:
1) Denormalize
2) Eliminate seeks
3) Design for read
Why is Cassandra steering toward this course, by making a language that does
not understand wide rows?
So what can be done? My suggestions:
Cassandra needs to lose the COMPACT STORAGE conversions. Each table needs a
virtual view that is compact storage with no work to migrate data and
recreate schemas. Every table should have a compact view for the schemaless,
or a simple query hint like /*transposed*/ should make this change.
Metadata should be definable by regex. For example, all columnes named tag*
are of type string.
CQL should have the column[slice_start] .. column[slice_end] operator from
cql2.
CQL should support
[jira] [Commented] (CASSANDRA-4794) cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException
[ https://issues.apache.org/jira/browse/CASSANDRA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481507#comment-13481507 ] Aleksey Yeschenko commented on CASSANDRA-4794: -- Thrift version is not the issue here. To get the latest version, run: git clone https://git-wip-us.apache.org/repos/asf/cassandra.git ant build cassandra 1.2.0 beta: atomic_batch_mutate fails with Default TException --- Key: CASSANDRA-4794 URL: https://issues.apache.org/jira/browse/CASSANDRA-4794 Project: Cassandra Issue Type: Bug Components: API Affects Versions: 1.2.0 beta 1 Environment: C++ Reporter: debadatta das Assignee: Aleksey Yeschenko Fix For: 1.2.0 beta 2 Attachments: InsertExample.java.txt, log_java.txt, sample_AtomicBatchMutate.cpp Hi, We have installed cassandra 1.2.0 beta with thrift 0.7.0. We are using cpp interface. When we use batch_mutate API, it works fine. But when we are using the new atomic_batch_mutate API with same parameters as batch_mutate, it fails with org::apache::cassandra::TimedOutException, what(): Default TException. We get the same TException error even after increasing Send/Reciv timeout values of Tsocket to 15 seconds or more. Details: cassandra ring: cassandra ring with single node consistency level paramter to atomic_batch_mutate ConsistencyLevel::ONE Thrift version: same results with thrift 0.5.0 and thrift 0.7.0. thrift 0.8.0 seems unsupported with cassanda 1.2.0. Gives compilation error for cpp interface build. We are calling atomic_batch_mutate() with same parameters as batch_mutate. cassclient.atomic_batch_mutate(outermap1, ConsistencyLevel::ONE); where outmap1 is mapstring, mapstring, vectorMutation outermap1; Please point out if anything is missing while using atomic_batch_mutate or the reason behind the failure. The logs in cassandra system.log we get during atomic_batch_mutate failure are: INFO [ScheduledTasks:1] 2012-10-10 04:47:30,604 MessagingService.java (line 800) 1 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-10-10 04:47:30,606 StatusLogger.java (line 53) Pool Name Active Pending Blocked INFO [ScheduledTasks:1] 2012-10-10 04:47:30,607 StatusLogger.java (line 68) ReadStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) RequestResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReadRepairStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) MutationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,608 StatusLogger.java (line 68) ReplicateOnWriteStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) GossipStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) AntiEntropyStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MigrationStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) StreamStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,609 StatusLogger.java (line 68) MemtablePostFlusher 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) FlushWriter 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) MiscStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) commitlog_archiver 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 68) InternalResponseStage 0 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,610 StatusLogger.java (line 73) CompactionManager 0 0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 85) MessagingService n/a 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 95) Cache Type Size Capacity KeysToSave Provider INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 96) KeyCache 227 74448896 all INFO [ScheduledTasks:1] 2012-10-10 04:47:30,611 StatusLogger.java (line 102) RowCache 0 0 all org.apache.cassandra.cache.SerializingCacheProvider INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 109) ColumnFamily Memtable ops,data INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) KeyspaceTest.CF_Test 1,71 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.local 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.peers 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612 StatusLogger.java (line 112) system.batchlog 0,0 INFO [ScheduledTasks:1] 2012-10-10 04:47:30,612
[jira] [Updated] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4815:
--
Comment: was deleted
(was: These are good questions and I wanted to reach a wider audience than the
Jira followers, so I wrote a blog post to address the questions here:
http://www.datastax.com/dev/blog/cql3-for-cassandra-experts
Please let me know if that clarifies things.
(Note that all the examples there work in 1.1 as well as 1.2, with the
exception of the cql3 CREATE and ALTER for song_tags. Those require 1.2. All
the pasted output is in fact from 1.1.)
)
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra Database, not looking for a seQL']['likesToday']=34;
set ['Cassandra Database, not looking for a
seQL']['credits-dir']='director:asf';
set ['Cassandra Database, not looking for a
seQL']['credits-jir]='jiraguy:bob';
set ['Cassandra Database, not looking for a seQL']['tags-action']='';
set ['Cassandra Database, not looking for a seQL']['tags-adventure']='';
set ['Cassandra Database, not looking for a seQL']['tags-romance']='';
set ['Cassandra Database, not looking for a seQL']['tags-programming']='';
This is the correct way to do it. 1 seek to find all the information related
to a movie. As long as this row does
not get large there is no reason to optimize by breaking data into other
column families. (Notice you can not transpose this
because movies is two 1-to-many relationships of potentially different types)
Lets look at the CQL3 way to do this design:
First, contrary to the original design of cassandra CQL does not like wide
rows. It also does not have a good way to dealing with dynamic rows together
with static rows either.
You have two options:
Option 1: lose all schema
create table movies ( name string, column blob, value blob, primary
key(name)) with compact storage.
This method is not so hot we have not lost all our validators, and by the way
you have to physically shutdown everything and rename files and recreate your
schema if you want to inform cassandra that a current table should be
compact. This could at very least be just a metadata change. Also you can not
add column schema either.
Option 2 Normalize (is even worse)
create table movie (name String, description string, likestoday int,
blacklisted int);
create table movecredits( name string, role string, personname string,
primary key(name,role) );
create table movetags( name string, tag string, primary key (name,tag) );
This is a terrible design, of the 4 key characteristics how cassandra data
should be designed it fails 3:
It does not:
1) Denormalize
2) Eliminate seeks
3) Design for read
Why is Cassandra steering toward this course, by making a language that does
not understand wide rows?
So what can be done? My suggestions:
Cassandra needs to lose the COMPACT STORAGE conversions. Each table needs a
virtual view that is compact storage with no work to migrate data and
recreate schemas. Every table should have a compact view for the schemaless,
or a simple query hint like /*transposed*/ should make this change.
Metadata should be definable by regex. For example, all columnes named tag*
are of type string.
CQL should have the column[slice_start] .. column[slice_end] operator from
cql2.
CQL should support current users, users should not have
[jira] [Commented] (CASSANDRA-4784) Create separate sstables for each token range handled by a node
[ https://issues.apache.org/jira/browse/CASSANDRA-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481512#comment-13481512 ] Sylvain Lebresne commented on CASSANDRA-4784: - bq. I do still see this as useful for backup/restore in the general case though While I kind of agree, I want to note that backuping data from only one replica runs the risk of missing whatever data on which the node is not consistent, unless you've run a repair just before. I suppose some may prefer just running the repair prior to the backing or, especially if they are running regular repairs, consider that good enough, but just wanted to note that at least in theory that's not completely bulletproof. Create separate sstables for each token range handled by a node --- Key: CASSANDRA-4784 URL: https://issues.apache.org/jira/browse/CASSANDRA-4784 Project: Cassandra Issue Type: Improvement Components: Core Reporter: sankalp kohli Priority: Minor Labels: perfomance Currently, each sstable has data for all the ranges that node is handling. If we change that and rather have separate sstables for each range that node is handling, it can lead to some improvements. Improvements 1) Node rebuild will be very fast as sstables can be directly copied over to the bootstrapping node. It will minimize any application level logic. We can directly use Linux native methods to transfer sstables without using CPU and putting less pressure on the serving node. I think in theory it will be the fastest way to transfer data. 2) Backup can only transfer sstables for a node which belong to its primary keyrange. 3) ETL process can only copy one replica of data and will be much faster. Changes: We can split the writes into multiple memtables for each range it is handling. The sstables being flushed from these can have details of which range of data it is handling. There will be no change I think for any reads as they work with interleaved data anyway. But may be we can improve there as well? Complexities: The change does not look very complicated. I am not taking into account how it will work when ranges are being changed for nodes. Vnodes might make this work more complicated. We can also have a bit on each sstable which says whether it is primary data or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4846) BulkLoader throws NPE at start up
Yuki Morishita created CASSANDRA-4846:
-
Summary: BulkLoader throws NPE at start up
Key: CASSANDRA-4846
URL: https://issues.apache.org/jira/browse/CASSANDRA-4846
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 2
Reporter: Yuki Morishita
BulkLoader in trunk throws below exception at start up and exit abnormally.
{code}
Exception in thread main java.lang.ExceptionInInitializerError
at
org.apache.cassandra.io.sstable.SSTableReader.init(SSTableReader.java:87)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:180)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:148)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:96)
at java.io.File.list(File.java:1010)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:117)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:63)
Caused by: java.lang.NullPointerException
at
org.apache.cassandra.service.CacheService.initRowCache(CacheService.java:154)
at
org.apache.cassandra.service.CacheService.init(CacheService.java:102)
at
org.apache.cassandra.service.CacheService.clinit(CacheService.java:83)
... 8 more
{code}
This comes from CASSANDRA-4732, which moved keyCache in SSTableReader
initialization at instance creation. This causes access to CacheService that
did not happen for v1.1 and ends up NPE because BulkLoader does not load
cassandra.yaml.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481513#comment-13481513
]
Jonathan Ellis commented on CASSANDRA-4815:
---
To add to that, if you want mostly static columns, but some schemaless then
you can throw the schemaless ones in a Map. This will NOT be easily
accessible from Thrift -- but it's a good example of the kinds of things that
cql3 makes easier.
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra Database, not looking for a seQL']['likesToday']=34;
set ['Cassandra Database, not looking for a
seQL']['credits-dir']='director:asf';
set ['Cassandra Database, not looking for a
seQL']['credits-jir]='jiraguy:bob';
set ['Cassandra Database, not looking for a seQL']['tags-action']='';
set ['Cassandra Database, not looking for a seQL']['tags-adventure']='';
set ['Cassandra Database, not looking for a seQL']['tags-romance']='';
set ['Cassandra Database, not looking for a seQL']['tags-programming']='';
This is the correct way to do it. 1 seek to find all the information related
to a movie. As long as this row does
not get large there is no reason to optimize by breaking data into other
column families. (Notice you can not transpose this
because movies is two 1-to-many relationships of potentially different types)
Lets look at the CQL3 way to do this design:
First, contrary to the original design of cassandra CQL does not like wide
rows. It also does not have a good way to dealing with dynamic rows together
with static rows either.
You have two options:
Option 1: lose all schema
create table movies ( name string, column blob, value blob, primary
key(name)) with compact storage.
This method is not so hot we have not lost all our validators, and by the way
you have to physically shutdown everything and rename files and recreate your
schema if you want to inform cassandra that a current table should be
compact. This could at very least be just a metadata change. Also you can not
add column schema either.
Option 2 Normalize (is even worse)
create table movie (name String, description string, likestoday int,
blacklisted int);
create table movecredits( name string, role string, personname string,
primary key(name,role) );
create table movetags( name string, tag string, primary key (name,tag) );
This is a terrible design, of the 4 key characteristics how cassandra data
should be designed it fails 3:
It does not:
1) Denormalize
2) Eliminate seeks
3) Design for read
Why is Cassandra steering toward this course, by making a language that does
not understand wide rows?
So what can be done? My suggestions:
Cassandra needs to lose the COMPACT STORAGE conversions. Each table needs a
virtual view that is compact storage with no work to migrate data and
recreate schemas. Every table should have a compact view for the schemaless,
or a simple query hint like /*transposed*/ should make this change.
Metadata should be definable by regex. For example, all columnes named tag*
are of type string.
CQL should have the column[slice_start] .. column[slice_end] operator from
cql2.
CQL should support current users, users should not have to
switch between CQL versions, and possibly thrift, to work with wide rows. The
language should work for them even if
it not expressly designed for them.
[jira] [Commented] (CASSANDRA-4784) Create separate sstables for each token range handled by a node
[ https://issues.apache.org/jira/browse/CASSANDRA-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481518#comment-13481518 ] sankalp kohli commented on CASSANDRA-4784: -- Yes it is required. But running repair is easier and cheaper than stores N copies of same data. Create separate sstables for each token range handled by a node --- Key: CASSANDRA-4784 URL: https://issues.apache.org/jira/browse/CASSANDRA-4784 Project: Cassandra Issue Type: Improvement Components: Core Reporter: sankalp kohli Priority: Minor Labels: perfomance Currently, each sstable has data for all the ranges that node is handling. If we change that and rather have separate sstables for each range that node is handling, it can lead to some improvements. Improvements 1) Node rebuild will be very fast as sstables can be directly copied over to the bootstrapping node. It will minimize any application level logic. We can directly use Linux native methods to transfer sstables without using CPU and putting less pressure on the serving node. I think in theory it will be the fastest way to transfer data. 2) Backup can only transfer sstables for a node which belong to its primary keyrange. 3) ETL process can only copy one replica of data and will be much faster. Changes: We can split the writes into multiple memtables for each range it is handling. The sstables being flushed from these can have details of which range of data it is handling. There will be no change I think for any reads as they work with interleaved data anyway. But may be we can improve there as well? Complexities: The change does not look very complicated. I am not taking into account how it will work when ranges are being changed for nodes. Vnodes might make this work more complicated. We can also have a bit on each sstable which says whether it is primary data or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4208) ColumnFamilyOutputFormat should support writing to multiple column families
[ https://issues.apache.org/jira/browse/CASSANDRA-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481523#comment-13481523 ] Michael Kjellman commented on CASSANDRA-4208: - Robbie - I'm okay with that. but not sure then we should have the BOF patch you provided applied if it doesn't work. I'm still working on debugging exactly why it doesn't stream but getting an environment setup to debug the whole process has been difficult. If anything maybe we should revert the change to BOF keep the other changes and then open another BOF bug for multiple output support? ColumnFamilyOutputFormat should support writing to multiple column families --- Key: CASSANDRA-4208 URL: https://issues.apache.org/jira/browse/CASSANDRA-4208 Project: Cassandra Issue Type: Improvement Components: Hadoop Affects Versions: 1.1.0 Reporter: Robbie Strickland Assignee: Robbie Strickland Fix For: 1.2.0 beta 2 Attachments: cassandra-1.1-4208.txt, cassandra-1.1-4208-v2.txt, cassandra-1.1-4208-v3.txt, cassandra-1.1-4208-v4.txt, trunk-4208.txt, trunk-4208-v2.txt, trunk-4208-v3.txt It is not currently possible to output records to more than one column family in a single reducer. Considering that writing values to Cassandra often involves multiple column families (i.e. updating your index when you insert a new value), this seems overly restrictive. I am submitting a patch that moves the specification of column family from the job configuration to the write() call in ColumnFamilyRecordWriter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481525#comment-13481525
]
Jonathan Ellis commented on CASSANDRA-4815:
---
bq. What does this mean? 'We can do that'. If we have an old style schema don't
we need to be able to alter a current table.
Only if you want to add meaningful names. That's what this next part is saying:
If we simply use the old schema directly as-is, Cassandra will give cell names
and values autogenerated CQL3 names: column1, column2, and so forth. Here I’m
accessing the data inserted earlier from CQL2, but with cqlsh --cql3:
{noformat}
SELECT * FROM song_tags;
id | column1 | value
--+-+---
8a172618-b121-4136-bb10-f665cfc469eb |2007 |
8a172618-b121-4136-bb10-f665cfc469eb | covers |
a3e64f8f-bd44-4f28-b8d9-6938726e34d4 |1973 |
a3e64f8f-bd44-4f28-b8d9-6938726e34d4 | blues |
{noformat}
... that said, as Sylvain points out we do have CASSANDRA-4822 open to allow
changing those default names without dropping and recreating the table
definition.
bq. Does it make sense to implement CLI like SET and GET?
Not in CQL-the-language, and I don't think even in cqlsh-the-utility. I
understand the appeal of the convenience, but the abstraction leakage it would
introduce threatens to undo all the work we're doing to make CQL3 something you
can use on its own terms.
(As far as performance goes, prepared statements make the length of the string
being parsed initially a non-issue.)
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra Database, not looking for a seQL']['likesToday']=34;
set ['Cassandra Database, not looking for a
seQL']['credits-dir']='director:asf';
set ['Cassandra Database, not looking for a
seQL']['credits-jir]='jiraguy:bob';
set ['Cassandra Database, not looking for a seQL']['tags-action']='';
set ['Cassandra Database, not looking for a seQL']['tags-adventure']='';
set ['Cassandra Database, not looking for a seQL']['tags-romance']='';
set ['Cassandra Database, not looking for a seQL']['tags-programming']='';
This is the correct way to do it. 1 seek to find all the information related
to a movie. As long as this row does
not get large there is no reason to optimize by breaking data into other
column families. (Notice you can not transpose this
because movies is two 1-to-many relationships of potentially different types)
Lets look at the CQL3 way to do this design:
First, contrary to the original design of cassandra CQL does not like wide
rows. It also does not have a good way to dealing with dynamic rows together
with static rows either.
You have two options:
Option 1: lose all schema
create table movies ( name string, column blob, value blob, primary
key(name)) with compact storage.
This method is not so hot we have not lost all our validators, and by the way
you have to physically shutdown everything and rename files and recreate your
schema if you want to inform cassandra that a current table should be
compact. This could at very least be just a metadata change. Also you can not
add column schema either.
Option 2 Normalize (is even worse)
create table movie (name String, description string, likestoday int,
blacklisted int);
create table movecredits( name string, role
[jira] [Commented] (CASSANDRA-4417) invalid counter shard detected
[ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481526#comment-13481526 ] Sylvain Lebresne commented on CASSANDRA-4417: - Quick question: do you always increment by the same value by any chance? I'm asking because the last log you've pasted indicates the conflicting information found correspond to 1 increment only, and in the first case the value is 1, on the second the value is 2. If you always increment by say 1, that would tell us which one is wrong (I'm not yet sure which conclusion I would draw from that but more info can't hurt :)). invalid counter shard detected --- Key: CASSANDRA-4417 URL: https://issues.apache.org/jira/browse/CASSANDRA-4417 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.1 Environment: Amazon Linux Reporter: Senthilvel Rangaswamy Seeing errors like these: 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 13) and (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 1) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard What does it mean ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4417) invalid counter shard detected
[ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481539#comment-13481539 ] Sylvain Lebresne commented on CASSANDRA-4417: - bq. Could this be related to: CASSANDRA-4071? I missed that earlier on, but yes, Bartłomiej is correct, CASSANDRA-4071 will totally trigger 'invalid counter shard detected' messages. As described in the ticket, if you don't use RF=1, this shouldn't actually create data loss, but it would still trigger the log until things get compacted away. invalid counter shard detected --- Key: CASSANDRA-4417 URL: https://issues.apache.org/jira/browse/CASSANDRA-4417 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.1 Environment: Amazon Linux Reporter: Senthilvel Rangaswamy Seeing errors like these: 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 13) and (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 1) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard What does it mean ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4764) Clean out STREAM_STAGE vestiges
[ https://issues.apache.org/jira/browse/CASSANDRA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481556#comment-13481556 ] Yuki Morishita commented on CASSANDRA-4764: --- For removing STREAM stage, +1. Attached patch seems against 1.1 and contains diff from different issue though. Clean out STREAM_STAGE vestiges --- Key: CASSANDRA-4764 URL: https://issues.apache.org/jira/browse/CASSANDRA-4764 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Nick Bailey Assignee: Jonathan Ellis Priority: Minor Labels: streaming Fix For: 1.2.0 beta 2 Attachments: 4764.txt Currently it appears as though bulk loading operations don't run in any stage. Seems like they should be running in STREAM_STAGE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4841) Exception thrown during nodetool repair -pr
[ https://issues.apache.org/jira/browse/CASSANDRA-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481564#comment-13481564 ] Michael Kjellman commented on CASSANDRA-4841: - jmx was dead, as was thrift and gossip. so the node was effectively in a crashed state imho. Exception thrown during nodetool repair -pr --- Key: CASSANDRA-4841 URL: https://issues.apache.org/jira/browse/CASSANDRA-4841 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6 Environment: Ubuntu 12.04 x64, 32GB RAM Reporter: Michael Kjellman Priority: Critical During a nodetool repair -pr an exception was thrown. root@scl-cas04:~# nodetool repair -pr Exception in thread main java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.io.EOFException at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:227) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:160) at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1017) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:305) at $Proxy0.forceTableRepairPrimaryRange(Unknown Source) at org.apache.cassandra.tools.NodeProbe.forceTableRepairPrimaryRange(NodeProbe.java:209) at org.apache.cassandra.tools.NodeCmd.optionalKSandCFs(NodeCmd.java:1044) at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:834) Caused by: java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:213) ... 9 more Logs: WARN 20:53:27,135 Heap is 0.9710037912028483 full. You may need to reduce memtable and/or cache sizes. Cassandr a will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold i n cassandra.yaml if you don't want Cassandra to do this automatically regardless of configuration issues a repair shouldn't crash a node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4847) Bad disk causes death of node despite disk_failure_policy
Kirk True created CASSANDRA-4847:
Summary: Bad disk causes death of node despite disk_failure_policy
Key: CASSANDRA-4847
URL: https://issues.apache.org/jira/browse/CASSANDRA-4847
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.0 beta 1
Reporter: Kirk True
Assignee: Kirk True
Steps:
# Create a bad disk via device mapper
# Specify good disk and bad disk is data directory
# Set {{disk_failure_policy}} to {{best_effort}} in cassandra.yaml
# Start node
Expected:
Attempts to create system directories to fail (as expected) on bad disk, and
have it added to blacklisted directories.
Actual:
Node start up aborts due to uncaught error:
{noformat}
FSWriteError in /mnt/bad_disk/system_traces/sessions
at
org.apache.cassandra.io.util.FileUtils.createDirectory(FileUtils.java:258)
at org.apache.cassandra.db.Directories.init(Directories.java:104)
at org.apache.cassandra.db.Directories.create(Directories.java:90)
at
org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:404)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:227)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:393)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:436)
Caused by: java.io.IOException: Failed to mkdirs
/mnt/bad_disk/system_traces/sessions
... 7 more
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4848) Expose black-listed directories via JMX
Kirk True created CASSANDRA-4848:
Summary: Expose black-listed directories via JMX
Key: CASSANDRA-4848
URL: https://issues.apache.org/jira/browse/CASSANDRA-4848
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.2.0 beta 1
Reporter: Kirk True
Assignee: Kirk True
JBOD support included failure modes (CASSANDRA-2118) that insert directories on
bad disks to the {{BlacklistedDirectories}} class. However, it doesn't appear
that the list of directories is exposed to administrators via JMX or other
means. So unless they're scouring the logs or being notified at the OS level,
they will remain unaware of the issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4842) DateType in Column MetaData causes server crash
[
https://issues.apache.org/jira/browse/CASSANDRA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich reassigned CASSANDRA-4842:
--
Assignee: Sylvain Lebresne (was: Pavel Yaskevich)
This is the problem with CompositeType, because DateType would convert input to
a human readable format with has *colons* inside and on
AbstractCompositeType.fromString it tries to spit composite into parts based on
colons as well.
DateType in Column MetaData causes server crash
---
Key: CASSANDRA-4842
URL: https://issues.apache.org/jira/browse/CASSANDRA-4842
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.0
Environment: All
Reporter: Russell Bradberry
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.1.7, 1.2.0 beta 2
when creating a column family with column metadata containing a date, there
is a server crash that will prevent startup.
To recreate from the cli:
{code}
create keyspace test;
use test;
create column family foo
with column_type = 'Standard'
and comparator = 'CompositeType(LongType,DateType)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'UTF8Type'
and column_metadata = [
{ column_name : '1234:1350695443433', validation_class : BooleanType}
];
{code}
Produces this error in the logs:
{code}
ERROR 21:11:18,795 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:373)
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:194)
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:141)
at
org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:931)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3410)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3398)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:369)
... 11 more
Caused by: org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.fromString(AbstractCompositeType.java:213)
at
org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:257)
at
org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1318)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1250)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:299)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:434)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:346)
at
org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:217)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.text.ParseException: Unable to parse the date: 2012-10-19 21
at org.apache.commons.lang.time.DateUtils.parseDate(DateUtils.java:285)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:113)
... 14 more
ERROR 21:11:18,795 Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
[jira] [Resolved] (CASSANDRA-4841) Exception thrown during nodetool repair -pr
[ https://issues.apache.org/jira/browse/CASSANDRA-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4841. --- Resolution: Invalid Sounds like you were GC storming then. Grep for GCInspector or top compared with thread IDs could confirm this. Often this happens because sstable structures outgrew the space Cassandra reserves for it in the heap. You can reduce these with the (global) index_interval and (per-CF) bloom_filter_fp_chance settings. Unfortunately Cassandra is not yet able to self-tune this. Exception thrown during nodetool repair -pr --- Key: CASSANDRA-4841 URL: https://issues.apache.org/jira/browse/CASSANDRA-4841 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.6 Environment: Ubuntu 12.04 x64, 32GB RAM Reporter: Michael Kjellman Priority: Critical During a nodetool repair -pr an exception was thrown. root@scl-cas04:~# nodetool repair -pr Exception in thread main java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.io.EOFException at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:227) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:160) at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1017) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:305) at $Proxy0.forceTableRepairPrimaryRange(Unknown Source) at org.apache.cassandra.tools.NodeProbe.forceTableRepairPrimaryRange(NodeProbe.java:209) at org.apache.cassandra.tools.NodeCmd.optionalKSandCFs(NodeCmd.java:1044) at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:834) Caused by: java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:213) ... 9 more Logs: WARN 20:53:27,135 Heap is 0.9710037912028483 full. You may need to reduce memtable and/or cache sizes. Cassandr a will now flush up to the two largest memtables to free up memory. Adjust flush_largest_memtables_at threshold i n cassandra.yaml if you don't want Cassandra to do this automatically regardless of configuration issues a repair shouldn't crash a node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/7] git commit: Merge branch 'cassandra-1.1' into trunk
Updated Branches: refs/heads/cassandra-1.1 f22e2c459 - 95fb613bf refs/heads/trunk a58b87020 - dd8a3c450 Merge branch 'cassandra-1.1' into trunk Conflicts: src/java/org/apache/cassandra/service/StorageService.java src/java/org/apache/cassandra/thrift/CassandraServer.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd8a3c45 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd8a3c45 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd8a3c45 Branch: refs/heads/trunk Commit: dd8a3c45072efbc0fea14e98552b4b5a9dab9de3 Parents: a58b870 95fb613 Author: Yuki Morishita yu...@apache.org Authored: Mon Oct 22 13:53:11 2012 -0500 Committer: Yuki Morishita yu...@apache.org Committed: Mon Oct 22 13:53:11 2012 -0500 -- CHANGES.txt|1 + .../apache/cassandra/thrift/CassandraServer.java | 12 ++- .../cassandra/service/CassandraServerTest.java | 91 ++- 3 files changed, 70 insertions(+), 34 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd8a3c45/CHANGES.txt -- diff --cc CHANGES.txt index 4026428,f309ef1..0f7caba --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -40,76 -5,8 +40,77 @@@ Merged from 1.1 (CASSANDRA-4765) * fix wrong leveled compaction progress calculation (CASSANDRA-4807) * add a close() method to CRAR to prevent leaking file descriptors (CASSANDRA-4820) + * fix potential infinite loop in get_count (CASSANDRA-4833) + +1.2-beta1 + * add atomic_batch_mutate (CASSANDRA-4542, -4635) + * increase default max_hint_window_in_ms to 3h (CASSANDRA-4632) + * include message initiation time to replicas so they can more + accurately drop timed-out requests (CASSANDRA-2858) + * fix clientutil.jar dependencies (CASSANDRA-4566) + * optimize WriteResponse (CASSANDRA-4548) + * new metrics (CASSANDRA-4009) + * redesign KEYS indexes to avoid read-before-write (CASSANDRA-2897) + * debug tracing (CASSANDRA-1123) + * parallelize row cache loading (CASSANDRA-4282) + * Make compaction, flush JBOD-aware (CASSANDRA-4292) + * run local range scans on the read stage (CASSANDRA-3687) + * clean up ioexceptions (CASSANDRA-2116) + * add disk_failure_policy (CASSANDRA-2118) + * Introduce new json format with row level deletion (CASSANDRA-4054) + * remove redundant name column from schema_keyspaces (CASSANDRA-4433) + * improve nodetool ring handling of multi-dc clusters (CASSANDRA-3047) + * update NTS calculateNaturalEndpoints to be O(N log N) (CASSANDRA-3881) + * add UseCondCardMark XX jvm settings on jdk 1.7 (CASSANDRA-4366) + * split up rpc timeout by operation type (CASSANDRA-2819) + * rewrite key cache save/load to use only sequential i/o (CASSANDRA-3762) + * update MS protocol with a version handshake + broadcast address id + (CASSANDRA-4311) + * multithreaded hint replay (CASSANDRA-4189) + * add inter-node message compression (CASSANDRA-3127) + * remove COPP (CASSANDRA-2479) + * Track tombstone expiration and compact when tombstone content is + higher than a configurable threshold, default 20% (CASSANDRA-3442, 4234) + * update MurmurHash to version 3 (CASSANDRA-2975) + * (CLI) track elapsed time for `delete' operation (CASSANDRA-4060) + * (CLI) jline version is bumped to 1.0 to properly support + 'delete' key function (CASSANDRA-4132) + * Save IndexSummary into new SSTable 'Summary' component (CASSANDRA-2392, 4289) + * Add support for range tombstones (CASSANDRA-3708) + * Improve MessagingService efficiency (CASSANDRA-3617) + * Avoid ID conflicts from concurrent schema changes (CASSANDRA-3794) + * Set thrift HSHA server thread limit to unlimited by default (CASSANDRA-4277) + * Avoids double serialization of CF id in RowMutation messages + (CASSANDRA-4293) + * stream compressed sstables directly with java nio (CASSANDRA-4297) + * Support multiple ranges in SliceQueryFilter (CASSANDRA-3885) + * Add column metadata to system column families (CASSANDRA-4018) + * (cql3) Always use composite types by default (CASSANDRA-4329) + * (cql3) Add support for set, map and list (CASSANDRA-3647) + * Validate date type correctly (CASSANDRA-4441) + * (cql3) Allow definitions with only a PK (CASSANDRA-4361) + * (cql3) Add support for row key composites (CASSANDRA-4179) + * improve DynamicEndpointSnitch by using reservoir sampling (CASSANDRA-4038) + * (cql3) Add support for 2ndary indexes (CASSANDRA-3680) + * (cql3) fix defining more than one PK to be invalid (CASSANDRA-4477) + * remove schema agreement checking from all external APIs (Thrift, CQL and CQL3) (CASSANDRA-4487) + * add Murmur3Partitioner and make it default for new installations (CASSANDRA-3772, 4621) + *
[3/7] git commit: fix potential infinite loop in get_count; patch by yukim reviewed by Tyler Hobbs for CASSANDRA-4833
fix potential infinite loop in get_count; patch by yukim reviewed by Tyler
Hobbs for CASSANDRA-4833
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95fb613b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95fb613b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95fb613b
Branch: refs/heads/trunk
Commit: 95fb613bf996c715392f9aa1f491609b6acaeff5
Parents: f22e2c4
Author: Yuki Morishita yu...@apache.org
Authored: Mon Oct 22 12:16:20 2012 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Oct 22 12:16:20 2012 -0500
--
CHANGES.txt|1 +
.../apache/cassandra/thrift/CassandraServer.java | 13 ++-
.../cassandra/service/CassandraServerTest.java | 88 ++-
3 files changed, 67 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8822c3b..f309ef1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
(CASSANDRA-4765)
* fix wrong leveled compaction progress calculation (CASSANDRA-4807)
* add a close() method to CRAR to prevent leaking file descriptors
(CASSANDRA-4820)
+ * fix potential infinite loop in get_count (CASSANDRA-4833)
1.1.6
* Wait for writes on synchronous read digest mismatch (CASSANDRA-4792)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 3bf155e..39b57f3 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -428,25 +428,28 @@ public class CassandraServer implements Cassandra.Iface
Integer.MAX_VALUE);
}
-int requestedCount = predicate.slice_range.count;
+final int requestedCount = predicate.slice_range.count;
+int remaining = requestedCount;
int pages = 0;
while (true)
{
-predicate.slice_range.count = Math.min(pageSize, requestedCount);
+predicate.slice_range.count = Math.min(pageSize, Math.max(2,
remaining)); // fetch at least two columns
columns = get_slice(key, column_parent, predicate,
consistency_level);
if (columns.isEmpty())
break;
-ColumnOrSuperColumn firstColumn = columns.get(columns.size() - 1);
ByteBuffer firstName = getName(columns.get(0));
int newColumns = pages == 0 ||
!firstName.equals(predicate.slice_range.start) ? columns.size() :
columns.size() - 1;
totalCount += newColumns;
-requestedCount -= newColumns;
+// if we over-counted, just return original limit
+if (totalCount requestedCount)
+return requestedCount;
+remaining -= newColumns;
pages++;
// We're done if either:
// - We've querying the number of columns requested by the user
// - The last page wasn't full
-if (requestedCount == 0 || columns.size()
predicate.slice_range.count)
+if (remaining == 0 || columns.size() predicate.slice_range.count)
break;
else
predicate.slice_range.start =
getName(columns.get(columns.size() - 1));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
--
diff --git a/test/unit/org/apache/cassandra/service/CassandraServerTest.java
b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
index 495d697..48701de 100644
--- a/test/unit/org/apache/cassandra/service/CassandraServerTest.java
+++ b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
@@ -21,40 +21,68 @@ package org.apache.cassandra.service;
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.Util;
+import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.RowMutation;
+import org.apache.cassandra.db.filter.QueryPath;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
public class CassandraServerTest extends SchemaLoader
{
+/**
+ * test get_count() to work correctly with 'count' settings around page
size.
+ * (CASSANDRA-4833)
+ */
[6/7] add describe_splits_ex providing improved split size estimate patch by Piotr Kolaczkowski; reviewed by jbellis for CASSANDRA-4803
http://git-wip-us.apache.org/repos/asf/cassandra/blob/533bf3f6/interface/thrift/gen-java/org/apache/cassandra/thrift/CfSplit.java
--
diff --git a/interface/thrift/gen-java/org/apache/cassandra/thrift/CfSplit.java
b/interface/thrift/gen-java/org/apache/cassandra/thrift/CfSplit.java
new file mode 100644
index 000..2519f9f
--- /dev/null
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/CfSplit.java
@@ -0,0 +1,549 @@
+/**
+ * Autogenerated by Thrift Compiler (0.7.0)
+ *
+ * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING
+ */
+package org.apache.cassandra.thrift;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+
+import org.apache.commons.lang.builder.HashCodeBuilder;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.EnumMap;
+import java.util.Set;
+import java.util.HashSet;
+import java.util.EnumSet;
+import java.util.Collections;
+import java.util.BitSet;
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Represents input splits used by hadoop ColumnFamilyRecordReaders
+ */
+public class CfSplit implements org.apache.thrift.TBaseCfSplit,
CfSplit._Fields, java.io.Serializable, Cloneable {
+ private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new
org.apache.thrift.protocol.TStruct(CfSplit);
+
+ private static final org.apache.thrift.protocol.TField
START_TOKEN_FIELD_DESC = new org.apache.thrift.protocol.TField(start_token,
org.apache.thrift.protocol.TType.STRING, (short)1);
+ private static final org.apache.thrift.protocol.TField END_TOKEN_FIELD_DESC
= new org.apache.thrift.protocol.TField(end_token,
org.apache.thrift.protocol.TType.STRING, (short)2);
+ private static final org.apache.thrift.protocol.TField ROW_COUNT_FIELD_DESC
= new org.apache.thrift.protocol.TField(row_count,
org.apache.thrift.protocol.TType.I64, (short)3);
+
+ public String start_token; // required
+ public String end_token; // required
+ public long row_count; // required
+
+ /** The set of fields this struct contains, along with convenience methods
for finding and manipulating them. */
+ public enum _Fields implements org.apache.thrift.TFieldIdEnum {
+START_TOKEN((short)1, start_token),
+END_TOKEN((short)2, end_token),
+ROW_COUNT((short)3, row_count);
+
+private static final MapString, _Fields byName = new HashMapString,
_Fields();
+
+static {
+ for (_Fields field : EnumSet.allOf(_Fields.class)) {
+byName.put(field.getFieldName(), field);
+ }
+}
+
+/**
+ * Find the _Fields constant that matches fieldId, or null if its not
found.
+ */
+public static _Fields findByThriftId(int fieldId) {
+ switch(fieldId) {
+case 1: // START_TOKEN
+ return START_TOKEN;
+case 2: // END_TOKEN
+ return END_TOKEN;
+case 3: // ROW_COUNT
+ return ROW_COUNT;
+default:
+ return null;
+ }
+}
+
+/**
+ * Find the _Fields constant that matches fieldId, throwing an exception
+ * if it is not found.
+ */
+public static _Fields findByThriftIdOrThrow(int fieldId) {
+ _Fields fields = findByThriftId(fieldId);
+ if (fields == null) throw new IllegalArgumentException(Field +
fieldId + doesn't exist!);
+ return fields;
+}
+
+/**
+ * Find the _Fields constant that matches name, or null if its not found.
+ */
+public static _Fields findByName(String name) {
+ return byName.get(name);
+}
+
+private final short _thriftId;
+private final String _fieldName;
+
+_Fields(short thriftId, String fieldName) {
+ _thriftId = thriftId;
+ _fieldName = fieldName;
+}
+
+public short getThriftFieldId() {
+ return _thriftId;
+}
+
+public String getFieldName() {
+ return _fieldName;
+}
+ }
+
+ // isset id assignments
+ private static final int __ROW_COUNT_ISSET_ID = 0;
+ private BitSet __isset_bit_vector = new BitSet(1);
+
+ public static final Map_Fields,
[7/7] git commit: fix progress counting in wide row iterator patch by Piotr Koalczkowski; reviewed by jbellis for CASSANDRA-4803
fix progress counting in wide row iterator
patch by Piotr Koalczkowski; reviewed by jbellis for CASSANDRA-4803
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0bb3a064
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0bb3a064
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0bb3a064
Branch: refs/heads/trunk
Commit: 0bb3a064f3dd34823145124360c049f5d29b91ad
Parents: 72dcc29
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Oct 19 17:59:09 2012 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Oct 19 18:00:25 2012 -0500
--
.../cassandra/hadoop/ColumnFamilyRecordReader.java | 23 +-
1 files changed, 21 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0bb3a064/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
index fc90e5c..73f9786 100644
--- a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
@@ -106,7 +106,9 @@ public class ColumnFamilyRecordReader extends
RecordReaderByteBuffer, SortedMap
public float getProgress()
{
-// TODO this is totally broken for wide rows
+if (!iter.hasNext())
+return 1.0F;
+
// the progress is likely to be reported slightly off the actual but
close enough
float progress = ((float) iter.rowsRead() / totalRowCount);
return progress 1.0F ? 1.0F : progress;
@@ -423,6 +425,7 @@ public class ColumnFamilyRecordReader extends
RecordReaderByteBuffer, SortedMap
{
private PeekingIteratorPairByteBuffer, SortedMapByteBuffer,
IColumn wideColumns;
private ByteBuffer lastColumn = ByteBufferUtil.EMPTY_BYTE_BUFFER;
+private ByteBuffer lastCountedKey = ByteBufferUtil.EMPTY_BYTE_BUFFER;
private void maybeInit()
{
@@ -476,12 +479,28 @@ public class ColumnFamilyRecordReader extends
RecordReaderByteBuffer, SortedMap
if (rows == null)
return endOfData();
-totalRead++;
PairByteBuffer, SortedMapByteBuffer, IColumn next =
wideColumns.next();
lastColumn = next.right.values().iterator().next().name();
+
+maybeCountRow(next);
return next;
}
+
+/**
+ * Increases the row counter only if we really moved to the next row.
+ * @param next just fetched row slice
+ */
+private void maybeCountRow(PairByteBuffer, SortedMapByteBuffer,
IColumn next)
+{
+ByteBuffer currentKey = next.left;
+if (!currentKey.equals(lastCountedKey))
+{
+totalRead++;
+lastCountedKey = currentKey;
+}
+}
+
private class WideColumnIterator extends
AbstractIteratorPairByteBuffer, SortedMapByteBuffer, IColumn
{
private final IteratorKeySlice rows;
[4/7] git commit: missing import
missing import Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f22e2c45 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f22e2c45 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f22e2c45 Branch: refs/heads/trunk Commit: f22e2c4596d67138b3da64d3b163743cbbbf82fc Parents: 533bf3f Author: Jonathan Ellis jbel...@apache.org Authored: Fri Oct 19 18:30:43 2012 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Oct 19 18:30:43 2012 -0500 -- .../cassandra/hadoop/ColumnFamilyInputFormat.java |5 + 1 files changed, 1 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f22e2c45/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java -- diff --git a/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java b/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java index c4c6570..4de6984 100644 --- a/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java +++ b/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java @@ -43,10 +43,7 @@ import org.apache.cassandra.db.IColumn; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.dht.Range; import org.apache.cassandra.dht.Token; -import org.apache.cassandra.thrift.Cassandra; -import org.apache.cassandra.thrift.InvalidRequestException; -import org.apache.cassandra.thrift.KeyRange; -import org.apache.cassandra.thrift.TokenRange; +import org.apache.cassandra.thrift.*; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.mapred.JobConf; import org.apache.hadoop.mapred.Reporter;
[2/7] git commit: fix potential infinite loop in get_count; patch by yukim reviewed by Tyler Hobbs for CASSANDRA-4833
fix potential infinite loop in get_count; patch by yukim reviewed by Tyler
Hobbs for CASSANDRA-4833
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95fb613b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95fb613b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95fb613b
Branch: refs/heads/cassandra-1.1
Commit: 95fb613bf996c715392f9aa1f491609b6acaeff5
Parents: f22e2c4
Author: Yuki Morishita yu...@apache.org
Authored: Mon Oct 22 12:16:20 2012 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Oct 22 12:16:20 2012 -0500
--
CHANGES.txt|1 +
.../apache/cassandra/thrift/CassandraServer.java | 13 ++-
.../cassandra/service/CassandraServerTest.java | 88 ++-
3 files changed, 67 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8822c3b..f309ef1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
(CASSANDRA-4765)
* fix wrong leveled compaction progress calculation (CASSANDRA-4807)
* add a close() method to CRAR to prevent leaking file descriptors
(CASSANDRA-4820)
+ * fix potential infinite loop in get_count (CASSANDRA-4833)
1.1.6
* Wait for writes on synchronous read digest mismatch (CASSANDRA-4792)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index 3bf155e..39b57f3 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -428,25 +428,28 @@ public class CassandraServer implements Cassandra.Iface
Integer.MAX_VALUE);
}
-int requestedCount = predicate.slice_range.count;
+final int requestedCount = predicate.slice_range.count;
+int remaining = requestedCount;
int pages = 0;
while (true)
{
-predicate.slice_range.count = Math.min(pageSize, requestedCount);
+predicate.slice_range.count = Math.min(pageSize, Math.max(2,
remaining)); // fetch at least two columns
columns = get_slice(key, column_parent, predicate,
consistency_level);
if (columns.isEmpty())
break;
-ColumnOrSuperColumn firstColumn = columns.get(columns.size() - 1);
ByteBuffer firstName = getName(columns.get(0));
int newColumns = pages == 0 ||
!firstName.equals(predicate.slice_range.start) ? columns.size() :
columns.size() - 1;
totalCount += newColumns;
-requestedCount -= newColumns;
+// if we over-counted, just return original limit
+if (totalCount requestedCount)
+return requestedCount;
+remaining -= newColumns;
pages++;
// We're done if either:
// - We've querying the number of columns requested by the user
// - The last page wasn't full
-if (requestedCount == 0 || columns.size()
predicate.slice_range.count)
+if (remaining == 0 || columns.size() predicate.slice_range.count)
break;
else
predicate.slice_range.start =
getName(columns.get(columns.size() - 1));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/95fb613b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
--
diff --git a/test/unit/org/apache/cassandra/service/CassandraServerTest.java
b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
index 495d697..48701de 100644
--- a/test/unit/org/apache/cassandra/service/CassandraServerTest.java
+++ b/test/unit/org/apache/cassandra/service/CassandraServerTest.java
@@ -21,40 +21,68 @@ package org.apache.cassandra.service;
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.Util;
+import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.RowMutation;
+import org.apache.cassandra.db.filter.QueryPath;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
public class CassandraServerTest extends SchemaLoader
{
+/**
+ * test get_count() to work correctly with 'count' settings around page
size.
+ * (CASSANDRA-4833)
+
[jira] [Created] (CASSANDRA-4849) typo in tuning cassandra doc
Michael Kjellman created CASSANDRA-4849: --- Summary: typo in tuning cassandra doc Key: CASSANDRA-4849 URL: https://issues.apache.org/jira/browse/CASSANDRA-4849 Project: Cassandra Issue Type: Bug Components: Documentation website Affects Versions: 1.1.6 Reporter: Michael Kjellman Priority: Trivial http://www.datastax.com/docs/1.1/operations/tuning#tuning-bloomfilters has a typo ALTER TABLE addamsFamily WITH bloomfilter_fp_chance = 0.01; should be ALTER TABLE addamsFamily WITH bloom_filter_fp_chance = 0.01; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4837) IllegalStateException when upgrading schema
[
https://issues.apache.org/jira/browse/CASSANDRA-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481641#comment-13481641
]
Wade Simmons commented on CASSANDRA-4837:
-
If I try ro recreate the schema with the CLI with gossip disabled (I used
join_ring=false) I get exceptions because it gets NPEs trying to send the
schema over gossip.
IllegalStateException when upgrading schema
---
Key: CASSANDRA-4837
URL: https://issues.apache.org/jira/browse/CASSANDRA-4837
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.6
Environment: Linux
Reporter: Wade Simmons
Assignee: Pavel Yaskevich
I am upgrading a cluster from 1.1.2 to 1.1.6. When restarting a node with new
code, I am seeing this exception repeat in the logs:
{code}
ERROR [InternalResponseStage:21] 2012-10-19 00:41:26,794
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[InternalResponseStage:21,5,main]
java.lang.IllegalStateException: One row required, 0 found
at
org.apache.cassandra.cql3.UntypedResultSet.one(UntypedResultSet.java:50)
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:258)
at
org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:406)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:355)
at
org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:329)
at
org.apache.cassandra.service.MigrationManager$MigrationTask$1.response(MigrationManager.java:449)
at
org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:45)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
I added in some debugging logging to see what Row it was trying to load, and
I see this:
{code}
Unable to load keyspace schema:
Row(key=DecoratedKey(112573196966143652100562749464385838776,
5365676d656e7473496e746567726174696f6e54657374),
cf=ColumnFamily(schema_keyspaces -deleted at 1350665377628000- []))
{code}
The hex key translates to a schema that exists in schema_keyspaces when I
query on the rest of the cluster. I tried restarting one of the other nodes
without upgrading the jar and it restarted without exceptions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4849) typo in tuning cassandra doc
[ https://issues.apache.org/jira/browse/CASSANDRA-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4849. --- Resolution: Fixed thanks, emailed d...@datastax.com typo in tuning cassandra doc Key: CASSANDRA-4849 URL: https://issues.apache.org/jira/browse/CASSANDRA-4849 Project: Cassandra Issue Type: Bug Components: Documentation website Affects Versions: 1.1.6 Reporter: Michael Kjellman Priority: Trivial http://www.datastax.com/docs/1.1/operations/tuning#tuning-bloomfilters has a typo ALTER TABLE addamsFamily WITH bloomfilter_fp_chance = 0.01; should be ALTER TABLE addamsFamily WITH bloom_filter_fp_chance = 0.01; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481658#comment-13481658 ] Yuki Morishita commented on CASSANDRA-4813: --- This is limitation of BulkOutputFormat right now. Currently, streaming session uses (IP, counter) for its ID. Since counter is per JVM, running two or more reducers on same node streaming to one cassandra node likely cause session conflict, and I think that is causing the issue here. To resolve this, we need to change the way to distinguish each session(possibly by changing to use UUID for session ID). [~mkjellman] Do you run your reducer on top of cassandra node? If that is the case, session conflict I described above may be the cause. If not, there is another issue in your one reducer case I think. Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node. - Key: CASSANDRA-4813 URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.3, 1.1.5 Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2. Reporter: Ralph Romanos Assignee: Yuki Morishita Labels: Bulkoutputformat, Hadoop, SSTables The issue occurs when streaming simultaneously SSTables from the same node to a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot handle receiving simultaneously SSTables from the same node. However, when it receives simultaneously SSTables from two different nodes, everything works fine. As a consequence, when using BulkOutputFormat to generate SSTables and stream them to a cassandra cluster, I cannot use more than one reducer per node otherwise I get a java.io.EOFException in the tasktracker's logs and a java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4764) Clean out STREAM_STAGE vestiges
[ https://issues.apache.org/jira/browse/CASSANDRA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4764: -- Attachment: 4764-v2.txt clean v2 attached against trunk Clean out STREAM_STAGE vestiges --- Key: CASSANDRA-4764 URL: https://issues.apache.org/jira/browse/CASSANDRA-4764 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Nick Bailey Assignee: Jonathan Ellis Priority: Minor Labels: streaming Fix For: 1.2.0 beta 2 Attachments: 4764.txt, 4764-v2.txt Currently it appears as though bulk loading operations don't run in any stage. Seems like they should be running in STREAM_STAGE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481673#comment-13481673 ] Michael Kjellman commented on CASSANDRA-4813: - [~nakagawayk] Yes - the single reducer was running on top of a Cassandra node. Why would having the reducer+the cassandra node cause a session ID collision if there is only one reducer? Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node. - Key: CASSANDRA-4813 URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.3, 1.1.5 Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2. Reporter: Ralph Romanos Assignee: Yuki Morishita Labels: Bulkoutputformat, Hadoop, SSTables The issue occurs when streaming simultaneously SSTables from the same node to a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot handle receiving simultaneously SSTables from the same node. However, when it receives simultaneously SSTables from two different nodes, everything works fine. As a consequence, when using BulkOutputFormat to generate SSTables and stream them to a cassandra cluster, I cannot use more than one reducer per node otherwise I get a java.io.EOFException in the tasktracker's logs and a java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4837) IllegalStateException when upgrading schema
[
https://issues.apache.org/jira/browse/CASSANDRA-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481681#comment-13481681
]
Wade Simmons commented on CASSANDRA-4837:
-
My next attempt will be to recreate the schema on a new 1-node cluster, and
then copy the resulting sstables over.
IllegalStateException when upgrading schema
---
Key: CASSANDRA-4837
URL: https://issues.apache.org/jira/browse/CASSANDRA-4837
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.6
Environment: Linux
Reporter: Wade Simmons
Assignee: Pavel Yaskevich
I am upgrading a cluster from 1.1.2 to 1.1.6. When restarting a node with new
code, I am seeing this exception repeat in the logs:
{code}
ERROR [InternalResponseStage:21] 2012-10-19 00:41:26,794
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[InternalResponseStage:21,5,main]
java.lang.IllegalStateException: One row required, 0 found
at
org.apache.cassandra.cql3.UntypedResultSet.one(UntypedResultSet.java:50)
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:258)
at
org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:406)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:355)
at
org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:329)
at
org.apache.cassandra.service.MigrationManager$MigrationTask$1.response(MigrationManager.java:449)
at
org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:45)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
I added in some debugging logging to see what Row it was trying to load, and
I see this:
{code}
Unable to load keyspace schema:
Row(key=DecoratedKey(112573196966143652100562749464385838776,
5365676d656e7473496e746567726174696f6e54657374),
cf=ColumnFamily(schema_keyspaces -deleted at 1350665377628000- []))
{code}
The hex key translates to a schema that exists in schema_keyspaces when I
query on the rest of the cluster. I tried restarting one of the other nodes
without upgrading the jar and it restarted without exceptions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4764) Clean out STREAM_STAGE vestiges
[ https://issues.apache.org/jira/browse/CASSANDRA-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481724#comment-13481724 ] Yuki Morishita commented on CASSANDRA-4764: --- +1 Clean out STREAM_STAGE vestiges --- Key: CASSANDRA-4764 URL: https://issues.apache.org/jira/browse/CASSANDRA-4764 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Nick Bailey Assignee: Jonathan Ellis Priority: Minor Labels: streaming Fix For: 1.2.0 beta 2 Attachments: 4764.txt, 4764-v2.txt Currently it appears as though bulk loading operations don't run in any stage. Seems like they should be running in STREAM_STAGE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4417) invalid counter shard detected
[ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481729#comment-13481729 ] Chris Herron commented on CASSANDRA-4417: - bq. Quick question: do you always increment by the same value by any chance? No, sorry just happened to pick that example. We have many other log entries where both values are higher and don't differ by 1. invalid counter shard detected --- Key: CASSANDRA-4417 URL: https://issues.apache.org/jira/browse/CASSANDRA-4417 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.1 Environment: Amazon Linux Reporter: Senthilvel Rangaswamy Seeing errors like these: 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 13) and (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 1) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard What does it mean ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4837) IllegalStateException when upgrading schema
[
https://issues.apache.org/jira/browse/CASSANDRA-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481785#comment-13481785
]
Pavel Yaskevich commented on CASSANDRA-4837:
Yes, this is what I wanted to suggest.
IllegalStateException when upgrading schema
---
Key: CASSANDRA-4837
URL: https://issues.apache.org/jira/browse/CASSANDRA-4837
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.6
Environment: Linux
Reporter: Wade Simmons
Assignee: Pavel Yaskevich
I am upgrading a cluster from 1.1.2 to 1.1.6. When restarting a node with new
code, I am seeing this exception repeat in the logs:
{code}
ERROR [InternalResponseStage:21] 2012-10-19 00:41:26,794
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[InternalResponseStage:21,5,main]
java.lang.IllegalStateException: One row required, 0 found
at
org.apache.cassandra.cql3.UntypedResultSet.one(UntypedResultSet.java:50)
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:258)
at
org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:406)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:355)
at
org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:329)
at
org.apache.cassandra.service.MigrationManager$MigrationTask$1.response(MigrationManager.java:449)
at
org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:45)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
I added in some debugging logging to see what Row it was trying to load, and
I see this:
{code}
Unable to load keyspace schema:
Row(key=DecoratedKey(112573196966143652100562749464385838776,
5365676d656e7473496e746567726174696f6e54657374),
cf=ColumnFamily(schema_keyspaces -deleted at 1350665377628000- []))
{code}
The hex key translates to a schema that exists in schema_keyspaces when I
query on the rest of the cluster. I tried restarting one of the other nodes
without upgrading the jar and it restarted without exceptions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4808) nodetool doesnt work well with negative tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4808: -- Fix Version/s: (was: 1.2.0 beta 1) 1.2.0 beta 2 nodetool doesnt work well with negative tokens -- Key: CASSANDRA-4808 URL: https://issues.apache.org/jira/browse/CASSANDRA-4808 Project: Cassandra Issue Type: Bug Components: Core Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.2.0 beta 2 Attachments: 0001-CASSANDRA-4808.patch ./apache-cassandra-1.2.0-beta1-SNAPSHOT/bin/nodetool move \-2253536297082652573 Unrecognized option: -2253536297082652573 usage: java org.apache.cassandra.tools.NodeCmd --host arg command -cf,--column-family arg only take a snapshot of the specified column family -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3799) cqlsh: ASSUME should also change how values are sent to cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481792#comment-13481792 ] Aleksey Yeschenko commented on CASSANDRA-3799: -- Implementing this will require cqlsh to parse every insert, update and batch statement (incl every statement inside it), transform the values in accordance with ASSUME rules and combining the query again before sending it to Cassandra. Is it worth the effort? cqlsh: ASSUME should also change how values are sent to cassandra - Key: CASSANDRA-3799 URL: https://issues.apache.org/jira/browse/CASSANDRA-3799 Project: Cassandra Issue Type: New Feature Components: Tools Affects Versions: 1.0.3 Reporter: paul cannon Assignee: Aleksey Yeschenko Priority: Minor Labels: cqlsh Fix For: 1.3 cqlsh's ASSUME command currently only changes how query *return* values are deserialized, and never transforms user CQL text before sending to Cassandra. Apparently cassandra-cli also changes how values are interpreted and marshaled for Cassandra, so user expectation is that cqlsh should also do this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4815) Make CQL work naturally with wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481793#comment-13481793
]
Edward Capriolo commented on CASSANDRA-4815:
{noformat}Not in CQL-the-language, and I don't think even in cqlsh-the-utility.
I understand the appeal of the convenience, but the abstraction leakage it
would introduce threatens to undo all the work we're doing to make CQL3
something you can use on its own terms.{noformat}
But what if I want to think of Cassandra as a memcache not a relational
database. This is one of my ticket points, CQL should support all the use cases
it can. You are calling it abstraction leakage but I think of it as a natural
way with working with Cassandra. But I do agree that SELECTS are better then
cli 'get' in most cases. What is missing is the SET side.
{noformat}
CREATE TABLE schemaless (
key blob,
column blob,
value blob,
PRIMARY KEY (key, column)
) WITH COMPACT STORAGE
{noformat}
Now to insert into this table I need to format everything as hex.
INSERT INTO SCHEMALESS (key,column,value) VALUES ('HEX','HEX','HEX');
The CLI has many useful functions like ascii(' '), or utf8(' '). Assume does
not seem to have an effect here. This is discussed in CASSANDRA-3799.
Make CQL work naturally with wide rows
--
Key: CASSANDRA-4815
URL: https://issues.apache.org/jira/browse/CASSANDRA-4815
Project: Cassandra
Issue Type: Wish
Reporter: Edward Capriolo
I find that CQL3 is quite obtuse and does not provide me a language useful
for accessing my data. First, lets point out how we should design Cassandra
data.
1) Denormalize
2) Eliminate seeks
3) Design for read
4) optimize for blind writes
So here is a schema that abides by these tried and tested rules large
production uses are employing today.
Say we have a table of movie objects:
Movie
Name
Description
- tags (string)
- credits composite(role string, name string )
-1 likesToday
-1 blacklisted
The above structure is a movie notice it hold a mix of static and dynamic
columns, but the other all number of columns is not very large. (even if it
was larger this is OK as well) Notice this table is not just
a single one to many relationship, it has 1 to 1 data and it has two sets of
1 to many data.
The schema today is declared something like this:
create column family movies
with default_comparator=UTF8Type and
column_metadata =
[
{column_name: blacklisted, validation_class: int},
{column_name: likestoday, validation_class: long},
{column_name: description, validation_class: UTF8Type}
];
We should be able to insert data like this:
set ['Cassandra Database, not looking for a seQL']['blacklisted']=1;
set ['Cassandra Database, not looking for a seQL']['likesToday']=34;
set ['Cassandra Database, not looking for a
seQL']['credits-dir']='director:asf';
set ['Cassandra Database, not looking for a
seQL']['credits-jir]='jiraguy:bob';
set ['Cassandra Database, not looking for a seQL']['tags-action']='';
set ['Cassandra Database, not looking for a seQL']['tags-adventure']='';
set ['Cassandra Database, not looking for a seQL']['tags-romance']='';
set ['Cassandra Database, not looking for a seQL']['tags-programming']='';
This is the correct way to do it. 1 seek to find all the information related
to a movie. As long as this row does
not get large there is no reason to optimize by breaking data into other
column families. (Notice you can not transpose this
because movies is two 1-to-many relationships of potentially different types)
Lets look at the CQL3 way to do this design:
First, contrary to the original design of cassandra CQL does not like wide
rows. It also does not have a good way to dealing with dynamic rows together
with static rows either.
You have two options:
Option 1: lose all schema
create table movies ( name string, column blob, value blob, primary
key(name)) with compact storage.
This method is not so hot we have not lost all our validators, and by the way
you have to physically shutdown everything and rename files and recreate your
schema if you want to inform cassandra that a current table should be
compact. This could at very least be just a metadata change. Also you can not
add column schema either.
Option 2 Normalize (is even worse)
create table movie (name String, description string, likestoday int,
blacklisted int);
create table movecredits( name string, role string, personname string,
primary key(name,role) );
create table movetags( name string, tag string, primary key (name,tag) );
This is a terrible design, of the 4 key characteristics how cassandra data
should be designed it fails 3:
It does not:
1) Denormalize
2) Eliminate seeks
3) Design for read
Why is
[jira] [Commented] (CASSANDRA-3799) cqlsh: ASSUME should also change how values are sent to cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481800#comment-13481800
]
Edward Capriolo commented on CASSANDRA-3799:
It seems like the entire statement could be processed server side.
INSERT INTO TABLE (name,column,value) VALUES
(ascii('yo'),ascii('donthis'),ascii('onserver'));
I can not image a JDBC client would parse something like
SELECT trim(concat('a','b')) from table;
on the client side.
cqlsh: ASSUME should also change how values are sent to cassandra
-
Key: CASSANDRA-3799
URL: https://issues.apache.org/jira/browse/CASSANDRA-3799
Project: Cassandra
Issue Type: New Feature
Components: Tools
Affects Versions: 1.0.3
Reporter: paul cannon
Assignee: Aleksey Yeschenko
Priority: Minor
Labels: cqlsh
Fix For: 1.3
cqlsh's ASSUME command currently only changes how query *return* values are
deserialized, and never transforms user CQL text before sending to Cassandra.
Apparently cassandra-cli also changes how values are interpreted and
marshaled for Cassandra, so user expectation is that cqlsh should also do
this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: clean up partitioner comments
Updated Branches:
refs/heads/trunk dd8a3c450 - 8f3d9b837
clean up partitioner comments
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8f3d9b83
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8f3d9b83
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8f3d9b83
Branch: refs/heads/trunk
Commit: 8f3d9b8371fa7c5dea83d45d83ec7fe4911a96c0
Parents: dd8a3c4
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Oct 22 16:15:01 2012 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Oct 22 16:15:08 2012 -0500
--
NEWS.txt |5 +
conf/cassandra.yaml| 11 ---
.../apache/cassandra/io/sstable/SSTableReader.java |4 ++--
3 files changed, 11 insertions(+), 9 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8f3d9b83/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 920940f..3736a2f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -20,6 +20,11 @@ Upgrading
the ability to read data files from Cassandra versions at least
back to 0.6, so a non-rolling upgrade remains possible with just
one step.
+- The default partitioner for new clusters is Murmur3Partitioner,
+ which is about 10% faster for index-intensive workloads. Partitioners
+ cannot be changed once data is in the cluster, however, so if you are
+ switching to the 1.2 cassandra.yaml, you should change this to
+ RandomPartitioner or whatever your old partitioner was.
- If you using counters and upgrading from a version prior to
1.1.6, you should drain existing Cassandra nodes prior to the
upgrade to prevent overcount during commitlog replay (see
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8f3d9b83/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 37fc572..a79e150 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -62,16 +62,13 @@ authority: org.apache.cassandra.auth.AllowAllAuthority
# The partitioner is responsible for distributing rows (by key) across
# nodes in the cluster. Any IPartitioner may be used, including your
# own as long as it is on the classpath. Out of the box, Cassandra
-# provides org.apache.cassandra.dht.RandomPartitioner
-# org.apache.cassandra.dht.ByteOrderedPartitioner,
-# org.apache.cassandra.dht.OrderPreservingPartitioner (deprecated),
-# and org.apache.cassandra.dht.CollatingOrderPreservingPartitioner
-# (deprecated).
+# provides org.apache.cassandra.dht.{Murmur3Partitioner, RandomPartitioner
+# ByteOrderedPartitioner, OrderPreservingPartitioner (deprecated)}.
#
# - RandomPartitioner distributes rows across the cluster evenly by md5.
-# When in doubt, this is the best option.
+# This is the default prior to 1.2 and is retained for compatibility.
# - Murmur3Partitioner is similar to RandomPartioner but uses Murmur3_128
-# Hash Function instead of md5
+# Hash Function instead of md5. When in doubt, this is the best option.
# - ByteOrderedPartitioner orders rows lexically by key bytes. BOP allows
# scanning rows in key order, but the ordering can generate hot spots
# for sequential insertion workloads.
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8f3d9b83/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index aee576e..1514a2c 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -172,8 +172,8 @@ public class SSTableReader extends SSTable
String partitionerName = partitioner.getClass().getCanonicalName();
if (sstableMetadata.partitioner != null
!partitionerName.equals(sstableMetadata.partitioner))
{
-logger.warn(Changing paritioner on a existing cluster can cause
data loose, Please verify your partitioner in cassandra.yaml);
-logger.error(String.format(Cannot open %s because partitioner
does not match %s != %s,descriptor, sstableMetadata.partitioner,
partitionerName));
+logger.error(String.format(Cannot open %s because partitioner
does not match %s != %s. Please verify your partitioner in cassandra.yaml,
+ descriptor,
sstableMetadata.partitioner, partitionerName));
System.exit(1);
}
[jira] [Commented] (CASSANDRA-4843) When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start
[ https://issues.apache.org/jira/browse/CASSANDRA-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481806#comment-13481806 ] Jonathan Ellis commented on CASSANDRA-4843: --- bq. maybe it's time to change the behavior so that if a partitioner is saved in the system table, we use that Partitioner is saved per-sstable so we can do this panic doublecheck, but you have to know (or think you know) the partitioner before you can actually open up a system table. I could be wrong, but I don't think it's a quick fix. In the meantime, I've updated the comments and error message in 8f3d9b8371fa7c5dea83d45d83ec7fe4911a96c0. When upgrading from 1.1.6 to 1.20 change in partitioner causes nodes not to start - Key: CASSANDRA-4843 URL: https://issues.apache.org/jira/browse/CASSANDRA-4843 Project: Cassandra Issue Type: Bug Reporter: Edward Capriolo Fix For: 1.2.0 beta 2 ERROR 10:17:20,341 Cannot open /home/edward/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hf-1 because partitioner does not match org.apache.cassandra.dht.RandomPartitioner != org.apache.cassandra.dht.Murmur3Partitioner This is because 1.2 has a new default partitioner, why are we changing the default? Is this wise? The current partitioner has been rock solid for years. Should the previously known partition be stored in the schema like the previously know seed nodes, and schema? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4750) Add jmx/nodetool methods to enable/disable hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4750: -- Priority: Minor (was: Major) Fix Version/s: (was: 1.2.0 beta 2) 1.2.1 Issue Type: New Feature (was: Improvement) Add jmx/nodetool methods to enable/disable hinted handoff - Key: CASSANDRA-4750 URL: https://issues.apache.org/jira/browse/CASSANDRA-4750 Project: Cassandra Issue Type: New Feature Reporter: Brandon Williams Assignee: Alexey Zotov Priority: Minor Labels: hintedhandoff, nodetool Fix For: 1.2.1 Attachments: cassandra-1.1-4750-enable_disable_hh.txt, cassandra-1.2-4750-enable_disable_hh.txt Title says it all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4783) AE in cql3 select
[
https://issues.apache.org/jira/browse/CASSANDRA-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481811#comment-13481811
]
Jonathan Ellis commented on CASSANDRA-4783:
---
+1
AE in cql3 select
-
Key: CASSANDRA-4783
URL: https://issues.apache.org/jira/browse/CASSANDRA-4783
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 1.2.0 beta 2
Attachments: 0001-Fix-validation-of-IN-queries.txt,
0002-Fix-mixing-list-set-operation-and-regular-updates.txt
Caused by 'select * from foo where key='blah' and column in (...)
{noformat}
ERROR 18:35:46,169 Exception in thread Thread[Thrift:11,5,main]
java.lang.AssertionError
at
org.apache.cassandra.cql3.statements.SelectStatement.getRequestedColumns(SelectStatement.java:443)
at
org.apache.cassandra.cql3.statements.SelectStatement.makeFilter(SelectStatement.java:312)
at
org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:200)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:125)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:61)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:130)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:138)
at
org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1658)
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3721)
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3709)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:196)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
Causes cqlsh to hang forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4788) streaming can put files in the wrong location
[ https://issues.apache.org/jira/browse/CASSANDRA-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481812#comment-13481812 ] Brandon Williams commented on CASSANDRA-4788: - Thanks. +1 streaming can put files in the wrong location - Key: CASSANDRA-4788 URL: https://issues.apache.org/jira/browse/CASSANDRA-4788 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 beta 1 Reporter: Brandon Williams Assignee: Yuki Morishita Fix For: 1.2.0 beta 2 Attachments: 4788.txt Some, but not all streaming incorrectly puts files in the top level data directory. Easiest way to repro that I've seen is bootstrap where it happens 100% of the time, but other operations like move and repair seem to do the right thing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4793) Commitlog files replayed but not in the order of their ids
[ https://issues.apache.org/jira/browse/CASSANDRA-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4793. --- Resolution: Fixed Assignee: Fabien Rousseau belatedly marking resolved Commitlog files replayed but not in the order of their ids -- Key: CASSANDRA-4793 URL: https://issues.apache.org/jira/browse/CASSANDRA-4793 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: Fabien Rousseau Assignee: Fabien Rousseau Priority: Minor Labels: commitlog Fix For: 1.2.0 beta 2 Attachments: 4793-potential-patch-for-correctly-ordering-commit-log-fi.patch, 4793-trunk-potential-patch-for-correctly-ordering-commit-log-fi.patch I noticed that the commitlog files were not replayed in the order of their ids. It seems that they are sorted by last modification date before being replayed, but this does not corresponds to their ids. Moreover, last modification date is changed when a file is copied, so, this could also change the order of archived commitlogs. Maybe it's safer to sort them using the id in the file name ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: remove vestiges of STREAM stage patch by jbellis; reviewed by yukim for CASSANDRA-4764
Updated Branches:
refs/heads/trunk 8f3d9b837 - 69ad77d8c
remove vestiges of STREAM stage
patch by jbellis; reviewed by yukim for CASSANDRA-4764
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/69ad77d8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/69ad77d8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/69ad77d8
Branch: refs/heads/trunk
Commit: 69ad77d8c6a1e2c78255d54477798d5619a3a84d
Parents: 8f3d9b8
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Oct 22 16:27:47 2012 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Oct 22 16:27:47 2012 -0500
--
.../org/apache/cassandra/concurrent/Stage.java |2 --
.../apache/cassandra/concurrent/StageManager.java |1 -
.../org/apache/cassandra/net/MessagingService.java |1 -
.../apache/cassandra/service/StorageService.java | 11 ++-
4 files changed, 2 insertions(+), 13 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/69ad77d8/src/java/org/apache/cassandra/concurrent/Stage.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/Stage.java
b/src/java/org/apache/cassandra/concurrent/Stage.java
index 062bbe6..f2907e2 100644
--- a/src/java/org/apache/cassandra/concurrent/Stage.java
+++ b/src/java/org/apache/cassandra/concurrent/Stage.java
@@ -21,7 +21,6 @@ public enum Stage
{
READ,
MUTATION,
-STREAM,
GOSSIP,
REQUEST_RESPONSE,
ANTI_ENTROPY,
@@ -41,7 +40,6 @@ public enum Stage
case MIGRATION:
case MISC:
case TRACING:
-case STREAM:
case INTERNAL_RESPONSE:
return internal;
case MUTATION:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/69ad77d8/src/java/org/apache/cassandra/concurrent/StageManager.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/StageManager.java
b/src/java/org/apache/cassandra/concurrent/StageManager.java
index 8359346..7ca45f4 100644
--- a/src/java/org/apache/cassandra/concurrent/StageManager.java
+++ b/src/java/org/apache/cassandra/concurrent/StageManager.java
@@ -51,7 +51,6 @@ public class StageManager
stages.put(Stage.INTERNAL_RESPONSE,
multiThreadedStage(Stage.INTERNAL_RESPONSE,
Runtime.getRuntime().availableProcessors()));
stages.put(Stage.REPLICATE_ON_WRITE,
multiThreadedConfigurableStage(Stage.REPLICATE_ON_WRITE,
getConcurrentReplicators(), MAX_REPLICATE_ON_WRITE_TASKS));
// the rest are all single-threaded
-stages.put(Stage.STREAM, new
JMXEnabledThreadPoolExecutor(Stage.STREAM));
stages.put(Stage.GOSSIP, new
JMXEnabledThreadPoolExecutor(Stage.GOSSIP));
stages.put(Stage.ANTI_ENTROPY, new
JMXEnabledThreadPoolExecutor(Stage.ANTI_ENTROPY));
stages.put(Stage.MIGRATION, new
JMXEnabledThreadPoolExecutor(Stage.MIGRATION));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/69ad77d8/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index 06a0270..41a42ec 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -133,7 +133,6 @@ public final class MessagingService implements
MessagingServiceMBean
put(Verb.READ, Stage.READ);
put(Verb.REQUEST_RESPONSE, Stage.REQUEST_RESPONSE);
put(Verb.STREAM_REPLY, Stage.MISC); // TODO does this really belong on
misc? I've just copied old behavior here
-put(Verb.STREAM_REQUEST, Stage.STREAM);
put(Verb.RANGE_SLICE, Stage.READ);
put(Verb.BOOTSTRAP_TOKEN, Stage.MISC);
put(Verb.TREE_REQUEST, Stage.ANTI_ENTROPY);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/69ad77d8/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 4cc8581..58ce112 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3352,15 +3352,8 @@ public class StorageService implements
IEndpointStateChangeSubscriber, StorageSe
}
};
-StageManager.getStage(Stage.STREAM).execute(new Runnable()
-{
-public void run()
-{
-// TODO
git commit: make TRACE verb droppable patch by David Alves; reviewed by jbellis for CASSANDRA-4672
Updated Branches: refs/heads/trunk 69ad77d8c - 269c0cdac make TRACE verb droppable patch by David Alves; reviewed by jbellis for CASSANDRA-4672 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/269c0cda Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/269c0cda Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/269c0cda Branch: refs/heads/trunk Commit: 269c0cdac3d0457e6f00f6f055b5aa280a21e8b2 Parents: 69ad77d Author: Jonathan Ellis jbel...@apache.org Authored: Mon Oct 22 16:31:52 2012 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Mon Oct 22 16:31:52 2012 -0500 -- CHANGES.txt|1 + .../org/apache/cassandra/net/MessagingService.java |1 + 2 files changed, 2 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/269c0cda/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 0f7caba..fc37d79 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.2-beta2 + * make TRACE verb droppable (CASSANDRA-4672) * fix BulkLoader recognition of CQL3 columnfamilies (CASSANDRA-4755) * Sort commitlog segments for replay by id instead of mtime (CASSANDRA-4793) * Make hint delivery asynchronous (CASSANDRA-4761) http://git-wip-us.apache.org/repos/asf/cassandra/blob/269c0cda/src/java/org/apache/cassandra/net/MessagingService.java -- diff --git a/src/java/org/apache/cassandra/net/MessagingService.java b/src/java/org/apache/cassandra/net/MessagingService.java index 41a42ec..263d5c0 100644 --- a/src/java/org/apache/cassandra/net/MessagingService.java +++ b/src/java/org/apache/cassandra/net/MessagingService.java @@ -269,6 +269,7 @@ public final class MessagingService implements MessagingServiceMBean * drop internal messages like bootstrap or repair notifications. */ public static final EnumSetVerb DROPPABLE_VERBS = EnumSet.of(Verb.BINARY, + Verb._TRACE, Verb.MUTATION, Verb.READ_REPAIR, Verb.READ,
[jira] [Updated] (CASSANDRA-4842) DateType in Column MetaData causes server crash
[
https://issues.apache.org/jira/browse/CASSANDRA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4842:
--
Fix Version/s: (was: 1.2.0 beta 2)
1.2.0
DateType in Column MetaData causes server crash
---
Key: CASSANDRA-4842
URL: https://issues.apache.org/jira/browse/CASSANDRA-4842
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.0
Environment: All
Reporter: Russell Bradberry
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.1.7, 1.2.0
when creating a column family with column metadata containing a date, there
is a server crash that will prevent startup.
To recreate from the cli:
{code}
create keyspace test;
use test;
create column family foo
with column_type = 'Standard'
and comparator = 'CompositeType(LongType,DateType)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'UTF8Type'
and column_metadata = [
{ column_name : '1234:1350695443433', validation_class : BooleanType}
];
{code}
Produces this error in the logs:
{code}
ERROR 21:11:18,795 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:373)
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:194)
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:141)
at
org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:931)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3410)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:3398)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException:
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:369)
... 11 more
Caused by: org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.fromString(AbstractCompositeType.java:213)
at
org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:257)
at
org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1318)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1250)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:299)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:434)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:346)
at
org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:217)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.text.ParseException: Unable to parse the date: 2012-10-19 21
at org.apache.commons.lang.time.DateUtils.parseDate(DateUtils.java:285)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:113)
... 14 more
ERROR 21:11:18,795 Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.db.marshal.MarshalException: unable to coerce
'2012-10-19 21' to a formatted date (long)
at
org.apache.cassandra.db.marshal.DateType.dateStringToTimestamp(DateType.java:117)
at org.apache.cassandra.db.marshal.DateType.fromString(DateType.java:85)
at
git commit: fix streaming to wrong directory
Updated Branches: refs/heads/trunk 269c0cdac - 8c99a3769 fix streaming to wrong directory Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c99a376 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c99a376 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c99a376 Branch: refs/heads/trunk Commit: 8c99a376980b90faf7b1ca1aa33d9fde0a356cda Parents: 269c0cd Author: Yuki Morishita yu...@apache.org Authored: Mon Oct 22 16:46:31 2012 -0500 Committer: Yuki Morishita yu...@apache.org Committed: Mon Oct 22 16:46:43 2012 -0500 -- .../org/apache/cassandra/streaming/StreamIn.java |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c99a376/src/java/org/apache/cassandra/streaming/StreamIn.java -- diff --git a/src/java/org/apache/cassandra/streaming/StreamIn.java b/src/java/org/apache/cassandra/streaming/StreamIn.java index a0d605d..b8f6b90 100644 --- a/src/java/org/apache/cassandra/streaming/StreamIn.java +++ b/src/java/org/apache/cassandra/streaming/StreamIn.java @@ -83,7 +83,7 @@ public class StreamIn Directories.DataDirectory localDir = Directories.getLocationCapableOfSize(remote.size); if (localDir == null) throw new RuntimeException(Insufficient disk space to store + remote.size + bytes); -Descriptor localdesc = Descriptor.fromFilename(cfStore.getTempSSTablePath(localDir.location)); +Descriptor localdesc = Descriptor.fromFilename(cfStore.getTempSSTablePath(cfStore.directories.getLocationForDisk(localDir.location))); return new PendingFile(localdesc, remote); }
[Cassandra Wiki] Trivial Update of JoannaItp by JoannaItp
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The JoannaItp page has been changed by JoannaItp: http://wiki.apache.org/cassandra/JoannaItp New page: Name: Joanna ShellBRAge: 22BRCountry: ItaliaBRTown: Averara BRZIP: 24010BRStreet: Strada Provinciale 65 48BRBRFeel free to visit my site; [[http://paddlingiowa.com/phpBB/privmsg.php?mode=postu=146845sid=4f489cca18d93c4edb21e72e84395129#http://Videos.guidehorse.com/uprofile.php?UID=408#austin divorce lawyer|main page]]
