Compressing data types

2011-06-21 Thread Timo Nentwig
Hi! Just wondering why this doesn't already exist: wouldn't it make sense to have decorating data types that compress (gzip, snappy) other data types (esp. UTF8Type, AsciiType) transparently? -tcn

Re: sstable2json2sstable bug with json data stored

2011-06-16 Thread Timo Nentwig
On 6/15/11 17:41, Timo Nentwig wrote: (json can likely be boiled down even more...) Any JSON (well, probably anything with quotes...) breaks it: { 74657374: [[data, {foo:bar}, 1308209845388000]] } [default@foo] set transactions[test][data]='{foo:bar}'; I feared that storing data

Re: sstable2json2sstable bug with json data stored

2011-06-16 Thread Timo Nentwig
Guess this could simply be done in the quote() method. { 74657374: [[data, {foo:bar}, 1308209845388000]] } Does this work? { 74657374: [[data, {foo:bar}, 1308209845388000]] } -sd On Thu, Jun 16, 2011 at 9:49 AM, Timo Nentwigtimo.nent...@toptarif.de wrote: On 6/15/11 17:41, Timo Nentwig wrote

Re: sstable2json2sstable bug with json data stored

2011-06-16 Thread Timo Nentwig
On 6/16/11 10:12, Timo Nentwig wrote: On 6/16/11 10:06, Sasha Dolgy wrote: The JSON you are showing below is an export from cassandra? Yes. Just posted the solution: https://issues.apache.org/jira/browse/CASSANDRA-2780?focusedCommentId=13050274page

sstable2json2sstable bug with json data stored

2011-06-15 Thread Timo Nentwig
Hi! Couldn't google anybody having yet experienced this, so I do (0.8): { foo:{ foo:{ foo:bar, foo:bar, foo:bar, foo:, foo:bar, foo:bar, id:123456 } }, foo:null } (json can likely be boiled down even more...) [default@foo] set

CQL/JDBC: Cannot locate cassandra.yaml

2011-06-05 Thread Timo Nentwig
$ CLASSPATH=~/sqlshell/lib/ ~/sqlshell/bin/sqlshell org.apache.cassandra.cql.jdbc.CassandraDriver,jdbc:cassandra:foo/bar@localhost:9160/ks 2011-06-05 16:21:54,452 INFO [main] org.apache.cassandra.cql.jdbc.Connection - Connected to localhost:9160 2011-06-05 16:21:54,517 ERROR [main]

Re: CQL/JDBC: Cannot locate cassandra.yaml

2011-06-05 Thread Timo Nentwig
On 6/5/11 16:26, Timo Nentwig wrote: $ CLASSPATH=~/sqlshell/lib/ ~/sqlshell/bin/sqlshell org.apache.cassandra.cql.jdbc.CassandraDriver,jdbc:cassandra:foo/bar@localhost:9160/ks 2011-06-05 16:21:54,452 INFO [main] org.apache.cassandra.cql.jdbc.Connection - Connected to localhost:9160 2011-06-05

Quorum + Token range confusion

2011-05-25 Thread Timo Nentwig
Hi! 5 nodes, replication factor of 2, fifth node down. As long as I write a single column with hector or pelops, it works. With 2 columns it fails because there are supposed to few servers to reach quorum. Confusing. If I decommission the fifth node with nodetool quorum works again and I can

Re: Quorum + Token range confusion

2011-05-25 Thread Timo Nentwig
On 5/25/11 13:45, Watanabe Maki wrote: I think I don't get your situation yet, but if you use RF=2, CL=QUORUM is identical with CL=ALL. Does it explain your experience? If it was CL=ALL, it would explain it, however I does not explain why it works when I decommission one node. RF=2 means

Re: Quorum + Token range confusion

2011-05-25 Thread Timo Nentwig
On 5/25/11 14:08, Timo Nentwig wrote: On 5/25/11 13:45, Watanabe Maki wrote: I think I don't get your situation yet, but if you use RF=2, CL=QUORUM is identical with CL=ALL. Does it explain your experience? If it was CL=ALL, it would explain it, however I does not explain why it works when

Re: suggestion: sstable2json to ignore TTL

2011-05-05 Thread Timo Nentwig
On Apr 27, 2011, at 16:59, Timo Nentwig wrote: On Apr 27, 2011, at 16:52, Edward Capriolo wrote: The method being private is not a deal-breaker.While not good software engineering practice you can copy and paste the code and renamed the class SSTable2MyJson or whatever. Sure I can do

Re: suggestion: sstable2json to ignore TTL

2011-04-29 Thread Timo Nentwig
On Apr 27, 2011, at 17:10, Edward Capriolo wrote: I would think most people who watch dev watch this list. http://wiki.apache.org/cassandra/HowToContribute So, here it is: https://issues.apache.org/jira/browse/CASSANDRA-2582

suggestion: sstable2json to ignore TTL

2011-04-27 Thread Timo Nentwig
Hi! What about a simple option for sstable2json to not print out expiration TTL+LocalDeletionTime (maybe even ignore isMarkedForDelete)? I want to move old data from a live cluster (with TTL) to an archive cluster (-data does not expire there). BTW is there a smarter way to do this? Actually

Re: suggestion: sstable2json to ignore TTL

2011-04-27 Thread Timo Nentwig
On Apr 27, 2011, at 15:58, Edward Capriolo wrote: Hacking a separate copy of SSTable2json is trivial. Just look for the section of the code that writes the data and change what it writes. If I did. The method's private... you can make it a knob --nottl then it could be included in Cassandra

Re: suggestion: sstable2json to ignore TTL

2011-04-27 Thread Timo Nentwig
On Apr 27, 2011, at 16:52, Edward Capriolo wrote: The method being private is not a deal-breaker.While not good software engineering practice you can copy and paste the code and renamed the class SSTable2MyJson or whatever. Sure I can do this but I'd like to have it just available in the

cassandra-cli (output) broken for super columns

2011-02-08 Thread Timo Nentwig
This is not what it's supposed to be like, is it? [default@foo] get foo[page-field]; = (super_column=20110208, (column=82f4c650-2d53-11e0-a08b-58b035f3f60d, value=msg1, timestamp=1297159430471000)

Re: cassandra-cli (output) broken for super columns

2011-02-08 Thread Timo Nentwig
On Feb 8, 2011, at 13:41, Stephen Connolly wrote: On 8 February 2011 10:38, Timo Nentwig timo.nent...@toptarif.de wrote: This is not what it's supposed to be like, is it? Looks alright: [default@foo] get foo[page-field]; = (super_column=20110208, (column=82f4c650-2d53-11e0-a08b

Re: Multiple indexes - how does Cassandra handle these internally?

2011-01-21 Thread Timo Nentwig
On Jan 21, 2011, at 13:55, buddhasystem wrote: if I use multiple secondary indexes in the query, what will Cassandra do? Some examples say it will index on first EQ and then loop on others. Does it ever do a proper index product to avoid inner loops? Just asked the same question on the

Re: Multiple indexes - how does Cassandra handle these internally?

2011-01-21 Thread Timo Nentwig
On Jan 21, 2011, at 16:46, Maxim Potekhin wrote: But Timo, this is even more mysterious! If both conditions are met, at least something must be returned in the second query. Have you tried this in CLI? That would allow you to at least alleviate client concerns. I did this on the CLI only so

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

2011-01-19 Thread Timo Nentwig
On Jan 18, 2011, at 18:53, Nate McCall wrote: When doing mixed types on slicing operations, you should use ByteArraySerializer and handle the conversions by hand. We have an issue open for making this more graceful. Pls. have a look at

cassandra-cli: where a and b (works) vs. where b and a (doesn't)

2011-01-18 Thread Timo Nentwig
I put a secondary index on rc (IntegerType) and user_agent (AsciiType). Don't understand this bevahiour at all, can somebody explain? [default@tracking] get crawler where user_agent=foo and rc=200; 0 Row Returned. [default@tracking] get crawler where rc=200 and user_agent=foo;

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

2011-01-18 Thread Timo Nentwig
); indexedSlicesQuery.addEqualsExpression(state, UT); indexedSlicesQuery.setColumnFamily(users); indexedSlicesQuery.setStartKey(); QueryResultOrderedRowsString, String, String result = indexedSlicesQuery.execute(); Aaron On 18/01/2011, at 11:54 PM, Timo Nentwig timo.nent...@toptarif.de wrote: I put a secondary index on rc

Re: cassandra-cli: where a and b (works) vs. where b and a (doesn't)

2011-01-18 Thread Timo Nentwig
On Jan 18, 2011, at 12:05, Timo Nentwig wrote: On Jan 18, 2011, at 12:02, Aaron Morton wrote: Does wrapping foo in single quotes help? No. Also, does this help http://www.datastax.com/blog/whats-new-cassandra-07-secondary-indexes Actually this doesn't even compile because

Re: Adding secondary index: java.lang.ArithmeticException: / by zero

2010-12-24 Thread Timo Nentwig
On Dec 23, 2010, at 12:34, Timo Nentwig wrote: On Dec 23, 2010, at 9:34, Timo Nentwig wrote: I was about to add a secondary index (which apparently failed) to existing data. When I restarted the node it crashed (!) with: It crashed because it ran out of heap space (2G). So I increased

Adding secondary index: java.lang.ArithmeticException: / by zero

2010-12-23 Thread Timo Nentwig
I was about to add a secondary index (which apparently failed) to existing data. When I restarted the node it crashed (!) with: INFO 09:21:36,510 Opening /var/lib/cassandra/data/test/tracking.6b6579-tmp-e-1 ERROR 09:21:36,512 Exception encountered during startup. java.lang.ArithmeticException:

Re: Adding secondary index: java.lang.ArithmeticException: / by zero

2010-12-23 Thread Timo Nentwig
On Dec 23, 2010, at 9:34, Timo Nentwig wrote: I was about to add a secondary index (which apparently failed) to existing data. When I restarted the node it crashed (!) with: It crashed because it ran out of heap space (2G). So I increased to 3.5G but after a whlie it's caught in full GC

Re: java.io.IOException: No space left on device

2010-12-22 Thread Timo Nentwig
On Dec 22, 2010, at 16:20, Peter Schuller wrote: And the data could be more evenly balanced, obviously. However the nodes fails to startup because due of lacking disk space (instead of starting up and denies further writes it appears to try to process the [6.6G!] commit logs). So, I

Re: java.io.IOException: No space left on device

2010-12-22 Thread Timo Nentwig
On Dec 22, 2010, at 16:20, Peter Schuller wrote: In any case: Monitoring disk-space is very very important. So, why doesn't cassandra monitor it itself and stop accepting writes if it runs out of space?

Re: Memory leak with Sun Java 1.6 ?

2010-12-14 Thread Timo Nentwig
:52 AM, Timo Nentwig timo.nent...@toptarif.de wrote: On Dec 10, 2010, at 19:37, Peter Schuller wrote: To cargo cult it: Are you running a modern JVM? (Not e.g. openjdk b17 in lenny or some such.) If it is a JVM issue, ensuring you're using a reasonably recent JVM is probably much easier

Re: Memory leak with Sun Java 1.6 ?

2010-12-14 Thread Timo Nentwig
On Dec 14, 2010, at 15:31, Timo Nentwig wrote: On Dec 14, 2010, at 14:41, Jonathan Ellis wrote: This is A row has grown too large section from that troubleshooting guide. Why? This is what a typical row (?) looks like: [defa...@test] list tracking limit 1; --- RowKey

Re: Memory leak with Sun Java 1.6 ?

2010-12-14 Thread Timo Nentwig
On Dec 14, 2010, at 19:38, Peter Schuller wrote: For debugging purposes you may want to switch Cassandra to standard IO mode instead of mmap. This will have a performance-penalty, but the virtual/resident sizes won't be polluted with mmap():ed data. Already did so. It *seems* to run more

Re: Memory leak with Sun Java 1.6 ?

2010-12-12 Thread Timo Nentwig
On Dec 10, 2010, at 19:37, Peter Schuller wrote: To cargo cult it: Are you running a modern JVM? (Not e.g. openjdk b17 in lenny or some such.) If it is a JVM issue, ensuring you're using a reasonably recent JVM is probably much easier than to start tracking it down... I had OOM problems

Re: Quorum: killing 1 out of 3 server kills the cluster (?)

2010-12-10 Thread Timo Nentwig
nodetool repair it held more data than all other nodes. So it copied data from the other nodes to this one? I assumed that data is replicated to q nodes not to all, is quorum 'only' about consistency and not about saving storage space? - Tyler On Thu, Dec 9, 2010 at 11:26 AM, Timo Nentwig

Quorum: killing 1 out of 3 server kills the cluster (?)

2010-12-09 Thread Timo Nentwig
Hi! I've 3 servers running (0.7rc1) with a replication_factor of 2 and use quorum for writes. But when I shut down one of them UnavailableExceptions are thrown. Why is that? Isn't that the sense of quorum and a fault-tolerant DB that it continues with the remaining 2 nodes and redistributes

Re: Quorum: killing 1 out of 3 server kills the cluster (?)

2010-12-09 Thread Timo Nentwig
. /d On Thu, Dec 9, 2010 at 4:40 PM, Timo Nentwig timo.nent...@toptarif.de wrote: Hi! I've 3 servers running (0.7rc1) with a replication_factor of 2 and use quorum for writes. But when I shut down one of them UnavailableExceptions are thrown. Why is that? Isn't that the sense of quorum

Re: Quorum: killing 1 out of 3 server kills the cluster (?)

2010-12-09 Thread Timo Nentwig
, 2010 at 5:00 PM, Timo Nentwig timo.nent...@toptarif.de wrote: On Dec 9, 2010, at 16:50, Daniel Lundin wrote: Quorum is really only useful when RF 2, since the for a quorum to succeed RF/2+1 replicas must be available. 2/2+1==2 and I killed 1 of 3, so... don't get it. This means

Re: Quorum: killing 1 out of 3 server kills the cluster (?)

2010-12-09 Thread Timo Nentwig
seeing). On Thu, Dec 9, 2010 at 5:00 PM, Timo Nentwig timo.nent...@toptarif.de wrote: On Dec 9, 2010, at 16:50, Daniel Lundin wrote: Quorum is really only useful when RF 2, since the for a quorum to succeed RF/2+1 replicas must be available. 2/2+1==2 and I killed 1 of 3, so... don't