[jira] [Created] (CASSANDRA-2830) Allow summing of counter columns in CQL
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the same
key, thus a query with a WHERE part that specifies multiple keys would return 1
result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2816) Repair doesn't synchronize merkle tree creation properly
[ https://issues.apache.org/jira/browse/CASSANDRA-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055421#comment-13055421 ] Sylvain Lebresne commented on CASSANDRA-2816: - bq. We have also spotted very noticable issues with full GCs when the merkle trees are passed around. Hopefully this could fix that too. This do make sure that we don't do multiple validation at the same time and that we keep a small number of merkle tree in memory at the same time. So I suppose this could help on the GC side. But overall I don't know if I am too optimistic about that, in part because I'm not sure what causes your issues. But this can't hurt on that side at least. bq. I will see if I can get this patch tested somewhere if it is ready for that. I believe it should be ready for that. bq. would it be an potential interesting idea to separate tombstones in different sstables. The thing is that some tombstones may be irrelevant become some update supersedes it (this is specially true of row tombstones). Hence basing a repair on tombstone only may transfer irrelevant data. I suppose it may depend on the use case this will be more or less a big deal. Also, this means that a read will be impacted in that we will often have to hit twice as many sstables. Given that it's not a crazy idea either to want to repair data regularly (if only for durability guarantee), I don't know if it is worth the trouble (we would have to separate tombstones from data at flush time, we'll have to maintain the two separate set of data/tombstone sstables, etc...). bq. make compaction deterministic or synchronized by a master across nodes Pretty sure we want to avoid going to a master architecture for everything if we can. Having master means that failure handling is more difficult (think network partition for instance) and require leader election and such, and the whole point of the fully distribution of Cassandra is to avoid those. Even without consider those, synchronizing compaction means synchronizing flush somehow and you want to be precise if you're going to use whole sstable md5s, which will be hard and quite probably inefficient. Repair doesn't synchronize merkle tree creation properly Key: CASSANDRA-2816 URL: https://issues.apache.org/jira/browse/CASSANDRA-2816 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.2 Attachments: 0001-Schedule-merkle-tree-request-one-by-one.patch Being a little slow, I just realized after having opened CASSANDRA-2811 and CASSANDRA-2815 that there is a more general problem with repair. When a repair is started, it will send a number of merkle tree to its neighbor as well as himself and assume for correction that the building of those trees will be started on every node roughly at the same time (if not, we end up comparing data snapshot at different time and will thus mistakenly repair a lot of useless data). This is bogus for many reasons: * Because validation compaction runs on the same executor that other compaction, the start of the validation on the different node is subject to other compactions. 0.8 mitigates this in a way by being multi-threaded (and thus there is less change to be blocked a long time by a long running compaction), but the compaction executor being bounded, its still a problem) * if you run a nodetool repair without arguments, it will repair every CFs. As a consequence it will generate lots of merkle tree requests and all of those requests will be issued at the same time. Because even in 0.8 the compaction executor is bounded, some of those validations will end up being queued behind the first ones. Even assuming that the different validation are submitted in the same order on each node (which isn't guaranteed either), there is no guarantee that on all nodes, the first validation will take the same time, hence desynchronizing the queued ones. Overall, it is important for the precision of repair that for a given CF and range (which is the unit at which trees are computed), we make sure that all node will start the validation at the same time (or, since we can't do magic, as close as possible). One (reasonably simple) proposition to fix this would be to have repair schedule validation compactions across nodes one by one (i.e, one CF/range at a time), waiting for all nodes to return their tree before submitting the next request. Then on each node, we should make sure that the node will start the validation compaction as soon as requested. For that, we probably want to have a specific executor for validation compaction and: * either we fail the
[jira] [Updated] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-2830:
Priority: Minor (was: Major)
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055445#comment-13055445
]
Sylvain Lebresne commented on CASSANDRA-2830:
-
It's not a crazy idea. Though we should at the very least make it generic
enough to have AVG(), MIN(), MAX() and such.
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2521: Attachment: 2521-v4.txt bq. why does DT.removeOldSSTableSize acquire/release around markCompacted? For the record, this is because this make the logic in SSTR.markCompacted and SSTR.releaseReference easier. If caller of markCompacted don't acquire a reference, markCompacted will have to deal with two cases: either no thread have a reference acquired, in which case the current thread should schedule the deletion, or other thread have a reference in which case it should let them the task of scheduling the deletion where they are done. But making this thread safe (so that we don't schedule twice or forget to schedule the deletion if the last thread holding a reference release it at the same time as the markCompacted is called) is a bit of annoying. Acquire a reference when markCompacted is called make this easier and move all the logic in releaseReference. bq. I believe currently, files are not deleted until the entire repair is finished The file should get deleted as soon as they are not useful anymore, that is as soon as they have been streamed. That being said, there was a bug, see below. bq. Did another repair overnight, one minor compaction included some 20 small sstables, all of them remains as well as a few from other compactions and the files from the repairs described before Yes, I did find a place where we were not correctly decrementing the reference count for streaming (repair was not unmarking sstable that were not streamed because they had nothing for the range to transfer). Attached v4 patch should fix that. bq. As for the last version of this patch, a quick look tonight shows access problems with markCurrentViewReferenced() v4 is based on v3 and fixes this (it reintroduce a specific method instead of making View public because I'm not too keen on doing that, but that can change if someone feels strongly about that). v4 also fix a bug in StreamingTransferTest and another one related to null segment in the unmmapping cleanup code. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055448#comment-13055448
]
Tomas Salfischberger commented on CASSANDRA-2830:
-
Good point, you could have a generic function implementation that is allowed to
do whatever it wants with an Iterator over the counter values and return a
single value. That would support easy implementation of SUM, MIN, MAX, AVG, but
also things like standard deviation and variance when the need arises.
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2816) Repair doesn't synchronize merkle tree creation properly
[ https://issues.apache.org/jira/browse/CASSANDRA-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055455#comment-13055455 ] Terje Marthinussen commented on CASSANDRA-2816: --- I don't know what causes GC when doing repairs either, but fire off repair on a few nodes with 100 million docs/node and there is a reasonable chance that a node here and there will log messages about reducing cache sizes due to memory pressure (I am not really sure it is a good idea to do this at all, reducing caches during stress rarely improves anything) or full GC. The thought about the master controlled compaction would not really affect network splits etc. Reconciliation after a network split is really as complex with or without a master. We need to get back to a state where all the nodes have the same data anyway which is a complex task anyway. This is more a consideration of the fact that we do not necessarily need to live in quorum based world during compaction and we are free to use alternative approaches in the compaction without changing read/write path or affecting availability. Master selection is not really a problem here. Start compaction, talk to other nodes with the same token ranges, select a leader. Does not even have to be the same master every time and could consider if we could make compaction part of a background read repair to reduce the amount of times we need to read/write data. For instance, if we can verify that the oldest/biggest sstables is 100% in sync with data on other replicas when it is compacted (why not do it during compaction when we go through the data anyway rather than later?),can we use that info to optimize the scans done during repairs by only using data in sstables with data received after some checkpoint in time as the starting point for the consistency check? Repair doesn't synchronize merkle tree creation properly Key: CASSANDRA-2816 URL: https://issues.apache.org/jira/browse/CASSANDRA-2816 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.2 Attachments: 0001-Schedule-merkle-tree-request-one-by-one.patch Being a little slow, I just realized after having opened CASSANDRA-2811 and CASSANDRA-2815 that there is a more general problem with repair. When a repair is started, it will send a number of merkle tree to its neighbor as well as himself and assume for correction that the building of those trees will be started on every node roughly at the same time (if not, we end up comparing data snapshot at different time and will thus mistakenly repair a lot of useless data). This is bogus for many reasons: * Because validation compaction runs on the same executor that other compaction, the start of the validation on the different node is subject to other compactions. 0.8 mitigates this in a way by being multi-threaded (and thus there is less change to be blocked a long time by a long running compaction), but the compaction executor being bounded, its still a problem) * if you run a nodetool repair without arguments, it will repair every CFs. As a consequence it will generate lots of merkle tree requests and all of those requests will be issued at the same time. Because even in 0.8 the compaction executor is bounded, some of those validations will end up being queued behind the first ones. Even assuming that the different validation are submitted in the same order on each node (which isn't guaranteed either), there is no guarantee that on all nodes, the first validation will take the same time, hence desynchronizing the queued ones. Overall, it is important for the precision of repair that for a given CF and range (which is the unit at which trees are computed), we make sure that all node will start the validation at the same time (or, since we can't do magic, as close as possible). One (reasonably simple) proposition to fix this would be to have repair schedule validation compactions across nodes one by one (i.e, one CF/range at a time), waiting for all nodes to return their tree before submitting the next request. Then on each node, we should make sure that the node will start the validation compaction as soon as requested. For that, we probably want to have a specific executor for validation compaction and: * either we fail the whole repair whenever one node is not able to execute the validation compaction right away (because no thread are available right away). * we simply tell the user that if he start too many repairs in parallel, he may start seeing some of those repairing more data than it should. -- This message is automatically
[jira] [Commented] (CASSANDRA-2804) expose dropped messages, exceptions over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055470#comment-13055470 ] Sylvain Lebresne commented on CASSANDRA-2804: - Looks good. That being said, the recent variant in JMX is essentially the same as the one the StatusLogger is logging, and I could see a point in preferring to log this in JMX rather than in the log file. Not sure what is the best way to accommodate those two however. In any case, the patch does improve on the current situation, so +1. expose dropped messages, exceptions over JMX Key: CASSANDRA-2804 URL: https://issues.apache.org/jira/browse/CASSANDRA-2804 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.7, 0.8.2 Attachments: 2804.txt, twttr-cassandra-0.8-counts-resync-droppedmsg-metric.diff Patch against 0.7. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2280) Request specific column families using StreamIn
[ https://issues.apache.org/jira/browse/CASSANDRA-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2280: Fix Version/s: (was: 0.8.1) 0.8.2 Request specific column families using StreamIn --- Key: CASSANDRA-2280 URL: https://issues.apache.org/jira/browse/CASSANDRA-2280 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Jonathan Ellis Fix For: 0.8.2 Attachments: 0001-Allow-specific-column-families-to-be-requested-for-str.txt, 0001-Allow-specific-column-families-to-be-requested-for-str.txt, 2280-v3.txt, 2280-v4.txt, 2280-v5.txt StreamIn.requestRanges only specifies a keyspace, meaning that requesting a range will request it for all column families: if you have a large number of CFs, this can cause quite a headache. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2576) Rewrite into new file post streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055486#comment-13055486 ] Sylvain Lebresne commented on CASSANDRA-2576: - Looks good, but this apparently already needs rebasing. Rewrite into new file post streaming Key: CASSANDRA-2576 URL: https://issues.apache.org/jira/browse/CASSANDRA-2576 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-CASSANDRA-2576-Don-t-depend-on-a-byte-for-byte-match-f.txt, 0002-CASSANDRA-2576-Rebuild-into-a-new-file-to-minimize-mag.txt Commutative/counter column families use a separate path to rebuild sstables post streaming, and that path currently rewrites the data within the streamed file. While this is great for space efficiency, it means a duplicated code path for writing sstables, which makes it more difficult to make changes like #674. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2816) Repair doesn't synchronize merkle tree creation properly
[ https://issues.apache.org/jira/browse/CASSANDRA-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055493#comment-13055493 ] Jonathan Ellis commented on CASSANDRA-2816: --- bq. I am not really sure it is a good idea to do this at all, reducing caches during stress rarely improves anything (This is on by default because the most common cause of OOMing is people configuring their caches too large.) It sounds odd to me that repair would balloon memory usage dramatically. Do you have monitoring graphs that show the difference in heap usage between normal and repair in progress? Repair doesn't synchronize merkle tree creation properly Key: CASSANDRA-2816 URL: https://issues.apache.org/jira/browse/CASSANDRA-2816 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.2 Attachments: 0001-Schedule-merkle-tree-request-one-by-one.patch Being a little slow, I just realized after having opened CASSANDRA-2811 and CASSANDRA-2815 that there is a more general problem with repair. When a repair is started, it will send a number of merkle tree to its neighbor as well as himself and assume for correction that the building of those trees will be started on every node roughly at the same time (if not, we end up comparing data snapshot at different time and will thus mistakenly repair a lot of useless data). This is bogus for many reasons: * Because validation compaction runs on the same executor that other compaction, the start of the validation on the different node is subject to other compactions. 0.8 mitigates this in a way by being multi-threaded (and thus there is less change to be blocked a long time by a long running compaction), but the compaction executor being bounded, its still a problem) * if you run a nodetool repair without arguments, it will repair every CFs. As a consequence it will generate lots of merkle tree requests and all of those requests will be issued at the same time. Because even in 0.8 the compaction executor is bounded, some of those validations will end up being queued behind the first ones. Even assuming that the different validation are submitted in the same order on each node (which isn't guaranteed either), there is no guarantee that on all nodes, the first validation will take the same time, hence desynchronizing the queued ones. Overall, it is important for the precision of repair that for a given CF and range (which is the unit at which trees are computed), we make sure that all node will start the validation at the same time (or, since we can't do magic, as close as possible). One (reasonably simple) proposition to fix this would be to have repair schedule validation compactions across nodes one by one (i.e, one CF/range at a time), waiting for all nodes to return their tree before submitting the next request. Then on each node, we should make sure that the node will start the validation compaction as soon as requested. For that, we probably want to have a specific executor for validation compaction and: * either we fail the whole repair whenever one node is not able to execute the validation compaction right away (because no thread are available right away). * we simply tell the user that if he start too many repairs in parallel, he may start seeing some of those repairing more data than it should. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1125) Filter out ColumnFamily rows that aren't part of the query
[ https://issues.apache.org/jira/browse/CASSANDRA-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mck SembWever updated CASSANDRA-1125: - Attachment: CASSANDRA-1125.patch Filter out ColumnFamily rows that aren't part of the query -- Key: CASSANDRA-1125 URL: https://issues.apache.org/jira/browse/CASSANDRA-1125 Project: Cassandra Issue Type: New Feature Components: Hadoop Reporter: Jeremy Hanna Assignee: Mck SembWever Priority: Minor Fix For: 1.0 Attachments: CASSANDRA-1125.patch Currently, when running a MapReduce job against data in a Cassandra data store, it reads through all the data for a particular ColumnFamily. This could be optimized to only read through those rows that have to do with the query. It's a small change but wanted to put it in Jira so that it didn't fall through the cracks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055507#comment-13055507 ] Sylvain Lebresne commented on CASSANDRA-2405: - I'm sorry but I still think we are still returning the wrong number to the user. To be clear, this is nothing against the code of the patch itself, I just think that given the way repair works, it is not so simple to have a time since last successful repair. The unit of a repair is for a given keyspace, column family and range. Because of that, I don't think we can return a single time since last successful repair for a given keyspace and column family. It has to include the range somehow. Granted, so far a nodetool repair repairs all the ranges of the node you launch it on, but I don't think this should be the case (CASSANDRA-2610). Moreover, even now, one of the range can fail without the other. So returning only one number for all ranges is wrong. The other problem is: I'm not convinced that recording the information only on the node coordinating the repair is necessarily super helpful. When you start a repair a node, you will also repair its neighbor (for only the range they share), so recording the time only on the initial node on which the nodetool command was connected is random, and will convey the idea that repair should be started for every range on every node (while I strongly thing that the short term goal should be to make it easy to NOT do that -- CASSANDRA-2610 again). Imho, we should hold back on this issue for now and at least wait for CASSANDRA-2610, CASSANDRA-2606 and CASSANDRA-2816 before committing to anything. I agree that having information to help people plan repair is nice, but it is at most a very minor improvement and exposing a misleading number is more harmful that no number. should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055511#comment-13055511 ] Pavel Yaskevich commented on CASSANDRA-2405: Sounds reasonable, maybe we should close this with Won't fix and create a more general one? should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.2 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2831) Creating or updating CF key_validation_class with the CLI doesn't works
Creating or updating CF key_validation_class with the CLI doesn't works
---
Key: CASSANDRA-2831
URL: https://issues.apache.org/jira/browse/CASSANDRA-2831
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.2
Environment: Ubuntu 10.10, 32 bits
java version 1.6.0_24
Brisk beta-2 installed from Debian packages
Reporter: Silvère Lestang
In the command line:
{code}
create column family test with key_validation_class = 'AsciiType' and
comparator = 'LongType' and default_validation_class = 'IntegerType';
describe keyspace;
Keyspace: Test:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:1]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.LongType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
{code}
The Default column value validator is BytesType instead of IntegerType. Also
tested with other types or with the update column family command, same
problem occur.
{code}
[default@Test] update column family test with default_validation_class =
'LongType';
51a37430-a0bb-11e0--ef8993101fdf
Waiting for schema agreement...
... schemas agree across the cluster
[default@Test] describe keyspace;
Keyspace: Test:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:1]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.LongType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
{code}
Btw, they are a typo in file
src/resources/org/apache/cassandra/cli/CliHelp.yaml line 642:
key_valiation_class key_validation_class
Very annoying for people like me who stupidly copy/paste the help.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2810) RuntimeException in Pig when using dump command on column name
[
https://issues.apache.org/jira/browse/CASSANDRA-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055521#comment-13055521
]
Silvère Lestang commented on CASSANDRA-2810:
I try again after applying [^2810.txt] and the patch from bug [CASSANDRA-2777]
and the bug is still here.
With the patch, you need to replace
{code}
test = LOAD 'cassandra://Test/test' USING CassandraStorage() AS
(rowkey:chararray, columns: bag {T: (name:long, value:int)});
{code}
by
{code}
test = LOAD 'cassandra://Test/test' USING CassandraStorage() AS ();
{code}
because CassandraStorage takes care of the schema.
I try:
{code}
grunt describe test;
test: {key: chararray,columns: {(name: long,value: int)}}
{code}
so we can see that the patch from bug 2777 works correctly (I also test with
different types for value).
But when I dump test, I still have the same exception.
RuntimeException in Pig when using dump command on column name
Key: CASSANDRA-2810
URL: https://issues.apache.org/jira/browse/CASSANDRA-2810
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.1
Environment: Ubuntu 10.10, 32 bits
java version 1.6.0_24
Brisk beta-2 installed from Debian packages
Reporter: Silvère Lestang
Assignee: Brandon Williams
Attachments: 2810.txt
This bug was previously report on [Brisk bug
tracker|https://datastax.jira.com/browse/BRISK-232].
In cassandra-cli:
{code}
[default@unknown] create keyspace Test
with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy'
and strategy_options = [{replication_factor:1}];
[default@unknown] use Test;
Authenticated to keyspace: Test
[default@Test] create column family test;
[default@Test] set test[ascii('row1')][long(1)]=integer(35);
set test[ascii('row1')][long(2)]=integer(36);
set test[ascii('row1')][long(3)]=integer(38);
set test[ascii('row2')][long(1)]=integer(45);
set test[ascii('row2')][long(2)]=integer(42);
set test[ascii('row2')][long(3)]=integer(33);
[default@Test] list test;
Using default limit of 100
---
RowKey: 726f7731
= (column=0001, value=35, timestamp=1308744931122000)
= (column=0002, value=36, timestamp=1308744931124000)
= (column=0003, value=38, timestamp=1308744931125000)
---
RowKey: 726f7732
= (column=0001, value=45, timestamp=1308744931127000)
= (column=0002, value=42, timestamp=1308744931128000)
= (column=0003, value=33, timestamp=1308744932722000)
2 Rows Returned.
[default@Test] describe keyspace;
Keyspace: Test:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:1]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.BytesType
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.BytesType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
{code}
In Pig command line:
{code}
grunt test = LOAD 'cassandra://Test/test' USING CassandraStorage() AS
(rowkey:chararray, columns: bag {T: (name:long, value:int)});
grunt value_test = foreach test generate rowkey, columns.name, columns.value;
grunt dump value_test;
{code}
In /var/log/cassandra/system.log, I have severals time this exception:
{code}
INFO [IPC Server handler 3 on 8012] 2011-06-22 15:03:28,533
TaskInProgress.java (line 551) Error from
attempt_201106210955_0051_m_00_3: java.lang.RuntimeException: Unexpected
data type -1 found in stream.
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at
org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
at
[jira] [Resolved] (CASSANDRA-2831) Creating or updating CF key_validation_class with the CLI doesn't works
[
https://issues.apache.org/jira/browse/CASSANDRA-2831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2831.
---
Resolution: Fixed
Fix Version/s: 0.8.1
fixed by r1137774
Creating or updating CF key_validation_class with the CLI doesn't works
---
Key: CASSANDRA-2831
URL: https://issues.apache.org/jira/browse/CASSANDRA-2831
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.2
Environment: Ubuntu 10.10, 32 bits
java version 1.6.0_24
Brisk beta-2 installed from Debian packages
Reporter: Silvère Lestang
Fix For: 0.8.1
In the command line:
{code}
create column family test with key_validation_class = 'AsciiType' and
comparator = 'LongType' and default_validation_class = 'IntegerType';
describe keyspace;
Keyspace: Test:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:1]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.LongType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
{code}
The Default column value validator is BytesType instead of IntegerType.
Also tested with other types or with the update column family command, same
problem occur.
{code}
[default@Test] update column family test with default_validation_class =
'LongType';
51a37430-a0bb-11e0--ef8993101fdf
Waiting for schema agreement...
... schemas agree across the cluster
[default@Test] describe keyspace;
Keyspace: Test:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:1]
Column Families:
ColumnFamily: test
Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.LongType
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 20.0/14400
Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: false
Built indexes: []
{code}
Btw, they are a typo in file
src/resources/org/apache/cassandra/cli/CliHelp.yaml line 642:
key_valiation_class key_validation_class
Very annoying for people like me who stupidly copy/paste the help.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-2823:
Attachment: 2823.patch
I think the problem is with the call to removeDeleted in resolveSuperset()
(which is fairly new). Basically, the code is fine with resolved being null as
long as this means that all the versions are null. But the removeDeleted call
make it possible to have a null removeDeleted even if the versions are not
null, if a row tombstone expires between the time it was returned by the node
and the time it is resolved by the coordinator for instance.
Attaching patch that skips the maybeScheduleRepair() call if resolved == null
since even in that case there is nothing to repair since the tombstone are now
expired.
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2811) Repair doesn't stagger flushes
[ https://issues.apache.org/jira/browse/CASSANDRA-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2811. - Resolution: Duplicate Marking this duplicate of CASSANDRA-2816 as the patch on the latter includes this. Repair doesn't stagger flushes -- Key: CASSANDRA-2811 URL: https://issues.apache.org/jira/browse/CASSANDRA-2811 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 0.8.2 When you do a nodetool repair (with no options), the following things occured: * For each keyspace, a call to SS.forceTableRepair is issued * In each of those calls: for each token range the node is responsible for, a repair session is created and started * Each of these session will request one merkle tree by column family (to each node for which it makes sense, which includes the node the repair is started on) All those merkle tree requests are done basically at the same time. And now that compaction is multi-threaded, this means that usually more than one validation compaction will be started at the same time. The problem is that a validation compaction starts by a flush. Given that by default the flush_queue_size is 4 and the number of compaction thread is the number of processors and given that on any recent machine the number of core will be = 4, this means that this will easily end up blocking write for some period of time. It turns out to also have a more subtle problem for repair itself. If two validation compaction for the same column family (but different range) are started in a very short time interval, the first validation will block on the flush, but the second one may not block at all if the memtable is clean when it request it's own flush. In which case that second validation will be executed on data older than it should. I think the simpler fix is to make sure we only ever do one validation compaction at a time. It's probably a better use of resources anyway. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2815) Bad timing in repair can transfer data it is not suppose to
[ https://issues.apache.org/jira/browse/CASSANDRA-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2815. - Resolution: Duplicate Marking as duplicate of CASSANDRA-2816 as the patch there fixes that issue too. Bad timing in repair can transfer data it is not suppose to Key: CASSANDRA-2815 URL: https://issues.apache.org/jira/browse/CASSANDRA-2815 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: repair Fix For: 0.8.2 The core of the problem is that the sstables used to construct a merkle tree are not necessarily the same than the ones for which streaming is initiated. This is usually not a big deal: newly compacted sstables don't matter since data hasn't change. Newly flushed data (between the start of the validation compaction and the start of the streaming) are a little bit more problematic, even though one could argue that on average this won't be not too much data. But there can be more problematic scenario: suppose a 3 nodes cluster with RF=3, all the data on node3 is removed and then repair is started on node3. Also suppose the cluster havs two CFs, cf1 and cf2, sharing evenly the data. Node3 will request the usual merkle trees and let's pretend validation compaction is mono-threaded to simplify. What can happen is the following: # node3 computes its merkle trees for all requests very quickly, having no data. It will thus wait on the other nodes trees (his own trees saying I have no data). # node1 starts computing its tree for say cf1 (queuing the computation for cf2). In the meantime, node2 may start computing the tree for cf2 (queuing the one for cf1). # when node1 completes his first tree, it sends it to node3. Node3 receives it, compare to his own tree and initiate transfer of all the data for cf1 from node1 to him. # not too long after that, node2 completes it's first tree, the one for cf2 and send it to node3. Based on it, transfer of all the data for cf2 from node2 to node3 starts. # An arbitrary long time after that (compaction validation can take time), node2 will finish his second tree (for cf1 that is and send it back to node3. Node3 will compare to his own (empty) tree and decide that all the ranges should be repaired with node2 for cf1. The problem is that when that happens, the transfer of cf1 from node1 may have been done already, or at least partly done. For that reason, node3 will start streaming all this data to node2. # For the same reasons, node3 may end up transfering all or part of cf2 to node1. So the problem is, even though at the beginning node1 and node2 may be perfectly consistent, we will end up streaming potentially huge amount of data to them. I think this affects both 0.7 and 0.8, though differently because compaction multi-threading and the fact that 0.8 differentiates the ranges make the timing different. One solution (in a way, the theoretically right solution) would be to grab references to the sstables we use for the validation compaction, and only initiate streaming on these sstables. However, in 0.8 where compaction is multi-threaded, this would mean retaining compacted sstables longer that necessary. This is also a bit complicated and in particular would require a network protocol change (because a streaming request message would have to contain some information allowing to decide what set of sstables to use). A maybe simpler solution could be to have the node coordinating the repair wait for the tree of all the remotes (in this obviously only for a given column family and range) before starting streaming. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2824) assert err on SystemTable.getCurrentLocalNodeId during a cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2824: Attachment: 2824.patch There is possibly a race here that triggers the assert. The code is relying on the fact that system tables have a gc_grace of 0 to assume it cannot get tombstone back, but given that gcbefore is at the precision of a second, and given than that on tie with the tombstone timestamp we do include the tombstone, we could get tombstone back. Attaching patch that takes a safer road. assert err on SystemTable.getCurrentLocalNodeId during a cleanup Key: CASSANDRA-2824 URL: https://issues.apache.org/jira/browse/CASSANDRA-2824 Project: Cassandra Issue Type: Bug Reporter: Jackson Chung Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 Attachments: 2824.patch when running nodetool cleanup the following happened: $ ./bin/nodetool cleanup --host localhost Exception in thread main java.lang.AssertionError at org.apache.cassandra.db.SystemTable.getCurrentLocalNodeId(SystemTable.java:383) at org.apache.cassandra.utils.NodeId$LocalNodeIdHistory.init(NodeId.java:179) at org.apache.cassandra.utils.NodeId.clinit(NodeId.java:38) at org.apache.cassandra.utils.NodeId$OneShotRenewer.init(NodeId.java:159) at org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1317) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2810) RuntimeException in Pig when using dump command on column name
[
https://issues.apache.org/jira/browse/CASSANDRA-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055585#comment-13055585
]
Silvère Lestang commented on CASSANDRA-2810:
After more test (with both patches), path [^2810.txt] doesn't seems to solve
the bug.
Here is a new test case:
Create a _Test_ keyspace and a _test_ column family with key_validation_class =
'AsciiType' and comparator = 'LongType' and default_validation_class =
'IntegerType' (don't use the cli because of [#CASSANDRA-2831]).
Insert some data:
{code}
set test[ascii('row1')][long(1)]=integer(35);
set test[ascii('row1')][long(2)]=integer(36);
set test[ascii('row1')][long(3)]=integer(38);
set test[ascii('row2')][long(1)]=integer(45);
set test[ascii('row2')][long(2)]=integer(42);
set test[ascii('row2')][long(3)]=integer(33);
{code}
In Pig cli:
{code}
test = LOAD 'cassandra://Test/test' USING CassandraStorage() AS ();
dump test;
{code}
The same exception as before is raised:
{code}
INFO [IPC Server handler 4 on 8012] 2011-06-27 16:40:28,562
TaskInProgress.java (line 551) Error from attempt_201106271436_0012_m_00_1:
java.lang.RuntimeException: Unexpected data type -1 found in stream.
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at
org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:224)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
{code}
RuntimeException in Pig when using dump command on column name
Key: CASSANDRA-2810
URL: https://issues.apache.org/jira/browse/CASSANDRA-2810
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.1
Environment: Ubuntu 10.10, 32 bits
java version 1.6.0_24
Brisk beta-2 installed from Debian packages
Reporter: Silvère Lestang
Assignee: Brandon Williams
Attachments: 2810.txt
This bug was previously report on [Brisk bug
tracker|https://datastax.jira.com/browse/BRISK-232].
In cassandra-cli:
{code}
[default@unknown] create keyspace Test
with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy'
and strategy_options = [{replication_factor:1}];
[default@unknown] use Test;
Authenticated to keyspace: Test
[default@Test] create column family test;
[default@Test] set test[ascii('row1')][long(1)]=integer(35);
set test[ascii('row1')][long(2)]=integer(36);
set test[ascii('row1')][long(3)]=integer(38);
set test[ascii('row2')][long(1)]=integer(45);
set test[ascii('row2')][long(2)]=integer(42);
set test[ascii('row2')][long(3)]=integer(33);
[default@Test] list test;
Using default limit of 100
---
RowKey: 726f7731
= (column=0001, value=35, timestamp=1308744931122000)
= (column=0002, value=36, timestamp=1308744931124000)
= (column=0003, value=38, timestamp=1308744931125000)
---
RowKey: 726f7732
= (column=0001, value=45, timestamp=1308744931127000)
= (column=0002, value=42,
[jira] [Commented] (CASSANDRA-2824) assert err on SystemTable.getCurrentLocalNodeId during a cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055590#comment-13055590 ] Jonathan Ellis commented on CASSANDRA-2824: --- why not just call removeDeleted? assert err on SystemTable.getCurrentLocalNodeId during a cleanup Key: CASSANDRA-2824 URL: https://issues.apache.org/jira/browse/CASSANDRA-2824 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Jackson Chung Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 Attachments: 2824.patch when running nodetool cleanup the following happened: $ ./bin/nodetool cleanup --host localhost Exception in thread main java.lang.AssertionError at org.apache.cassandra.db.SystemTable.getCurrentLocalNodeId(SystemTable.java:383) at org.apache.cassandra.utils.NodeId$LocalNodeIdHistory.init(NodeId.java:179) at org.apache.cassandra.utils.NodeId.clinit(NodeId.java:38) at org.apache.cassandra.utils.NodeId$OneShotRenewer.init(NodeId.java:159) at org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1317) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055591#comment-13055591
]
Jonathan Ellis commented on CASSANDRA-2823:
---
+1
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055593#comment-13055593
]
Jonathan Ellis commented on CASSANDRA-2823:
---
although I slightly prefer the if == null return version immediately after
initializing resolved to keep those two pieces of logic together.
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055593#comment-13055593
]
Jonathan Ellis edited comment on CASSANDRA-2823 at 6/27/11 3:06 PM:
although I slightly prefer the if == null return immediately after
initializing resolved, to keep those two pieces of logic together.
was (Author: jbellis):
although I slightly prefer the if == null return version immediately
after initializing resolved to keep those two pieces of logic together.
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055596#comment-13055596
]
Sylvain Lebresne commented on CASSANDRA-2823:
-
Yeah, I didn't do that mostly because there is still a few lines of code
(besides maybe scheduling repair) that we need to do even if resolved is null
(debugging message in RowRepairResolver and more importantly, the clear of
versions and versionSources in RangeSliceResolver).
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2823) NPE during range slices with rowrepairs
[
https://issues.apache.org/jira/browse/CASSANDRA-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055597#comment-13055597
]
Jonathan Ellis commented on CASSANDRA-2823:
---
ah, right -- skipping the clear would be buggy. +1 again. :)
NPE during range slices with rowrepairs
---
Key: CASSANDRA-2823
URL: https://issues.apache.org/jira/browse/CASSANDRA-2823
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8.2
Environment: This is a trunk build with 2521 and 2433
I somewhat doubt that is related however.
Reporter: Terje Marthinussen
Assignee: Sylvain Lebresne
Attachments: 2823.patch
Doing some heavy testing of relatively fast feeding (5000+ mutations/sec) +
repair on all node + range slices.
Then occasionally killing a node here and there and restarting it.
Triggers the following NPE
ERROR [pool-2-thread-3] 2011-06-24 20:56:27,289 Cassandra.java (line 3210)
Internal error processing get_range_slices
java.lang.NullPointerException
at
org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:109)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:112)
at
org.apache.cassandra.service.RangeSliceResponseResolver$2.getReduced(RangeSliceResponseResolver.java:83)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:161)
at
org.apache.cassandra.utils.MergeIterator.computeNext(MergeIterator.java:88)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:120)
at
org.apache.cassandra.service.RangeSliceResponseResolver.resolve(RangeSliceResponseResolver.java:43)
Looking at the code in getReduced:
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
{noformat}
seems like resolved becomes null when this happens and versions.size is
larger than 1.
RowRepairResolver.resolveSuperset() does actually return null if it cannot
resolve anything, so there is definately a case here which can occur and is
not handled.
It may also be an interesting question if it is guaranteed that
versions.add(current.left.cf);
can never return null?
Jonathan suggested on IRC that maybe
{noformat}
ColumnFamily resolved = versions.size() 1
?
RowRepairResolver.resolveSuperset(versions)
: versions.get(0);
if (resolved == null)
return new Row(key, resolved);
{noformat}
could be a fix.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2820) Re-introduce FastByteArrayInputStream (and Output equivalent)
[ https://issues.apache.org/jira/browse/CASSANDRA-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Loy updated CASSANDRA-2820: Attachment: fast_bytearray_iostreams_harmony-patch-3.txt This patch has: * FastByteArrayIn/OutputStream impls that extend the base Harmony versions. * Same formatting of the Fast versions and the Harmony version for easy diffing if Harmony bug fixes need to be applied. * A full carbon-copy of Harmony code to ensure cross-JRE support. * Other import changes (eclipse CTRL+SHIFT+Os) reverted. Re-introduce FastByteArrayInputStream (and Output equivalent) - Key: CASSANDRA-2820 URL: https://issues.apache.org/jira/browse/CASSANDRA-2820 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.0 Environment: n/a Reporter: Paul Loy Priority: Minor Labels: bytearrayinputstream, bytearrayoutputstream, license, synchronized Attachments: fast_bytearray_iostreams_harmony-patch-2.txt, fast_bytearray_iostreams_harmony-patch-3.txt In https://issues.apache.org/jira/browse/CASSANDRA-37 FastByteArrayInputStream and FastByteArrayOutputStream were removed due to being code copied from the JDK and then subsequently modified. The JDK license is incompatible with Apache 2 license so the code had to go. I have since had a look at the performance of the JDK ByteArrayInputStream and a FastByteArrayInputStream (i.e. one with synchronized methods made un-synchronized) and seen the difference is significant. After a warmup-period of 1 loops I get the following for 1 loops through a 128000 byte array: bais : 3513ms fbais: 72ms This varies depending on the OS, machine and Java version, but it's always in favour of the FastByteArrayInputStream as you might expect. Then, at Jonathan Ellis' suggestion, I tried this using a modified Apache Harmony ByteArrayInputStream - i.e. one whose license is compatible - and the results were the same. A significant boost. I will attach a patch with changes for the 0.8.0 tag. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2773) Index manager cannot support deleting and inserting into a row in the same mutation
[ https://issues.apache.org/jira/browse/CASSANDRA-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055611#comment-13055611 ] Sylvain Lebresne commented on CASSANDRA-2773: - Actually, there isn't really much risk for data loss given that as Jonathan said, if you hit that, it's fairly easy to go back to 0.7.5, fix client code and upgrade again. Granted this is not user friendly and not something you should expect from a minor upgrade, but let at least set the record straight on the data loss part. That being said, I don't think the patch on this ticket could screw up indexes more that we use to prior to 0.7.6, so maybe we can commit to 0.7.7 on that ground. I'd still suggest fixing client code in the meantime. Index manager cannot support deleting and inserting into a row in the same mutation - Key: CASSANDRA-2773 URL: https://issues.apache.org/jira/browse/CASSANDRA-2773 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Boris Yen Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8.2 Attachments: 2773-v2.txt, 2773.txt, cassandra.log I use hector 0.8.0-1 and cassandra 0.8. 1. create mutator by using hector api, 2. Insert a few columns into the mutator for key key1, cf standard. 3. add a deletion to the mutator to delete the record of key1, cf standard. 4. repeat 2 and 3 5. execute the mutator. the result: the connection seems to be held by the sever forever, it never returns. when I tried to restart the cassandra I saw unsupportedexception : Index manager cannot support deleting and inserting into a row in the same mutation. and the cassandra is dead forever, unless I delete the commitlog. I would expect to get an exception when I execute the mutator, not after I restart the cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2773) Index manager cannot support deleting and inserting into a row in the same mutation
[ https://issues.apache.org/jira/browse/CASSANDRA-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055615#comment-13055615 ] Jonathan Ellis commented on CASSANDRA-2773: --- bq. I don't think the patch on this ticket could screw up indexes more that we use to prior to 0.7.6 That's a valid way to frame the issue. I'm good to commit for 0.7.7 if Jim can test the patch first, since he's the only one we've heard of hitting this in 0.7.x. (Specifically, we want to make sure that if we query WHERE foo = X we don't get results back where foo is something other than X. Ideally you'd start with an empty database, or at least drop + recreate indexes first to make sure the results aren't contaminated w/ corrupt entries from pre-0.7.6.) Index manager cannot support deleting and inserting into a row in the same mutation - Key: CASSANDRA-2773 URL: https://issues.apache.org/jira/browse/CASSANDRA-2773 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Boris Yen Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8.2 Attachments: 2773-v2.txt, 2773.txt, cassandra.log I use hector 0.8.0-1 and cassandra 0.8. 1. create mutator by using hector api, 2. Insert a few columns into the mutator for key key1, cf standard. 3. add a deletion to the mutator to delete the record of key1, cf standard. 4. repeat 2 and 3 5. execute the mutator. the result: the connection seems to be held by the sever forever, it never returns. when I tried to restart the cassandra I saw unsupportedexception : Index manager cannot support deleting and inserting into a row in the same mutation. and the cassandra is dead forever, unless I delete the commitlog. I would expect to get an exception when I execute the mutator, not after I restart the cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1608) Redesigned Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Coverston updated CASSANDRA-1608: -- Attachment: 1608-v7.txt Added an interval tree to cull sstables that are not needed for point and range queries. Redesigned Compaction - Key: CASSANDRA-1608 URL: https://issues.apache.org/jira/browse/CASSANDRA-1608 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Benjamin Coverston Attachments: 0001-leveldb-style-compaction.patch, 1608-v2.txt, 1608-v3.txt, 1608-v4.txt, 1608-v5.txt, 1608-v7.txt After seeing the I/O issues in CASSANDRA-1470, I've been doing some more thinking on this subject that I wanted to lay out. I propose we redo the concept of how compaction works in Cassandra. At the moment, compaction is kicked off based on a write access pattern, not read access pattern. In most cases, you want the opposite. You want to be able to track how well each SSTable is performing in the system. If we were to keep statistics in-memory of each SSTable, prioritize them based on most accessed, and bloom filter hit/miss ratios, we could intelligently group sstables that are being read most often and schedule them for compaction. We could also schedule lower priority maintenance on SSTable's not often accessed. I also propose we limit the size of each SSTable to a fix sized, that gives us the ability to better utilize our bloom filters in a predictable manner. At the moment after a certain size, the bloom filters become less reliable. This would also allow us to group data most accessed. Currently the size of an SSTable can grow to a point where large portions of the data might not actually be accessed as often. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of Counters by SylvainLebresne
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The Counters page has been changed by SylvainLebresne: http://wiki.apache.org/cassandra/Counters?action=diffrev1=12rev2=13 == Technical limitations == * If a write fails unexpectedly (timeout or loss of connection to the coordinator node) the client will not know if the operation has been performed. A retry can result in an over count [[https://issues.apache.org/jira/browse/CASSANDRA-2495|CASSANDRA-2495]]. - * Counter removal is intrinsically limited. For instance, if you issue very quickly the sequence increment, remove, increment it is possible for the removal to be lost (if for some reason the remove happens to be the last received messages). Hence, removal of counters is provided for definitive removal, that is when the deleted counter is not increment afterwards. Note that if you need to reset a counter, you can read its ''value'' and insert ''-value''. + * Counter removal is intrinsically limited. For instance, if you issue very quickly the sequence increment, remove, increment it is possible for the removal to be lost (if for some reason the remove happens to be the last received messages). Hence, removal of counters is provided for definitive removal only, that is when the deleted counter is not increment afterwards. Note that if you need to reset a counter, you can read its ''value'' and insert ''-value''. == Further reading == - See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]] and especially the [[https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf|design doc]] for further information about how this works internally. + See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]] and especially the [[https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf|design doc]] for further information about how this works internally (but note that some of the limitation fixed in these technical documents have been fixed since then, for instance all consistency level '''are''' supported, for both reads and writes).
[jira] [Updated] (CASSANDRA-2653) index scan errors out when zero columns are requested
[
https://issues.apache.org/jira/browse/CASSANDRA-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-2653:
Attachment: 2653_v2.patch
I actually agree with taking the impact. Especially given that there is
actually very little cases where it will make an actual difference anyway.
Attaching patch (2653_v2, based on 0.7) that implement the idea and add back
the sanity check.
index scan errors out when zero columns are requested
-
Key: CASSANDRA-2653
URL: https://issues.apache.org/jira/browse/CASSANDRA-2653
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.6, 0.8.0 beta 2
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 0.7.7, 0.8.1
Attachments:
0001-Handle-data-get-returning-null-in-secondary-indexes.patch,
0001-Handle-null-returns-in-data-index-query-v0.7.patch,
0001-Reset-SSTII-in-EchoedRow-constructor.patch, 2653_v2.patch,
v1-0001-CASSANDRA-2653-reproduce-regression.txt
As reported by Tyler Hobbs as an addendum to CASSANDRA-2401,
{noformat}
ERROR 16:13:38,864 Fatal exception in thread Thread[ReadStage:16,5,main]
java.lang.AssertionError: No data found for
SliceQueryFilter(start=java.nio.HeapByteBuffer[pos=10 lim=10 cap=30],
finish=java.nio.HeapByteBuffer[pos=17 lim=17 cap=30], reversed=false,
count=0] in DecoratedKey(81509516161424251288255223397843705139,
6b657931):QueryPath(columnFamilyName='cf', superColumnName='null',
columnName='null') (original filter
SliceQueryFilter(start=java.nio.HeapByteBuffer[pos=10 lim=10 cap=30],
finish=java.nio.HeapByteBuffer[pos=17 lim=17 cap=30], reversed=false,
count=0]) from expression 'cf.626972746864617465 EQ 1'
at
org.apache.cassandra.db.ColumnFamilyStore.scan(ColumnFamilyStore.java:1517)
at
org.apache.cassandra.service.IndexScanVerbHandler.doVerb(IndexScanVerbHandler.java:42)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2824) assert err on SystemTable.getCurrentLocalNodeId during a cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2824: Attachment: 2824_v2.patch You're right, I skipped the fact that gcBefore==0 is different from gc_grace==0. Patch v2 attached. assert err on SystemTable.getCurrentLocalNodeId during a cleanup Key: CASSANDRA-2824 URL: https://issues.apache.org/jira/browse/CASSANDRA-2824 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Jackson Chung Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 Attachments: 2824.patch, 2824_v2.patch when running nodetool cleanup the following happened: $ ./bin/nodetool cleanup --host localhost Exception in thread main java.lang.AssertionError at org.apache.cassandra.db.SystemTable.getCurrentLocalNodeId(SystemTable.java:383) at org.apache.cassandra.utils.NodeId$LocalNodeIdHistory.init(NodeId.java:179) at org.apache.cassandra.utils.NodeId.clinit(NodeId.java:38) at org.apache.cassandra.utils.NodeId$OneShotRenewer.init(NodeId.java:159) at org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1317) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mck SembWever reassigned CASSANDRA-2807:
Assignee: Mck SembWever
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.8.0
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mck SembWever updated CASSANDRA-2807:
-
Attachment: CASSANDRA-2807.patch
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.8.0
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
Attachments: CASSANDRA-2807.patch
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2824) assert err on SystemTable.getCurrentLocalNodeId during a cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055712#comment-13055712 ] Jonathan Ellis commented on CASSANDRA-2824: --- +1 (can you include a link here in the comment when you commit?) assert err on SystemTable.getCurrentLocalNodeId during a cleanup Key: CASSANDRA-2824 URL: https://issues.apache.org/jira/browse/CASSANDRA-2824 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.0 Reporter: Jackson Chung Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8.2 Attachments: 2824.patch, 2824_v2.patch when running nodetool cleanup the following happened: $ ./bin/nodetool cleanup --host localhost Exception in thread main java.lang.AssertionError at org.apache.cassandra.db.SystemTable.getCurrentLocalNodeId(SystemTable.java:383) at org.apache.cassandra.utils.NodeId$LocalNodeIdHistory.init(NodeId.java:179) at org.apache.cassandra.utils.NodeId.clinit(NodeId.java:38) at org.apache.cassandra.utils.NodeId$OneShotRenewer.init(NodeId.java:159) at org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1317) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2773) Index manager cannot support deleting and inserting into a row in the same mutation
[ https://issues.apache.org/jira/browse/CASSANDRA-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Ancona updated CASSANDRA-2773: -- Attachment: v1-0002-CASSANDRA-2773-Add-unit-tests-to-verfy-fix-cherry-pick.txt v1-0001-allow-deleting-a-rowand-updating-indexed-columns-init-.txt Index manager cannot support deleting and inserting into a row in the same mutation - Key: CASSANDRA-2773 URL: https://issues.apache.org/jira/browse/CASSANDRA-2773 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Boris Yen Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8.2 Attachments: 2773-v2.txt, 2773.txt, cassandra.log, v1-0001-allow-deleting-a-rowand-updating-indexed-columns-init-.txt, v1-0002-CASSANDRA-2773-Add-unit-tests-to-verfy-fix-cherry-pick.txt I use hector 0.8.0-1 and cassandra 0.8. 1. create mutator by using hector api, 2. Insert a few columns into the mutator for key key1, cf standard. 3. add a deletion to the mutator to delete the record of key1, cf standard. 4. repeat 2 and 3 5. execute the mutator. the result: the connection seems to be held by the sever forever, it never returns. when I tried to restart the cassandra I saw unsupportedexception : Index manager cannot support deleting and inserting into a row in the same mutation. and the cassandra is dead forever, unless I delete the commitlog. I would expect to get an exception when I execute the mutator, not after I restart the cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2773) Index manager cannot support deleting and inserting into a row in the same mutation
[ https://issues.apache.org/jira/browse/CASSANDRA-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055721#comment-13055721 ] Jim Ancona commented on CASSANDRA-2773: --- I applied the 0.8 patch and added a couple of tests to ColumnFamilyStoreTest. The tests trigger the UnsupportedOperationException in 0.7.6 and return the correct values with the patch applied. Would you like me to test the same scenario against an actual patched server? Index manager cannot support deleting and inserting into a row in the same mutation - Key: CASSANDRA-2773 URL: https://issues.apache.org/jira/browse/CASSANDRA-2773 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Boris Yen Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8.2 Attachments: 2773-v2.txt, 2773.txt, cassandra.log, v1-0001-allow-deleting-a-rowand-updating-indexed-columns-init-.txt, v1-0002-CASSANDRA-2773-Add-unit-tests-to-verfy-fix-cherry-pick.txt I use hector 0.8.0-1 and cassandra 0.8. 1. create mutator by using hector api, 2. Insert a few columns into the mutator for key key1, cf standard. 3. add a deletion to the mutator to delete the record of key1, cf standard. 4. repeat 2 and 3 5. execute the mutator. the result: the connection seems to be held by the sever forever, it never returns. when I tried to restart the cassandra I saw unsupportedexception : Index manager cannot support deleting and inserting into a row in the same mutation. and the cassandra is dead forever, unless I delete the commitlog. I would expect to get an exception when I execute the mutator, not after I restart the cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055726#comment-13055726 ] David Allsopp commented on CASSANDRA-2383: -- In cassandra.bat, change: -Dlog4j.configuration=log4j-server.properties^ to: -Dlog4j.configuration=file:///%CASSANDRA_HOME%/conf/log4j-server.properties^ I'm hopeful there's a less ugly way, but this seems to work. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055734#comment-13055734 ] David Allsopp commented on CASSANDRA-2383: -- OK, simpler solution is: -Dlog4j.configuration=file:conf/log4j-server.properties^ cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055734#comment-13055734 ] David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 8:39 PM: --- OK, simpler solution is: -Dlog4j.configuration=file:conf/log4j-server.properties^ But note that this only works if the current directory is CASSANDRA_HOME, so double-clicking on the batch file won't work, whereas the previous solution will work from the 'bin' directory, so double-clicking is OK. was (Author: dallsopp): OK, simpler solution is: -Dlog4j.configuration=file:conf/log4j-server.properties^ cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-2818: Attachment: 2818-v3.txt v2 has two problems: * It shuts the connection down slightly too aggressively, causing an exception on the remote side before setVersion gets called. * It stores the remote's version even when it is greater, causing the lower version node to always report itself as the newer version to the newer node. v3 address the first problem by sleeping for a half second before closing, and addresses the second by only calling setVersion if the remote side is compatible, otherwise it calls addSavedEndpoint before disconnecting so that it will reconnect. 0.8.0 is unable to participate with nodes using a _newer_ protocol version -- Key: CASSANDRA-2818 URL: https://issues.apache.org/jira/browse/CASSANDRA-2818 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Michael Allen Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818.txt When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of these in system.log: INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103) Received connection from newer protocol version. Ignorning message. and the node never joins the ring. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of HowToContribute by Joe Stein
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The HowToContribute page has been changed by Joe Stein: http://wiki.apache.org/cassandra/HowToContribute?action=diffrev1=37rev2=38 Comment: failed nosetests without the cql driver being installed first Setting up and running system tests: === Running the functional tests for Thrift RPC === + 1. Install CQL: `svn checkout https://svn.apache.org/repos/asf/cassandra/drivers; cd drivers/py; python setup.py build; sudo python setup.py install`. 1. Install the [[http://somethingaboutorange.com/mrl/projects/nose/0.11.1/|nose]] test runner (`aptitude install python-nose`, `easy_install nose`, etc). 1. Install the Thrift compiler (see InstallThrift) and Python libraries (`cd thrift/lib/py python setup.py install`). 1. Generate Cassandra's Python code using `ant gen-thrift-py`. + 1. Build the source `ant clean build`. 1. Run `nosetests test/system/` from the top-level source directory. If you need to modify the system tests, you probably only need to care about test/system/test_thrift_server.py. (test/system/__init__.py takes care of spawning new cassandra instances for each test and cleaning up afterwards so they are isolated.)
[jira] [Commented] (CASSANDRA-2773) Index manager cannot support deleting and inserting into a row in the same mutation
[ https://issues.apache.org/jira/browse/CASSANDRA-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055771#comment-13055771 ] Jonathan Ellis commented on CASSANDRA-2773: --- Thanks for the test case, Jim. If by the same scenario, you mean the workload that left your commitlog throwing exceptions, then yes please. Index manager cannot support deleting and inserting into a row in the same mutation - Key: CASSANDRA-2773 URL: https://issues.apache.org/jira/browse/CASSANDRA-2773 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Boris Yen Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8.2 Attachments: 2773-v2.txt, 2773.txt, cassandra.log, v1-0001-allow-deleting-a-rowand-updating-indexed-columns-init-.txt, v1-0002-CASSANDRA-2773-Add-unit-tests-to-verfy-fix-cherry-pick.txt I use hector 0.8.0-1 and cassandra 0.8. 1. create mutator by using hector api, 2. Insert a few columns into the mutator for key key1, cf standard. 3. add a deletion to the mutator to delete the record of key1, cf standard. 4. repeat 2 and 3 5. execute the mutator. the result: the connection seems to be held by the sever forever, it never returns. when I tried to restart the cassandra I saw unsupportedexception : Index manager cannot support deleting and inserting into a row in the same mutation. and the cassandra is dead forever, unless I delete the commitlog. I would expect to get an exception when I execute the mutator, not after I restart the cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-1125) Filter out ColumnFamily rows that aren't part of the query
[ https://issues.apache.org/jira/browse/CASSANDRA-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055503#comment-13055503 ] Mck SembWever edited comment on CASSANDRA-1125 at 6/27/11 9:31 PM: --- can this go into 0.8.1 ? ( and can we split this issue into two: 1) for KeyRange and 2) for IndexClause ) was (Author: michaelsembwever): can this go into 0.8.1 ? Filter out ColumnFamily rows that aren't part of the query -- Key: CASSANDRA-1125 URL: https://issues.apache.org/jira/browse/CASSANDRA-1125 Project: Cassandra Issue Type: New Feature Components: Hadoop Reporter: Jeremy Hanna Assignee: Mck SembWever Priority: Minor Fix For: 1.0 Attachments: CASSANDRA-1125.patch Currently, when running a MapReduce job against data in a Cassandra data store, it reads through all the data for a particular ColumnFamily. This could be optimized to only read through those rows that have to do with the query. It's a small change but wanted to put it in Jira so that it didn't fall through the cracks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055775#comment-13055775 ] Jonathan Ellis commented on CASSANDRA-2818: --- bq. It shuts the connection down slightly too aggressively, causing an exception on the remote side before setVersion gets called I can see that the initiating side could get pissed that the target closes the socket uncleanly -- what I don't get is how a sleep could make a difference. Is it on the reconnect? In which case the sleep is going to be fragile with a bigger cluster, since we depend on gossip to spread the version info. Do you have a sample stacktrace? 0.8.0 is unable to participate with nodes using a _newer_ protocol version -- Key: CASSANDRA-2818 URL: https://issues.apache.org/jira/browse/CASSANDRA-2818 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Michael Allen Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818.txt When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of these in system.log: INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103) Received connection from newer protocol version. Ignorning message. and the node never joins the ring. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1140333 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Author: jbellis
Date: Mon Jun 27 21:40:36 2011
New Revision: 1140333
URL: http://svn.apache.org/viewvc?rev=1140333view=rev
Log:
Add support for multiple (comma-delimited) coordinator addresses to CFIF
patch by Mck SembWever; reviewed by jbellis for CASSANDRA-2807
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1140333r1=1140332r2=1140333view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Jun 27 21:40:36 2011
@@ -5,6 +5,8 @@
* Expose number of threads blocked on submitting memtable to flush
(CASSANDRA-2817)
* add ability to return endpoints to nodetool (CASSANDRA-2776)
+ * Add support for multiple (comma-delimited) coordinator addresses
+ to ColumnFamilyInputFormat (CASSANDRA-2807)
0.8.1
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java?rev=1140333r1=1140332r2=1140333view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Mon Jun 27 21:40:36 2011
@@ -214,7 +214,30 @@ public class ColumnFamilyInputFormat ext
private ListTokenRange getRangeMap(Configuration conf) throws IOException
{
-Cassandra.Client client =
createConnection(ConfigHelper.getInitialAddress(conf),
ConfigHelper.getRpcPort(conf), true);
+String[] addresses = ConfigHelper.getInitialAddress(conf).split(,);
+Cassandra.Client client = null;
+ListIOException exceptions = new ArrayListIOException();
+for (String address : addresses)
+{
+try
+{
+client = createConnection(address,
ConfigHelper.getRpcPort(conf), true);
+break;
+}
+catch (IOException ioe)
+{
+exceptions.add(ioe);
+}
+}
+if (client == null)
+{
+logger.error(failed to connect to any initial addresses);
+for (IOException ioe : exceptions)
+{
+logger.error(, ioe);
+}
+throw exceptions.get(exceptions.size() - 1);
+}
ListTokenRange map;
try
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055778#comment-13055778 ] Jonathan Ellis commented on CASSANDRA-2383: --- log4j should already find it because of set CLASSPATH=%CASSANDRA_HOME%\conf cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2807:
--
Affects Version/s: (was: 0.8.0)
0.6
Fix Version/s: 0.8.1
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.6
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
Fix For: 0.8.1
Attachments: CASSANDRA-2807.patch
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055780#comment-13055780
]
Jeremy Hanna commented on CASSANDRA-2807:
-
is this an easy thing to commit to 0.7-branch as well?
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.6
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
Fix For: 0.8.1
Attachments: CASSANDRA-2807.patch
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055782#comment-13055782 ] David Allsopp commented on CASSANDRA-2383: -- Another solution that seems to work regardless of working directory is leave the original log4j.configuration line, but remove the line: -Dlog4j.defaultInitOverride=true^ This gets the classloader to find the config file as a resource, rather than supplying a file reference directly. However, I'm unsure why the defaultInitOverride was there in the first place... cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2653) index scan errors out when zero columns are requested
[
https://issues.apache.org/jira/browse/CASSANDRA-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055786#comment-13055786
]
Jonathan Ellis commented on CASSANDRA-2653:
---
doesn't this assert still have the the query to the index and the data is not
atomic problem?
index scan errors out when zero columns are requested
-
Key: CASSANDRA-2653
URL: https://issues.apache.org/jira/browse/CASSANDRA-2653
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.6, 0.8.0 beta 2
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 0.7.7, 0.8.1
Attachments:
0001-Handle-data-get-returning-null-in-secondary-indexes.patch,
0001-Handle-null-returns-in-data-index-query-v0.7.patch,
0001-Reset-SSTII-in-EchoedRow-constructor.patch, 2653_v2.patch,
v1-0001-CASSANDRA-2653-reproduce-regression.txt
As reported by Tyler Hobbs as an addendum to CASSANDRA-2401,
{noformat}
ERROR 16:13:38,864 Fatal exception in thread Thread[ReadStage:16,5,main]
java.lang.AssertionError: No data found for
SliceQueryFilter(start=java.nio.HeapByteBuffer[pos=10 lim=10 cap=30],
finish=java.nio.HeapByteBuffer[pos=17 lim=17 cap=30], reversed=false,
count=0] in DecoratedKey(81509516161424251288255223397843705139,
6b657931):QueryPath(columnFamilyName='cf', superColumnName='null',
columnName='null') (original filter
SliceQueryFilter(start=java.nio.HeapByteBuffer[pos=10 lim=10 cap=30],
finish=java.nio.HeapByteBuffer[pos=17 lim=17 cap=30], reversed=false,
count=0]) from expression 'cf.626972746864617465 EQ 1'
at
org.apache.cassandra.db.ColumnFamilyStore.scan(ColumnFamilyStore.java:1517)
at
org.apache.cassandra.service.IndexScanVerbHandler.doVerb(IndexScanVerbHandler.java:42)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055789#comment-13055789
]
Jonathan Ellis commented on CASSANDRA-2807:
---
It's time to be rigorous about keeping 0.7 stable -- if you want new features
that are probably safe, like this one, you should be using 0.8.x.
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.6
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
Fix For: 0.8.1
Attachments: CASSANDRA-2807.patch
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792 ] David Allsopp commented on CASSANDRA-2383: -- @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792 ] David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 10:00 PM: @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. was (Author: dallsopp): @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For:
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792 ] David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 9:59 PM: --- @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. was (Author: dallsopp): @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does no t exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Documents/Work/Coding/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792 ] David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 10:00 PM: @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. was (Author: dallsopp): @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.prop erties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/co nf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7
[jira] [Commented] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[
https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055796#comment-13055796
]
Brandon Williams commented on CASSANDRA-2818:
-
This repeats infinitely:
{noformat}
TRACE 22:02:38,187 cassandra-2/10.179.64.227 sending GOSSIP_DIGEST_SYN to
9@/10.179.65.102
DEBUG 22:02:38,188 error writing to /10.179.65.102
java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:114)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:90)
{noformat}
0.8.0 is unable to participate with nodes using a _newer_ protocol version
--
Key: CASSANDRA-2818
URL: https://issues.apache.org/jira/browse/CASSANDRA-2818
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.0
Reporter: Michael Allen
Assignee: Brandon Williams
Priority: Minor
Fix For: 0.8.2
Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818.txt
When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of
these in system.log:
INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103)
Received connection from newer protocol version. Ignorning message.
and the node never joins the ring.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055798#comment-13055798
]
Tomas Salfischberger commented on CASSANDRA-2830:
-
I did a really crude trail of adding the SUM keyword to the Cql.g definition
and handle it in the same way as COUNT is done. It works, I can SUM my counter
columns using CQL, but this is of course not a great way to implement it.
Any pointers for the right way to implement this? How do we want to define
generic aggregate functions in the grammar and what would be the best way to
handle them?
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[
https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055797#comment-13055797
]
Jonathan Ellis commented on CASSANDRA-2383:
---
Weird. Could you dig into the code in AbstractCassandraDaemon a little?
Here's what's trying to do load by classpath:
{code}
// load from the classpath.
configLocation =
AbstractCassandraDaemon.class.getClassLoader().getResource(config);
if (configLocation == null)
throw new RuntimeException(Couldn't figure out log4j
configuration.);
{code}
cassandra.bat does not handle folder names with space characters on windows
---
Key: CASSANDRA-2383
URL: https://issues.apache.org/jira/browse/CASSANDRA-2383
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.4
Environment: OS : windows
java : 1.6.0.23
Reporter: david lee
Assignee: Benjamin Coverston
Priority: Minor
Fix For: 0.7.7
when cassandra home folder is placed inside a folder which has space
characters in its name,
log4j settings are not properly loaded and warning messages are shown.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792 ] David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 10:04 PM: @Jonathan - yes, I thought so too, but it doesn't. The default init process for log4j is described at http://logging.apache.org/log4j/1.2/manual.html, but it doesn't really explain what happens if defaultInitOverride is set! With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. was (Author: dallsopp): @Jonathan - yes, I thought so too, but it doesn't. With -Dlog4j.debug=true and the original batch file, i.e. -Dlog4j.configuration=log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I see: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. If I remove the defaultInitOverride, I get: Starting Cassandra Server log4j: [/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] does not exist. log4j: Trying to find [log4j-server.properties] using context classloader sun.misc.Launcher$AppClassLoader@1a45a877. log4j: Using URL [file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties] for automatic log4j configuration. log4j: Reading configuration from URL file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties [etc...] Finally, with: -Dlog4j.configuration=file:conf/log4j-server.properties^ -Dlog4j.defaultInitOverride=true^ I get this, if current directory is CASSANDRA_HOME: Starting Cassandra Server log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. [etc...] But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking the batch file): Starting Cassandra Server log4j: [conf/log4j-server.properties] does not exist. log4j: Default initialization of overridden by log4j.defaultInitOverrideproperty. log4j:WARN No appenders could be found for logger (org.apache.cassandra.service.AbstractCassandraDaemon). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4
[jira] [Issue Comment Edited] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055798#comment-13055798
]
Tomas Salfischberger edited comment on CASSANDRA-2830 at 6/27/11 10:06 PM:
---
I did a really crude trial of adding the SUM keyword to the Cql.g definition
and handle it in the same way as COUNT is done. It works, I can SUM my counter
columns using CQL, but this is of course not a great way to implement it.
Any pointers for the right way to implement this? How do we want to define
generic aggregate functions in the grammar and what would be the best way to
handle them?
was (Author: t0mas):
I did a really crude trail of adding the SUM keyword to the Cql.g
definition and handle it in the same way as COUNT is done. It works, I can SUM
my counter columns using CQL, but this is of course not a great way to
implement it.
Any pointers for the right way to implement this? How do we want to define
generic aggregate functions in the grammar and what would be the best way to
handle them?
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055800#comment-13055800 ] David Allsopp commented on CASSANDRA-2383: -- http://www.vipan.com/htdocs/log4jhelp.html has the following: log4j.configuration=app_config.properties: First call to Category.getRoot() or Category.getInstance(...) method makes Log4j go through an initialization process. (You can watch that happening by setting log4j.debug=true.) During this process, Log4j looks in the application's classpath for a log4j.properties file *or the properties file you specify* via the this property key. However, you need to set this as a system property, for example by running your program with java -Dlog4j.configuration=app_config.properties This is because, if you set it in the configuration file, it is too late. Log4j would have already started to read the log4j.properties file by default, if available! cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: Benjamin Coverston Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[
https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055792#comment-13055792
]
David Allsopp edited comment on CASSANDRA-2383 at 6/27/11 10:13 PM:
@Jonathan - yes, I thought so too, but it doesn't. The default init process for
log4j is described at http://logging.apache.org/log4j/1.2/manual.html, but it
doesn't really explain what happens if defaultInitOverride is set!
With -Dlog4j.debug=true and the original batch file, i.e.
{quote}
-Dlog4j.configuration=log4j-server.properties^
-Dlog4j.defaultInitOverride=true^}}
{quote}
I see:
{quote}
Starting Cassandra Server
log4j:
[/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
does not exist.
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
log4j:WARN No appenders could be found for logger
(org.apache.cassandra.service.AbstractCassandraDaemon).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
{quote}
If I remove the defaultInitOverride, I get:
{quote}
Starting Cassandra Server
log4j:
[/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
does not exist.
log4j: Trying to find [log4j-server.properties] using context classloader
sun.misc.Launcher$AppClassLoader@1a45a877.
log4j: Using URL
[file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
for automatic log4j configuration.
log4j: Reading configuration from URL
file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties
[etc...]
{quote}
Finally, with:
{quote}
-Dlog4j.configuration=file:conf/log4j-server.properties^
-Dlog4j.defaultInitOverride=true^
{quote}
I get this, if current directory is CASSANDRA_HOME:
{quote}
Starting Cassandra Server
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
[etc...]
{quote}
But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking
the batch file):
{quote}
Starting Cassandra Server
log4j: [conf/log4j-server.properties] does not exist.
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
log4j:WARN No appenders could be found for logger
(org.apache.cassandra.service.AbstractCassandraDaemon).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
{quote}
was (Author: dallsopp):
@Jonathan - yes, I thought so too, but it doesn't. The default init process
for log4j is described at http://logging.apache.org/log4j/1.2/manual.html, but
it doesn't really explain what happens if defaultInitOverride is set!
With -Dlog4j.debug=true and the original batch file, i.e.
-Dlog4j.configuration=log4j-server.properties^
-Dlog4j.defaultInitOverride=true^
I see:
Starting Cassandra Server
log4j:
[/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
does not exist.
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
log4j:WARN No appenders could be found for logger
(org.apache.cassandra.service.AbstractCassandraDaemon).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
If I remove the defaultInitOverride, I get:
Starting Cassandra Server
log4j:
[/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
does not exist.
log4j: Trying to find [log4j-server.properties] using context classloader
sun.misc.Launcher$AppClassLoader@1a45a877.
log4j: Using URL
[file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties]
for automatic log4j configuration.
log4j: Reading configuration from URL
file:/C:/Users/David/Key%20Value/apache-cassandra-0.7.6-2/conf/log4j-server.properties
[etc...]
Finally, with:
-Dlog4j.configuration=file:conf/log4j-server.properties^
-Dlog4j.defaultInitOverride=true^
I get this, if current directory is CASSANDRA_HOME:
Starting Cassandra Server
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
[etc...]
But this, if current directory is CASSANDRA_HOME/bin (e.g. if double-clicking
the batch file):
Starting Cassandra Server
log4j: [conf/log4j-server.properties] does not exist.
log4j: Default initialization of overridden by
log4j.defaultInitOverrideproperty.
log4j:WARN No appenders could be found for logger
(org.apache.cassandra.service.AbstractCassandraDaemon).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
cassandra.bat does not handle folder names with space characters on windows
[jira] [Commented] (CASSANDRA-2830) Allow summing of counter columns in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055802#comment-13055802
]
Jonathan Ellis commented on CASSANDRA-2830:
---
Aggregate functions really needs to wait for CASSANDRA-2474 and its
treat-a-row-as-a-table feature.
Allow summing of counter columns in CQL
---
Key: CASSANDRA-2830
URL: https://issues.apache.org/jira/browse/CASSANDRA-2830
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Tomas Salfischberger
Priority: Minor
Labels: CQL
CQL could be extended with a method to calculate the sum of a set of counter
columns. This avoids transferring a long list of counter columns to be summed
by the client, while the server could calculate the total and instead only
transfer that result. My proposal for the syntax (based on the COUNT()
suggestion in the comments of CASSANDRA-1704):
{code}SELECT SUM(columnFrom..columnTo) FROM CF WHERE ...{code}
The simplest approach would be to only allow summing of counters under the
same key, thus a query with a WHERE part that specifies multiple keys would
return 1 result per key. This avoids summing values from different nodes.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2383) cassandra.bat does not handle folder names with space characters on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2383: - Assignee: T Jake Luciani (was: Benjamin Coverston) cassandra.bat does not handle folder names with space characters on windows --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2383) log4j unable to load properties file from classpath
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2383: -- Summary: log4j unable to load properties file from classpath (was: cassandra.bat does not handle folder names with space characters on windows) log4j unable to load properties file from classpath --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2383) log4j unable to load properties file from classpath
[ https://issues.apache.org/jira/browse/CASSANDRA-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055807#comment-13055807 ] David Allsopp commented on CASSANDRA-2383: -- @Jonathan - will try to take a look soon. The getResource() stuff might perhaps be affected by the classes being within a jarfile. log4j unable to load properties file from classpath --- Key: CASSANDRA-2383 URL: https://issues.apache.org/jira/browse/CASSANDRA-2383 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.4 Environment: OS : windows java : 1.6.0.23 Reporter: david lee Assignee: T Jake Luciani Priority: Minor Fix For: 0.7.7 when cassandra home folder is placed inside a folder which has space characters in its name, log4j settings are not properly loaded and warning messages are shown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2807) ColumnFamilyInputFormat configuration should support multiple initial addresses
[
https://issues.apache.org/jira/browse/CASSANDRA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055881#comment-13055881
]
Hudson commented on CASSANDRA-2807:
---
Integrated in Cassandra-0.8 #194 (See
[https://builds.apache.org/job/Cassandra-0.8/194/])
Add support for multiple (comma-delimited) coordinator addresses to CFIF
patch by Mck SembWever; reviewed by jbellis for CASSANDRA-2807
jbellis :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1140333
Files :
* /cassandra/branches/cassandra-0.8/CHANGES.txt
*
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
ColumnFamilyInputFormat configuration should support multiple initial
addresses
---
Key: CASSANDRA-2807
URL: https://issues.apache.org/jira/browse/CASSANDRA-2807
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.6
Reporter: Greg Katz
Assignee: Mck SembWever
Priority: Minor
Fix For: 0.8.1
Attachments: CASSANDRA-2807.patch
The {{ColumnFamilyInputFormat}} class only allows a single initial node to be
specified through the cassandra.thrift.address configuration property. The
configuration should support a list of nodes in order to account for the
possibility that the initial node becomes unavailable.
By contrast, the {{RingCache}} class used by the {{ColumnFamilyRecordWriter}}
reads the exact same {{cassandra.thrift.address}} property but splits its
value on commas to allow multiple initial nodes to be specified.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055885#comment-13055885 ] Jonathan Ellis commented on CASSANDRA-2818: --- v4 also changes the current-version to 3, so we don't create a version exhaustion problem for ourselves (see comment in MS). 0.8.0 is unable to participate with nodes using a _newer_ protocol version -- Key: CASSANDRA-2818 URL: https://issues.apache.org/jira/browse/CASSANDRA-2818 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Michael Allen Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818-v4.txt, 2818.txt When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of these in system.log: INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103) Received connection from newer protocol version. Ignorning message. and the node never joins the ring. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2818: -- Attachment: 2818-v4.txt That makes sense for v2, yeah. I realized that we don't actually need to reconnect to send old-version messages -- version is per-Message, the connection itself is basically just a queue around a socket. v4 attached that doesn't drop (non-streaming) connections at all. (This is part of the how did it possibly work on 0.7? answer, I think.) 0.8.0 is unable to participate with nodes using a _newer_ protocol version -- Key: CASSANDRA-2818 URL: https://issues.apache.org/jira/browse/CASSANDRA-2818 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Michael Allen Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818-v4.txt, 2818.txt When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of these in system.log: INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103) Received connection from newer protocol version. Ignorning message. and the node never joins the ring. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055887#comment-13055887 ] Jonathan Ellis commented on CASSANDRA-2521: --- +1 from me in principle, suggest waiting for Terje's test before committing tho :) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055888#comment-13055888 ] Jonathan Ellis commented on CASSANDRA-2521: --- (suggest adding a comment for the markCompacted acquire/release code on commit.) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2832) reduce variance in HH impact between wide and narrow rows
reduce variance in HH impact between wide and narrow rows - Key: CASSANDRA-2832 URL: https://issues.apache.org/jira/browse/CASSANDRA-2832 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.7, 0.8.2 Attachments: 2832.txt default page_size of 1 is huge, and makes it impossible to set a hinted_handoff_throttle_delay_in_ms that works well for both wide rows and narrow. at the same time you don't want to make it TOO small because that will hurt performance on the source node (seeking to the hinted row repeatedly). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2832) reduce variance in HH impact between wide and narrow rows
[ https://issues.apache.org/jira/browse/CASSANDRA-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2832: -- Attachment: 2832.txt patch to set max page_size to 128. reduce variance in HH impact between wide and narrow rows - Key: CASSANDRA-2832 URL: https://issues.apache.org/jira/browse/CASSANDRA-2832 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.7, 0.8.2 Attachments: 2832.txt default page_size of 1 is huge, and makes it impossible to set a hinted_handoff_throttle_delay_in_ms that works well for both wide rows and narrow. at the same time you don't want to make it TOO small because that will hurt performance on the source node (seeking to the hinted row repeatedly). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2818) 0.8.0 is unable to participate with nodes using a _newer_ protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056191#comment-13056191 ] Brandon Williams commented on CASSANDRA-2818: - +1 0.8.0 is unable to participate with nodes using a _newer_ protocol version -- Key: CASSANDRA-2818 URL: https://issues.apache.org/jira/browse/CASSANDRA-2818 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.0 Reporter: Michael Allen Assignee: Brandon Williams Priority: Minor Fix For: 0.8.2 Attachments: 2818-disconnect.txt, 2818-v2.txt, 2818-v3.txt, 2818-v4.txt, 2818.txt When a 0.8.1 node tries to join a 0.8.0 ring, we see an endless supply of these in system.log: INFO [Thread-4] 2011-06-23 21:14:04,149 IncomingTcpConnection.java (line 103) Received connection from newer protocol version. Ignorning message. and the node never joins the ring. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1140385 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java
Author: jbellis Date: Tue Jun 28 00:14:09 2011 New Revision: 1140385 URL: http://svn.apache.org/viewvc?rev=1140385view=rev Log: reduce max HH pageSize to 1024 patch by jbellis; reviewed by brandonwilliams for CASSANDRA-2832 Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java?rev=1140385r1=1140384r2=1140385view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java Tue Jun 28 00:14:09 2011 @@ -92,7 +92,7 @@ public class HintedHandOffManager implem public static final String HINTS_CF = HintsColumnFamily; private static final Logger logger_ = LoggerFactory.getLogger(HintedHandOffManager.class); -private static final int PAGE_SIZE = 1; +private static final int PAGE_SIZE = 1024; private static final String SEPARATOR = -; private static final int LARGE_NUMBER = 65536; // 64k nodes ought to be enough for anybody.
[jira] [Created] (CASSANDRA-2833) CounterColumn should have an optional binary field so that double can be incremented/decremented along with long
CounterColumn should have an optional binary field so that double can be incremented/decremented along with long Key: CASSANDRA-2833 URL: https://issues.apache.org/jira/browse/CASSANDRA-2833 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Joe Stein Currently CounterColumn only has a long making it not feasible to track increment/decrement of durations or other values common to analytics represented as a double The change I am proposing to implement, after some discussions/advice in the irc to issues raised, is to add a new optional binary field to CounterColumn (thrift). I was thinking we could call it *operand* Under the hood (src/java/org/apache/cassandra/db/CounterColumn.java) I would handle things with byte[] moving between long and double as internal helper functions with case switch on type of operand we are setting might also need an optional enum for type perhaps too so the client can let the server know how it should materialize the bytes for when it += the value stored The clients should continue to work as expected and folks looking to use this can just do so. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056239#comment-13056239 ] Terje Marthinussen commented on CASSANDRA-2521: --- First impression is good. I merged this with 2816 last night and tested overnight with a dataset that normally uses 32-40% disk space when compacted. Calling repair on a handful of nodes, I would normally have a good chance of seeing a node or two with 90%+ full disk and regular full GC's. With these patches, disk use seem to peak in the 55-65% disk area which is a very significant improvement. However, unfortunately, I started repairs on 5 of 12 nodes and one of them has gone crazy however and filled the disk 100%. :( I did happen to start repair twice on this node by accident. I somewhat do not think that is the real problem however, but I am not sure. Will make a ticket anyway on adding functionality to prevent repairs from getting started twice on the same CF and I will test more to see if this happens again. However I noticed this in the log: INFO [Thread-185] 2011-06-28 05:01:15,390 StorageService.java (line 2083) requesting GC to free disk space I guess we can get rid of that? Should be nothing good from calling full GC there anymore? If anything, maybe we should consider a variation here where cassandra instead aborts all repairs and compactions and clean up everything before trying again with smaller amounts of data in each compaction/repair attempt? Another ticket on that maybe? Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2834) Avoid repair getting started twice at the same time for the same CF
Avoid repair getting started twice at the same time for the same CF --- Key: CASSANDRA-2834 URL: https://issues.apache.org/jira/browse/CASSANDRA-2834 Project: Cassandra Issue Type: Improvement Reporter: Terje Marthinussen It may seem like it is possible to start repair twice at the same time on the same CF? Not 100% verified, but if this is indeed the case, we may want to consider avoiding that including making nodetool repair abort and return and error if repair is attempted on the same CF as one which already have repair running. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2832) reduce variance in HH impact between wide and narrow rows
[ https://issues.apache.org/jira/browse/CASSANDRA-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056241#comment-13056241 ] Hudson commented on CASSANDRA-2832: --- Integrated in Cassandra-0.7 #513 (See [https://builds.apache.org/job/Cassandra-0.7/513/]) reduce max HH pageSize to 1024 patch by jbellis; reviewed by brandonwilliams for CASSANDRA-2832 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1140385 Files : * /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/HintedHandOffManager.java reduce variance in HH impact between wide and narrow rows - Key: CASSANDRA-2832 URL: https://issues.apache.org/jira/browse/CASSANDRA-2832 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.7, 0.8.2 Attachments: 2832.txt default page_size of 1 is huge, and makes it impossible to set a hinted_handoff_throttle_delay_in_ms that works well for both wide rows and narrow. at the same time you don't want to make it TOO small because that will hurt performance on the source node (seeking to the hinted row repeatedly). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056263#comment-13056263 ] Terje Marthinussen commented on CASSANDRA-2521: --- Looking a bit more on files and logs from the node with full disk. I suspect things mainly went wrong due to 2 repairs (if they actually both ran) 1. Seems there are some Compacted files around which does not get cleaned up, but I guess these may be results of references acquired which are not freed as the streaming fills up the disk and fails. 2. there are no less but 53 -tmp- files. A lot of concurrent streams here! I still think it may be a good idea if we could detect that we were heading for full disk and abort everything before things crashed. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang reassigned CASSANDRA-2819: -- Assignee: Melvin Wang Split rpc timeout for read and write ops Key: CASSANDRA-2819 URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Stu Hood Assignee: Melvin Wang Fix For: 1.0 Given the vastly different latency characteristics of reads and writes, it makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056268#comment-13056268 ] Jonathan Ellis commented on CASSANDRA-2521: --- bq. I still think it may be a good idea if we could detect that we were heading for full disk and abort everything before things crashed That's a good feature to have, but out of scope here. :) CASSANDRA-809 is open as a general deal better with running out of space ticket, but we can open more specific ones. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-957) convenience workflow for replacing dead node
[ https://issues.apache.org/jira/browse/CASSANDRA-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Goffinet updated CASSANDRA-957: - Assignee: (was: Chris Goffinet) convenience workflow for replacing dead node Key: CASSANDRA-957 URL: https://issues.apache.org/jira/browse/CASSANDRA-957 Project: Cassandra Issue Type: Wish Components: Core, Tools Reporter: Jonathan Ellis Fix For: 1.0 Attachments: 0001-Support-bringing-back-a-node-to-the-cluster-that-exi.patch, 0002-Do-not-include-local-node-when-computing-workMap.patch Original Estimate: 24h Remaining Estimate: 24h Replacing a dead node with a new one is a common operation, but nodetool removetoken followed by bootstrap is inefficient (re-replicating data first to the remaining nodes, then to the new one) and manually bootstrapping to a token just less than the old one's, followed by nodetool removetoken is slightly painful and prone to manual errors. First question: how would you expose this in our tool ecosystem? It needs to be a startup-time option to the new node, so it can't be nodetool, and messing with the config xml definitely takes the convenience out. A one-off -DreplaceToken=XXY argument? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-957) convenience workflow for replacing dead node
[ https://issues.apache.org/jira/browse/CASSANDRA-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056293#comment-13056293 ] Chris Goffinet commented on CASSANDRA-957: -- I am going to defer this to anyone else who would like to pick up this ticket. I just do not have the spare time to focus on this. convenience workflow for replacing dead node Key: CASSANDRA-957 URL: https://issues.apache.org/jira/browse/CASSANDRA-957 Project: Cassandra Issue Type: Wish Components: Core, Tools Reporter: Jonathan Ellis Fix For: 1.0 Attachments: 0001-Support-bringing-back-a-node-to-the-cluster-that-exi.patch, 0002-Do-not-include-local-node-when-computing-workMap.patch Original Estimate: 24h Remaining Estimate: 24h Replacing a dead node with a new one is a common operation, but nodetool removetoken followed by bootstrap is inefficient (re-replicating data first to the remaining nodes, then to the new one) and manually bootstrapping to a token just less than the old one's, followed by nodetool removetoken is slightly painful and prone to manual errors. First question: how would you expose this in our tool ecosystem? It needs to be a startup-time option to the new node, so it can't be nodetool, and messing with the config xml definitely takes the convenience out. A one-off -DreplaceToken=XXY argument? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of UseCases by TimSmith
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The UseCases page has been changed by TimSmith: http://wiki.apache.org/cassandra/UseCases?action=diffrev1=14rev2=15 Comment: typo [[https://www.cloudkick.com/blog/2010/mar/02/4_months_with_cassandra/|Cloudkick]] implements time-series down at the second-level with roll-ups. == An implementation of some DBMS rules written in python using pycassa == - We have created a DBMS layer that handles references to other columnfamilys (foreign keys), Automatic reverse linking. required fields in columnfamilys and datatypes (long and datetime). It wraps the get, get_range, insert, remove funtions of pycassas columnfamilys. At this time it is limited to: on delete cascade and positive long numbers but this could change if there is enough interest. It suits our project. + We have created a DBMS layer that handles references to other columnfamilys (foreign keys), Automatic reverse linking. required fields in columnfamilys and datatypes (long and datetime). It wraps the get, get_range, insert, remove functions of pycassas columnfamilys. At this time it is limited to: on delete cascade and positive long numbers but this could change if there is enough interest. It suits our project. [[ThomasBoose/dbms implementation]]
[Cassandra Wiki] Update of Operations by TimSmith
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The Operations page has been changed by TimSmith:
http://wiki.apache.org/cassandra/Operations?action=diffrev1=94rev2=95
Comment:
Fix typos
See PerformanceTuning
== Schema management ==
- Server clocks should be synchronized with something like ntp. Otherwise,
schema changes may be rejected as being obsolete.
+ Server clocks should be synchronized with something like NTP. Otherwise,
schema changes may be rejected as being obsolete.
See LiveSchemaUpdates [refers to functionality in 0.7]
@@ -23, +23 @@
=== Token selection ===
Using a strong hash function means !RandomPartitioner keys will, on average,
be evenly spread across the Token space, but you can still have imbalances if
your Tokens do not divide up the range evenly, so you should specify
!InitialToken to your first nodes as `i * (2**127 / N)` for i = 0 .. N-1. In
Cassandra 0.7, you should specify `initial_token` in `cassandra.yaml`.
- With !NetworkTopologyStrategy, you should calculate the tokens the nodes in
each DC independantly. Tokens still neded to be unique, so you can add 1 to the
tokens in the 2nd DC, add 2 in the 3rd, and so on. Thus, for a 4-node cluster
in 2 datacenters, you would have
+ With !NetworkTopologyStrategy, you should calculate the tokens the nodes in
each DC independently. Tokens still needed to be unique, so you can add 1 to
the tokens in the 2nd DC, add 2 in the 3rd, and so on. Thus, for a 4-node
cluster in 2 datacenters, you would have
{{{
DC1
node 1 = 0
@@ -91, +91 @@
Important things to note:
1. You should wait long enough for all the nodes in your cluster to become
aware of the bootstrapping node via gossip before starting another bootstrap.
The new node will log Bootstrapping when this is safe, 2 minutes after
starting. (90s to make sure it has accurate load information, and 30s waiting
for other nodes to start sending it inserts happening in its to-be-assumed part
of the token ring.)
- 1. Relating to point 1, one can only boostrap N nodes at a time with
automatic token picking, where N is the size of the existing cluster. If you
need to more than double the size of your cluster, you have to wait for the
first N nodes to finish until your cluster is size 2N before bootstrapping more
nodes. So if your current cluster is 5 nodes and you want add 7 nodes,
bootstrap 5 and let those finish before boostrapping the last two.
+ 1. Relating to point 1, one can only bootstrap N nodes at a time with
automatic token picking, where N is the size of the existing cluster. If you
need to more than double the size of your cluster, you have to wait for the
first N nodes to finish until your cluster is size 2N before bootstrapping more
nodes. So if your current cluster is 5 nodes and you want add 7 nodes,
bootstrap 5 and let those finish before bootstrapping the last two.
1. As a safety measure, Cassandra does not automatically remove data from
nodes that lose part of their Token Range to a newly added node. Run
`nodetool cleanup` on the source node(s) (neighboring nodes that shared the
same subrange) when you are satisfied the new node is up and working. If you do
not do this the old data will still be counted against the load on that node
and future bootstrap attempts at choosing a location will be thrown off.
1. When bootstrapping a new node, existing nodes have to divide the key
space before beginning replication. This can take awhile, so be patient.
1. During bootstrap, a node will drop the Thrift port and will not be
accessible from `nodetool`.
@@ -148, +148 @@
Consider how to schedule your repairs. A repair causes additional disk and
CPU activity on the nodes participating in the repair, and it will typically be
a good idea to spread repairs out over time so as to minimize the chances of
repairs running concurrently on many nodes.
Dealing with the consequences of nodetool repair not running within
GCGraceSeconds
- If `nodetool repair` has not been run often enough to the pointthat
GCGraceSeconds has passed, you risk forgotten deletes (see DistributedDeletes).
In addition to data popping up that has been deleted, you may see
inconsistencies in data return from different nodes that will not self-heal by
read-repair or further `nodetool repair`. Some further details on this latter
effect is documented in
[[https://issues.apache.org/jira/browse/CASSANDRA-1316|CASSANDRA-1316]].
+ If `nodetool repair` has not been run often enough to the point that
GCGraceSeconds has passed, you risk forgotten deletes (see DistributedDeletes).
In addition to data popping up that has been deleted, you may see
inconsistencies in data return from different nodes that will not self-heal by
read-repair or further `nodetool repair`. Some further details on this latter
effect is documented in
