[jira] [Commented] (CASSANDRA-2540) Data reads by default
[
https://issues.apache.org/jira/browse/CASSANDRA-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026880#comment-13026880
]
Terje Marthinussen commented on CASSANDRA-2540:
---
Would it make sense to make dynamic snitch react to digest but not data
error directly to improve recovery time and why do we need to wait for the
100ms to make dynamic snitch work?
Couldn't we add a load balancer like function that would detect if a node has a
significant number of outstanding requests on node 1 vs. 2 and 3, then send to
node 2 instead?
Overall, I am not asking for a round robin load balancer though... (not good
for caching)
The digest function may also in some cases maybe be made dynamic based on the
size of the data being read. That is, just send the data for small data sizes
and use digests for large responses?
No, I don't know all the details on how this part of the code work, so my
suggestions may be totally wrong :)
I do wonder however if we get enough of these timeouts to actually be a
problem. If there is a couple of delays in latencies for a few seconds 2-3
times a week/month, no problem.
However, if we have so many of these errors that people see them many times a
day, it would seem like this is a performance problem somewhere in cassandra
which should be fixed rather than applying some patchwork which hide it...
Data reads by default
-
Key: CASSANDRA-2540
URL: https://issues.apache.org/jira/browse/CASSANDRA-2540
Project: Cassandra
Issue Type: Wish
Reporter: Stu Hood
Priority: Minor
The intention of digest vs data reads is to save bandwidth in the read path
at the cost of latency, but I expect that this has been a premature
optimization.
* Data requested by a read will often be within an order of magnitude of the
digest size, and a failed digest means extra roundtrips, more bandwidth
* The [digest reads but not your data
read|https://issues.apache.org/jira/browse/CASSANDRA-2282?focusedCommentId=13004656page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13004656]
problem means failing QUORUM reads because a single node is unavailable, and
would require eagerly re-requesting at some fraction of your timeout
* Saving bandwidth in cross datacenter usecases comes at huge cost to
latency, but since both constraints change proportionally (enough), the
tradeoff is not clear
Some options:
# Add an option to use digest reads
# Remove digest reads entirely (and/or punt and make them a runtime
optimization based on data size in the future)
# Continue to use digest reads, but send them to {{N - R}} nodes for
(somewhat) more predicatable behavior with QUORUM
\\
The outcome of data-reads-by-default should be significantly improved
latency, with a moderate increase in bandwidth usage for large reads.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2477) CQL support for describing keyspaces / column familes
[ https://issues.apache.org/jira/browse/CASSANDRA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026889#comment-13026889 ] Vivek Mishra commented on CASSANDRA-2477: - This is to do with CQL jdbc support? CQL support for describing keyspaces / column familes - Key: CASSANDRA-2477 URL: https://issues.apache.org/jira/browse/CASSANDRA-2477 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2475) Prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026890#comment-13026890 ] Vivek Mishra commented on CASSANDRA-2475: - Eric, Can you please provide more information on this JIRA? what needs to be done? I can take up this issue. Vivek Prepared statements --- Key: CASSANDRA-2475 URL: https://issues.apache.org/jira/browse/CASSANDRA-2475 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2536) Schema disagreements when using connections to multiple hosts
[ https://issues.apache.org/jira/browse/CASSANDRA-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026896#comment-13026896 ] Sylvain Lebresne commented on CASSANDRA-2536: - I'll hijack this conversation by saying that I think we should start advertising that people should try to keep their server clocks in sync unless they have a good reason not too (which would legitimize the fact that timestamp comparison is good enough). Counter removes for instance use server side timestamps and would be screwed up by diverging clocks (and by that I mean more screwed up than they already are by design). And really, is there any reason not to install a ntpd server in the first place anyway? Schema disagreements when using connections to multiple hosts - Key: CASSANDRA-2536 URL: https://issues.apache.org/jira/browse/CASSANDRA-2536 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 beta 1 Environment: Two node 0.8-beta1 cluster with one seed and JNA. Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 2536-compare-timestamp.txt, schema_disagree.py If you have two thrift connections open to different nodes and you create a KS using the first, then a CF in that KS using the second, you wind up with a schema disagreement even if you wait/sleep after creating the KS. The attached script reproduces the issue using pycassa (1.0.6 should work fine, although it has the 0.7 thrift-gen code). It's also reproducible by hand with two cassandra-cli sessions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of DebianPackaging by SylvainLebresne
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The DebianPackaging page has been changed by SylvainLebresne. http://wiki.apache.org/cassandra/DebianPackaging?action=diffrev1=20rev2=21 -- gpg --keyserver pgp.mit.edu --recv-keys F758CE318D77295D gpg --export --armor F758CE318D77295D | sudo apt-key add - }}} + + Starting with the 0.7.5 debian package, you will also need to add public key 2B5C1B00 using the same commands as above (the list of Apache contributors public keys is available at [[http://www.apache.org/dist/cassandra/KEYS]]). And then
[jira] [Commented] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026918#comment-13026918 ] Stu Hood commented on CASSANDRA-1278: - Very interesting work here! * Did you accidentally remove the buffering for non-streaming connections in IncomingTCPConnection? * What kind of failures are supported during a load? What's the proper behavior for flush_proxy when some hosts fail? * Could we avoid coding in knowledge of the file format in the inner loop of IncomingLoaderStreamReader? I would much, much prefer that non-file-format-specific framing be added, and it would have the added benefit of not requiring as many system calls (4 per row vs 2 per frame) * What is the benefit of using an independent protocol for Loader streams? Again, awesome. Make bulk loading into Cassandra less crappy, more pluggable Key: CASSANDRA-1278 URL: https://issues.apache.org/jira/browse/CASSANDRA-1278 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jeremy Hanna Assignee: Matthew F. Dennis Fix For: 0.8.1 Attachments: 1278-cassandra-0.7-v2.txt, 1278-cassandra-0.7.1.txt, 1278-cassandra-0.7.txt Original Estimate: 40h Time Spent: 40h 40m Remaining Estimate: 0h Currently bulk loading into Cassandra is a black art. People are either directed to just do it responsibly with thrift or a higher level client, or they have to explore the contrib/bmt example - http://wiki.apache.org/cassandra/BinaryMemtable That contrib module requires delving into the code to find out how it works and then applying it to the given problem. Using either method, the user also needs to keep in mind that overloading the cluster is possible - which will hopefully be addressed in CASSANDRA-685 This improvement would be to create a contrib module or set of documents dealing with bulk loading. Perhaps it could include code in the Core to make it more pluggable for external clients of different types. It is just that this is something that many that are new to Cassandra need to do - bulk load their data into Cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2580) stress will not use existing keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-2580: --- Attachment: CASSANDRA-2580.patch System.exit(1); was added by Gary Dusbabek at 2011-04-15 12:48:15 +, only cassandra-0.8 and trunk are affected this patch applies for both of them. stress will not use existing keyspaces -- Key: CASSANDRA-2580 URL: https://issues.apache.org/jira/browse/CASSANDRA-2580 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8 beta 1 Reporter: Brandon Williams Assignee: Pavel Yaskevich Fix For: 0.8.0, 0.8.1 Attachments: CASSANDRA-2580.patch cassandra-3:/srv/cassandra# tools/stress/bin/stress -n 1 -d cassandra-2 -i 1 -o insert Created keyspaces. Sleeping 1s for propagation. total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time 1,1,1,0.049,0 cassandra-3:/srv/cassandra# tools/stress/bin/stress -n 1 -d cassandra-2 -i 1 -o insert Unable to create stress keyspace: Keyspace already exists. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2536) Schema disagreements when using connections to multiple hosts
[ https://issues.apache.org/jira/browse/CASSANDRA-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026953#comment-13026953 ] Gary Dusbabek commented on CASSANDRA-2536: -- I think timestamp comparisons will be fine. Schema disagreements when using connections to multiple hosts - Key: CASSANDRA-2536 URL: https://issues.apache.org/jira/browse/CASSANDRA-2536 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 beta 1 Environment: Two node 0.8-beta1 cluster with one seed and JNA. Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 2536-compare-timestamp.txt, schema_disagree.py If you have two thrift connections open to different nodes and you create a KS using the first, then a CF in that KS using the second, you wind up with a schema disagreement even if you wait/sleep after creating the KS. The attached script reproduces the issue using pycassa (1.0.6 should work fine, although it has the 0.7 thrift-gen code). It's also reproducible by hand with two cassandra-cli sessions. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2540) Data reads by default
[
https://issues.apache.org/jira/browse/CASSANDRA-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026955#comment-13026955
]
Chris Burroughs commented on CASSANDRA-2540:
All of the examples here are QUORUM, but I think ONE should have the most
dramatic latency benefit from this. Option (1) sounds like the best choice
now, with something clever added later if the benefits are demonstrated.
Data reads by default
-
Key: CASSANDRA-2540
URL: https://issues.apache.org/jira/browse/CASSANDRA-2540
Project: Cassandra
Issue Type: Wish
Reporter: Stu Hood
Priority: Minor
The intention of digest vs data reads is to save bandwidth in the read path
at the cost of latency, but I expect that this has been a premature
optimization.
* Data requested by a read will often be within an order of magnitude of the
digest size, and a failed digest means extra roundtrips, more bandwidth
* The [digest reads but not your data
read|https://issues.apache.org/jira/browse/CASSANDRA-2282?focusedCommentId=13004656page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13004656]
problem means failing QUORUM reads because a single node is unavailable, and
would require eagerly re-requesting at some fraction of your timeout
* Saving bandwidth in cross datacenter usecases comes at huge cost to
latency, but since both constraints change proportionally (enough), the
tradeoff is not clear
Some options:
# Add an option to use digest reads
# Remove digest reads entirely (and/or punt and make them a runtime
optimization based on data size in the future)
# Continue to use digest reads, but send them to {{N - R}} nodes for
(somewhat) more predicatable behavior with QUORUM
\\
The outcome of data-reads-by-default should be significantly improved
latency, with a moderate increase in bandwidth usage for large reads.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2408) Faulty memory causes adjacent nodes to have invalid data and fail compactation (java.io.IOException: Keys must be written in ascending order.)
[ https://issues.apache.org/jira/browse/CASSANDRA-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2408. --- Resolution: Duplicate Fix Version/s: (was: 0.7.6) CASSANDRA-1717 will address not propagating corrupt data. Faulty memory causes adjacent nodes to have invalid data and fail compactation (java.io.IOException: Keys must be written in ascending order.) -- Key: CASSANDRA-2408 URL: https://issues.apache.org/jira/browse/CASSANDRA-2408 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Reporter: Thibaut Hi, We had to replace a node with faulty ram. Besides the cassandra cluster getting unresponsive, we also observerd a keys must be written in ascending order exception on all adjacent quorum nodes, causing their compactation to fail. We had another node with faulty ram a week earlier, and it was causing the same errors on it's neighbours. A faulty node shouldn't affect the key ordering of adjacent nodes? INFO [CompactionExecutor:1] 2011-03-31 09:57:29,529 CompactionManager.java (line 396) Compacting [SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1793-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1798-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1803-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1808-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1832-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1841-Data.db'),SSTableReader(path='/cassandra/data/table_lists/table_lists-f-1852-Data.db')] INFO [CompactionExecutor:1] 2011-03-31 09:58:26,437 SSTableWriter.java (line 108) Last written key : DecoratedKey(ba578bed5e75a06d1156dabccc68abd5_www.abc.com_hostinlinks_9223370735413988125_870cc25d-7e4f-437b-aa83-a311dfbf7f88, 62613537386265643565373561303664313135366461626363633638616264355f772e6b6f7475736f7a6c756b2e636f6d5f686f7374696e6c696e6b735f39323237303733353431333938383132355f38373063633235642d376534662d343337622d616138332d613331316466626637663838) INFO [CompactionExecutor:1] 2011-03-31 09:58:26,515 SSTableWriter.java (line 109) Current key : DecoratedKey(ba578bed5e75a06d1156dabccc68abd5_www.abc.com_hostinlinks_9223370735413988136_17070303-ab24-4207-83bd-1604e2c159e4, 62613537386265643565373561303664313135366461626363633638616264355f772e6b677475736f7a6c756b2e636f6d5f686f7374696e6c696e6b735f39323237303733353431333938383133365f31373037303330332d616232342d343230372d383362642d313630346532633135396534) INFO [CompactionExecutor:1] 2011-03-31 09:58:26,515 SSTableWriter.java (line 110) Writing into file /cassandra/data/table_lists/table_lists-tmp-f-1861-Data.db ERROR [CompactionExecutor:1] 2011-03-31 09:58:26,516 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.IOException: Keys must be written in ascending order. at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:111) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:128) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:452) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2498) Improve read performance in update-intensive workload
[ https://issues.apache.org/jira/browse/CASSANDRA-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2498: - Assignee: Pavel Yaskevich We could add this metadata into the -statistics component instead of adding a new one. (Just default it to zero on old-version sstables.) Improve read performance in update-intensive workload - Key: CASSANDRA-2498 URL: https://issues.apache.org/jira/browse/CASSANDRA-2498 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Labels: ponies Fix For: 1.0 Read performance in an update-heavy environment relies heavily on compaction to maintain good throughput. (This is not the case for workloads where rows are only inserted once, because the bloom filter keeps us from having to check sstables unnecessarily.) Very early versions of Cassandra attempted to mitigate this by checking sstables in descending generation order (mostly equivalent to descending mtime): once all the requested columns were found, it would not check any older sstables. This was incorrect, because data timestamp will not correspond to sstable timestamp, both because compaction has the side effect of refreshing data to a newer sstable, and because hintead handoff may send us data older than what we already have. Instead, we could create a per-sstable piece of metadata containing the most recent (client-specified) timestamp for any column in the sstable. We could then sort sstables by this timestamp instead, and perform a similar optimization (if the remaining sstable client-timestamps are older than the oldest column found in the desired result set so far, we don't need to look further). Since under almost every workload, client timestamps of data in a given sstable will tend to be similar, we expect this to cut the number of sstables down proportionally to how frequently each column in the row is updated. (If each column is updated with each write, we only have to check a single sstable.) This may also be useful information when deciding which SSTables to compact. (Note that this optimization is only appropriate for named-column queries, not slice queries, since we don't know what non-overlapping columns may exist in older sstables.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2122) Replace messages with a serialization framework for easy backwards compatibility
[ https://issues.apache.org/jira/browse/CASSANDRA-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2122. --- Resolution: Won't Fix Fix Version/s: (was: 1.0) Consensus is that spending effort to make our intra-cluster messaging less flexible is not a good use of time. Replace messages with a serialization framework for easy backwards compatibility Key: CASSANDRA-2122 URL: https://issues.apache.org/jira/browse/CASSANDRA-2122 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Gary Dusbabek Labels: ponies CASSANDRA-1015 provided a way to keep 0.8 messages compatible with 0.7 messages. The next step will be to replace messaging with something that is more easily backward compatible. Avro, thrift and msgpack have all been suggested as frameworks. If done properly, this will amount to rewriting MessagingService, though I'm not sure that's possible at this point (we don't have backwards-compatible disk structures, and those are sent inside the RowMutations). It may be preferred to keep MessagingService, and use it to package up messages that are serialized by whatever framework. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2424) Failed compactation doesn't delete temporary files causing hd to fill
[ https://issues.apache.org/jira/browse/CASSANDRA-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2424. - Resolution: Duplicate dupe of CASSANDRA-2468 Failed compactation doesn't delete temporary files causing hd to fill - Key: CASSANDRA-2424 URL: https://issues.apache.org/jira/browse/CASSANDRA-2424 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.4 Reporter: Thibaut Fix For: 0.8.1 Hi, We seem to have a few incorrect keys in one of our columns (Seems to be a memory bit flip). Cassandra will try to compact and compact those tables again but won't delete any temporary files. INFO [CompactionExecutor:1] 2011-04-05 17:52:45,494 SSTableWriter.java (line 108) Last written key : DecoratedKey(dsearch_1300960988716456398199492444161_1301855754714_4564397053271578441, 647365617263685f313330303936303938383731363435363c3339383139393439323434343136315f313330313835353735343731345f34353634333937303533323731353738343431) INFO [CompactionExecutor:1] 2011-04-05 17:52:45,521 SSTableWriter.java (line 109) Current key : DecoratedKey(dsearch_13009609887164564398199492444161_1301855754760_4564395937046043595, 647365617263685f31333030393630393838373136343536343339383139393439323434343136315f313330313835353735343736305f34353634333935393337303436303433353935) INFO [CompactionExecutor:1] 2011-04-05 17:52:45,521 SSTableWriter.java (line 110) Writing into file /cassandra/data/table_userentries/table_userentries-tmp-f-3550-Data.db ERROR [CompactionExecutor:1] 2011-04-05 17:52:45,522 AbstractCassandraDaemon.java (line 112) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.IOException: Keys must be written in ascending order. at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:111) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:128) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:452) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) ls -l /cassandra/data/table_userentries/table_userentries-tmp* -rw-r--r-- 1 root root 13G 2011-04-05 16:09 /cassandra/data/table_userentries/table_userentries-tmp-f-3507-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 16:09 /cassandra/data/table_userentries/table_userentries-tmp-f-3507-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 16:21 /cassandra/data/table_userentries/table_userentries-tmp-f-3512-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 16:21 /cassandra/data/table_userentries/table_userentries-tmp-f-3512-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 16:33 /cassandra/data/table_userentries/table_userentries-tmp-f-3517-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 16:33 /cassandra/data/table_userentries/table_userentries-tmp-f-3517-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 16:44 /cassandra/data/table_userentries/table_userentries-tmp-f-3521-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 16:44 /cassandra/data/table_userentries/table_userentries-tmp-f-3521-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 16:55 /cassandra/data/table_userentries/table_userentries-tmp-f-3527-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 16:55 /cassandra/data/table_userentries/table_userentries-tmp-f-3527-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 17:05 /cassandra/data/table_userentries/table_userentries-tmp-f-3531-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 17:06 /cassandra/data/table_userentries/table_userentries-tmp-f-3531-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 17:16 /cassandra/data/table_userentries/table_userentries-tmp-f-3535-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 17:16 /cassandra/data/table_userentries/table_userentries-tmp-f-3535-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 17:28 /cassandra/data/table_userentries/table_userentries-tmp-f-3540-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 17:28 /cassandra/data/table_userentries/table_userentries-tmp-f-3540-Index.db -rw-r--r-- 1 root root 13G 2011-04-05 17:39 /cassandra/data/table_userentries/table_userentries-tmp-f-3545-Data.db -rw-r--r-- 1 root root 512M 2011-04-05 17:40
[jira] [Updated] (CASSANDRA-1983) Make sstable filenames contain a UUID instead of increasing integer
[ https://issues.apache.org/jira/browse/CASSANDRA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1983: -- Affects Version/s: (was: 0.7.0) Fix Version/s: (was: 1.0) Make sstable filenames contain a UUID instead of increasing integer --- Key: CASSANDRA-1983 URL: https://issues.apache.org/jira/browse/CASSANDRA-1983 Project: Cassandra Issue Type: Improvement Reporter: David King Priority: Minor sstable filenames look like CFName-1569-Index.db, containing an integer for uniqueness. This makes it possible (however unlikely) that the integer could overflow, which could be a problem. It also makes it difficult to collapse multiple nodes into a single one with rsync. I do this occasionally for testing: I'll copy our 20 node cluster into only 3 nodes by copying all of the data files and running cleanup; at present this requires a manual step of uniqifying the overlapping sstable names. If instead of an incrementing integer, it would be handy if these contained a UUID or somesuch that guarantees uniqueness across the cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2558) Add concurrent_compactions configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026991#comment-13026991 ] Sylvain Lebresne commented on CASSANDRA-2558: - The thing is, ThreadPoolExecutor throws an IllegalArgumentException if maxPoolSize is 0. So I do agree that having concurrent_compactors=0 disable compaction would be coherent, but is it worth the extra code that will be needed (I don't see a super easy way to do this right off the bat) ? And after all disabling compaction is not something we want to encourage. An alternative would be to simply throw a ConfigurationException if the provided number is = 0, if what we want is avoiding silent correction. Add concurrent_compactions configuration -- Key: CASSANDRA-2558 URL: https://issues.apache.org/jira/browse/CASSANDRA-2558 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 beta 1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.1 Attachments: 0001-Make-compaction-thread-number-configurable.patch Original Estimate: 2h Remaining Estimate: 2h We should expose a way to configure the max number of thread to use when multi_threaded compaction is turned on. So far, it uses nb_of_processors thread, which if you have many cores may be unreasonably high (as far as random IO is concerned and thus independently of compaction throttling)... at least unless you have SSD. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2544) build produces unneeded/confusing checksum files
[ https://issues.apache.org/jira/browse/CASSANDRA-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2544. - Resolution: Won't Fix build produces unneeded/confusing checksum files Key: CASSANDRA-2544 URL: https://issues.apache.org/jira/browse/CASSANDRA-2544 Project: Cassandra Issue Type: Bug Components: Packaging, Tools Affects Versions: 0.7.5 Reporter: Eric Evans Assignee: Stephen Connolly The artifacts recently created for 0.7.5 (vote pending), have checksum files generated for the signatures (apache-cassandra-0.7.5-bin.tar.gz.asc.md5, apache-cassandra-0.7.5-bin.tar.gz.asc.sha1, apache-cassandra-0.7.5-src.tar.gz.asc.md5, and apache-cassandra-0.7.5-src.tar.gz.asc.sha1). These should be removed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2558) Add concurrent_compactions configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027010#comment-13027010 ] Jonathan Ellis commented on CASSANDRA-2558: --- bq. An alternative would be to simply throw a ConfigurationException if the provided number is = 0, if what we want is avoiding silent correction. Sounds good. Add concurrent_compactions configuration -- Key: CASSANDRA-2558 URL: https://issues.apache.org/jira/browse/CASSANDRA-2558 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 beta 1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.1 Attachments: 0001-Make-compaction-thread-number-configurable.patch Original Estimate: 2h Remaining Estimate: 2h We should expose a way to configure the max number of thread to use when multi_threaded compaction is turned on. So far, it uses nb_of_processors thread, which if you have many cores may be unreasonably high (as far as random IO is concerned and thus independently of compaction throttling)... at least unless you have SSD. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2476) CQL support for column TTLs
[ https://issues.apache.org/jira/browse/CASSANDRA-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2476: - Assignee: Pavel Yaskevich Syntax ideas? I'd be fine with just making this another USING clause. Usually you want all the columns in a single update to have the same TTL; if you don't, you can always break it into a batch. CQL support for column TTLs --- Key: CASSANDRA-2476 URL: https://issues.apache.org/jira/browse/CASSANDRA-2476 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Assignee: Pavel Yaskevich Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2476) CQL support for column TTLs
[ https://issues.apache.org/jira/browse/CASSANDRA-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027016#comment-13027016 ] Pavel Yaskevich commented on CASSANDRA-2476: USING TTL number seems reasonable to me also. CQL support for column TTLs --- Key: CASSANDRA-2476 URL: https://issues.apache.org/jira/browse/CASSANDRA-2476 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Assignee: Pavel Yaskevich Fix For: 0.8.1 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2473) CQL support for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2473: - Assignee: Pavel Yaskevich IMO the most natural way to express this is to start to allow the syntax UPDATE foo SET X = X + N WHERE key = K; (for counter column X) The alternatives would be to represent it as a procedure call: SELECT increment(foo, K, X, N); I don't like that much though, since it's a semantic mismatch (there is no actual function named increment involved). I'd rather save that for when we actually introduce functions/stored procedures. CQL support for counters Key: CASSANDRA-2473 URL: https://issues.apache.org/jira/browse/CASSANDRA-2473 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Assignee: Pavel Yaskevich Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2583) optimize batches containing multiple updates to the same rows
optimize batches containing multiple updates to the same rows - Key: CASSANDRA-2583 URL: https://issues.apache.org/jira/browse/CASSANDRA-2583 Project: Cassandra Issue Type: Improvement Components: API Affects Versions: 0.8 beta 1 Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8.1 Currently we turn batches into one RowMutation per statement. This is suboptimal if we have multiple statements operating on the same row. This could happen either with generated code, or deliberately to allow different options to different columns in the row, e.g. for TTLs (CASSANDRA-2476). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2558) Add concurrent_compactions configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027027#comment-13027027 ] Jonathan Ellis commented on CASSANDRA-2558: --- +1 Add concurrent_compactions configuration -- Key: CASSANDRA-2558 URL: https://issues.apache.org/jira/browse/CASSANDRA-2558 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 beta 1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.1 Attachments: 0001-Make-compaction-thread-number-configurable-v2.patch, 0001-Make-compaction-thread-number-configurable.patch Original Estimate: 2h Remaining Estimate: 2h We should expose a way to configure the max number of thread to use when multi_threaded compaction is turned on. So far, it uses nb_of_processors thread, which if you have many cores may be unreasonably high (as far as random IO is concerned and thus independently of compaction throttling)... at least unless you have SSD. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097843 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/db/migration/Migration.java
Author: jbellis Date: Fri Apr 29 15:35:18 2011 New Revision: 1097843 URL: http://svn.apache.org/viewvc?rev=1097843view=rev Log: refuse to apply migrations with older timestamps than the current schema patch by Tyler Hobbs; reviewed by jbellis and gdusbabek for CASSANDRA-2536 Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1097843r1=1097842r2=1097843view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Fri Apr 29 15:35:18 2011 @@ -4,6 +4,8 @@ * fix incorrect use of NBHM.size in ReadCallback that could cause reads to time out even when responses were received (CASSAMDRA-2552) * trigger read repair correctly for LOCAL_QUORUM reads (CASSANDRA-2556) + * refuse to apply migrations with older timestamps than the current + schema (CASSANDRA-2536) 0.7.5 Modified: cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java?rev=1097843r1=1097842r2=1097843view=diff == --- cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java (original) +++ cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/migration/Migration.java Fri Apr 29 15:35:18 2011 @@ -98,6 +98,8 @@ public abstract class Migration // ensure migration is serial. don't apply unless the previous version matches. if (!DatabaseDescriptor.getDefsVersion().equals(lastVersion)) throw new ConfigurationException(Previous version mismatch. cannot apply.); +if (newVersion.timestamp() = lastVersion.timestamp()) +throw new ConfigurationException(New version timestamp is not newer than the current version timestamp.); // write to schema assert rm != null; if (!clientMode)
svn commit: r1097845 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/migration/
Author: jbellis Date: Fri Apr 29 15:38:17 2011 New Revision: 1097845 URL: http://svn.apache.org/viewvc?rev=1097845view=rev Log: merge from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/migration/Migration.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 15:38:17 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455 +/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097843 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/trunk:1090978-1090979 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1097845r1=1097844r2=1097845view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Apr 29 15:38:17 2011 @@ -14,6 +14,8 @@ * forceUserDefinedCompaction will attempt to compact what it is given even if the pessimistic estimate is that there is not enough disk space; automatic compactions will only compact 2 or more sstables (CASSANDRA-2575) + * refuse to apply migrations with older timestamps than the current + schema (CASSANDRA-2536) 0.8.0-beta1 Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 15:38:17 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455 +/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097843 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/trunk/contrib:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 15:38:17 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097843 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 15:38:17 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097843
[jira] [Commented] (CASSANDRA-2473) CQL support for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027040#comment-13027040 ] Sylvain Lebresne commented on CASSANDRA-2473: - I agree with using native syntax, not a procedure call. But maybe UPDATE foo SET X += N WHERE key = K; would make it more clear that you can really only increment X, not set any value you want (it's easy to check and reject the request, but I like it when the syntax make is easy to not screw up) CQL support for counters Key: CASSANDRA-2473 URL: https://issues.apache.org/jira/browse/CASSANDRA-2473 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Assignee: Pavel Yaskevich Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2473) CQL support for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027042#comment-13027042 ] Pavel Yaskevich commented on CASSANDRA-2473: I agree with Sylvain on this. CQL support for counters Key: CASSANDRA-2473 URL: https://issues.apache.org/jira/browse/CASSANDRA-2473 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Assignee: Pavel Yaskevich Fix For: 1.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2581) Rebuffer called excessively during seeks
[ https://issues.apache.org/jira/browse/CASSANDRA-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2581: -- Attachment: 2581.txt Rebuffer called excessively during seeks Key: CASSANDRA-2581 URL: https://issues.apache.org/jira/browse/CASSANDRA-2581 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.3 Reporter: Chris Goffinet Assignee: Chris Goffinet Priority: Minor Fix For: 0.7.6, 0.8.1 Attachments: 0001-Rebuffer-called-excessively-during-seeks.patch, 2581.txt When doing an strace tonight, I noticed during memtable flushes that we were only writing 1KB per every write() system call...After diving more into it, it's because of a bug in the seek() code. if (newPosition = bufferOffset + validBufferBytes || newPosition bufferOffset) vs. if (newPosition (bufferOffset + validBufferBytes) || newPosition bufferOffset) Two things I noticed, we shouldn't need to rebuffer if newPosition is equal to bufferOffset + validBufferBytes, second the evaluation was doing (newPosition = bufferOffset) + validBufferBytes which always seemed to be true. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2581) Rebuffer called excessively during seeks
[ https://issues.apache.org/jira/browse/CASSANDRA-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2581: -- Reviewer: xedin Affects Version/s: (was: 0.8.1) 0.7.3 Fix Version/s: 0.8.1 0.7.6 bq. we shouldn't need to rebuffer if newPosition is equal to bufferOffset + validBufferBytes I think that is correct, patch attached. I will ask Pavel to also review since this is extremely important not to break. bq. second the evaluation was doing (newPosition = bufferOffset) + validBufferBytes No, addition is higher precedence than comparison (or the logical operations). In fact if you force the grouping you suggest, javac will reject it since you cannot add a boolean and an int. Rebuffer called excessively during seeks Key: CASSANDRA-2581 URL: https://issues.apache.org/jira/browse/CASSANDRA-2581 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.3 Reporter: Chris Goffinet Assignee: Chris Goffinet Priority: Minor Fix For: 0.7.6, 0.8.1 Attachments: 0001-Rebuffer-called-excessively-during-seeks.patch, 2581.txt When doing an strace tonight, I noticed during memtable flushes that we were only writing 1KB per every write() system call...After diving more into it, it's because of a bug in the seek() code. if (newPosition = bufferOffset + validBufferBytes || newPosition bufferOffset) vs. if (newPosition (bufferOffset + validBufferBytes) || newPosition bufferOffset) Two things I noticed, we shouldn't need to rebuffer if newPosition is equal to bufferOffset + validBufferBytes, second the evaluation was doing (newPosition = bufferOffset) + validBufferBytes which always seemed to be true. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097870 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ tools/stress/src/org/apache/cassandra/stress/operations/
Author: jbellis Date: Fri Apr 29 16:24:07 2011 New Revision: 1097870 URL: http://svn.apache.org/viewvc?rev=1097870view=rev Log: merge from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/operations/IndexedRangeSlicer.java cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 16:24:07 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097843 +/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097668,1097843 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/trunk:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 16:24:07 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097843 +/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097668,1097843 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/trunk/contrib:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 16:24:07 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097843 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 16:24:07 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097843 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 16:24:07 2011 @@ -1,5 +1,5 @@
[jira] [Commented] (CASSANDRA-2571) Check for null super column for SC CF in ThriftValidation (and always validate the sc key)
[
https://issues.apache.org/jira/browse/CASSANDRA-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027063#comment-13027063
]
Hudson commented on CASSANDRA-2571:
---
Integrated in Cassandra-0.7 #463 (See
[https://builds.apache.org/hudson/job/Cassandra-0.7/463/])
Reject queries with missing mandatory super column and always validate
super column name
patch by jbellis and slebresne for CASSANDRA-2571
Check for null super column for SC CF in ThriftValidation (and always
validate the sc key)
--
Key: CASSANDRA-2571
URL: https://issues.apache.org/jira/browse/CASSANDRA-2571
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0, 0.8 beta 1
Reporter: Mike Bulman
Assignee: Sylvain Lebresne
Fix For: 0.7.6, 0.8.0
Attachments: 0001-Improve-ThriftValidation.patch, 2571.txt
Run the following via cli:
{noformat}
[default@test] use test;
Authenticated to keyspace: test
[default@test] create column family super with column_type=Super and
default_validation_class=CounterColumnType;
d41df8e0-7055-11e0--242d50cf1fbf
Waiting for schema agreement...
... schemas agree across the cluster
[default@test] incr super['0']['0'];
Value incremented.
[default@test] incr super['0']['0']['0'];
null
{noformat}
Obviously the first incr call is invalid, even though it reports otherwise,
as well as generates this exception:
{noformat}
ERROR 17:38:05,871 Fatal exception in thread Thread[COMMIT-LOG-WRITER,5,main]
java.lang.RuntimeException: java.lang.ClassCastException:
org.apache.cassandra.db.CounterColumn cannot be cast to
org.apache.cassandra.db.SuperColumn
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.ClassCastException:
org.apache.cassandra.db.CounterColumn cannot be cast to
org.apache.cassandra.db.SuperColumn
at
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:353)
at
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:337)
at
org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:88)
at
org.apache.cassandra.db.ColumnFamilySerializer.serialize(ColumnFamilySerializer.java:74)
at
org.apache.cassandra.db.RowMutation$RowMutationSerializer.serialize(RowMutation.java:353)
at
org.apache.cassandra.db.RowMutation.getSerializedBuffer(RowMutation.java:236)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.write(CommitLogSegment.java:111)
at
org.apache.cassandra.db.commitlog.CommitLog$LogRecordAdder.run(CommitLog.java:480)
at
org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:49)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 1 more
{noformat}
But the second, proper incr call results in a bunch of exceptions and not a
real increment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1291 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1097884 Blamelist: slebresne Build succeeded! sincerely, -The Buildbot
svn commit: r1097887 - in /cassandra/branches/cassandra-0.8: ./ conf/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/
Author: slebresne Date: Fri Apr 29 17:00:26 2011 New Revision: 1097887 URL: http://svn.apache.org/viewvc?rev=1097887view=rev Log: Add option to configure number of compaction thread patch by slebresne; reviewed by stuhood and jbellis for CASSANDRA-2558 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/conf/cassandra.yaml cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/CompactionManager.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1097887r1=1097886r2=1097887view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Apr 29 17:00:26 2011 @@ -16,6 +16,7 @@ automatic compactions will only compact 2 or more sstables (CASSANDRA-2575) * refuse to apply migrations with older timestamps than the current schema (CASSANDRA-2536) + * Allow configuring the number of compaction thread (CASSANDRA-2558) 0.8.0-beta1 Modified: cassandra/branches/cassandra-0.8/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra.yaml?rev=1097887r1=1097886r2=1097887view=diff == --- cassandra/branches/cassandra-0.8/conf/cassandra.yaml (original) +++ cassandra/branches/cassandra-0.8/conf/cassandra.yaml Fri Apr 29 17:00:26 2011 @@ -249,10 +249,15 @@ column_index_size_in_kb: 64 # will be logged specifying the row key. in_memory_compaction_limit_in_mb: 64 -# Enables multiple compactions to execute at once. This is highly recommended -# for preserving read performance in a mixed read/write workload as this -# avoids sstables from accumulating during long running compactions. -compaction_multithreading: false +# Number of compaction threads. This default to the number of processors, +# enabling multiple compactions to execute at once. Using more than one +# thread is highly recommended to preserve read performance in a mixed +# read/write workload as this avoids sstables from accumulating during long +# running compactions. The default is usually fine and if you experience +# problems with compaction running too slowly or too fast, you should look at +# compaction_throughput_mb_per_sec first. +# Uncomment to make compaction mono-threaded. +#concurrent_compacters: 1 # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java?rev=1097887r1=1097886r2=1097887view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java Fri Apr 29 17:00:26 2011 @@ -82,7 +82,7 @@ public class Config /* if the size of columns or super-columns are more than this, indexing will kick in */ public Integer column_index_size_in_kb = 64; public Integer in_memory_compaction_limit_in_mb = 256; -public Boolean compaction_multithreading = true; +public Integer concurrent_compactors = Runtime.getRuntime().availableProcessors(); public Integer compaction_throughput_mb_per_sec = 16; public String[] data_file_directories; Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1097887r1=1097886r2=1097887view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java Fri Apr 29 17:00:26 2011 @@ -341,8 +341,11 @@ public class DatabaseDescriptor throw new ConfigurationException(in_memory_compaction_limit_in_mb must be a positive integer); } -if (conf.compaction_multithreading == null) -conf.compaction_multithreading = true; +if (conf.concurrent_compactors == null) +conf.concurrent_compactors = Runtime.getRuntime().availableProcessors(); + +if (conf.concurrent_compactors = 0) +throw new
[jira] [Commented] (CASSANDRA-2576) Rewrite into new file post streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027079#comment-13027079 ] Sylvain Lebresne commented on CASSANDRA-2576: - Looks good, but I would want to remove the addition of the close() function to SSTableWriter (because it calls closeAndOpenReader). I fear people would be tempted to call close in a final block, which could remove the temporary marker, even though some error would have left the sstable corrupted. And as far as I can tell, this is a gratuitous addition, that has nothing to do with that patch. Writing a new file was in the first versions of the counter patch and was changed for the space efficiency it provides (or at least I don't think there was another reason). Nevertheless I do agree that it is cleaner that way, so let's hope the temporary increased space needed won't bite anyone (since we delete the streamed file as soon as the new file is created, it's probably fine). Rewrite into new file post streaming Key: CASSANDRA-2576 URL: https://issues.apache.org/jira/browse/CASSANDRA-2576 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Fix For: 1.0 Attachments: 0001-Don-t-depend-on-a-byte-for-byte-match-for-SSTable-rebu.txt, 0002-Rebuild-into-a-new-file-to-minimize-magic.txt Commutative/counter column families use a separate path to rebuild sstables post streaming, and that path currently rewrites the data within the streamed file. While this is great for space efficiency, it means a duplicated code path for writing sstables, which makes it more difficult to make changes like #674. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097889 - in /cassandra/trunk: ./ conf/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/
Author: slebresne Date: Fri Apr 29 17:09:26 2011 New Revision: 1097889 URL: http://svn.apache.org/viewvc?rev=1097889view=rev Log: merge CASSANDRA-2558 from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/conf/cassandra.yaml cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/config/Config.java cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/trunk/src/java/org/apache/cassandra/db/CompactionManager.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:09:26 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 /cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097852 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090935-1097471,1097883 +/cassandra/branches/cassandra-0.8:1090935-1097471,1097883,1097887 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1097889r1=1097888r2=1097889view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Fri Apr 29 17:09:26 2011 @@ -17,6 +17,7 @@ * fix incorrect use of NBHM.size in ReadCallback that could cause reads to time out even when responses were received (CASSAMDRA-2552) * trigger read repair correctly for LOCAL_QUORUM reads (CASSANDRA-2556) + * Allow configuring the number of compaction thread (CASSANDRA-2558) 0.8.0-beta1 Modified: cassandra/trunk/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1097889r1=1097888r2=1097889view=diff == --- cassandra/trunk/conf/cassandra.yaml (original) +++ cassandra/trunk/conf/cassandra.yaml Fri Apr 29 17:09:26 2011 @@ -249,10 +249,15 @@ column_index_size_in_kb: 64 # will be logged specifying the row key. in_memory_compaction_limit_in_mb: 64 -# Enables multiple compactions to execute at once. This is highly recommended -# for preserving read performance in a mixed read/write workload as this -# avoids sstables from accumulating during long running compactions. -compaction_multithreading: false +# Number of compaction threads. This default to the number of processors, +# enabling multiple compactions to execute at once. Using more than one +# thread is highly recommended to preserve read performance in a mixed +# read/write workload as this avoids sstables from accumulating during long +# running compactions. The default is usually fine and if you experience +# problems with compaction running too slowly or too fast, you should look at +# compaction_throughput_mb_per_sec first. +# Uncomment to make compaction mono-threaded. +#concurrent_compacters: 1 # Throttles compaction to the given total throughput across the entire # system. The faster you insert data, the faster you need to compact in Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:09:26 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097852 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090935-1097471,1097883 +/cassandra/branches/cassandra-0.8/contrib:1090935-1097471,1097883,1097887 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 /incubator/cassandra/branches/cassandra-0.4/contrib:810145-810987,810994-834239,834349-834350 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original)
svn commit: r1097892 - /cassandra/branches/cassandra-0.7/build.xml
Author: slebresne
Date: Fri Apr 29 17:23:01 2011
New Revision: 1097892
URL: http://svn.apache.org/viewvc?rev=1097892view=rev
Log:
Deploy parent POM to maven central
patch by stephenc for CASSANDRA-2562
Modified:
cassandra/branches/cassandra-0.7/build.xml
Modified: cassandra/branches/cassandra-0.7/build.xml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/build.xml?rev=1097892r1=1097891r2=1097892view=diff
==
--- cassandra/branches/cassandra-0.7/build.xml (original)
+++ cassandra/branches/cassandra-0.7/build.xml Fri Apr 29 17:23:01 2011
@@ -280,6 +280,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
groupId=org.apache.cassandra
artifactId=cassandra-parent
version=${version}
+packaging=pom
url=http://cassandra.apache.org;
name=Apache Cassandra
inceptionYear=2009
@@ -472,6 +473,7 @@ url=${svn.entry.url}?pathrev=${svn.entry
target name=maven-write-poms depends=maven-declare-dependencies
description=Create Maven POM files
artifact:writepom pomRefId=all-pom
file=${build.dir}/${final.name}.pom/
+ artifact:writepom pomRefId=parent-pom
file=${build.dir}/${final.name}-parent.pom/
artifact:writepom pomRefId=dist-pom
file=${build.dir}/${final.name}-dist.pom/
/target
@@ -1114,6 +1116,11 @@ url=${svn.entry.url}?pathrev=${svn.entry
depends=mvn-install
if=release
description=Publishes the artifacts to the Maven repository
+!-- the parent pom --
+deploy pomFile=${build.dir}/${final.name}-parent.pom
+file=${build.dir}/${final.name}-parent.pom
+packaging=pom/
+
!-- the distribution --
deploy pomFile=${build.dir}/${final.name}-dist.pom
file=${build.dir}/${final.name}-dist.pom
svn commit: r1097893 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/
Author: slebresne Date: Fri Apr 29 17:26:23 2011 New Revision: 1097893 URL: http://svn.apache.org/viewvc?rev=1097893view=rev Log: merge --record-only of CASSANDRA-2562 from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:26:23 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852 +/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/trunk:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:26:23 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852 +/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/trunk/contrib:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:26:23 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:26:23 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090978-1090979 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:26:23 2011 @@ -1,5 +1,5 @@
svn commit: r1097894 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/
Author: slebresne Date: Fri Apr 29 17:28:13 2011 New Revision: 1097894 URL: http://svn.apache.org/viewvc?rev=1097894view=rev Log: merge records of CASSANDRA-2562 from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:28:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097852 +/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097852,1097892 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090935-1097471,1097883,1097887 +/cassandra/branches/cassandra-0.8:1090935-1097471,1097883,1097887,1097893 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:28:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097852 +/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097852,1097892 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090935-1097471,1097883,1097887 +/cassandra/branches/cassandra-0.8/contrib:1090935-1097471,1097883,1097887,1097893 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 /incubator/cassandra/branches/cassandra-0.4/contrib:810145-810987,810994-834239,834349-834350 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:28:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097852 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097852,1097892 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090935-1097471,1097883,1097887 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090935-1097471,1097883,1097887,1097893 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 /incubator/cassandra/branches/cassandra-0.4/interface/gen-java/org/apache/cassandra/service/Cassandra.java:810145-834239,834349-834350 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 17:28:13 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097852 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1096531,1096805,1097448,1097455,1097852,1097892
[jira] [Resolved] (CASSANDRA-2562) Parent POM does not get deployed to the maven repository
[ https://issues.apache.org/jira/browse/CASSANDRA-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-2562. - Resolution: Fixed Assignee: Stephen Connolly Committed Parent POM does not get deployed to the maven repository Key: CASSANDRA-2562 URL: https://issues.apache.org/jira/browse/CASSANDRA-2562 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.5 Reporter: Stephen Connolly Assignee: Stephen Connolly Fix For: 0.7.6 Attachments: CASSANDRA-2562.patch, CASSANDRA-2562.patch The parent pom does not get deployed to the Maven Central (for 0.7.5 I am fixing this by hand) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2584) When disabling or reducing compaction frequency, CopactionManager still shows frequent activity
When disabling or reducing compaction frequency, CopactionManager still shows frequent activity --- Key: CASSANDRA-2584 URL: https://issues.apache.org/jira/browse/CASSANDRA-2584 Project: Cassandra Issue Type: Improvement Reporter: Patricio Echague Priority: Trivial I was playing with different set of configurations to test performance. One of those was with compaction disabled. besides the fact that it is a bad practice to disable compaction, even when I did disable it, JConsole CompactionManager#completedTasks showed 30 tasks completed during the duration of my test. I think it would be good to stop the tasks from being executed. It looks like the executor is still executing the tasks with NO-OP. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2555) Explicit timestamp support for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027091#comment-13027091 ] Hudson commented on CASSANDRA-2555: --- Integrated in Cassandra #872 (See [https://builds.apache.org/hudson/job/Cassandra/872/]) Explicit timestamp support for CQL -- Key: CASSANDRA-2555 URL: https://issues.apache.org/jira/browse/CASSANDRA-2555 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.8.1 Attachments: CASSANDRA-2555-v2.patch, CASSANDRA-2555.patch To make CQL BATCHes useful for the normal purpose of idempotent retries, we need to add support for client-supplied timestamps. At a minimum this needs to be available for a batch (BEGIN BATCH USING TIMESTAMP 1303743619771318) but might also be nice at the statement level (UPDATE ... USING TIMESTAMP ...). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2558) Add concurrent_compactions configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027092#comment-13027092 ] Hudson commented on CASSANDRA-2558: --- Integrated in Cassandra #872 (See [https://builds.apache.org/hudson/job/Cassandra/872/]) merge CASSANDRA-2558 from 0.8 Add concurrent_compactions configuration -- Key: CASSANDRA-2558 URL: https://issues.apache.org/jira/browse/CASSANDRA-2558 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 beta 1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.0 Attachments: 0001-Make-compaction-thread-number-configurable-v2.patch, 0001-Make-compaction-thread-number-configurable.patch Original Estimate: 2h Remaining Estimate: 2h We should expose a way to configure the max number of thread to use when multi_threaded compaction is turned on. So far, it uses nb_of_processors thread, which if you have many cores may be unreasonably high (as far as random IO is concerned and thus independently of compaction throttling)... at least unless you have SSD. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2571) Check for null super column for SC CF in ThriftValidation (and always validate the sc key)
[
https://issues.apache.org/jira/browse/CASSANDRA-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027093#comment-13027093
]
Hudson commented on CASSANDRA-2571:
---
Integrated in Cassandra #872 (See
[https://builds.apache.org/hudson/job/Cassandra/872/])
merge CASSANDRA-2571 from 0.8
Check for null super column for SC CF in ThriftValidation (and always
validate the sc key)
--
Key: CASSANDRA-2571
URL: https://issues.apache.org/jira/browse/CASSANDRA-2571
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.0, 0.8 beta 1
Reporter: Mike Bulman
Assignee: Sylvain Lebresne
Fix For: 0.7.6, 0.8.0
Attachments: 0001-Improve-ThriftValidation.patch, 2571.txt
Run the following via cli:
{noformat}
[default@test] use test;
Authenticated to keyspace: test
[default@test] create column family super with column_type=Super and
default_validation_class=CounterColumnType;
d41df8e0-7055-11e0--242d50cf1fbf
Waiting for schema agreement...
... schemas agree across the cluster
[default@test] incr super['0']['0'];
Value incremented.
[default@test] incr super['0']['0']['0'];
null
{noformat}
Obviously the first incr call is invalid, even though it reports otherwise,
as well as generates this exception:
{noformat}
ERROR 17:38:05,871 Fatal exception in thread Thread[COMMIT-LOG-WRITER,5,main]
java.lang.RuntimeException: java.lang.ClassCastException:
org.apache.cassandra.db.CounterColumn cannot be cast to
org.apache.cassandra.db.SuperColumn
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.ClassCastException:
org.apache.cassandra.db.CounterColumn cannot be cast to
org.apache.cassandra.db.SuperColumn
at
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:353)
at
org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:337)
at
org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:88)
at
org.apache.cassandra.db.ColumnFamilySerializer.serialize(ColumnFamilySerializer.java:74)
at
org.apache.cassandra.db.RowMutation$RowMutationSerializer.serialize(RowMutation.java:353)
at
org.apache.cassandra.db.RowMutation.getSerializedBuffer(RowMutation.java:236)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.write(CommitLogSegment.java:111)
at
org.apache.cassandra.db.commitlog.CommitLog$LogRecordAdder.run(CommitLog.java:480)
at
org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:49)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 1 more
{noformat}
But the second, proper incr call results in a bunch of exceptions and not a
real increment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2452) New EC2 Snitch to use public ip and hence natively support for EC2 mult-region's.
[ https://issues.apache.org/jira/browse/CASSANDRA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027096#comment-13027096 ] Vijay commented on CASSANDRA-2452: -- Jonathan, yes it overlaps... the idea is to set the private and public address automatically via this patch In #2491 The broadcast address is just used for identification similarly to getLocalID()... i think this patch is less intrusive... but if we can commit the other patch i can port this to automatically set the broadcast address... New EC2 Snitch to use public ip and hence natively support for EC2 mult-region's. - Key: CASSANDRA-2452 URL: https://issues.apache.org/jira/browse/CASSANDRA-2452 Project: Cassandra Issue Type: New Feature Components: Core Affects Versions: 0.8 beta 1 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 0.8.1 Attachments: 2452-EC2Snitch-Changes.txt, 2452-Intro-EC2MultiRegionSnitch-V2.txt Make cassandra talk identify itself using the public ip (To avoid any future conflicts of private ips). 1) Split the logic of identification vs listen Address in the code. 2) Move the logic to assign IP address to the node into EndPointSnitch. 3) Make EC2 Snitch query for its public ip and use it for identification. 4) Make EC2 snitch to use InetAddress.getLocal to listen to the private ip. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2562) Parent POM does not get deployed to the maven repository
[ https://issues.apache.org/jira/browse/CASSANDRA-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027098#comment-13027098 ] Hudson commented on CASSANDRA-2562: --- Integrated in Cassandra-0.7 #464 (See [https://builds.apache.org/hudson/job/Cassandra-0.7/464/]) Deploy parent POM to maven central patch by stephenc for CASSANDRA-2562 Parent POM does not get deployed to the maven repository Key: CASSANDRA-2562 URL: https://issues.apache.org/jira/browse/CASSANDRA-2562 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.5 Reporter: Stephen Connolly Assignee: Stephen Connolly Fix For: 0.7.6 Attachments: CASSANDRA-2562.patch, CASSANDRA-2562.patch The parent pom does not get deployed to the Maven Central (for 0.7.5 I am fixing this by hand) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2562) Parent POM does not get deployed to the maven repository
[ https://issues.apache.org/jira/browse/CASSANDRA-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027120#comment-13027120 ] Hudson commented on CASSANDRA-2562: --- Integrated in Cassandra #873 (See [https://builds.apache.org/hudson/job/Cassandra/873/]) merge records of CASSANDRA-2562 from 0.8 Parent POM does not get deployed to the maven repository Key: CASSANDRA-2562 URL: https://issues.apache.org/jira/browse/CASSANDRA-2562 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.5 Reporter: Stephen Connolly Assignee: Stephen Connolly Fix For: 0.7.6 Attachments: CASSANDRA-2562.patch, CASSANDRA-2562.patch The parent pom does not get deployed to the Maven Central (for 0.7.5 I am fixing this by hand) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2540) Data reads by default
[
https://issues.apache.org/jira/browse/CASSANDRA-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027140#comment-13027140
]
Ryan King commented on CASSANDRA-2540:
--
Sylvain-
It don't think its the average performance that matters here, but the worse
case. For our deployments we have latency targets at the 99th percentile. Some
of those are quite low ( 10ms), so even a small number of requests that have
to wait for the rpc timeout make our goals difficult, even if we lower the rpc
timeout.
Data reads by default
-
Key: CASSANDRA-2540
URL: https://issues.apache.org/jira/browse/CASSANDRA-2540
Project: Cassandra
Issue Type: Wish
Reporter: Stu Hood
Priority: Minor
The intention of digest vs data reads is to save bandwidth in the read path
at the cost of latency, but I expect that this has been a premature
optimization.
* Data requested by a read will often be within an order of magnitude of the
digest size, and a failed digest means extra roundtrips, more bandwidth
* The [digest reads but not your data
read|https://issues.apache.org/jira/browse/CASSANDRA-2282?focusedCommentId=13004656page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13004656]
problem means failing QUORUM reads because a single node is unavailable, and
would require eagerly re-requesting at some fraction of your timeout
* Saving bandwidth in cross datacenter usecases comes at huge cost to
latency, but since both constraints change proportionally (enough), the
tradeoff is not clear
Some options:
# Add an option to use digest reads
# Remove digest reads entirely (and/or punt and make them a runtime
optimization based on data size in the future)
# Continue to use digest reads, but send them to {{N - R}} nodes for
(somewhat) more predicatable behavior with QUORUM
\\
The outcome of data-reads-by-default should be significantly improved
latency, with a moderate increase in bandwidth usage for large reads.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2315) mmap segment underflow when building secondary index after joining ring
[
https://issues.apache.org/jira/browse/CASSANDRA-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2315.
---
Resolution: Duplicate
Fix Version/s: (was: 0.7.6)
I believe this is fixed by CASSANDRA-2283
mmap segment underflow when building secondary index after joining ring
---
Key: CASSANDRA-2315
URL: https://issues.apache.org/jira/browse/CASSANDRA-2315
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.3
Reporter: Matthew F. Dennis
there was a 7 node ring (N0-N6) that was upgraded to 0.7.3
all the data was scrubbed
7 additional nodes were added (N7-N13) with no problems
some compactions ran, some things were restarted (but there was no read/write
load)
after a while 7 more nodes (N14-N20) were added
all joined successfully except one
that node was pulling data from one of the nodes in the N7-N13 set (i.e. a
node that got all it's data from the original nodes)
the EC2 instance that was running was completely destroyed, a new instance
created and the bootstrap attempted again but still failed
{code}
INFO [Thread-65] 2011-03-10 17:16:59,448 ColumnFamilyStore.java (line 377)
Index build of service_id, complete
INFO [Thread-65] 2011-03-10 17:16:59,449 ColumnFamilyStore.java (line 359)
Submitting index build of service_id, for data in
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata-f-1-Data.db')
INFO [Thread-75] 2011-03-10 17:17:45,757 ColumnFamilyStore.java (line 377)
Index build of service_id, complete
INFO [Thread-75] 2011-03-10 17:17:45,767 ColumnFamilyStore.java (line 359)
Submitting index build of service_id, for data in
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata-f-2-Data.db'),
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata-f-3-Data.db'),
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata-f-4-Data.db'),
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata-f-5-Data.db')
INFO [ScheduledTasks:1] 2011-03-10 17:17:46,550 ColumnFamilyStore.java (line
1020) Enqueuing flush of
Memtable-GoogleDocsMetadata.736572766963655f6964@1078361799(353352 bytes,
4539 operations)
INFO [FlushWriter:1] 2011-03-10 17:17:46,552 Memtable.java (line 157) Writing
Memtable-GoogleDocsMetadata.736572766963655f6964@1078361799(353352 bytes,
4539 operations)
INFO [FlushWriter:1] 2011-03-10 17:17:46,736 Memtable.java (line 164)
Completed flushing
/mnt/cassandra/data/BackupifyMetadata/GoogleDocsMetadata.736572766963655f6964-f-1-Data.db
(483139 bytes)
INFO [Thread-65] 2011-03-10 17:18:04,278 ColumnFamilyStore.java (line 377)
Index build of service_id, complete
INFO [Thread-65] 2011-03-10 17:18:04,279 ColumnFamilyStore.java (line 359)
Submitting index build of service_id, for data in
SSTableReader(path='/mnt/cassandra/data/BackupifyMetadata/BloggerMetadata-f-1-Data.db')
ERROR [Thread-75] 2011-03-10 17:18:21,781 AbstractCassandraDaemon.java (line
114) Fatal exception in thread Thread[Thread-75,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: mmap segment underflow; remaining is 43239499 but
1768777055 requested
at
org.apache.cassandra.db.ColumnFamilyStore.buildSecondaryIndexes(ColumnFamilyStore.java:375)
at
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:159)
at
org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:91)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError:
mmap segment underflow; remaining is 43239499 but 1768777055 requested
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.cassandra.db.ColumnFamilyStore.buildSecondaryIndexes(ColumnFamilyStore.java:365)
... 3 more
Caused by: java.lang.AssertionError: mmap segment underflow; remaining is
43239499 but 1768777055 requested
at
org.apache.cassandra.io.util.MappedFileDataInput.readBytes(MappedFileDataInput.java:119)
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:310)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:267)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
at
[jira] [Commented] (CASSANDRA-2129) removetoken after removetoken rf error fails to work
[ https://issues.apache.org/jira/browse/CASSANDRA-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027148#comment-13027148 ] Jonathan Ellis commented on CASSANDRA-2129: --- Possibly the same bug was reported on the user list: http://permalink.gmane.org/gmane.comp.db.cassandra.user/15803 removetoken after removetoken rf error fails to work Key: CASSANDRA-2129 URL: https://issues.apache.org/jira/browse/CASSANDRA-2129 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Mike Bulman Assignee: Brandon Williams Priority: Minor Fix For: 0.7.6 Attachments: 2129.txt Original Estimate: 4h Remaining Estimate: 4h 2 node cluster, a keyspace existed with rf=2. Tried removetoken and got: mbulman@ripcord-maverick1:/usr/src/cassandra/tags/cassandra-0.7.0$ bin/nodetool -h localhost removetoken 159559397954378837828954138596956659794 Exception in thread main java.lang.IllegalStateException: replication factor (2) exceeds number of endpoints (1) Deleted the keyspace, and tried again: mbulman@ripcord-maverick1:/usr/src/cassandra/tags/cassandra-0.7.0$ bin/nodetool -h localhost removetoken 159559397954378837828954138596956659794 Exception in thread main java.lang.UnsupportedOperationException: This node is already processing a removal. Wait for it to complete. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2034) Make Read Repair unnecessary when Hinted Handoff is enabled
[ https://issues.apache.org/jira/browse/CASSANDRA-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027151#comment-13027151 ] Jonathan Ellis commented on CASSANDRA-2034: --- bq. add a scheduled task this is the wrong approach, as we found out when we tried something similar for read repair, which we fixed in CASSANDRA-2069. Better would be to add a hook to messagingservice callback expiration, and fire hint recording from there if MS expires the callback before all acks are received. (We could refactor the dynamic snitch latency update into a similar hook for reads.) bq. This would need a separate executor for local writes that doesn't drop writes when it's behind I'm more worried about this; there is the potential to take us back to the Bad Old Days when HH could cause cascading failure. (Of course the right answer is, Don't run your cluster so close to the edge of capacity, but we still want to degrade gracefully when this is ignored.) Make Read Repair unnecessary when Hinted Handoff is enabled --- Key: CASSANDRA-2034 URL: https://issues.apache.org/jira/browse/CASSANDRA-2034 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Original Estimate: 8h Remaining Estimate: 8h Currently, HH is purely an optimization -- if a machine goes down, enabling HH means RR/AES will have less work to do, but you can't disable RR entirely in most situations since HH doesn't kick in until the FailureDetector does. Let's add a scheduled task to the mutate path, such that we return to the client normally after ConsistencyLevel is achieved, but after RpcTimeout we check the responseHandler write acks and write local hints for any missing targets. This would making disabling RR when HH is enabled a much more reasonable option, which has a huge impact on read throughput. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2487) Pig example script no longer working
[ https://issues.apache.org/jira/browse/CASSANDRA-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2487: -- Reviewer: brandon.williams Pig example script no longer working Key: CASSANDRA-2487 URL: https://issues.apache.org/jira/browse/CASSANDRA-2487 Project: Cassandra Issue Type: Bug Reporter: Jeremy Hanna Assignee: Jeremy Hanna Priority: Minor Labels: hadoop, pig Fix For: 0.7.6 Attachments: 2487.txt There is a strange error given when trying to run the example-script.pig. java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:870) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:573) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) Looks like it has to do with PIG-919 and PIG-1277. For now we can just cast the var as a chararray and it works though. Will attach a patch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097922 - in /cassandra/branches/cassandra-0.7/contrib/pig: README.txt example-script.pig
Author: brandonwilliams
Date: Fri Apr 29 19:28:06 2011
New Revision: 1097922
URL: http://svn.apache.org/viewvc?rev=1097922view=rev
Log:
Update pig example script to work again.
Patch by Jeremy Hanna, reviewed by brandonwilliams for CASSANDRA-2487
Modified:
cassandra/branches/cassandra-0.7/contrib/pig/README.txt
cassandra/branches/cassandra-0.7/contrib/pig/example-script.pig
Modified: cassandra/branches/cassandra-0.7/contrib/pig/README.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/README.txt?rev=1097922r1=1097921r2=1097922view=diff
==
--- cassandra/branches/cassandra-0.7/contrib/pig/README.txt (original)
+++ cassandra/branches/cassandra-0.7/contrib/pig/README.txt Fri Apr 29 19:28:06
2011
@@ -18,17 +18,22 @@ also set PIG_CONF_DIR to the location of
Finally, set the following as environment variables (uppercase,
underscored), or as Hadoop configuration variables (lowercase, dotted):
-* PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening on
* PIG_INITIAL_ADDRESS or cassandra.thrift.address : initial address to connect
to
+* PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening on
* PIG_PARTITIONER or cassandra.partitioner.class : cluster partitioner
-Run:
+For example, against a local node with the default settings, you'd use:
+export PIG_INITIAL_ADDRESS=localhost
+export PIG_RPC_PORT=9160
+export PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
+
+Then you can build and run it like this:
contrib/pig$ ant
contrib/pig$ bin/pig_cassandra -x local example-script.pig
This will run the test script against your Cassandra instance
-and will assume that there is a Keyspace1/Standard1 with some
+and will assume that there is a MyKeyspace/MyColumnFamily with some
data in it. It will run in local mode (see pig docs for more info).
If you'd like to get to a 'grunt' shell prompt, run:
@@ -38,24 +43,24 @@ contrib/pig$ bin/pig_cassandra -x local
Once the 'grunt' shell has loaded, try a simple program like the
following, which will determine the top 50 column names:
-grunt rows = LOAD 'cassandra://Keyspace1/Standard1' USING CassandraStorage();
-grunt cols = FOREACH rows GENERATE flatten($1);
+grunt rows = LOAD 'cassandra://MyKeyspace/MyColumnFamily' USING
CassandraStorage() AS (key, columns: bag {T: tuple(name, value)});
+grunt cols = FOREACH rows GENERATE flatten(columns);
grunt colnames = FOREACH cols GENERATE $0;
-grunt namegroups = GROUP colnames BY $0;
+grunt namegroups = GROUP colnames BY (chararray) $0;
grunt namecounts = FOREACH namegroups GENERATE COUNT($1), group;
grunt orderednames = ORDER namecounts BY $0;
grunt topnames = LIMIT orderednames 50;
grunt dump topnames;
Slices on columns can also be specified:
-grunt rows = LOAD
'cassandra://Keyspace1/Standard1slice_start=C2slice_end=C4ilimit=1reversed=true'
USING CassandraStorage();
+grunt rows = LOAD
'cassandra://MyKeyspace/MyColumnFamilyslice_start=C2slice_end=C4ilimit=1reversed=true'
USING CassandraStorage() AS (key, columns: bag {T: tuple(name, value)});
Binary values for slice_start and slice_end can be escaped such as '\u0255'
Outputting to Cassandra requires the same format from input, so the simplest
example is:
-grunt rows = LOAD 'cassandra://Keyspace1/Standard1' USING CassandraStorage();
-grunt STORE rows into 'cassandra://Keyspace1/Standard2' USING
CassandraStorage();
+grunt rows = LOAD 'cassandra://MyKeyspace/MyColumnFamily' USING
CassandraStorage();
+grunt STORE rows into 'cassandra://MyKeyspace/MyColumnFamily' USING
CassandraStorage();
Which will copy the ColumnFamily. Note that the destination ColumnFamily must
already exist for this to work.
Modified: cassandra/branches/cassandra-0.7/contrib/pig/example-script.pig
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/example-script.pig?rev=1097922r1=1097921r2=1097922view=diff
==
--- cassandra/branches/cassandra-0.7/contrib/pig/example-script.pig (original)
+++ cassandra/branches/cassandra-0.7/contrib/pig/example-script.pig Fri Apr 29
19:28:06 2011
@@ -1,7 +1,7 @@
-rows = LOAD 'cassandra://Keyspace1/Standard1' USING CassandraStorage();
-cols = FOREACH rows GENERATE flatten($1);
+rows = LOAD 'cassandra://MyKeyspace/MyColumnFamily' USING CassandraStorage()
AS (key, columns: bag {T: tuple(name, value)});
+cols = FOREACH rows GENERATE flatten(columns);
colnames = FOREACH cols GENERATE $0;
-namegroups = GROUP colnames BY $0;
+namegroups = GROUP colnames BY (chararray) $0;
namecounts = FOREACH namegroups GENERATE COUNT($1), group;
orderednames = ORDER namecounts BY $0;
topnames = LIMIT orderednames 50;
[jira] [Commented] (CASSANDRA-2118) Provide failure modes if issues with the underlying filesystem of a node
[ https://issues.apache.org/jira/browse/CASSANDRA-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027161#comment-13027161 ] Jonathan Ellis commented on CASSANDRA-2118: --- is this a complete patch? i don't see FSReadError class anywhere or where we turn IOException (i assume) into FSRE. Provide failure modes if issues with the underlying filesystem of a node Key: CASSANDRA-2118 URL: https://issues.apache.org/jira/browse/CASSANDRA-2118 Project: Cassandra Issue Type: Sub-task Affects Versions: 0.8 beta 1 Reporter: Chris Goffinet Assignee: Chris Goffinet Attachments: 0001-Provide-failure-modes-if-issues-with-the-underlying-.patch, 0001-Provide-failure-modes-if-issues-with-the-underlying-v2.patch, 0001-Provide-failure-modes-if-issues-with-the-underlying-v3.patch CASSANDRA-2116 introduces the ability to detect FS errors. Let's provide a mode in cassandra.yaml so operators can decide that in the event of failure what to do: 1) standard - means continue on all errors (default) 2) read - means only stop gossip/rpc server if 'reads' fail from drive, writes can fail but not kill gossip/rpc server 3) readwrite - means stop gossip/rpc server if any read or write errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097930 - /cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Author: brandonwilliams
Date: Fri Apr 29 19:43:33 2011
New Revision: 1097930
URL: http://svn.apache.org/viewvc?rev=1097930view=rev
Log:
Remove errant System.exit from stress so it can reuse keyspaces.
Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-2580
Modified:
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Modified:
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java?rev=1097930r1=1097929r2=1097930view=diff
==
---
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
(original)
+++
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Fri Apr 29 19:43:33 2011
@@ -438,7 +438,6 @@ public class Session
catch (InvalidRequestException e)
{
out.println(Unable to create stress keyspace: + e.getWhy());
-System.exit(1);
}
catch (Exception e)
{
[jira] [Updated] (CASSANDRA-1996) Add to the hadoop integration contrib stuff - more complicated set of text
[ https://issues.apache.org/jira/browse/CASSANDRA-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1996: -- Fix Version/s: (was: 0.7.6) Add to the hadoop integration contrib stuff - more complicated set of text -- Key: CASSANDRA-1996 URL: https://issues.apache.org/jira/browse/CASSANDRA-1996 Project: Cassandra Issue Type: Improvement Components: Contrib, Hadoop Reporter: Jeremy Hanna Assignee: Jeremy Hanna Priority: Trivial It would be nice to have a more complicated set of text to do the word count over - perhaps a more interesting example too. Based on CASSANDRA-1993, it would be nice for sanity checking purposes as well. This could be for the word count example and perhaps have an easier way to test the pig and hadoop output streaming examples as well. I've noticed that I forget and people in the channel don't get through the README for the Pig example - maybe there are just too many steps and we could do better with the defaults. Anyway, so other improvement could go in this ticket as well. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1740: -- Fix Version/s: (was: 0.7.6) 0.8.1 Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Stu Hood Priority: Minor Fix For: 0.8.1 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2487) Pig example script no longer working
[ https://issues.apache.org/jira/browse/CASSANDRA-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027163#comment-13027163 ] Hudson commented on CASSANDRA-2487: --- Integrated in Cassandra-0.7 #465 (See [https://builds.apache.org/hudson/job/Cassandra-0.7/465/]) Update pig example script to work again. Patch by Jeremy Hanna, reviewed by brandonwilliams for CASSANDRA-2487 Pig example script no longer working Key: CASSANDRA-2487 URL: https://issues.apache.org/jira/browse/CASSANDRA-2487 Project: Cassandra Issue Type: Bug Reporter: Jeremy Hanna Assignee: Jeremy Hanna Priority: Minor Labels: hadoop, pig Fix For: 0.7.6 Attachments: 2487.txt There is a strange error given when trying to run the example-script.pig. java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:870) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:573) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) Looks like it has to do with PIG-919 and PIG-1277. For now we can just cast the var as a chararray and it works though. Will attach a patch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2520) use -z to test for empty variables to make sh and dash happy
[ https://issues.apache.org/jira/browse/CASSANDRA-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2520. --- Resolution: Not A Problem Fix Version/s: (was: 0.7.6) Reviewer: (was: cywjackson) Assignee: (was: Jackson Chung) this isn't actually causing problems with dash that we've seen. use -z to test for empty variables to make sh and dash happy Key: CASSANDRA-2520 URL: https://issues.apache.org/jira/browse/CASSANDRA-2520 Project: Cassandra Issue Type: Bug Components: Packaging Reporter: Jonathan Ellis Priority: Minor Attachments: 2520-0.7.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1097950 - /cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java
Author: jbellis Date: Fri Apr 29 20:52:58 2011 New Revision: 1097950 URL: http://svn.apache.org/viewvc?rev=1097950view=rev Log: add empty cf_defs list to satisfy thrift patch by jbellis Modified: cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java Modified: cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java?rev=1097950r1=1097949r2=1097950view=diff == --- cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java (original) +++ cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java Fri Apr 29 20:52:58 2011 @@ -84,12 +84,11 @@ public abstract class TestBase Cassandra.Client client = controller.createClient(hosts.get(0)); MapString,String stratOptions = new HashMapString,String(); stratOptions.put(replication_factor, + rf); -client.system_add_keyspace( -new KsDef( -name, -org.apache.cassandra.locator.SimpleStrategy, -Arrays.asList(cfdef)) -.setStrategy_options(stratOptions)); +client.system_add_keyspace(new KsDef(name, + org.apache.cassandra.locator.SimpleStrategy, + Arrays.asList(cfdef)) + .setStrategy_options(stratOptions) + .setCf_defs(Collections.CfDefemptyList())); // poll, until KS added for (InetAddress host : hosts)
svn commit: r1097952 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/migration/ test/distri
Author: jbellis Date: Fri Apr 29 20:59:49 2011 New Revision: 1097952 URL: http://svn.apache.org/viewvc?rev=1097952view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/CompactionManager.java cassandra/trunk/src/java/org/apache/cassandra/db/migration/Migration.java cassandra/trunk/test/distributed/org/apache/cassandra/TestBase.java cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/Session.java cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/operations/IndexedRangeSlicer.java cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 20:59:49 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097852,1097892 +/cassandra/branches/cassandra-0.7:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090935-1097471,1097883,1097887,1097893 +/cassandra/branches/cassandra-0.8:1090935-1097951 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1097952r1=1097951r2=1097952view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Fri Apr 29 20:59:49 2011 @@ -18,6 +18,11 @@ reads to time out even when responses were received (CASSAMDRA-2552) * trigger read repair correctly for LOCAL_QUORUM reads (CASSANDRA-2556) * Allow configuring the number of compaction thread (CASSANDRA-2558) + * forceUserDefinedCompaction will attempt to compact what it is given + even if the pessimistic estimate is that there is not enough disk space; + automatic compactions will only compact 2 or more sstables (CASSANDRA-2575) + * refuse to apply migrations with older timestamps than the current + schema (CASSANDRA-2536) 0.8.0-beta1 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 20:59:49 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097852,1097892 +/cassandra/branches/cassandra-0.7/contrib:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090935-1097471,1097883,1097887,1097893 +/cassandra/branches/cassandra-0.8/contrib:1090935-1097951 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 /incubator/cassandra/branches/cassandra-0.4/contrib:810145-810987,810994-834239,834349-834350 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Apr 29 20:59:49 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097852,1097892 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1096531,1096805,1097448,1097455,1097668,1097843,1097852,1097892 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1294 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1097952 Blamelist: jbellis BUILD FAILED: failed compile sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-2398) Type specific compression
[
https://issues.apache.org/jira/browse/CASSANDRA-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027198#comment-13027198
]
Stu Hood commented on CASSANDRA-2398:
-
Planning to update the signatures to:
{code:java}
public ByteBuffer compress(int version, final ListByteBuffer from, ByteBuffer
to)
public void decompress(int version, ByteBuffer from, ListByteBuffer to)
{code}
Which would make skip a native operation, and remove a lot of copies and object
creation.
Type specific compression
-
Key: CASSANDRA-2398
URL: https://issues.apache.org/jira/browse/CASSANDRA-2398
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.8 beta 1
Reporter: Stu Hood
Labels: compression
Attachments:
0001-CASSANDRA-2398-Add-basic-type-specific-compression-to-.txt,
0002-CASSANDRA-2398-LZF-for-AbstractType-base.txt
Cassandra has a lot of locations that are ripe for type specific compression.
A short list:
Indexes
* Keys compressed as BytesType, which could default to LZO/LZMA
* Offsets (delta and varint encoding)
* Column names added by 2319
Data
* Keys, columns, timestamps: see
http://wiki.apache.org/cassandra/FileFormatDesignDoc
A basic interface for type specific compression could be as simple as:
{code:java}
public void compress(int version, final ListByteBuffer from, DataOutput to)
throws IOException
public void decompress(int version, DataInput from, ListByteBuffer to)
throws IOException
public void skip(int version, DataInput from) throws IOException
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2468) Clean up after failed compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027205#comment-13027205 ] Aaron Morton commented on CASSANDRA-2468: - should have time this weekend. Clean up after failed compaction Key: CASSANDRA-2468 URL: https://issues.apache.org/jira/browse/CASSANDRA-2468 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jonathan Ellis Assignee: Aaron Morton Priority: Minor Fix For: 0.7.6, 0.8.1 (Started in CASSANDRA-2088.) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew F. Dennis updated CASSANDRA-1278: - Attachment: (was: 1278-cassandra-0.7-v2.txt) Make bulk loading into Cassandra less crappy, more pluggable Key: CASSANDRA-1278 URL: https://issues.apache.org/jira/browse/CASSANDRA-1278 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jeremy Hanna Assignee: Matthew F. Dennis Fix For: 0.8.1 Attachments: 1278-cassandra-0.7.1.txt, 1278-cassandra-0.7.txt Original Estimate: 40h Time Spent: 40h 40m Remaining Estimate: 0h Currently bulk loading into Cassandra is a black art. People are either directed to just do it responsibly with thrift or a higher level client, or they have to explore the contrib/bmt example - http://wiki.apache.org/cassandra/BinaryMemtable That contrib module requires delving into the code to find out how it works and then applying it to the given problem. Using either method, the user also needs to keep in mind that overloading the cluster is possible - which will hopefully be addressed in CASSANDRA-685 This improvement would be to create a contrib module or set of documents dealing with bulk loading. Perhaps it could include code in the Core to make it more pluggable for external clients of different types. It is just that this is something that many that are new to Cassandra need to do - bulk load their data into Cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
[
https://issues.apache.org/jira/browse/CASSANDRA-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew F. Dennis updated CASSANDRA-1278:
-
Attachment: 1278-cassandra-0.7-v2.txt
{quote}Did you accidentally remove the buffering for non-streaming connections
in IncomingTCPConnection?{quote}
It was intentional as previously only the streaming was buffered (at 4k) but
the bulk of the work uses the socket channel; only the size and header are read
from input and the header uses readFully. It adds an extra call when
constructing the stream because of the size but avoids copying the data into
the buffer (in the BufferedStream) and then into the byte array. We could
lower some of those calls by reading both the magic and the header int at the
same time into a ByteBuffer and then viewing it as an IntBuffer but I don't
think that buys you anything as it only happens on a new connection. It also
avoids bugs where something has been read from the socket into the buffer and
then the socket channel is used later even though the buffer may not have been
fully drained.
{quote}What kind of failures are supported during a load?{quote}
On the server side all failures result in the same behaviour: close socket,
delete temp files.
On the client side if flushing of a BareMemtable to the server fails the proxy
will log it and continue running.
In both cases any data that was being loaded via the proxy needs to be reloaded.
{quote}What's the proper behaviour for flush_proxy when some hosts fail?{quote}
log failed flushes and continue running; any data that was being loaded via the
proxy needs to be reloaded.
{quote}Could we avoid coding in knowledge of the file format in the inner loop
of IncomingLoaderStreamReader? I would much, much prefer that
non-file-format-specific framing be added, and it would have the added benefit
of not requiring as many system calls (4 per row vs 2 per frame){quote}
We could construct something that buffers up X amount of data and then frames
the data being sent and change the inner loop to decompose that but it's extra
complexity, code and overhead. If we buffer it on the other side we consume
more memory for a longer period of time (thus giving it a better chance that it
needs to promoted and/or compacted) adding to the already problematic GC
pressure. If we don't buffer the rows we end up framing every row which is
additional data and still doing 2 out of the 4 transfers we do now on data of
the same size (since the frames wouldn't be any bigger). BTW, 2 or 4 xfes in
this situation doesn't affect the performance; the latency on the network and
CPU of compaction and indexing building dwarf any gains to be made here. The
current approach has the added benefit that debugging is easy because it's
clear where the key and row boundaries are.
{quote}What is the benefit of using an independent protocol for Loader
streams?{quote}
If you're comparing to the streams we use for repair and similar, they require
table names and byte ranges be known up front. While a proxy could just
generate a random name, it doesn't know the sizes because it doesn't have a
SSTable on disk (or buffered in memory). There is also no way for a node to
request a retry from a proxy if the stream fails because the proxy won't have
the data and in general is probably firewalled off C*-to-Proxy connections.
And even if we did, we'd still have a bunch of small sessions because the proxy
doesn't know when a client is going to stop sending data to it. In the most
general sense it could be a constant thing; a client may just continually pump
an RSS feed or stock ticks or something into it. tl;dr simplicity and code
reduction.
{quote}Again, awesome.{quote}
thanks
Make bulk loading into Cassandra less crappy, more pluggable
Key: CASSANDRA-1278
URL: https://issues.apache.org/jira/browse/CASSANDRA-1278
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jeremy Hanna
Assignee: Matthew F. Dennis
Fix For: 0.8.1
Attachments: 1278-cassandra-0.7-v2.txt, 1278-cassandra-0.7.1.txt,
1278-cassandra-0.7.txt
Original Estimate: 40h
Time Spent: 40h 40m
Remaining Estimate: 0h
Currently bulk loading into Cassandra is a black art. People are either
directed to just do it responsibly with thrift or a higher level client, or
they have to explore the contrib/bmt example -
http://wiki.apache.org/cassandra/BinaryMemtable That contrib module requires
delving into the code to find out how it works and then applying it to the
given problem. Using either method, the user also needs to keep in mind that
overloading the cluster is possible - which will hopefully be addressed in
CASSANDRA-685
This
[jira] [Resolved] (CASSANDRA-2548) CQL: After running DELETE without specifying columns, the key is still returned in queries
[
https://issues.apache.org/jira/browse/CASSANDRA-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cathy Daw resolved CASSANDRA-2548.
--
Resolution: Not A Problem
http://wiki.apache.org/cassandra/FAQ#range_ghosts
CQL: After running DELETE without specifying columns, the key is still
returned in queries
--
Key: CASSANDRA-2548
URL: https://issues.apache.org/jira/browse/CASSANDRA-2548
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.8 beta 1
Reporter: Cathy Daw
Priority: Minor
Labels: cql
I am not sure if this is expected because of how cassandra-cli works, but
coming from SQL I would expect that if I delete a row it doesn't show up in a
subsequent select statement.
*Test Case*
{code}
cqlsh DELETE FROM users where KEY = 'user2';
cqlsh select * from users where KEY='user2';
u'user2'
{code}
*Cassandra CLI shows no rows returned*
{code}
[default@cqldb] get users['user2'];
Returned 0 results.
[default@cqldb] list users;
RowKey: user2
[default@cqldb] del users ['user2'];
row removed.
[default@cqldb] list users;
RowKey: user2
{code}
*Test Setup*
{code}
CREATE COLUMNFAMILY users (
KEY varchar PRIMARY KEY,
password varchar,
gender varchar,
session_token varchar,
state varchar,
birth_year bigint);
INSERT INTO users (KEY, password, gender, state, birth_year) VALUES ('user2',
'ch@ngem3', 'f', 'CA', '1972');
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2569) Deleting a record for a key that doesn't exist, creates a key for the record
[
https://issues.apache.org/jira/browse/CASSANDRA-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cathy Daw resolved CASSANDRA-2569.
--
Resolution: Not A Problem
http://wiki.apache.org/cassandra/FAQ#range_ghosts
Deleting a record for a key that doesn't exist, creates a key for the record
Key: CASSANDRA-2569
URL: https://issues.apache.org/jira/browse/CASSANDRA-2569
Project: Cassandra
Issue Type: Bug
Reporter: Cathy Daw
Priority: Minor
This bug is similar to CASSANDRA-2548 where a delete did not do a full
delete. This is filed since the delete itself did an insert.
*Test Setup*
{code}
CREATE KEYSPACE cqldb with strategy_class =
'org.apache.cassandra.locator.SimpleStrategy'
and strategy_options:replication_factor=1;
use cqldb;
CREATE COLUMNFAMILY users (KEY varchar PRIMARY KEY, password varchar, gender
varchar,
session_token varchar, state varchar, birth_year bigint);
INSERT INTO users (KEY, password, gender, state, birth_year) VALUES ('user1',
'ch@ngem3', 'f', 'CA', '1971');
{code}
*CQLSH: Delete a row and a column using non-existent keys*
{code}
cqlsh DELETE bad_col FROM users where KEY = 'user2';
cqlsh DELETE from users where KEY = 'user3';
cqlsh select * from users;
u'user1' | u'birth_year',1971 | u'gender',u'f' | u'password',u'ch@ngem3' |
u'state',u'CA'
u'user3'
u'user2'
{code}
*Same is reproducible in cassandra-cli*
{code}
Authenticated to keyspace: cqldb
[default@cqldb] del users ['user11'];
row removed.
[default@cqldb] del users['user12']['badcol2'];
column removed.
[default@cqldb] list users;
Using default limit of 100
---
RowKey: user11
---
RowKey: user1
= (column=birth_year, value=1971, timestamp=1303849935934)
= (column=gender, value=f, timestamp=1303849935936)
= (column=password, value=ch@ngem3, timestamp=1303849935936)
= (column=state, value=CA, timestamp=1303849935936)
---
RowKey: user12
---
RowKey: user3
---
RowKey: user2
5 Rows Returned.
[default@cqldb] get users['user12'];
Returned 0 results.
[default@cqldb] get users['user2'];
Returned 0 results.
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2585) Inserting integer value using cassandra cli result in byte array of size 1 insteand of 4
Inserting integer value using cassandra cli result in byte array of size 1
insteand of 4
Key: CASSANDRA-2585
URL: https://issues.apache.org/jira/browse/CASSANDRA-2585
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.5
Environment: mac os, java 6
Reporter: Olivier Smadja
I create a column family Stream like this :
create column family Stream with comparator=AsciiType and column_metadata=[
{column_name: 'type', validation_class: IntegerType}];
Then I create some data using cassandra-cli :
set Stream['oid'][status]=1;
cli stores a byte array of length 1 in Stream['oid'][status].
As the status field is marked as integer, my understanding is that the data
stored should be an array of 4 bytes representing the number '1'.
This causes problems when using hector object mapping api.
Thanks
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13027291#comment-13027291 ] Stu Hood commented on CASSANDRA-1278: - It was intentional as previously only the streaming was buffered (at 4k) It was the other way around IIRC: (non-encrypted) streaming used channel.transferTo, which bypassed the buffering entirely. The buffering was for internode messaging: see CASSANDRA-1943. We could construct something that buffers up X amount of data and then frames the data being sent and change the inner loop to decompose that but it's extra complexity, code and overhead. You're already buffering rows in StreamingProxyFlusher.bufferRow: the change would simply be to continue to buffer rows until a threshold was reached. The benefit here is that the code on the receiving side doesn't need to change when the proxy starts sending it a different SSTable version/format. I've never heard of somebody regretting having framing in a protocol: it's always the other way around. Also, an SSTable version (as usually held by Descriptor) should be added to the header of your protocol so that clients don't break by sending unversioned blobs: not having versioning is my primary complaint vis-a-vis BinaryMemtables. If we buffer it on the other side we consume more memory for a longer period of time I was talking about buffering on the client side: the server side can do one system call to flush to disk, such that it never enters userspace. Thanks! Make bulk loading into Cassandra less crappy, more pluggable Key: CASSANDRA-1278 URL: https://issues.apache.org/jira/browse/CASSANDRA-1278 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jeremy Hanna Assignee: Matthew F. Dennis Fix For: 0.8.1 Attachments: 1278-cassandra-0.7-v2.txt, 1278-cassandra-0.7.1.txt, 1278-cassandra-0.7.txt Original Estimate: 40h Time Spent: 40h 40m Remaining Estimate: 0h Currently bulk loading into Cassandra is a black art. People are either directed to just do it responsibly with thrift or a higher level client, or they have to explore the contrib/bmt example - http://wiki.apache.org/cassandra/BinaryMemtable That contrib module requires delving into the code to find out how it works and then applying it to the given problem. Using either method, the user also needs to keep in mind that overloading the cluster is possible - which will hopefully be addressed in CASSANDRA-685 This improvement would be to create a contrib module or set of documents dealing with bulk loading. Perhaps it could include code in the Core to make it more pluggable for external clients of different types. It is just that this is something that many that are new to Cassandra need to do - bulk load their data into Cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
