[jira] [Updated] (CASSANDRA-2156) Compaction Throttling
[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2156:
Attachment: (was:
0006-Throttle-total-compaction-to-a-configurable-throughput.txt)
Compaction Throttling
-
Key: CASSANDRA-2156
URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
Project: Cassandra
Issue Type: New Feature
Reporter: Stu Hood
Assignee: Stu Hood
Fix For: 0.8
Attachments:
for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
for-0.6-0002-Make-compaction-throttling-configurable.txt
Compaction is currently relatively bursty: we compact as fast as we can, and
then we wait for the next compaction to be possible (hurry up and wait).
Instead, to properly amortize compaction, you'd like to compact exactly as
fast as you need to to keep the sstable count under control.
For every new level of compaction, you need to increase the rate that you
compact at: a rule of thumb that we're testing on our clusters is to
determine the maximum number of buckets a node can support (aka, if the 15th
bucket holds 750 GB, we're not going to have more than 15 buckets), and then
multiply the flush throughput by the number of buckets to get a minimum
compaction throughput to maintain your sstable count.
Full explanation: for a min compaction threshold of {{T}}, the bucket at
level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
causing the bucket at level N to fill. If the bucket at level N fills, it
causes {{SsubN}} units to be compacted. So, for each active level in your
system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
time a new unit is added.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2156) Compaction Throttling
[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2156:
Attachment: 0007-Throttle-total-compaction-to-a-configurable-throughput.txt
Compaction Throttling
-
Key: CASSANDRA-2156
URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
Project: Cassandra
Issue Type: New Feature
Reporter: Stu Hood
Assignee: Stu Hood
Fix For: 0.8
Attachments:
0007-Throttle-total-compaction-to-a-configurable-throughput.txt,
for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
for-0.6-0002-Make-compaction-throttling-configurable.txt
Compaction is currently relatively bursty: we compact as fast as we can, and
then we wait for the next compaction to be possible (hurry up and wait).
Instead, to properly amortize compaction, you'd like to compact exactly as
fast as you need to to keep the sstable count under control.
For every new level of compaction, you need to increase the rate that you
compact at: a rule of thumb that we're testing on our clusters is to
determine the maximum number of buckets a node can support (aka, if the 15th
bucket holds 750 GB, we're not going to have more than 15 buckets), and then
multiply the flush throughput by the number of buckets to get a minimum
compaction throughput to maintain your sstable count.
Full explanation: for a min compaction threshold of {{T}}, the bucket at
level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
causing the bucket at level N to fill. If the bucket at level N fills, it
causes {{SsubN}} units to be compacted. So, for each active level in your
system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
time a new unit is added.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0005-Add-a-harness-to-allow-compaction-tasks-that-need-to-a.txt) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0001-Add-a-compacting-set-to-DataTracker.txt) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0004-Allow-multithread-compaction-to-be-disabled.txt) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: 0006-Prevent-cache-saves-from-occuring-concurrently.txt 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt 0004-Allow-multithread-compaction-to-be-disabled.txt 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt 0001-Add-a-compacting-set-to-DataTracker.txt * Inlined stopTheWorld in 0005. Yes, I agree that the name sucked, but whether or not it is possible for a lock acquisition to fail on a server that is not already screwed, and whether an abstraction is in order here is still up for debate * Removed the 'forceMajor' parameter: will open a ticket post-commit to allow for guaranteeing that a manually triggered compaction is major * Moved ksname/cfname into getters. I didn't do this initially because the CFS is sometimes null, but I guess you'd get the NPE in either case * Added an AtomicBoolean to AutoSavingCache in 0006. I reeeally think this should go to the flush stage, since the tasks have almost identical lifetimes, and we don't really need progress for either of them * Wrapped the IdentityHashMap into an IdentityHashSet * Returned printCompactionStats to its former glory * Removed OperationType from SSTableWriter.Builder's task type Thanks! CASSANDRA-2156 has been rebased as well. Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt, 0004-Allow-multithread-compaction-to-be-disabled.txt, 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt, 0006-Prevent-cache-saves-from-occuring-concurrently.txt This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of API07 by AhmetEkremSaban
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The API07 page has been changed by AhmetEkremSaban. The comment on this change is: Started a new page for API 0.7. http://wiki.apache.org/cassandra/API07 -- New page: ## page was copied from API == Overview == The Cassandra Thrift API changed between [[API03|0.3]], [[API04|0.4]], [[API|0.5]] and [[API|0.6]]; this document explains the 0.7 version. Cassandra's client API is built entirely on top of Thrift. It should be noted that these documents mention default values, but these are not generated in all of the languages that Thrift supports. Full examples of using Cassandra from Thrift, including setup boilerplate, are found on ThriftExamples. Higher-level clients are linked from ClientOptions. '''WARNING:''' Some SQL/RDBMS terms are used in this documentation for analogy purposes. They should be thought of as just that; analogies. There are few similarities between how data is managed in a traditional RDBMS and Cassandra. Please see DataModel for more information. '''This article is a stub for the Apache Cassandra API 0.7'''
[Cassandra Wiki] Trivial Update of ClientOptions by AhmetEkremSaban
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The ClientOptions page has been changed by AhmetEkremSaban. The comment on this change is: Ordered entries alphabetically. http://wiki.apache.org/cassandra/ClientOptions?action=diffrev1=124rev2=125 -- If no high-level client exists for your environment, you may be able to update an [[ClientOptions06|older client]]; failing that, you'll have to use the raw Thrift [[API]]. * Python: + * Pycassa: http://github.com/pycassa/pycassa * Telephus: http://github.com/driftx/Telephus (Twisted) - * Pycassa: http://github.com/pycassa/pycassa * Java: + * Datanucleus JDO: http://github.com/tnine/Datanucleus-Cassandra-Plugin * Hector: http://github.com/rantav/hector (Examples https://github.com/zznate/hector-examples ) + * Kundera http://code.google.com/p/kundera/ * Pelops: http://github.com/s7/scale7-pelops - * Kundera http://code.google.com/p/kundera/ - * Datanucleus JDO: http://github.com/tnine/Datanucleus-Cassandra-Plugin * Grails: * grails-cassandra: https://github.com/wolpert/grails-cassandra * .NET: + * Aquiles: http://aquiles.codeplex.com/ * FluentCassandra: http://github.com/managedfusion/fluentcassandra - * Aquiles: http://aquiles.codeplex.com/ * Ruby: * Cassandra: http://github.com/fauna/cassandra * PHP:
svn commit: r1090979 - in /cassandra/trunk: ./ conf/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/io/ src/java/org/apache/cassandra/service/ sr
Author: slebresne
Date: Mon Apr 11 08:45:07 2011
New Revision: 1090979
URL: http://svn.apache.org/viewvc?rev=1090979view=rev
Log:
Compaction throttling
patch by stuhood; reviewed by slebresne for CASSANDRA-2156
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/conf/cassandra.yaml
cassandra/trunk/src/java/org/apache/cassandra/config/Config.java
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
cassandra/trunk/src/java/org/apache/cassandra/db/CompactionManager.java
cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1090979r1=1090978r2=1090979view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Apr 11 08:45:07 2011
@@ -20,7 +20,7 @@
* push replication_factor into strategy_options (CASSANDRA-1263)
* give snapshots the same name on each node (CASSANDRA-1791)
* multithreaded compaction (CASSANDRA-2191)
-
+ * compaction throttling (CASSANDRA-2156)
0.7.5
* Avoid seeking when sstable2json exports the entire file (CASSANDRA-2318)
Modified: cassandra/trunk/conf/cassandra.yaml
URL:
http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1090979r1=1090978r2=1090979view=diff
==
--- cassandra/trunk/conf/cassandra.yaml (original)
+++ cassandra/trunk/conf/cassandra.yaml Mon Apr 11 08:45:07 2011
@@ -250,9 +250,17 @@ column_index_size_in_kb: 64
in_memory_compaction_limit_in_mb: 64
# Enables multiple compactions to execute at once. This is highly recommended
-# for preserving read performance in a mixed read/write workload.
+# for preserving read performance in a mixed read/write workload as this
+# avoids sstables from accumulating during long running compactions.
compaction_multithreading: true
+# Throttles compaction to the given total throughput across the entire
+# system. The faster you insert data, the faster you need to compact in
+# order to keep the sstable count down, but in general, setting this to
+# 16 to 32 times the rate you are inserting data is more than sufficient.
+# Setting this to 0 disables throttling.
+compaction_throughput_mb_per_sec: 16
+
# Track cached row keys during compaction, and re-cache their new
# positions in the compacted sstable. Disable if you use really large
# key caches.
Modified: cassandra/trunk/src/java/org/apache/cassandra/config/Config.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/Config.java?rev=1090979r1=1090978r2=1090979view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/config/Config.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/config/Config.java Mon Apr 11
08:45:07 2011
@@ -83,6 +83,7 @@ public class Config
public Integer column_index_size_in_kb = 64;
public Integer in_memory_compaction_limit_in_mb = 256;
public Boolean compaction_multithreading = true;
+public Integer compaction_throughput_mb_per_sec = 16;
public String[] data_file_directories;
Modified:
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1090979r1=1090978r2=1090979view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
Mon Apr 11 08:45:07 2011
@@ -344,6 +344,9 @@ public class DatabaseDescriptor
if (conf.compaction_multithreading == null)
conf.compaction_multithreading = true;
+if (conf.compaction_throughput_mb_per_sec == null)
+conf.compaction_throughput_mb_per_sec = 16;
+
/* data file and commit log directories. they get created later,
when they're needed. */
if (conf.commitlog_directory != null conf.data_file_directories
!= null conf.saved_caches_directory != null)
{
@@ -731,6 +734,16 @@ public class DatabaseDescriptor
return conf.compaction_multithreading;
}
+public static int getCompactionThroughputMbPerSec()
+{
+return conf.compaction_throughput_mb_per_sec;
+}
+
+public static void setCompactionThroughputMbPerSec(int value)
+
[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018251#comment-13018251 ] Sylvain Lebresne commented on CASSANDRA-2191: - For the record: bq. Inlined stopTheWorld in 0005. Yes, I agree that the name sucked, but whether or not it is possible for a lock acquisition to fail on a server that is not already screwed, and whether an abstraction is in order here is still up for debate I do like the inlined version much more. I did not pretended that the previous version wasn't working. It was just hard to check that the umarking was happening correctly and even though I agree lock acquisition is unlikely to fail, it would have been easy for someone else to add lines inside stopTheWorld at the wrong place that could fail. And the name sucked :) bq. Added an AtomicBoolean to AutoSavingCache in 0006. I reeeally think this should go to the flush stage, since the tasks have almost identical lifetimes, and we don't really need progress for either of them I just don't want for cache saving to block flush too long. So I'm not saying it should not go to flush stage ever, but I'm inconfortable putting it there without some proper testing of its impact. We could make the flush stage multithreaded (with throttling), then I would have no problem with moving cache saving there (but then we would still have to make sure only one saving happen at a time). Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt, 0004-Allow-multithread-compaction-to-be-disabled.txt, 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt, 0006-Prevent-cache-saves-from-occuring-concurrently.txt This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2156) Compaction Throttling
[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018255#comment-13018255
]
Hudson commented on CASSANDRA-2156:
---
Integrated in Cassandra #848 (See
[https://hudson.apache.org/hudson/job/Cassandra/848/])
Compaction throttling
patch by stuhood; reviewed by slebresne for CASSANDRA-2156
Compaction Throttling
-
Key: CASSANDRA-2156
URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
Project: Cassandra
Issue Type: New Feature
Reporter: Stu Hood
Assignee: Stu Hood
Fix For: 0.8
Attachments:
0007-Throttle-total-compaction-to-a-configurable-throughput.txt,
for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
for-0.6-0002-Make-compaction-throttling-configurable.txt
Compaction is currently relatively bursty: we compact as fast as we can, and
then we wait for the next compaction to be possible (hurry up and wait).
Instead, to properly amortize compaction, you'd like to compact exactly as
fast as you need to to keep the sstable count under control.
For every new level of compaction, you need to increase the rate that you
compact at: a rule of thumb that we're testing on our clusters is to
determine the maximum number of buckets a node can support (aka, if the 15th
bucket holds 750 GB, we're not going to have more than 15 buckets), and then
multiply the flush throughput by the number of buckets to get a minimum
compaction throughput to maintain your sstable count.
Full explanation: for a min compaction threshold of {{T}}, the bucket at
level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
causing the bucket at level N to fill. If the bucket at level N fills, it
causes {{SsubN}} units to be compacted. So, for each active level in your
system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
time a new unit is added.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018254#comment-13018254 ] Hudson commented on CASSANDRA-2191: --- Integrated in Cassandra #848 (See [https://hudson.apache.org/hudson/job/Cassandra/848/]) Multithreaded compactions patch by stuhood; reviewed by slebresne for CASSANDRA-2191 Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt, 0004-Allow-multithread-compaction-to-be-disabled.txt, 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt, 0006-Prevent-cache-saves-from-occuring-concurrently.txt This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018271#comment-13018271 ] Pavel Yaskevich commented on CASSANDRA-2441: +1 Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2420) row cache / streaming aren't aware of each other
[ https://issues.apache.org/jira/browse/CASSANDRA-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne reassigned CASSANDRA-2420: --- Assignee: Sylvain Lebresne row cache / streaming aren't aware of each other Key: CASSANDRA-2420 URL: https://issues.apache.org/jira/browse/CASSANDRA-2420 Project: Cassandra Issue Type: Bug Affects Versions: 0.6 Reporter: Matthew F. Dennis Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 SSTableWriter.Builder.build() takes tables that resulted from streaming, repair, bootstrapping, et cetera and builds the indexes and bloom filters before adding it so the current node is aware of it. However, if there is data present in the cache for a row that is also present in the streamed table the row cache can over shadow the data in the newly built table. In other words, until the row in row cache is removed from the cache (e.g. because it's pushed out because of size, the node is restarted, the cache is manually cleared) the data in the newly built table will never be returned to clients. The solution that seems most reasonable at this point is to have SSTableWriter.Builder.build() (or something below it) update the row cache if the row key in the table being built is also present in the cache. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2326) stress.java indexed range slicing is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018276#comment-13018276 ] Pavel Yaskevich commented on CASSANDRA-2326: We can maybe offer users to provide list of the values for indexes range slices, don't have any other solution right now... stress.java indexed range slicing is broken --- Key: CASSANDRA-2326 URL: https://issues.apache.org/jira/browse/CASSANDRA-2326 Project: Cassandra Issue Type: Bug Components: Contrib Reporter: Brandon Williams Assignee: Pavel Yaskevich Priority: Trivial I probably broke it when I fixed the build that CASSANDRA-2312 broke. Now it compiles, but never works. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018303#comment-13018303 ] Pavel Yaskevich commented on CASSANDRA-1740: Stu Hood: do you want to be in charge of this one since your changes in compaction mechanism? Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.5 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018310#comment-13018310 ] Sylvain Lebresne commented on CASSANDRA-1740: - I committed CASSANDRA-2191 so as stu said, this will need some rebasing to handle it. For repair, let's just create another ticket. If stu wants/have time to do it, fine, otherwise I may do it. Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.5 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018315#comment-13018315 ] Sylvain Lebresne commented on CASSANDRA-1740: - As for handling the multi-threaded compactions, the hashcode would more or less work but since it's not totally safe I would prefer assigning a name to each compaction which could be: * a uuid assigned to each compaction when created * a simple (atomically) increasing number * a simple (atomically) increasing number for each type of compaction, the name being something like major-42 or minor-3012. Nice thing is it tells you how many minor, major, validata, ... compaction you have run already. Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.5 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018316#comment-13018316 ] Pavel Yaskevich commented on CASSANDRA-1740: I like the last option! Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.5 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2056) Need a way of flattening schemas.
[ https://issues.apache.org/jira/browse/CASSANDRA-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2056: - Attachment: v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt v2-0001-convert-MigrationManager-into-a-singleton.txt Need a way of flattening schemas. - Key: CASSANDRA-2056 URL: https://issues.apache.org/jira/browse/CASSANDRA-2056 Project: Cassandra Issue Type: Improvement Reporter: Gary Dusbabek Assignee: Gary Dusbabek Fix For: 0.8 Attachments: v2-0001-convert-MigrationManager-into-a-singleton.txt, v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt, v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt For all of our trying not to, we still managed to screw this up. Schema updates currently contain a serialized RowMutation stored as a column value. When a node needs updated schema, it requests these values, deserializes them and applies them. As the serialization scheme for RowMutation changes over time (this is inevitable), those old migrations will become incompatible with newer implementations of the RowMutation deserializer. This means that when new nodes come online, they'll get migration messages that they have trouble deserializing. (Remember, we've only made the promise that we'll be backwards compatible for one version--see CASSANDRA-1015--even though we'd eventually have this problem without that guarantee.) What I propose is a cluster command to flatten the schema prior to upgrading. This would basically purge the old schema updates and replace them with a single serialized migration (serialized in the current protocol version). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2056) Need a way of flattening schemas.
[ https://issues.apache.org/jira/browse/CASSANDRA-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018339#comment-13018339 ] Gary Dusbabek commented on CASSANDRA-2056: -- Attached rebased v2. CliTest fails though, so not committing yet. Need a way of flattening schemas. - Key: CASSANDRA-2056 URL: https://issues.apache.org/jira/browse/CASSANDRA-2056 Project: Cassandra Issue Type: Improvement Reporter: Gary Dusbabek Assignee: Gary Dusbabek Fix For: 0.8 Attachments: v2-0001-convert-MigrationManager-into-a-singleton.txt, v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt, v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt For all of our trying not to, we still managed to screw this up. Schema updates currently contain a serialized RowMutation stored as a column value. When a node needs updated schema, it requests these values, deserializes them and applies them. As the serialization scheme for RowMutation changes over time (this is inevitable), those old migrations will become incompatible with newer implementations of the RowMutation deserializer. This means that when new nodes come online, they'll get migration messages that they have trouble deserializing. (Remember, we've only made the promise that we'll be backwards compatible for one version--see CASSANDRA-1015--even though we'd eventually have this problem without that guarantee.) What I propose is a cluster command to flatten the schema prior to upgrading. This would basically purge the old schema updates and replace them with a single serialized migration (serialized in the current protocol version). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2056) Need a way of flattening schemas.
[ https://issues.apache.org/jira/browse/CASSANDRA-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2056: - Attachment: (was: v1-0001-convert-MigrationManager-into-a-singleton.txt) Need a way of flattening schemas. - Key: CASSANDRA-2056 URL: https://issues.apache.org/jira/browse/CASSANDRA-2056 Project: Cassandra Issue Type: Improvement Reporter: Gary Dusbabek Assignee: Gary Dusbabek Fix For: 0.8 Attachments: v2-0001-convert-MigrationManager-into-a-singleton.txt, v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt, v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt For all of our trying not to, we still managed to screw this up. Schema updates currently contain a serialized RowMutation stored as a column value. When a node needs updated schema, it requests these values, deserializes them and applies them. As the serialization scheme for RowMutation changes over time (this is inevitable), those old migrations will become incompatible with newer implementations of the RowMutation deserializer. This means that when new nodes come online, they'll get migration messages that they have trouble deserializing. (Remember, we've only made the promise that we'll be backwards compatible for one version--see CASSANDRA-1015--even though we'd eventually have this problem without that guarantee.) What I propose is a cluster command to flatten the schema prior to upgrading. This would basically purge the old schema updates and replace them with a single serialized migration (serialized in the current protocol version). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2056) Need a way of flattening schemas.
[ https://issues.apache.org/jira/browse/CASSANDRA-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2056: - Attachment: (was: v1-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt) Need a way of flattening schemas. - Key: CASSANDRA-2056 URL: https://issues.apache.org/jira/browse/CASSANDRA-2056 Project: Cassandra Issue Type: Improvement Reporter: Gary Dusbabek Assignee: Gary Dusbabek Fix For: 0.8 Attachments: v2-0001-convert-MigrationManager-into-a-singleton.txt, v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt, v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt For all of our trying not to, we still managed to screw this up. Schema updates currently contain a serialized RowMutation stored as a column value. When a node needs updated schema, it requests these values, deserializes them and applies them. As the serialization scheme for RowMutation changes over time (this is inevitable), those old migrations will become incompatible with newer implementations of the RowMutation deserializer. This means that when new nodes come online, they'll get migration messages that they have trouble deserializing. (Remember, we've only made the promise that we'll be backwards compatible for one version--see CASSANDRA-1015--even though we'd eventually have this problem without that guarantee.) What I propose is a cluster command to flatten the schema prior to upgrading. This would basically purge the old schema updates and replace them with a single serialized migration (serialized in the current protocol version). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2056) Need a way of flattening schemas.
[ https://issues.apache.org/jira/browse/CASSANDRA-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2056: - Attachment: (was: v1-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt) Need a way of flattening schemas. - Key: CASSANDRA-2056 URL: https://issues.apache.org/jira/browse/CASSANDRA-2056 Project: Cassandra Issue Type: Improvement Reporter: Gary Dusbabek Assignee: Gary Dusbabek Fix For: 0.8 Attachments: v2-0001-convert-MigrationManager-into-a-singleton.txt, v2-0002-bail-on-migrations-originating-from-newer-protocol-ver.txt, v2-0003-a-way-to-upgrade-schema-when-protocol-version-changes.txt For all of our trying not to, we still managed to screw this up. Schema updates currently contain a serialized RowMutation stored as a column value. When a node needs updated schema, it requests these values, deserializes them and applies them. As the serialization scheme for RowMutation changes over time (this is inevitable), those old migrations will become incompatible with newer implementations of the RowMutation deserializer. This means that when new nodes come online, they'll get migration messages that they have trouble deserializing. (Remember, we've only made the promise that we'll be backwards compatible for one version--see CASSANDRA-1015--even though we'd eventually have this problem without that guarantee.) What I propose is a cluster command to flatten the schema prior to upgrading. This would basically purge the old schema updates and replace them with a single serialized migration (serialized in the current protocol version). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of Operations_JP by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The Operations_JP page has been changed by MakiWatanabe. The comment on this change is: fix typo. http://wiki.apache.org/cassandra/Operations_JP?action=diffrev1=104rev2=105 -- レプリケーションファクタを減らすのは簡単です。レプリケーションファクタを減らした後、余分なレプリカデータを削除するためにcleanupを実行して下さい。 === ネットワークトポロジー === - レプリケーションストラテジーによってデータセンター間のレプリカ配置を制御できますが、これに加えてできますが、データセンター内でどのノードが同じラックに設置されているかをCassandraに認識させることができます。Cassandraはreadやトークン範囲変更のためのデータの移動の際にこの情報を使用して最も近いレプリカを使用します。近接ノード検出の挙動は設定ファイルで差し替え可能な!EndpointSnitchクラスで変更可能です。 + レプリケーションストラテジーによってデータセンター間のレプリカ配置を制御できますが、これに加えてデータセンター内でどのノードが同じラックに設置されているかをCassandraに認識させることができます。Cassandraはreadやトークン範囲変更のためのデータの移動の際にこの情報を使用して最も近いレプリカを使用します。近接ノード検出の挙動は設定ファイルで差し替え可能な!EndpointSnitchクラスで変更可能です。 !EndpointSnitchはレプリケーションストラテジーに関係していますが、レプリケーションストラテジーそのものとは異なるものです。!RackAwareStrategyが適切にレプリカを配置するには正しく構成されたSnitch が必要です。しかしデータセンターを意識したレプリケーションストラテジーを使用しない場合もCassandraはノード間の近接情報を必要としています。
svn commit: r1091080 - in /cassandra/branches/cassandra-0.8: ./ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache/cassandra/config/ src/java/org/apache/ca
Author: jbellis
Date: Mon Apr 11 14:11:23 2011
New Revision: 1091080
URL: http://svn.apache.org/viewvc?rev=1091080view=rev
Log:
add optional key alias to CFMetaData
patch by jhermes; reviewed by jbellis for CASSANDRA-2396
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/interface/cassandra.thrift
cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Constants.java
cassandra/branches/cassandra-0.8/src/avro/internode.genavro
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/CFMetaData.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1091080r1=1091079r2=1091080view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 14:11:23 2011
@@ -18,7 +18,7 @@
* purge tombstones from row cache (CASSANDRA-2305)
* push replication_factor into strategy_options (CASSANDRA-1263)
* give snapshots the same name on each node (CASSANDRA-1791)
- * add key type information (CASSANDRA-2311)
+ * add key type information and alias (CASSANDRA-2311, 2396)
0.7.5
Modified: cassandra/branches/cassandra-0.8/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/interface/cassandra.thrift?rev=1091080r1=1091079r2=1091080view=diff
==
--- cassandra/branches/cassandra-0.8/interface/cassandra.thrift (original)
+++ cassandra/branches/cassandra-0.8/interface/cassandra.thrift Mon Apr 11
14:11:23 2011
@@ -46,7 +46,7 @@ namespace rb CassandraThrift
# for every edit that doesn't result in a change to major/minor.
#
# See the Semantic Versioning Specification (SemVer) http://semver.org.
-const string VERSION = 20.0.0
+const string VERSION = 20.1.0
#
@@ -394,6 +394,7 @@ struct CfDef {
25: optional double merge_shards_chance,
26: optional string key_validation_class,
27: optional string
row_cache_provider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider,
+28: optional binary key_alias,
}
/* describes a keyspace. */
Modified:
cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java?rev=1091080r1=1091079r2=1091080view=diff
==
---
cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
(original)
+++
cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
Mon Apr 11 14:11:23 2011
@@ -69,6 +69,7 @@ public class CfDef implements org.apache
private static final org.apache.thrift.protocol.TField
MERGE_SHARDS_CHANCE_FIELD_DESC = new
org.apache.thrift.protocol.TField(merge_shards_chance,
org.apache.thrift.protocol.TType.DOUBLE, (short)25);
private static final org.apache.thrift.protocol.TField
KEY_VALIDATION_CLASS_FIELD_DESC = new
org.apache.thrift.protocol.TField(key_validation_class,
org.apache.thrift.protocol.TType.STRING, (short)26);
private static final org.apache.thrift.protocol.TField
ROW_CACHE_PROVIDER_FIELD_DESC = new
org.apache.thrift.protocol.TField(row_cache_provider,
org.apache.thrift.protocol.TType.STRING, (short)27);
+ private static final org.apache.thrift.protocol.TField KEY_ALIAS_FIELD_DESC
= new org.apache.thrift.protocol.TField(key_alias,
org.apache.thrift.protocol.TType.STRING, (short)28);
public String keyspace;
public String name;
@@ -94,6 +95,7 @@ public class CfDef implements org.apache
public double merge_shards_chance;
public String key_validation_class;
public String row_cache_provider;
+ public ByteBuffer key_alias;
/** The set of fields this struct contains, along with convenience methods
for finding and manipulating them. */
public enum _Fields implements org.apache.thrift.TFieldIdEnum {
@@ -120,7 +122,8 @@ public class CfDef implements org.apache
REPLICATE_ON_WRITE((short)24, replicate_on_write),
MERGE_SHARDS_CHANCE((short)25, merge_shards_chance),
KEY_VALIDATION_CLASS((short)26, key_validation_class),
-ROW_CACHE_PROVIDER((short)27, row_cache_provider);
+ROW_CACHE_PROVIDER((short)27, row_cache_provider),
+KEY_ALIAS((short)28, key_alias);
private static final MapString, _Fields byName = new HashMapString,
_Fields();
@@ -183,6 +186,8 @@ public class CfDef implements org.apache
svn commit: r1091087 [1/3] - in /cassandra/branches/cassandra-0.8/drivers: py/cql/cassandra/constants.py py/cql/cassandra/ttypes.py txpy/txcql/cassandra/Cassandra.py txpy/txcql/cassandra/constants.py
Author: jbellis
Date: Mon Apr 11 14:23:30 2011
New Revision: 1091087
URL: http://svn.apache.org/viewvc?rev=1091087view=rev
Log:
update generated .py code
Modified:
cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/constants.py
cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/ttypes.py
cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/Cassandra.py
cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/constants.py
cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/ttypes.py
Modified: cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/constants.py
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/constants.py?rev=1091087r1=1091086r2=1091087view=diff
==
--- cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/constants.py
(original)
+++ cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/constants.py Mon
Apr 11 14:23:30 2011
@@ -7,4 +7,4 @@
from thrift.Thrift import *
from ttypes import *
-VERSION = 20.0.0
+VERSION = 20.1.0
Modified: cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/ttypes.py
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/ttypes.py?rev=1091087r1=1091086r2=1091087view=diff
==
--- cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/ttypes.py
(original)
+++ cassandra/branches/cassandra-0.8/drivers/py/cql/cassandra/ttypes.py Mon Apr
11 14:23:30 2011
@@ -2324,6 +2324,7 @@ class CfDef:
- merge_shards_chance
- key_validation_class
- row_cache_provider
+ - key_alias
thrift_spec = (
@@ -2355,9 +2356,10 @@ class CfDef:
(25, TType.DOUBLE, 'merge_shards_chance', None, None, ), # 25
(26, TType.STRING, 'key_validation_class', None, None, ), # 26
(27, TType.STRING, 'row_cache_provider', None,
org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider, ), # 27
+(28, TType.STRING, 'key_alias', None, None, ), # 28
)
- def __init__(self, keyspace=None, name=None, column_type=thrift_spec[3][4],
comparator_type=thrift_spec[5][4], subcomparator_type=None, comment=None,
row_cache_size=thrift_spec[9][4], key_cache_size=thrift_spec[11][4],
read_repair_chance=thrift_spec[12][4], column_metadata=None,
gc_grace_seconds=None, default_validation_class=None, id=None,
min_compaction_threshold=None, max_compaction_threshold=None,
row_cache_save_period_in_seconds=None, key_cache_save_period_in_seconds=None,
memtable_flush_after_mins=None, memtable_throughput_in_mb=None,
memtable_operations_in_millions=None, replicate_on_write=None,
merge_shards_chance=None, key_validation_class=None,
row_cache_provider=thrift_spec[27][4],):
+ def __init__(self, keyspace=None, name=None, column_type=thrift_spec[3][4],
comparator_type=thrift_spec[5][4], subcomparator_type=None, comment=None,
row_cache_size=thrift_spec[9][4], key_cache_size=thrift_spec[11][4],
read_repair_chance=thrift_spec[12][4], column_metadata=None,
gc_grace_seconds=None, default_validation_class=None, id=None,
min_compaction_threshold=None, max_compaction_threshold=None,
row_cache_save_period_in_seconds=None, key_cache_save_period_in_seconds=None,
memtable_flush_after_mins=None, memtable_throughput_in_mb=None,
memtable_operations_in_millions=None, replicate_on_write=None,
merge_shards_chance=None, key_validation_class=None,
row_cache_provider=thrift_spec[27][4], key_alias=None,):
self.keyspace = keyspace
self.name = name
self.column_type = column_type
@@ -2382,6 +2384,7 @@ class CfDef:
self.merge_shards_chance = merge_shards_chance
self.key_validation_class = key_validation_class
self.row_cache_provider = row_cache_provider
+self.key_alias = key_alias
def read(self, iprot):
if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is
not None and fastbinary is not None:
@@ -2518,6 +2521,11 @@ class CfDef:
self.row_cache_provider = iprot.readString();
else:
iprot.skip(ftype)
+ elif fid == 28:
+if ftype == TType.STRING:
+ self.key_alias = iprot.readString();
+else:
+ iprot.skip(ftype)
else:
iprot.skip(ftype)
iprot.readFieldEnd()
@@ -2627,6 +2635,10 @@ class CfDef:
oprot.writeFieldBegin('row_cache_provider', TType.STRING, 27)
oprot.writeString(self.row_cache_provider)
oprot.writeFieldEnd()
+if self.key_alias != None:
+ oprot.writeFieldBegin('key_alias', TType.STRING, 28)
+ oprot.writeString(self.key_alias)
+ oprot.writeFieldEnd()
oprot.writeFieldStop()
oprot.writeStructEnd()
def validate(self):
svn commit: r1091087 [3/3] - in /cassandra/branches/cassandra-0.8/drivers: py/cql/cassandra/constants.py py/cql/cassandra/ttypes.py txpy/txcql/cassandra/Cassandra.py txpy/txcql/cassandra/constants.py
Modified:
cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/constants.py
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/constants.py?rev=1091087r1=1091086r2=1091087view=diff
==
--- cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/constants.py
(original)
+++ cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/constants.py
Mon Apr 11 14:23:30 2011
@@ -7,4 +7,4 @@
from thrift.Thrift import *
from ttypes import *
-VERSION = 20.0.0
+VERSION = 20.1.0
Modified:
cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/ttypes.py
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/ttypes.py?rev=1091087r1=1091086r2=1091087view=diff
==
--- cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/ttypes.py
(original)
+++ cassandra/branches/cassandra-0.8/drivers/txpy/txcql/cassandra/ttypes.py Mon
Apr 11 14:23:30 2011
@@ -2324,6 +2324,7 @@ class CfDef:
- merge_shards_chance
- key_validation_class
- row_cache_provider
+ - key_alias
thrift_spec = (
@@ -2355,9 +2356,10 @@ class CfDef:
(25, TType.DOUBLE, 'merge_shards_chance', None, None, ), # 25
(26, TType.STRING, 'key_validation_class', None, None, ), # 26
(27, TType.STRING, 'row_cache_provider', None,
org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider, ), # 27
+(28, TType.STRING, 'key_alias', None, None, ), # 28
)
- def __init__(self, keyspace=None, name=None, column_type=thrift_spec[3][4],
comparator_type=thrift_spec[5][4], subcomparator_type=None, comment=None,
row_cache_size=thrift_spec[9][4], key_cache_size=thrift_spec[11][4],
read_repair_chance=thrift_spec[12][4], column_metadata=None,
gc_grace_seconds=None, default_validation_class=None, id=None,
min_compaction_threshold=None, max_compaction_threshold=None,
row_cache_save_period_in_seconds=None, key_cache_save_period_in_seconds=None,
memtable_flush_after_mins=None, memtable_throughput_in_mb=None,
memtable_operations_in_millions=None, replicate_on_write=None,
merge_shards_chance=None, key_validation_class=None,
row_cache_provider=thrift_spec[27][4],):
+ def __init__(self, keyspace=None, name=None, column_type=thrift_spec[3][4],
comparator_type=thrift_spec[5][4], subcomparator_type=None, comment=None,
row_cache_size=thrift_spec[9][4], key_cache_size=thrift_spec[11][4],
read_repair_chance=thrift_spec[12][4], column_metadata=None,
gc_grace_seconds=None, default_validation_class=None, id=None,
min_compaction_threshold=None, max_compaction_threshold=None,
row_cache_save_period_in_seconds=None, key_cache_save_period_in_seconds=None,
memtable_flush_after_mins=None, memtable_throughput_in_mb=None,
memtable_operations_in_millions=None, replicate_on_write=None,
merge_shards_chance=None, key_validation_class=None,
row_cache_provider=thrift_spec[27][4], key_alias=None,):
self.keyspace = keyspace
self.name = name
self.column_type = column_type
@@ -2382,6 +2384,7 @@ class CfDef:
self.merge_shards_chance = merge_shards_chance
self.key_validation_class = key_validation_class
self.row_cache_provider = row_cache_provider
+self.key_alias = key_alias
def read(self, iprot):
if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is
not None and fastbinary is not None:
@@ -2518,6 +2521,11 @@ class CfDef:
self.row_cache_provider = iprot.readString();
else:
iprot.skip(ftype)
+ elif fid == 28:
+if ftype == TType.STRING:
+ self.key_alias = iprot.readString();
+else:
+ iprot.skip(ftype)
else:
iprot.skip(ftype)
iprot.readFieldEnd()
@@ -2627,6 +2635,10 @@ class CfDef:
oprot.writeFieldBegin('row_cache_provider', TType.STRING, 27)
oprot.writeString(self.row_cache_provider)
oprot.writeFieldEnd()
+if self.key_alias != None:
+ oprot.writeFieldBegin('key_alias', TType.STRING, 28)
+ oprot.writeString(self.key_alias)
+ oprot.writeFieldEnd()
oprot.writeFieldStop()
oprot.writeStructEnd()
def validate(self):
[Cassandra Wiki] Update of MultinodeCluster by MakiWatanabe
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The MultinodeCluster page has been changed by MakiWatanabe.
The comment on this change is: update for 0.7 format.
http://wiki.apache.org/cassandra/MultinodeCluster?action=diffrev1=5rev2=6
--
+ Prior to the 0.7 release, Cassandra storage configuration is described by the
''conf/storage-conf.xml'' file. As of 0.7, it is described by the
''conf/cassandra.yaml'' file. Please refer to MultinodeCluster06 for about
pre-0.7 configuration.
+
= Creating a multinode cluster =
- The default storage-conf.xml provided with cassandra is great for getting up
and running on a single node. However, it is inappropriate for use in a
multi-node cluster. The configuration and process here are the ''simplest''
way to create a multi-node cluster, but may not be the ''best'' way in
production deployments.
+ The default cassandra.yaml provided with cassandra is great for getting up
and running on a single node. However, it is inappropriate for use in a
multi-node cluster. The configuration and process here are the ''simplest''
way to create a multi-node cluster, but may not be the ''best'' way in
production deployments.
== Preparing the first node ==
- The default storage-conf.xml uses the local, loopback address as its listen
(inter-node) and Thrift (client access) addresses:
+ The default cassandra.yaml uses the local, loopback address as its listen
(inter-node) and Thrift (client access) addresses:
{{{
- ListenAddresslocalhost/ListenAddress
+ listen_address: localhost
- ThriftAddresslocalhost/ThriftAddress
+ rpc_address: localhost
}}}
As the listen address is used for intra-cluster communication, it must be
changed to a routable address so the other nodes can reach it. For example,
assuming you have an Ethernet interface with address 192.168.1.1, you would
change the listen address like so:
{{{
- ListenAddress192.168.1.1/ListenAddress
+ listen_address: 192.168.1.1
}}}
The Thrift interface can be configured using either a specified address, like
the listen address, or using the wildcard 0.0.0.0, which causes cassandra to
listen for clients on all available interfaces. Update it as either:
{{{
- ThriftAddress192.168.1.1/ThriftAddress
+ rpc_address: 192.168.1.1
}}}
Or:
{{{
- ThriftAddress0.0.0.0/ThriftAddress
+ rpc_address: 0.0.0.0
}}}
If the DNS entry for your host is correct, it is safe to use a hostname
instead of an IP address. Similarly, the seed information should be changed
from the loopback address:
{{{
- Seeds
- Seed127.0.0.1/Seed
- /Seeds
+ seeds:
+ - 127.0.0.1
+
}}}
Becomes:
{{{
- Seeds
- Seed192.168.1.1/Seed
- /Seeds
+ seeds:
+ - 192.168.1.1
+
}}}
Once these changes are made, simply restart cassandra on this node. Use
netstat to verify cassandra is listening on the right address. Look for a line
like this:
{{{tcp4 0 0 192.168.1.1.7000 *.*
LISTEN}}}
- If netstat still shows cassandra listening on 127.0.0.1.7000, then either the
previous cassandra process was not properly killed or you are not editing the
storage-conf.xml file cassandra is actually using.
+ If netstat still shows cassandra listening on 127.0.0.1.7000, then either the
previous cassandra process was not properly killed or you are not editing the
cassandra.yaml file cassandra is actually using.
== Preparing the rest of the nodes ==
- The other nodes in the ring will use a storage-conf.xml almost identical to
the one on your first node, so use that configuration as the base for these
changes rather than the default storage-conf.xml. The first change is to turn
on automatic bootstrapping. This will cause the node to join the ring and
attempt to take control of a range of the token space:
+ The other nodes in the ring will use a cassandra.yaml almost identical to the
one on your first node, so use that configuration as the base for these changes
rather than the default cassandra.yaml. The first change is to turn on
automatic bootstrapping. This will cause the node to join the ring and attempt
to take control of a range of the token space:
{{{
- AutoBootstraptrue/AutoBootstrap
+ auto_bootstrap: true
}}}
The second change is to the listen address, as it must also not be the
loopback and cannot be the same as any other node. Assuming your second node
has an Ethernet interface with the address 192.168.2.34, set its listen address
with:
{{{
- ListenAddress192.168.2.34/ListenAddress
+ listen_address: 192.168.2.34
}}}
Finally, update the the Thrift address to accept client connections, as with
the first node, either with a specific address or the wildcard:
{{{
- ThriftAddress192.168.2.34/ThriftAddress
+ rpc_address: 192.168.2.34
}}}
Or:
{{{
-
[Cassandra Wiki] Update of MultinodeCluster06_JP by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The MultinodeCluster06_JP page has been changed by MakiWatanabe. http://wiki.apache.org/cassandra/MultinodeCluster06_JP?action=diffrev1=12rev2=13 -- ## page was copied from MultinodeCluster_JP ## page was copied from MultinodeCluster - '''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくはStorageConfigurationを参照してください。''' + '''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくはStorageConfiguration_JPを参照してください。''' = マルチノードクラスタの作成 =
svn commit: r1091090 - in /cassandra/branches/cassandra-0.8: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/
Author: jbellis Date: Mon Apr 11 14:32:45 2011 New Revision: 1091090 URL: http://svn.apache.org/viewvc?rev=1091090view=rev Log: merge from 0.7 Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 14:32:45 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7:1026516-1090647 +/cassandra/branches/cassandra-0.7:1026516-1091087 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1091090r1=1091089r2=1091090view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 14:32:45 2011 @@ -45,7 +45,9 @@ index (CASSANDRA-2376) * fix race condition that could leave orphaned data files when dropping CF or KS (CASSANDRA-2381) + * convert mmap assertion to if/throw so scrub can catch it (CASSANDRA-2417) * Try harder to close files after compaction (CASSANDRA-2431) + * re-set bootstrapped flag after move finishes (CASSANDRA-2435) 0.7.4 Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 14:32:45 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1090647 +/cassandra/branches/cassandra-0.7/contrib:1026516-1091087 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 14:32:45 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1090647 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1091087 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 14:32:45 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1090647 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1091087 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
[Cassandra Wiki] Update of MultinodeCluster06_JP by MakiWatanabe
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The MultinodeCluster06_JP page has been changed by MakiWatanabe.
http://wiki.apache.org/cassandra/MultinodeCluster06_JP?action=diffrev1=13rev2=14
--
## page was copied from MultinodeCluster_JP
## page was copied from MultinodeCluster
-
'''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくはStorageConfiguration_JPを参照してください。'''
+
'''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくは[[StorageConfiguration|StorageConfiguration_JP]]を参照してください。'''
= マルチノードクラスタの作成 =
@@ -14, +14 @@
標準のstorage-conf.xmlはloopbackアドレスをlistenアドレス（ノード間通信用）及びThriftアドレス（クライアントアクセス用）に使用しています：
- 0.7より前のバージョン
{{{
ListenAddresslocalhost/ListenAddress
ThriftAddresslocalhost/ThriftAddress
}}}
- 0.7
- {{{
- listen_address: localhost
- rpc_address: localhost
- }}}
listenアドレスはノード間通信に使用されるので、他のノードからアクセス可能なアドレスに変更する必要があります。
例えば、そのノードが192.168.1.1のEthernetインターフェースを持っている場合、listenアドレスを次のように変更すればいいでしょう：
- 0.7より前のバージョン
{{{
ListenAddress192.168.1.1/ListenAddress
- }}}
-
- 0.7
- {{{
- listen_address: 192.168.1.1
}}}
Thriftインターフェースには特定のIPアドレス、あるいはワイルドカードアドレス0.0.0.0を指定できます。ワイルドカードアドレスを指定すると、cassandraは使用可能なすべてのインターフェースでクライアントからの要求を受け付けます。Thrfitアドレスを次のように指定して下さい：
- 0.7より前のバージョン
{{{
ThriftAddress192.168.1.1/ThriftAddress
}}}
- 0.7
- {{{
- rpc_address: 192.168.1.1
- }}}
-
あるいは：
- 0.7より前のバージョン
{{{
ThriftAddress0.0.0.0/ThriftAddress
}}}
- 0.7
- {{{
- rpc_address: 0.0.0.0
- }}}
-
そのホストのDNSエントリが正しければ、IPアドレスよりホスト名を使った方が安全です。同様に、seed情報もloopbackアドレスから変更する必要があります：
- 0.7より前のバージョン
{{{
Seeds
Seed127.0.0.1/Seed
/Seeds
}}}
- 0.7
- {{{
- seeds:
- - 127.0.0.1
- }}}
-
これを次のように変更:
- 0.7より前のバージョン
{{{
Seeds
Seed192.168.1.1/Seed
/Seeds
}}}
- 0.7
- {{{
- seeds:
- - 192.168.1.1
- }}}
これらの変更が済んだら、cassandraをリスタートしてください。netstatを使用してcassandraが正しいアドレスをlistenしていることを確認してください。
設定が正しければ次のように表示されるはです：
@@ -105, +68 @@
Ring内の他のノードでは最初のノードで設定したstorage-conf.xmlとほぼ同一のものを使用します。従って最初のノードで編集したstorage-conf.xmlをベースに変更を加えていくことにしましょう。最初の変更は、自動ブートストラップを有効にすることです。この設定により、ノードはRingに参加し、トークン空間における一定範囲を担当範囲にすることを試みます：
- 0.7より前のバージョン
{{{
AutoBootstraptrue/AutoBootstrap
}}}
- 0.7
- {{{
- auto_bootstrap: true
- }}}
-
二つ目の変更はlistenアドレスです。listenアドレスはloopbackアドレスでもなく、また他のノードと重複してもいけません。二つ目のノードが192.168.2.34のEthernetインターフェースを持っている場合、listenアドレスを次のように設定します：
- 0.7より前のバージョン
{{{
ListenAddress192.168.2.34/ListenAddress
}}}
- 0.7
- {{{
- listen_address: 192.168.2.34
- }}}
-
最後に、Thriftアドレスを変更し、クライアントアクセスを受け付け可能にします。最初のノードと同様に、特定のアドレス、もしくはワイルドカードを指定します：
- 0.7より前のバージョン
{{{
ThriftAddress192.168.2.34/ThriftAddress
}}}
- 0.7
- {{{
- rpc_address: 192.168.2.34
- }}}
-
または：
- 0.7より前のバージョン
{{{
ThriftAddress0.0.0.0/ThriftAddress
}}}
- 0.7
- {{{
- rpc_address: 0.0.0.0
- }}}
設定のSeedsセクションをそのまま残さなければいけないことに注意してください。この設定は追加するノードがブートストラップで最初のノードを参照するために必要です。これらの設定変更が済んだら、新しいノードでcassandraを起動してください。新しいノードは自動的にRingに参加し、初期トークンを自分自身に割り当て、要求待ちになります。
[Cassandra Wiki] Trivial Update of MultinodeCluster06_JP by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The MultinodeCluster06_JP page has been changed by MakiWatanabe. http://wiki.apache.org/cassandra/MultinodeCluster06_JP?action=diffrev1=14rev2=15 -- ## page was copied from MultinodeCluster_JP ## page was copied from MultinodeCluster - '''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくは[[StorageConfiguration|StorageConfiguration_JP]]を参照してください。''' + '''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくは[[StorageConfiguration_JP|StorageConfiguration]]を参照してください。''' = マルチノードクラスタの作成 =
[Cassandra Wiki] Update of MultinodeCluster_JP by MakiWatanabe
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The MultinodeCluster_JP page has been changed by MakiWatanabe.
The comment on this change is: update for 0.7 format.
http://wiki.apache.org/cassandra/MultinodeCluster_JP?action=diffrev1=11rev2=12
--
## page was copied from MultinodeCluster
-
'''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。詳しくはStorageConfigurationを参照してください。'''
+
'''0.7より前のバージョンではストレージ設定はconf/storage-conf.xmlに記述されていましたが、0.7ではconf/cassandra.yamlに記述されます。
+ 0.6以前のバージョンの設定方法については[[MultinodeCluster06_JP|MultinodeCluster06]]を参照してください。
+ パラメータの詳細については[[StorageConfiguration_JP]]を参照してください。'''
= マルチノードクラスタの作成 =
-
Cassandraパッケージに含まれている標準のstorage-conf.xmlはシングルノード環境を構築するには便利ですが、マルチノードクラスタの構築には適当ではありません。ここではマルチノードクラスタを構築するための最も簡単な手順と設定を説明します。ただし、ここで述べる方法は本番環境の構築においては必ずしも最善ではないでしょう。
+
Cassandraパッケージに含まれている標準のcassandra.yamlはシングルノード環境を構築するには便利ですが、マルチノードクラスタの構築には適当ではありません。ここではマルチノードクラスタを構築するための最も簡単な手順と設定を説明します。ただし、ここで述べる方法は本番環境の構築においては必ずしも最善ではないでしょう。
== 最初のノードの準備 ==
-
標準のstorage-conf.xmlはloopbackアドレスをlistenアドレス（ノード間通信用）及びThriftアドレス（クライアントアクセス用）に使用しています：
+
標準のcassandra.yamlはloopbackアドレスをlistenアドレス（ノード間通信用）及びThriftアドレス（クライアントアクセス用）に使用しています：
- 0.7より前のバージョン
- {{{
- ListenAddresslocalhost/ListenAddress
- ThriftAddresslocalhost/ThriftAddress
- }}}
-
- 0.7
{{{
listen_address: localhost
rpc_address: localhost
@@ -28, +23 @@
listenアドレスはノード間通信に使用されるので、他のノードからアクセス可能なアドレスに変更する必要があります。
例えば、そのノードが192.168.1.1のEthernetインターフェースを持っている場合、listenアドレスを次のように変更すればいいでしょう：
- 0.7より前のバージョン
- {{{
- ListenAddress192.168.1.1/ListenAddress
- }}}
-
- 0.7
{{{
listen_address: 192.168.1.1
}}}
@@ -41, +30 @@
Thriftインターフェースには特定のIPアドレス、あるいはワイルドカードアドレス0.0.0.0を指定できます。ワイルドカードアドレスを指定すると、cassandraは使用可能なすべてのインターフェースでクライアントからの要求を受け付けます。Thrfitアドレスを次のように指定して下さい：
- 0.7より前のバージョン
- {{{
- ThriftAddress192.168.1.1/ThriftAddress
- }}}
-
- 0.7
{{{
rpc_address: 192.168.1.1
}}}
あるいは：
- 0.7より前のバージョン
- {{{
- ThriftAddress0.0.0.0/ThriftAddress
- }}}
-
- 0.7
{{{
rpc_address: 0.0.0.0
}}}
そのホストのDNSエントリが正しければ、IPアドレスよりホスト名を使った方が安全です。同様に、seed情報もloopbackアドレスから変更する必要があります：
- 0.7より前のバージョン
- {{{
- Seeds
- Seed127.0.0.1/Seed
- /Seeds
- }}}
-
- 0.7
{{{
seeds:
- 127.0.0.1
@@ -80, +49 @@
これを次のように変更:
- 0.7より前のバージョン
- {{{
- Seeds
- Seed192.168.1.1/Seed
- /Seeds
- }}}
-
- 0.7
{{{
seeds:
- 192.168.1.1
@@ -98, +59 @@
{{{tcp4 0 0 192.168.1.1.7000 *.*
LISTEN}}}
-
cassandraが依然として127.0.0.1.7000をlistenしているなら、前のcassandraプロセスが正常にkillされなかったか、あなたが編集したstorage-conf.xmlをcassandraが参照していないかのいずれかでしょう。
+
cassandraが依然として127.0.0.1.7000をlistenしているなら、前のcassandraプロセスが正常にkillされなかったか、あなたが編集したcassandra.yamlをcassandraが参照していないかのいずれかでしょう。
== 残りのノードの準備 ==
-
Ring内の他のノードでは最初のノードで設定したstorage-conf.xmlとほぼ同一のものを使用します。従って最初のノードで編集したstorage-conf.xmlをベースに変更を加えていくことにしましょう。最初の変更は、自動ブートストラップを有効にすることです。この設定により、ノードはRingに参加し、トークン空間における一定範囲を担当範囲にすることを試みます：
+
Ring内の他のノードでは最初のノードで設定したcassandra.yamlとほぼ同一のものを使用します。従って最初のノードで編集したcassandra.yamlをベースに変更を加えていくことにしましょう。最初の変更は、自動ブートストラップを有効にすることです。この設定により、ノードはRingに参加し、トークン空間における一定範囲を担当範囲にすることを試みます：
- 0.7より前のバージョン
- {{{
- AutoBootstraptrue/AutoBootstrap
- }}}
-
- 0.7
{{{
auto_bootstrap: true
}}}
二つ目の変更はlistenアドレスです。listenアドレスはloopbackアドレスでもなく、また他のノードと重複してもいけません。二つ目のノードが192.168.2.34のEthernetインターフェースを持っている場合、listenアドレスを次のように設定します：
- 0.7より前のバージョン
- {{{
- ListenAddress192.168.2.34/ListenAddress
- }}}
-
- 0.7
{{{
listen_address: 192.168.2.34
}}}
最後に、Thriftアドレスを変更し、クライアントアクセスを受け付け可能にします。最初のノードと同様に、特定のアドレス、もしくはワイルドカードを指定します：
- 0.7より前のバージョン
- {{{
- ThriftAddress192.168.2.34/ThriftAddress
- }}}
-
- 0.7
{{{
rpc_address: 192.168.2.34
}}}
または：
- 0.7より前のバージョン
- {{{
- ThriftAddress0.0.0.0/ThriftAddress
- }}}
-
- 0.7
{{{
rpc_address: 0.0.0.0
}}}
[jira] [Updated] (CASSANDRA-2342) Add range slice support for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2342: Attachment: 0001-Range-slice-support-for-counters.patch CASSANDRA-2440 almost fixed that, this patch really only remove the check that rejects range_slice query on counter CF (and fix the cli and add a system test). Add range slice support for counters Key: CASSANDRA-2342 URL: https://issues.apache.org/jira/browse/CASSANDRA-2342 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8 Attachments: 0001-Range-slice-support-for-counters.patch There is no equivalent for get_range_slice() for counters right now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of Operations_JP by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The Operations_JP page has been changed by MakiWatanabe. The comment on this change is: typo. http://wiki.apache.org/cassandra/Operations_JP?action=diffrev1=105rev2=106 -- * !RackUnawareStrategy: レプリカはトークンを昇順で並べた場合の「次のN-1個のノード」に配置されます。 - * !RackAwareStrategy: 二つめのレプリカはリングを順次辿る上でで最初に見つかる「別のデータセンターに設置されているノード」に配置されます。残りのN-2個のレプリカはできる限り一つめのレプリカが格納されたノードと同一のラックに設置されているノードに配置されます。 + * !RackAwareStrategy: 二つめのレプリカはリングを順次辿る上で最初に見つかる「別のデータセンターに設置されているノード」に配置されます。残りのN-2個のレプリカはできる限り一つめのレプリカが格納されたノードと同一のラックに設置されているノードに配置されます。 !RackAwareStrategyを使用する際はデータ配置の偏りを避けるため、リング上で隣接するノードは異なるデータセンターに配置するべきであることに注意してください。例えばノードA、B、C、Dからなるリングがあり、この順番でトークンが設定されているとします。ここでノードA、Bがデータセンター1.、ノードC、Dがデータセンター2に配置されている場合、ノードA、Cは常に「他のデータセンターで最初に見つかるノード」になります。このためノードA、CにはB、Dより多くのデータが蓄積されるでしょう。
[jira] [Created] (CASSANDRA-2447) Remove auto-bootstrap option
Remove auto-bootstrap option Key: CASSANDRA-2447 URL: https://issues.apache.org/jira/browse/CASSANDRA-2447 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Priority: Minor Fix For: 0.8 We already optimize auto-bootstrap to be no-op if there are no non-system tables. Given that, the only penalty imposed by autobootstrap is a 30s sleep waiting for gossip. Feels worth it to avoid the confusion this option causes, and the problems if you don't turn it on when it should be. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018379#comment-13018379 ] Sylvain Lebresne commented on CASSANDRA-2191: - Forgot the magical +1 (better late than never) Multithread across compaction buckets - Key: CASSANDRA-2191 URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Stu Hood Priority: Critical Labels: compaction Fix For: 0.8 Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt, 0004-Allow-multithread-compaction-to-be-disabled.txt, 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt, 0006-Prevent-cache-saves-from-occuring-concurrently.txt This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and reasoning are different enough to open a separate issue. The problem with compactions currently is that they compact the set of sstables that existed the moment the compaction started. This means that for longer running compactions (even when running as fast as possible on the hardware), a very large number of new sstables might be created in the meantime. We have observed this proliferation of sstables killing performance during major/high-bucketed compactions. One approach would be to pause compactions in upper buckets (containing larger files) when compactions in lower buckets become possible. While this would likely solve the problem with read performance, it does not actually help us perform compaction any faster, which is a reasonable requirement for other situations. Instead, we need to be able to perform any compactions that are currently required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of FAQ by jab_doa
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The FAQ page has been changed by jab_doa. The comment on this change is: big commit log. http://wiki.apache.org/cassandra/FAQ?action=diffrev1=112rev2=113 -- * [[#mmap|Why does top report that Cassandra is using a lot more memory than the Java heap max?]] * [[#jna|I'm getting java.io.IOException: Cannot run program ln when trying to snapshot or update a keyspace]] * [[#replicaplacement|How does Cassandra decide which nodes have what data?]] - * [[#cachehitrateunits| I have a row or key cache hit rate of 0.XX123456789. Is that XX% or 0.XX% ?]] + * [[#cachehitrateunits|I have a row or key cache hit rate of 0.XX123456789. Is that XX% or 0.XX% ?]] + * [[#bigcommitlog|Commit Log gets very big. Cassandra does not delete old commit logs. Why?]] Anchor(cant_listen_on_ip_any) @@ -408, +409 @@ Anchor(replicaplacement) == How does Cassandra decide which nodes have what data? == - The set of nodes (a single node, or several) responsible for any given piece of data is determined by: * The row key (data is partitioned on row key) @@ -422, +422 @@ Anchor(cachehitrateunits) == I have a row or key cache hit rate of 0.XX123456789 reported by JMX. Is that XX% or 0.XX% ? == - XX% + Anchor(bigcommitlog) + + == Commit Log gets very big. Cassandra does not delete old commit logs. Why? == + You probably got a few Column Families with very low throughput than others. They will not get flushed very frequently and the commit log doesnt get deleted. There is one option to work around the problem: You can set the per-cf-option memtable_flush_after_min. This will flush your CF at least every x minutes. 60 minutes i a good value to start. To set this via cassandra-cli use: + + update column family XXX with memtable_flush_after=60; +
[Cassandra Wiki] Update of FAQ by jab_doa
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The FAQ page has been changed by jab_doa. http://wiki.apache.org/cassandra/FAQ?action=diffrev1=113rev2=114 -- Anchor(bigcommitlog) == Commit Log gets very big. Cassandra does not delete old commit logs. Why? == - You probably got a few Column Families with very low throughput than others. They will not get flushed very frequently and the commit log doesnt get deleted. There is one option to work around the problem: You can set the per-cf-option memtable_flush_after_min. This will flush your CF at least every x minutes. 60 minutes i a good value to start. To set this via cassandra-cli use: + You probably got a few Column Families with very low throughput. They will not get flushed very frequently and the commit log doesnt get deleted. There is one option to work around the problem: You can set the per-cf-option memtable_flush_after_min. This will flush your CF at least every x minutes. 60 minutes i a good value to start. To set this via cassandra-cli use: update column family XXX with memtable_flush_after=60;
[jira] [Resolved] (CASSANDRA-1585) Support renaming columnfamilies and keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-1585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1585. --- Resolution: Later Support renaming columnfamilies and keyspaces - Key: CASSANDRA-1585 URL: https://issues.apache.org/jira/browse/CASSANDRA-1585 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Stu Hood Priority: Minor Renames were briefly supported but were race-prone. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-785) Improve load balancing when using rack aware or DC strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-785. -- Resolution: Won't Fix Fix Version/s: (was: 0.8) there is pretty general agreement now that node-at-a-time loadbalance is a dead end. Improve load balancing when using rack aware or DC strategy --- Key: CASSANDRA-785 URL: https://issues.apache.org/jira/browse/CASSANDRA-785 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.5 Reporter: Jaakko Laine Attachments: 785.patch Current load balancing functionality does not consider data centers. This may result in new nodes bootstrapping in wrong place if most loaded node is in another DC than the bootstrapping node. Explore possibilities to make load balance work better in multi-DC configuration without making load balancing complicated. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-974) SASL authentication support
[ https://issues.apache.org/jira/browse/CASSANDRA-974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-974. -- Resolution: Later Fix Version/s: (was: 0.8) Tabling this for now. If CQL is the future, and we write a new transport for it, we'll need to retarget for that. SASL authentication support --- Key: CASSANDRA-974 URL: https://issues.apache.org/jira/browse/CASSANDRA-974 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Looking at http://java.sun.com/j2se/1.5.0/docs/guide/security/sasl/sasl-refguide.html I am skeptical that we can make this work with Thrift. Someone please tell me how I am wrong. :) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of FrontPage_ZH by JavenWang
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The FrontPage_ZH page has been changed by JavenWang. The comment on this change is: Threshold 应该翻译成阈值, 据我所知中文里面没有阀值这个词.. http://wiki.apache.org/cassandra/FrontPage_ZH?action=diffrev1=8rev2=9 -- * [[GettingStarted_ZH| Cassandra起步]] * [[RunningCassandra|运行Cassandra]] - * [[MemtableThresholds|内存表阀值]] + * [[MemtableThresholds|内存表阈值]] * [[StorageConfiguration|存储配置]] * [[ArchitectureOverview|架构总览]] * [[FAQ|常见问题]]
[jira] [Resolved] (CASSANDRA-1825) Separation of Data (Cached/Non-Cached)
[ https://issues.apache.org/jira/browse/CASSANDRA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1825. --- Resolution: Later Fix Version/s: (was: 0.8) Separation of Data (Cached/Non-Cached) -- Key: CASSANDRA-1825 URL: https://issues.apache.org/jira/browse/CASSANDRA-1825 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet At the moment Cassandra goes through the ROW-READ stage to fetch data from the page cache, and if it's not in the page cache, it goes to disk. Data that is currently hot (in page cache) will block if all I/O threads are busy reading from disk. We should seriously look at implementing a buffer pool similar to MySQL for storing data in-memory, and our I/O threads be dedicated to just going to disk. I suggest studying how InnoDB does scheduling as well, they have good lessons to learn from. Scaling I/O by thread's isn't going to be a good solution here either. I would argue that going past 64 threads for I/O is just going to hurt overall performance based on context switching. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-1980) Add support for client-side encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1980. --- Resolution: Later Fix Version/s: (was: 0.8) as with CASSANDRA-974, let's table this until we see how the CQL transport evolves Add support for client-side encryption -- Key: CASSANDRA-1980 URL: https://issues.apache.org/jira/browse/CASSANDRA-1980 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jeremy Hanna With thrift-106 in place coming in thrift version 0.6, it will be necessary to add a way for making the server aware of that. We could allow the option in the server configuration (cassandra.yaml) to use encrypted or non-encrypted client connections. However if they choose encrypted, the only clients able to connect would be java clients at this point. It might be better to take some more time and look at how to enable accepting both encrypted and unencrypted connections at the same time. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1473) Implement a Cassandra aware Hadoop mapreduce.Partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018396#comment-13018396 ] Jeremy Hanna commented on CASSANDRA-1473: - This still seems relevant in order to optimize for output for hadoop related tasks. If people are seriously using a portion of their cluster for analytics, this seems like a good cost/benefit type of ticket so that their cluster isn't as saturated just because of inefficiencies in output. It would be nice to know about magnitude of the gain, but personally I think this is worthwhile. Implement a Cassandra aware Hadoop mapreduce.Partitioner Key: CASSANDRA-1473 URL: https://issues.apache.org/jira/browse/CASSANDRA-1473 Project: Cassandra Issue Type: Improvement Components: Hadoop Reporter: Stu Hood Fix For: 0.8 When using a IPartitioner that does not sort data in byte order (RandomPartitioner for example) with Cassandra's Hadoop integration, Hadoop is unaware of the output order of the data. We can make Hadoop aware of the proper order of the output data by implementing Hadoop's mapreduce.Partitioner interface: then Hadoop will handle sorting all of the data according to Cassandra's IPartitioner, and the writing clients will be able to connect to smaller numbers of Cassandra nodes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of HowToPublishToMavenCentral by StephenConnolly
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The HowToPublishToMavenCentral page has been changed by StephenConnolly.
http://wiki.apache.org/cassandra/HowToPublishToMavenCentral?action=diffrev1=7rev2=8
--
== Prerequisites ==
- 1. You need to have a GPG signature set-up. (Open issue: Is there a place in
Cassandra SVN for storing the official GPG keys of committers to allow third
parties to verify that the releases are officical? I would suggest somewhere
like http://svn.apache.org/repos/asf/cassandra/site/src/content/keys.html so
that it can be part of the Cassandra website)
+ 1. You need to have a GPG private key set-up and listed in the list of
offical keys: http://www.apache.org/dist/cassandra/KEYS
1. You have a Subversion 1.5+ client installed and on your shell's path. See
http://subversion.apache.org/. (Note: Ideally you would use a Subversion 1.6+
client but the minimum is 1.5+)
1. You have JDK 6 installed and on your shell's path.
1. If you receive an OutOfMemoryError during the build, make sure to have
set the environment variable ANT_OPTS=-Xmx512m
1. Follow the environment configuration steps outlined at:
[[http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env|Publishing
Maven Artifacts]].
+ 1. You need to have a minimal {{{~/.m2/settings.xml}}} file which at least
defines the following
+ {{{
+ settings
+ servers
+ server
+ idapache.snapshots.https/id
+ username!-- your apache ldap username --/username
+ password!-- your apache ldap password --/password
+ /server
+ server
+ idapache.releases.https/id
+ username!-- your apache ldap username --/username
+ password!-- your apache ldap password --/password
+ /server
+ /servers
+ profiles
+ profile
+ idapache-release/id
+ properties
+ gpg.keyname!-- enough of the key id to id it --/gpg.keyname
+ !-- either you feel comfortable with the passphrase on disk --
+ gpg.passphrase!-- your passphrase for your gpg key goes
here--/gpg.passphrase
+ !-- or you use an agent--
+ gpg.useagenttrue/gpg.useagent
+ /properties
+ /profile
+ /profiles
+ /settings
+ }}}
== Using repository.apache.org ==
[jira] [Resolved] (CASSANDRA-2103) expiring counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2103. --- Resolution: Won't Fix Fix Version/s: (was: 0.8) Assignee: (was: Ryan King) expiring counter columns Key: CASSANDRA-2103 URL: https://issues.apache.org/jira/browse/CASSANDRA-2103 Project: Cassandra Issue Type: New Feature Components: Core Affects Versions: 0.8 Reporter: Kelvin Kakugawa Attachments: 0001-CASSANDRA-2103-expiring-counters-logic-tests.patch add ttl functionality to counter columns. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of HowToPublishToMavenCentral by StephenConnolly
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The HowToPublishToMavenCentral page has been changed by StephenConnolly.
http://wiki.apache.org/cassandra/HowToPublishToMavenCentral?action=diffrev1=8rev2=9
--
{{{
settings
servers
+ !-- once you have got things working you can encrypt the passwords for
the servers --
server
idapache.snapshots.https/id
username!-- your apache ldap username --/username
@@ -33, +34 @@
idapache-release/id
properties
gpg.keyname!-- enough of the key id to id it --/gpg.keyname
+
+ !-- pick one of the following --
+
!-- either you feel comfortable with the passphrase on disk --
gpg.passphrase!-- your passphrase for your gpg key goes
here--/gpg.passphrase
+
!-- or you use an agent--
gpg.useagenttrue/gpg.useagent
+
/properties
/profile
/profiles
[jira] [Updated] (CASSANDRA-2220) It would be nice to be able to rollback to a specific schema version.
[ https://issues.apache.org/jira/browse/CASSANDRA-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2220: -- Priority: Minor (was: Major) Labels: ponies (was: ) It would be nice to be able to rollback to a specific schema version. - Key: CASSANDRA-2220 URL: https://issues.apache.org/jira/browse/CASSANDRA-2220 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Labels: ponies Fix For: 0.8 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-815) Generalize RackAwareStrategy to support writing to N of M datacenters
[ https://issues.apache.org/jira/browse/CASSANDRA-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-815. -- Resolution: Duplicate Fix Version/s: (was: 0.8) This feels like a ConsistencyLevel problem, not a Strategy one. (CASSANDRA-2338) Generalize RackAwareStrategy to support writing to N of M datacenters - Key: CASSANDRA-815 URL: https://issues.apache.org/jira/browse/CASSANDRA-815 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Priority: Minor As requested on the mailing list, and at PyCon. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-1037) Improve load balancing to take into account load in terms of operations
[ https://issues.apache.org/jira/browse/CASSANDRA-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1037. --- Resolution: Won't Fix Fix Version/s: (was: 0.8) node-at-a-time loadbalance feels like a dead end. Improve load balancing to take into account load in terms of operations --- Key: CASSANDRA-1037 URL: https://issues.apache.org/jira/browse/CASSANDRA-1037 Project: Cassandra Issue Type: Improvement Reporter: Jeremy Hanna Priority: Minor Currently in cassandra, the load balancing takes into account disk space. When using an order-preserving partitioner, there can be hot spots in the various ranges of tokens in terms of operations. We would like to propose improving the load balancing so that it takes that the number of operations into account. There are two places where this can be handled: 1. when the cluster decides on which nodes need to be balanced out. 2. how to balance an individual node - where to split For number 1, the number of operations that a node performed could be factored in to how important it is to balance that node. For number 2, we are already using a midpoint in the node when trying to load balance with respect to space. We would propose adding a weight to the midpoint to lean towards splitting so that the operational load could be better handled, not just space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2013) Add CL.TWO, CL.THREE; tweak CL documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018404#comment-13018404 ] Jonathan Ellis commented on CASSANDRA-2013: --- Planning to supercede these in CASSANDRA-2338 Add CL.TWO, CL.THREE; tweak CL documentation Key: CASSANDRA-2013 URL: https://issues.apache.org/jira/browse/CASSANDRA-2013 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Peter Schuller Priority: Minor Fix For: 0.7.4 Attachments: 2013-assert.txt, 2013.txt Attaching draft patch to add CL.TWO and CL.THREE. Motivation for adding is that having to select between either ONE or QUORUM is too narrow a choice for clusters with RF 3. In such a case, it makes particular sense to want to do writes at e.g. CL.TWO for durability purposes even though you are not looking to get strong consistency with QUORUM. CL.THREE is the same argument. TWO and THREE felt reasonable; there is no objective reason why stopping at THREE is the obvious choice. Technically one would want to specify an arbitrary number, but that is a much more significant change. Two open questions: (1) I adjusted the documentation of ConsistencyLevel to be more consistent and also to reflect what I believe to be reality (for example, as far as I can tell QUORUM doesn't send requests to all nodes as claimed in the .thrift file). I'm not terribly confident that I have not missed something though. (2) There is at least one unresolved issue, which is this assertion check WriteResponseHandler: assert 1 = blockFor blockFor = 2 * Table.open(table).getReplicationStrategy().getReplicationFactor() : String.format(invalid response count %d for replication factor %d, blockFor, Table.open(table).getReplicationStrategy().getReplicationFactor()); At THREE, this causes an assertion failure on keyspace with RF=1. I would, as a user, expect UnavailableException. However I am uncertain as to what to do about this assertion. I think this highlights one TWO/THREE are different from previously existing CL:s, in that they essentially hard-code replicate counts rather than expressing them in terms that can by definition be served by the cluster at any RF. Given that with THREE (and not TWO, but that is only due to the implementation detail that bootstrapping is involved) implies a replicate count that is independent of the replication factor, there is essentially a new failure mode. It is suddenly possible for a consistency level to be fundamentally incompatible with the RF. My gut reaction is to want UnavailableException still, and that the assertion check can essentially be removed (other than the = 1 part). If a different failure mode is desired, presumably it would not be an assertion failure (which should indicate a Cassandra bug). Maybe UnstisfiableConsistencyLevel? I propose just adjusting the assertion (which has no equivalent in ReadCallback btw); giving a friendlier error message in case of a CL/RF mismatch would be good, but doesn't feel worth introducing extra complexity to deal with it. 'ant test' passes. I have tested w/ py_stress with a three-node cluster and an RF=3 keyspace and with 1 and 2 nodes down, and get expected behavior (available or unavailable as a function of nodes that are up). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-1159) For contrib modules that use Java, have a consistent build mechanism
[ https://issues.apache.org/jira/browse/CASSANDRA-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1159. --- Resolution: Duplicate Fix Version/s: (was: 0.8) See CASSANDRA-1805 for refactoring contrib. For contrib modules that use Java, have a consistent build mechanism Key: CASSANDRA-1159 URL: https://issues.apache.org/jira/browse/CASSANDRA-1159 Project: Cassandra Issue Type: Improvement Components: Contrib Reporter: Jeremy Hanna Priority: Minor Contrib modules have a habit of periodically not working for some reason. To some extent that's expected - they are optional contrib modules. However, I think it's reasonable to at least have some way to perform a periodic sanity check on them if we can. This improvement would make sure there is a consistent build mechanism - build.xml - for each of the contrib modules that use Java. That way, there could be a hudson build perhaps nightly or weekly, that could inform the devs if the contrib modules are not even compiling. It's not like it would be a huge priority to fix immediately, but they would at least be aware that changes in the code/config have broken a contrib module. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-1283) Make the row cache pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1283. --- Resolution: Duplicate Fix Version/s: (was: 0.8) Done in CASSANDRA-1969 Make the row cache pluggable - Key: CASSANDRA-1283 URL: https://issues.apache.org/jira/browse/CASSANDRA-1283 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Daniel Kluesing Priority: Minor Attachments: trunk-pluggableCache.txt -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of 首页 by JavenWang
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The 首页 page has been changed by JavenWang.
The comment on this change is: should use 首页 as the FrontPage rather than
FrontPage_ZH.
http://wiki.apache.org/cassandra/%E9%A6%96%E9%A1%B5?action=diffrev1=1rev2=2
--
## Please edit system and help pages ONLY in the moinmaster wiki! For more
## information, please see MoinMaster:MoinPagesEditorGroup.
##master-page:FrontPage
- ##master-date:2004-11-21 15:15:01
#format wiki
#language zh
#pragma section-numbers off
+ = Cassandra Wiki =
- = 维基链接名 维基 =
+
Cassandra是一套高度可扩展、最终一致、分布式的结构化键值存储系统。Cassandra结合了[[http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf|Dynamo]]的分布技术和GOOGLE的[[http://labs.google.com/papers/bigtable-osdi06.pdf|BigTable]]数据模型。
Cassandra具有Dynamo的[[http://www.allthingsdistributed.com/2008/12/eventually_consistent.html|最终一致性]]；
同时Cassandra相比典型的键/值数据存储数据模型更为丰富，它提供与Google的Bigtable相似的、基于ColumnFamily的数据模型。
+ Cassandra最初由Avinash Lakshman (Amazon's Dynamo的作者之一) 和 Prashant Malik (
Facebook工程师)在Facebook设计开发，在2008年Facebook把它贡献给了开源社区。
在很多的地方你可以把Cassandra看成是Dynamo的升级版本2.0或者是Dynamo与!BigTable的结合。Cassandra在Facebook投入实际应用运行，但仍然处于大量开发进展阶段。
- 这个维基是有关什么的?
-
- 您也许可以从这些连接开始:
- * [最新改动]: 谁最近改动了什么
- * [维基沙盘演练]: 您可以随意改动编辑，热身演练
- * [查找网页]: 用多种方法搜索浏览这个站点
- * [语法参考]: 维基语法简便参考
- * [站点导航]: 本站点内容概要
- == 如何使用这个站点 ==
+ == 通用信息 ==
+ * [[http://cassandra.apache.org/|Cassandra官方站点]] 下载，问题追踪，邮件列表，等等
+ * [[ArticlesAndPresentations| Cassandra相关文章与演示 ]]
+ * [[DataModel|数据模型]]
+ * [[CassandraLimitations|Cassandra的局限性]]: 什么场合不适用Cassandra
- 维基(wiki)是一种协同合作网站，任何人都可以参与网站的建立、编辑和维护并分享网站的内容：
- * 点击每个网页页眉或页尾中的'''GetText(Edit)'''就可以随意编辑改动这个网页。
- *
创建一个链接简单的不能再简单了：您可以使用连在一起的，每个单词第一个字母大写，但不用空格分隔的词组(比如WikiSandBox)，也可以用{{{[quoted
words in brackets]}}}。简体中文的链接可以使用后者，比如{{{[维基沙盘演练]}}}。
- * 每页的页眉中的搜索框可以用来将进行网页标题搜索或者进行全文检索。
- * 新手可以参阅[帮助-新手入门]，如需要详尽的帮助，参阅[帮助目录]。
- 如需要更多有关[维基网]的信息，请参考[维基好坏说]和MoinMoin:WikiNature
(英文)。也请参考MoinMoin:WikiWikiWebFaq (英文)。
+ == 用户文档 ==
- 本维基使用[简体中文MoinMoin]系统。[简体中文MoinMoin]是MoinMoin的简体中文版本。
+ * [[GettingStarted_ZH| Cassandra起步]]
+ * [[RunningCassandra|运行Cassandra]]
+ * [[MemtableThresholds|内存表阈值]]
+ * [[StorageConfiguration|存储配置]]
+ * [[ArchitectureOverview|架构总览]]
+ * [[FAQ|常见问题]]
+ * [[API|Thrift API文档]] （完善中）
+ * [[Operations | Cassandra的维护]]
+ * [[Embedding | 嵌入式Cassandra ]]
+ * [[CassandraHardware_ZH|Cassandra硬件]]
- 此页的英文版本：FrontPage
+ == 开发者文档 ==
+ * [[ArchitectureInternals_ZH | 内部构架]]
+ * [[CLI Design | 客户端应用设计]]
+ * [[HowToContribute|怎么为Cassandra做自己的贡献？]]
+ * [[HowToCommit|补丁提交流程]]
+
+ == 邮件列表 ==
+ * 用户: u...@cassandra.apache.org
[[mailto:user-subscr...@cassandra.apache.org|(subscribe)]]
[[http://mail-archives.apache.org/mod_mbox/incubator-cassandra-user/|(archives)]]
+ * 开发者: d...@cassandra.apache.org
[[mailto:dev-subscr...@cassandra.apache.org|(subscribe)]]
[[http://mail-archives.apache.org/mod_mbox/incubator-cassandra-dev/|(archives)]]
+ * 代码提交者: commits@cassandra.apache.org
[[mailto:commits-subscr...@cassandra.apache.org|(subscribe)]]
+
+ == 相关信息 ==
+
+ * [[http://incubator.apache.org/thrift|Cassandra客户端连接依赖的Thrift项目]]
+
+ == Google SoC 2010 Page ==
+ * [[GoogleSoc2010|Google SoC]]
+
+ This wiki is powered by MoinMoin. With the exception of a few immutable
pages, anyone
+ can edit it. Try SyntaxReference if you need help on wiki markup, and
+ FindPage or SiteNavigation to search for existing pages before creating a
+ new one. If you aren't sure where to begin, checkout RecentChanges to see
+ what others have been working on, or RandomPage if you are feeling lucky.
+
+ == 其他语言 ==
+ * [[FrontPage|English 英文版]]
+ * [[FrontPage_JP|Japanese 日文版]]
+
[jira] [Commented] (CASSANDRA-2371) Removed/Dead Node keeps reappearing
[ https://issues.apache.org/jira/browse/CASSANDRA-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018412#comment-13018412 ] Jonathan Ellis commented on CASSANDRA-2371: --- bq. The problem with adding it, however, is that SS.handleStateLeft gets called every gossip round and runs through the hint removal process But it only runs hint removal once per node, right? Or is onChange not an accurate method name? bq. One option may be to check if the node is locally persisted and if not, just ignore the message since we never knew about it anyway. Locally persisted... with a token in SystemTable? Ignore the ... hint message? bq. Another is to just not remove hints when we see the LEFT state We should issue a delete to the hints row, but we should not force a major compaction. The rule of thumb is, avoiding inflicting a performance hit on the cluster trumps immediate disk space cleanup. Removed/Dead Node keeps reappearing --- Key: CASSANDRA-2371 URL: https://issues.apache.org/jira/browse/CASSANDRA-2371 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.7.3, 0.7.4 Environment: Large Amazon EC2 instances. Ubuntu 10.04.2 Reporter: techlabs Assignee: Brandon Williams Priority: Minor Fix For: 0.7.5 Attachments: 2371.txt The removetoken option does not seem to work. The original node 10.240.50.63 comes back into the ring, even after the EC2 instance is no longer in existence. Originally I tried to add a new node 10.214.103.224 with the same token, but there were some complications with that. I have pasted below all the INFO log entries found with greping the system log files. Seems to be a similar issue seen with http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Ghost-node-showing-up-in-the-ring-td6198180.html INFO [GossipStage:1] 2011-03-16 00:54:31,590 StorageService.java (line 745) Nodes /10.214.103.224 and /10.240.50.63 have the same token 957044156965139000. /10.214.103.224 is the new owner INFO [GossipStage:1] 2011-03-16 17:26:51,083 StorageService.java (line 865) Removing token 957044156965139000 for /10.214.103.224 INFO [GossipStage:1] 2011-03-19 17:27:24,767 StorageService.java (line 865) Removing token 957044156965139000 for /10.214.103.224 INFO [GossipStage:1] 2011-03-19 17:29:30,191 StorageService.java (line 865) Removing token 957044156965139000 for /10.214.103.224 INFO [GossipStage:1] 2011-03-19 17:31:35,609 StorageService.java (line 865) Removing token 957044156965139000 for /10.214.103.224 INFO [GossipStage:1] 2011-03-19 17:33:39,440 StorageService.java (line 865) Removing token 957044156965139000 for /10.214.103.224 INFO [GossipStage:1] 2011-03-23 17:22:55,520 StorageService.java (line 865) Removing token 957044156965139000 for /10.240.50.63 INFO [GossipStage:1] 2011-03-10 03:52:37,299 Gossiper.java (line 608) Node /10.240.50.63 is now part of the cluster INFO [GossipStage:1] 2011-03-10 03:52:37,545 Gossiper.java (line 600) InetAddress /10.240.50.63 is now UP INFO [HintedHandoff:1] 2011-03-10 03:53:36,168 HintedHandOffManager.java (line 304) Started hinted handoff for endpoint /10.240.50.63 INFO [HintedHandoff:1] 2011-03-10 03:53:36,169 HintedHandOffManager.java (line 360) Finished hinted handoff of 0 rows to endpoint /10.240.50.63 INFO [GossipStage:1] 2011-03-15 23:23:43,770 Gossiper.java (line 623) Node /10.240.50.63 has restarted, now UP again INFO [GossipStage:1] 2011-03-15 23:23:43,771 StorageService.java (line 726) Node /10.240.50.63 state jump to normal INFO [HintedHandoff:1] 2011-03-15 23:28:48,957 HintedHandOffManager.java (line 304) Started hinted handoff for endpoint /10.240.50.63 INFO [HintedHandoff:1] 2011-03-15 23:28:48,958 HintedHandOffManager.java (line 360) Finished hinted handoff of 0 rows to endpoint /10.240.50.63 INFO [ScheduledTasks:1] 2011-03-15 23:37:25,071 Gossiper.java (line 226) InetAddress /10.240.50.63 is now dead. INFO [GossipStage:1] 2011-03-16 00:54:31,590 StorageService.java (line 745) Nodes /10.214.103.224 and /10.240.50.63 have the same token 957044156965139000. /10.214.103.224 is the new owner WARN [GossipStage:1] 2011-03-16 00:54:31,590 TokenMetadata.java (line 115) Token 957044156965139000 changing ownership from /10.240.50.63 to /10.214.103.224 INFO [GossipStage:1] 2011-03-18 23:37:09,158 Gossiper.java (line 610) Node /10.240.50.63 is now part of the cluster INFO [GossipStage:1] 2011-03-21 23:37:10,421 Gossiper.java (line 610) Node /10.240.50.63 is now part of the cluster
[Cassandra Wiki] Update of FrontPage by JavenWang
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The FrontPage page has been changed by JavenWang. The comment on this change is: should use 首页 as the FrontPage rather than FrontPage_ZH. http://wiki.apache.org/cassandra/FrontPage?action=diffrev1=59rev2=60 -- This wiki is powered by MoinMoin. With the exception of a few immutable pages, anyone can edit it. Try SyntaxReference if you need help on wiki markup, and FindPage or SiteNavigation to search for existing pages before creating a new one. If you aren't sure where to begin, checkout RecentChanges to see what others have been working on, or RandomPage if you are feeling lucky. == Other Languages == - * [[FrontPage_ZH|SimpleChinese 简体中文]] + * [[首页|SimpleChinese 简体中文]] * [[FrontPage_JP|Japanese 日本語]]
svn commit: r1091113 - in /cassandra/branches/cassandra-0.8: ./ conf/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandr
Author: slebresne Date: Mon Apr 11 16:28:38 2011 New Revision: 1091113 URL: http://svn.apache.org/viewvc?rev=1091113view=rev Log: Merge CASSANDRA-2156 from trunk Modified: cassandra/branches/cassandra-0.8/ (props changed) cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/conf/cassandra.yaml cassandra/branches/cassandra-0.8/contrib/ (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/CompactionManager.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/io/CompactionIterator.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeProbe.java Propchange: cassandra/branches/cassandra-0.8/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 16:28:38 2011 @@ -2,7 +2,7 @@ /cassandra/branches/cassandra-0.7:1026516-1091087 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 -/cassandra/trunk:1090978 +/cassandra/trunk:1090978-1090979 /incubator/cassandra/branches/cassandra-0.3:774578-796573 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 /incubator/cassandra/branches/cassandra-0.5:72-915439 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1091113r1=1091112r2=1091113view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 16:28:38 2011 @@ -20,7 +20,7 @@ * give snapshots the same name on each node (CASSANDRA-1791) * add key type information and alias (CASSANDRA-2311, 2396) * multithreaded compaction (CASSANDRA-2191) - + * compaction throttling (CASSANDRA-2156) 0.7.5 * Avoid seeking when sstable2json exports the entire file (CASSANDRA-2318) Modified: cassandra/branches/cassandra-0.8/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra.yaml?rev=1091113r1=1091112r2=1091113view=diff == --- cassandra/branches/cassandra-0.8/conf/cassandra.yaml (original) +++ cassandra/branches/cassandra-0.8/conf/cassandra.yaml Mon Apr 11 16:28:38 2011 @@ -250,9 +250,17 @@ column_index_size_in_kb: 64 in_memory_compaction_limit_in_mb: 64 # Enables multiple compactions to execute at once. This is highly recommended -# for preserving read performance in a mixed read/write workload. +# for preserving read performance in a mixed read/write workload as this +# avoids sstables from accumulating during long running compactions. compaction_multithreading: true +# Throttles compaction to the given total throughput across the entire +# system. The faster you insert data, the faster you need to compact in +# order to keep the sstable count down, but in general, setting this to +# 16 to 32 times the rate you are inserting data is more than sufficient. +# Setting this to 0 disables throttling. +compaction_throughput_mb_per_sec: 16 + # Track cached row keys during compaction, and re-cache their new # positions in the compacted sstable. Disable if you use really large # key caches. Propchange: cassandra/branches/cassandra-0.8/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 16:28:38 2011 @@ -2,7 +2,7 @@ /cassandra/branches/cassandra-0.7/contrib:1026516-1091087 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 -/cassandra/trunk/contrib:1090978 +/cassandra/trunk/contrib:1090978-1090979
[jira] [Updated] (CASSANDRA-2448) Remove loadbalance command
[ https://issues.apache.org/jira/browse/CASSANDRA-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey updated CASSANDRA-2448: --- Attachment: 0001-Remove-loadbalance-command-from-nodetool.patch Remove loadbalance command -- Key: CASSANDRA-2448 URL: https://issues.apache.org/jira/browse/CASSANDRA-2448 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 0.8 Attachments: 0001-Remove-loadbalance-command-from-nodetool.patch With the update to how the move command works, the loadbalance command is even less useful that it was previously. The loadbalance command now calculates the token it is going to move to before it leaves which means it isn't considering the load it is giving away. Given that, I think we should just remove the loadbalance command entirely. Anyone who wants to do an old style loadbalance can just do decommission then bootstrap. This is a minor change, and honestly I think it might count as a 'bug' so I think we should squeeze it into 0.8, post-freeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1091139 - in /cassandra/branches/cassandra-0.8: ./ src/java/org/apache/cassandra/service/ src/java/org/apache/cassandra/tools/
Author: jbellis
Date: Mon Apr 11 17:35:42 2011
New Revision: 1091139
URL: http://svn.apache.org/viewvc?rev=1091139view=rev
Log:
r/m nodetool loadbalance
patch by Nick Bailey; reviewed by jbellis for CASSANDRA-2448
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/NEWS.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeCmd.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/tools/NodeProbe.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1091139r1=1091138r2=1091139view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 17:35:42 2011
@@ -19,9 +19,11 @@
* push replication_factor into strategy_options (CASSANDRA-1263)
* give snapshots the same name on each node (CASSANDRA-1791)
* add key type information and alias (CASSANDRA-2311, 2396)
+ * remove nodetool loadbalance (CASSANDRA-2448)
* multithreaded compaction (CASSANDRA-2191)
* compaction throttling (CASSANDRA-2156)
+
0.7.5
* Avoid seeking when sstable2json exports the entire file (CASSANDRA-2318)
* fix tombstone handling in repair and sstable2json (CASSANDRA-2279)
Modified: cassandra/branches/cassandra-0.8/NEWS.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/NEWS.txt?rev=1091139r1=1091138r2=1091139view=diff
==
--- cassandra/branches/cassandra-0.8/NEWS.txt (original)
+++ cassandra/branches/cassandra-0.8/NEWS.txt Mon Apr 11 17:35:42 2011
@@ -1,5 +1,5 @@
-Whatever
-
+0.8
+===
Upgrading
-
@@ -10,6 +10,9 @@ Upgrading
Upgrading from version 0.7.1 or later can be done with a rolling
restart,
one node at a time. You do not need to bring down the whole cluster.
+The loadbalance command has been removed from nodetool. For similar
+behavior, decommission then rebootstrap with empty initial_token.
+
Other
-
In the past, sstable2json would write column names and values as hex
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java?rev=1091139r1=1091138r2=1091139view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java
Mon Apr 11 17:35:42 2011
@@ -815,10 +815,9 @@ public class StorageService implements I
}
/**
- * Handle node leaving the ring. This can be either because of
decommission or loadbalance
+ * Handle node leaving the ring. This will happen when a node is
decommissioned
*
- * @param endpoint If reason for leaving is decommission or loadbalance
- * endpoint is the leaving node.
+ * @param endpoint If reason for leaving is decommission, endpoint is the
leaving node.
* @param pieces STATE_LEFT,token
*/
private void handleStateLeft(InetAddress endpoint, String[] pieces)
@@ -1800,15 +1799,6 @@ public class StorageService implements I
}
/**
- * Generates balanced token and calls load balance operation to move
current node to that token
- * @throws IOException on any I/O operation error
- */
-public void loadBalance() throws IOException
-{
-move(BootStrapper.getBalancedToken(tokenMetadata_,
StorageLoadBalancer.instance.getLoadInfo()));
-}
-
-/**
* move the node to new token or find a new token to boot to according to
load
*
* @param newToken new token to boot to, or if null, find balanced token
to boot to
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java?rev=1091139r1=1091138r2=1091139view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageServiceMBean.java
Mon Apr 11 17:35:42 2011
@@ -198,13 +198,6 @@ public interface StorageServiceMBean
public void move(String newToken) throws IOException, InterruptedException;
[jira] [Commented] (CASSANDRA-2167) Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018453#comment-13018453 ] Jonathan Ellis commented on CASSANDRA-2167: --- Revisiting this, I don't think it's actually a valid optimization -- if we free the backing Memory object before shipping the read buffers off over the network, we'll segfault. No? Add a deserialize(ByteBuffer) method to ColumnFamilySerializer -- Key: CASSANDRA-2167 URL: https://issues.apache.org/jira/browse/CASSANDRA-2167 Project: Cassandra Issue Type: Sub-task Components: Core Affects Versions: 0.8 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 0.8 Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF. This can be done using ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length); This is an improvement on https://issues.apache.org/jira/browse/CASSANDRA-1969; Plz see jonathan's comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2342) Add range slice support for counters
[ https://issues.apache.org/jira/browse/CASSANDRA-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018457#comment-13018457 ] Jonathan Ellis commented on CASSANDRA-2342: --- +1 Add range slice support for counters Key: CASSANDRA-2342 URL: https://issues.apache.org/jira/browse/CASSANDRA-2342 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.8 Attachments: 0001-Range-slice-support-for-counters.patch There is no equivalent for get_range_slice() for counters right now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1091148 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/cli/CliClient.java src/java/org/apache/cassandra/thrift/CassandraServer.java test/system/test_thr
Author: slebresne
Date: Mon Apr 11 17:52:13 2011
New Revision: 1091148
URL: http://svn.apache.org/viewvc?rev=1091148view=rev
Log:
Fix range slice for counters
patch by slebresne; reviewed by jbellis for CASSANDRA-2342
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
cassandra/branches/cassandra-0.8/test/system/test_thrift_server.py
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1091148r1=1091147r2=1091148view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 17:52:13 2011
@@ -1,7 +1,7 @@
0.8-dev
* remove Avro RPC support (CASSANDRA-926)
* adds support for columns that act as incr/decr counters
- (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384, 2236)
+ (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384, 2236, 2342)
* CQL (CASSANDRA-1703, 1704, 1705, 1706, 1707, 1708, 1710, 1711, 1940,
2124, 2302, 2277)
* avoid double RowMutation serialization on write path (CASSANDRA-1800)
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java?rev=1091148r1=1091147r2=1091148view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java
Mon Apr 11 17:52:13 2011
@@ -2206,6 +2206,24 @@ public class CliClient extends CliUserHe
sessionState.out.println());
}
+else if (columnOrSuperColumn.counter_column != null)
+{
+CounterColumn col = columnOrSuperColumn.counter_column;
+
+sessionState.out.printf(= (counter=%s, value=%s)%n,
formatColumnName(keySpace, columnFamilyName, col.name), col.value);
+}
+else if (columnOrSuperColumn.counter_super_column != null)
+{
+CounterSuperColumn superCol =
columnOrSuperColumn.counter_super_column;
+sessionState.out.printf(= (super_column=%s,,
formatColumnName(keySpace, columnFamilyName, superCol.name));
+
+for (CounterColumn col : superCol.columns)
+{
+sessionState.out.printf(%n (counter=%s,
value=%s), formatSubcolumnName(keySpace, columnFamilyName, col.name),
col.value);
+}
+
+sessionState.out.println());
+}
}
}
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java?rev=1091148r1=1091147r2=1091148view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
Mon Apr 11 17:52:13 2011
@@ -575,7 +575,7 @@ public class CassandraServer implements
String keyspace = state().getKeyspace();
state().hasColumnFamilyAccess(column_parent.column_family,
Permission.READ);
-CFMetaData metadata = ThriftValidation.validateColumnFamily(keyspace,
column_parent.column_family, false);
+CFMetaData metadata = ThriftValidation.validateColumnFamily(keyspace,
column_parent.column_family);
ThriftValidation.validateColumnParent(metadata, column_parent);
ThriftValidation.validatePredicate(metadata, column_parent, predicate);
ThriftValidation.validateKeyRange(range);
Modified: cassandra/branches/cassandra-0.8/test/system/test_thrift_server.py
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/test/system/test_thrift_server.py?rev=1091148r1=1091147r2=1091148view=diff
==
--- cassandra/branches/cassandra-0.8/test/system/test_thrift_server.py
(original)
+++ cassandra/branches/cassandra-0.8/test/system/test_thrift_server.py Mon Apr
11 17:52:13 2011
@@ -105,6 +105,12 @@ def _insert_range():
client.insert('key1', ColumnParent('Standard1'), Column('c3', 'value3',
0), ConsistencyLevel.ONE)
time.sleep(0.1)
+def _insert_counter_range():
+client.add('key1',
svn commit: r1091151 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/thrift/ test/system/
Author: slebresne Date: Mon Apr 11 18:00:23 2011 New Revision: 1091151 URL: http://svn.apache.org/viewvc?rev=1091151view=rev Log: Merge CASSANDRA-2342 from 0.8. Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java cassandra/trunk/test/system/test_thrift_server.py Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 18:00:23 2011 @@ -1,6 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 /cassandra/branches/cassandra-0.7:1026516-1090647 /cassandra/branches/cassandra-0.7.0:1053690-1055654 +/cassandra/branches/cassandra-0.8:1091148 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 /incubator/cassandra/branches/cassandra-0.4:810145-834239,834349-834350 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1091151r1=1091150r2=1091151view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Apr 11 18:00:23 2011 @@ -1,16 +1,16 @@ 0.8-dev * remove Avro RPC support (CASSANDRA-926) - * avoid double RowMutation serialization on write path (CASSANDRA-1800) * adds support for columns that act as incr/decr counters - (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384) + (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384, 2236, 2342) + * CQL (CASSANDRA-1703, 1704, 1705, 1706, 1707, 1708, 1710, 1711, 1940, + 2124, 2302, 2277) + * avoid double RowMutation serialization on write path (CASSANDRA-1800) * make NetworkTopologyStrategy the default (CASSANDRA-1960) * configurable internode encryption (CASSANDRA-1567) * human readable column names in sstable2json output (CASSANDRA-1933) * change default JMX port to 7199 (CASSANDRA-2027) * backwards compatible internal messaging (CASSANDRA-1015) * check for null encryption in MessagingService (CASSANDRA-2152) - * Fix for Cli to support updating replicate_on_write (CASSANDRA-2236) - * JDBC driver for CQL (CASSANDRA-2124, 2302, 2277) * atomic switch of memtables and sstables (CASSANDRA-2284) * add pluggable SeedProvider (CASSANDRA-1669) * Fix clustertool to not throw exception when calling get_endpoints (CASSANDRA-2437) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 18:00:23 2011 @@ -1,6 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1090647 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 +/cassandra/branches/cassandra-0.8/contrib:1091148 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 /incubator/cassandra/branches/cassandra-0.4/contrib:810145-834239,834349-834350 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Apr 11 18:00:23 2011 @@ -1,6 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1090647 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1091148 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1263 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1091151 Blamelist: slebresne BUILD FAILED: failed compile sincerely, -The Buildbot
[jira] [Created] (CASSANDRA-2449) Deprecate or modify per-cf memtable sizes in favor of the global threshold
Deprecate or modify per-cf memtable sizes in favor of the global threshold -- Key: CASSANDRA-2449 URL: https://issues.apache.org/jira/browse/CASSANDRA-2449 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Fix For: 0.8 The new memtable_total_space_in_mb setting is an excellent way to cap memory usage for memtables, and one could argue that it should replace the per-cf memtable sizes entirely. On the other hand, people may still want a knob to tune to flush certain cfs less frequently. I think a best of both worlds approach might be to deprecate the memtable_(throughput|operations) settings, and replace them with a preference value, which controls the relative memory usage of one CF versus another (all CFs at 1 would mean equal preference). For backwards compatibility, we could continue to read from the _throughput value and treat it as the preference value, while logging a warning. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2448) Remove loadbalance command
[ https://issues.apache.org/jira/browse/CASSANDRA-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018514#comment-13018514 ] Nick Bailey commented on CASSANDRA-2448: this should also be applied to trunk Remove loadbalance command -- Key: CASSANDRA-2448 URL: https://issues.apache.org/jira/browse/CASSANDRA-2448 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 0.8 Attachments: 0001-Remove-loadbalance-command-from-nodetool.patch With the update to how the move command works, the loadbalance command is even less useful that it was previously. The loadbalance command now calculates the token it is going to move to before it leaves which means it isn't considering the load it is giving away. Given that, I think we should just remove the loadbalance command entirely. Anyone who wants to do an old style loadbalance can just do decommission then bootstrap. This is a minor change, and honestly I think it might count as a 'bug' so I think we should squeeze it into 0.8, post-freeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2252: Attachment: (was: merged-2252.tgz) off-heap memtables -- Key: CASSANDRA-2252 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Attachments: 0001-add-MemtableAllocator.txt, 0002-add-off-heap-MemtableAllocator-support.txt Original Estimate: 0.4h Remaining Estimate: 0.4h The memtable design practically actively fights Java's GC design. Todd Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2252: Attachment: (was: 2252-alternate-v2.tgz) off-heap memtables -- Key: CASSANDRA-2252 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Attachments: 0001-add-MemtableAllocator.txt, 0002-add-off-heap-MemtableAllocator-support.txt Original Estimate: 0.4h Remaining Estimate: 0.4h The memtable design practically actively fights Java's GC design. Todd Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2252: Attachment: merged-2252.tgz Rebased for trunk. I took the easy way out and just included the Allocator's allocation count in the memtable live size: this will overcount, but it accounts for fragmentation due to updates. I've started a patch for JAMM to add a mode that will allow us to ignore shared buffers: see [jamm/buffer-behavior|https://github.com/stuhood/jamm/commits/buffer-behavior]. off-heap memtables -- Key: CASSANDRA-2252 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 1.0 Attachments: 0001-add-MemtableAllocator.txt, 0002-add-off-heap-MemtableAllocator-support.txt, merged-2252.tgz Original Estimate: 0.4h Remaining Estimate: 0.4h The memtable design practically actively fights Java's GC design. Todd Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1091176 - /cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Author: jbellis
Date: Mon Apr 11 19:27:03 2011
New Revision: 1091176
URL: http://svn.apache.org/viewvc?rev=1091176view=rev
Log:
update stress.java for replication_factor-to-strategy_options change
Modified:
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Modified:
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java?rev=1091176r1=1091175r2=1091176view=diff
==
---
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
(original)
+++
cassandra/branches/cassandra-0.8/tools/stress/src/org/apache/cassandra/stress/Session.java
Mon Apr 11 19:27:03 2011
@@ -414,7 +414,9 @@ public class Session
keyspace.setName(Keyspace1);
keyspace.setStrategy_class(replicationStrategy);
-keyspace.setReplication_factor(replicationFactor);
+MapString,String options = new HashMapString, String();
+options.put(replication_factor, String.valueOf(replicationFactor));
+keyspace.setStrategy_options(options);
if (!replicationStrategyOptions.isEmpty())
{
[jira] [Updated] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2441: -- Attachment: 2441.txt Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Attachments: 2441.txt Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2441: -- Attachment: jamm-0.2.1.jar attached. (requires jamm 0.2.1 in lib/, also attached.) (most of the patch is svn deleting the 0.2 jar. silly svn.) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Attachments: 2441.txt, jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2450) incompatibility w/ 0.7 schemas
incompatibility w/ 0.7 schemas
--
Key: CASSANDRA-2450
URL: https://issues.apache.org/jira/browse/CASSANDRA-2450
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8
Reporter: Jonathan Ellis
Assignee: Jon Hermes
Fix For: 0.8
If you create a SimpleStrategy keyspace under 0.7, then switch to 0.8, you will
get this error on startup:
{noformat}
ERROR 14:31:41,725 Exception encountered during startup.
java.lang.RuntimeException: org.apache.cassandra.config.ConfigurationException:
SimpleStrategy requires a replication_factor strategy option.
at org.apache.cassandra.db.Table.init(Table.java:277)
at org.apache.cassandra.db.Table.open(Table.java:109)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:160)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:314)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80)
Caused by: org.apache.cassandra.config.ConfigurationException: SimpleStrategy
requires a replication_factor strategy option.
at
org.apache.cassandra.locator.SimpleStrategy.validateOptions(SimpleStrategy.java:75)
at
org.apache.cassandra.locator.AbstractReplicationStrategy.createReplicationStrategy(AbstractReplicationStrategy.java:262)
at
org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:327)
at org.apache.cassandra.db.Table.init(Table.java:273)
... 4 more
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2449) Deprecate or modify per-cf memtable sizes in favor of the global threshold
[ https://issues.apache.org/jira/browse/CASSANDRA-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018528#comment-13018528 ] Jonathan Ellis commented on CASSANDRA-2449: --- bq. replace them with a preference value, which controls the relative memory usage of one CF versus another I'm not a fan of adding additional complexity here. At best, you'll get substantially the current behavior; at worst (when you tell it to prefer keeping the largest CF in memory), you'll create a flush storm of smaller CFs. Deprecate or modify per-cf memtable sizes in favor of the global threshold -- Key: CASSANDRA-2449 URL: https://issues.apache.org/jira/browse/CASSANDRA-2449 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Fix For: 0.8 The new memtable_total_space_in_mb setting is an excellent way to cap memory usage for memtables, and one could argue that it should replace the per-cf memtable sizes entirely. On the other hand, people may still want a knob to tune to flush certain cfs less frequently. I think a best of both worlds approach might be to deprecate the memtable_(throughput|operations) settings, and replace them with a preference value, which controls the relative memory usage of one CF versus another (all CFs at 1 would mean equal preference). For backwards compatibility, we could continue to read from the _throughput value and treat it as the preference value, while logging a warning. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2448) Remove loadbalance command
[ https://issues.apache.org/jira/browse/CASSANDRA-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018530#comment-13018530 ] Jonathan Ellis commented on CASSANDRA-2448: --- merging to trunk is asynchronous but will happen Remove loadbalance command -- Key: CASSANDRA-2448 URL: https://issues.apache.org/jira/browse/CASSANDRA-2448 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8 Reporter: Nick Bailey Assignee: Nick Bailey Priority: Minor Fix For: 0.8 Attachments: 0001-Remove-loadbalance-command-from-nodetool.patch With the update to how the move command works, the loadbalance command is even less useful that it was previously. The loadbalance command now calculates the token it is going to move to before it leaves which means it isn't considering the load it is giving away. Given that, I think we should just remove the loadbalance command entirely. Anyone who wants to do an old style loadbalance can just do decommission then bootstrap. This is a minor change, and honestly I think it might count as a 'bug' so I think we should squeeze it into 0.8, post-freeze. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2167) Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018537#comment-13018537 ] Vijay commented on CASSANDRA-2167: -- Initial assumption was that, it will be handled with by the Memory.free seems like it is not so we should add some kind of locks before it frees up the memory? as a separate thread (mem.free)? or is it too much overhead? if we agree on lock... i can add some code to do so. Add a deserialize(ByteBuffer) method to ColumnFamilySerializer -- Key: CASSANDRA-2167 URL: https://issues.apache.org/jira/browse/CASSANDRA-2167 Project: Cassandra Issue Type: Sub-task Components: Core Affects Versions: 0.8 Reporter: Vijay Assignee: Vijay Priority: Minor Fix For: 1.0 Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF. This can be done using ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length); This is an improvement on https://issues.apache.org/jira/browse/CASSANDRA-1969; Plz see jonathan's comment. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2420) row cache / streaming aren't aware of each other
[ https://issues.apache.org/jira/browse/CASSANDRA-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2420: Attachment: 0001-Handle-the-row-cache-for-streamed-row.patch There is a very simple patch for this issue. It consists in invalidating the cache for each key we index. The downside is that this will invalidate all key that gets repaired, but updating the cache (instead of invalidating) implies reading on disk so doing this during the indexing or at the next read may not matter much. In any case, this is better that the current situation and after all . I however attached a patch (against trunk for now) that 'do the right thing' and will update the cache in the case of repair instead of invalidating. I mentioned the first solution in case we consider that the 'right one' is too disruptive for 0.7 for instance (not that the patch is very complicated). Note that the patch fixes a tiny unrelated issue: the writeStat are not updated during a write if the used cache has 'isPutCopying' (this could be fixed separately). row cache / streaming aren't aware of each other Key: CASSANDRA-2420 URL: https://issues.apache.org/jira/browse/CASSANDRA-2420 Project: Cassandra Issue Type: Bug Affects Versions: 0.6 Reporter: Matthew F. Dennis Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 Attachments: 0001-Handle-the-row-cache-for-streamed-row.patch SSTableWriter.Builder.build() takes tables that resulted from streaming, repair, bootstrapping, et cetera and builds the indexes and bloom filters before adding it so the current node is aware of it. However, if there is data present in the cache for a row that is also present in the streamed table the row cache can over shadow the data in the newly built table. In other words, until the row in row cache is removed from the cache (e.g. because it's pushed out because of size, the node is restarted, the cache is manually cleared) the data in the newly built table will never be returned to clients. The solution that seems most reasonable at this point is to have SSTableWriter.Builder.build() (or something below it) update the row cache if the row key in the table being built is also present in the cache. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2451) Make clean compactions cleanup the row cache
Make clean compactions cleanup the row cache Key: CASSANDRA-2451 URL: https://issues.apache.org/jira/browse/CASSANDRA-2451 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor We uselessly keep in cache keys that are cleanup, which is not a big deal because they will get expunged eventually but there is no point in wasting the memory in the meantime. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2452) EC2 Snitch to friendly to public ip and natively support for EC2 mult-region.
EC2 Snitch to friendly to public ip and natively support for EC2 mult-region. - Key: CASSANDRA-2452 URL: https://issues.apache.org/jira/browse/CASSANDRA-2452 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8 Environment: JVM Reporter: Vijay Assignee: Vijay Priority: Minor Make cassandra talk identify itself using the public ip (To avoid any future conflicts of private ips). 1) Split the logic of identification vs listen Address in the code. 2) Move the logic to assign IP address to the node into EndPointSnitch. 3) Make EC2 Snitch query for its public ip and use it for identification. 4) Make EC2 snitch to use InetAddress.getLocal to listen to the private ip. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2441: -- Attachment: (was: 2441.txt) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8 Attachments: jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2451) Make clean compactions cleanup the row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2451: Attachment: 0001-Cleanup-cache-during-cleanup-compaction.patch Patch is again 0.7 Make clean compactions cleanup the row cache Key: CASSANDRA-2451 URL: https://issues.apache.org/jira/browse/CASSANDRA-2451 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 Attachments: 0001-Cleanup-cache-during-cleanup-compaction.patch Original Estimate: 1h Remaining Estimate: 1h We uselessly keep in cache keys that are cleanup, which is not a big deal because they will get expunged eventually but there is no point in wasting the memory in the meantime. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2441: -- Attachment: 2441.txt On Brandon's advice I moved the entire warning into the log4j call, even though this makes it unwieldy and we don't have perfect information as to what the cause is at that point. Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8 Attachments: 2441.txt, jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018554#comment-13018554 ] Jonathan Ellis commented on CASSANDRA-1278: --- go ahead and use git to split this up into logical pieces (e.g., internals changes, proxy server, tests). where does throughput max out in rows/s as you add clients (proxy nodes), vs stress-against-plain-cassandra? Make bulk loading into Cassandra less crappy, more pluggable Key: CASSANDRA-1278 URL: https://issues.apache.org/jira/browse/CASSANDRA-1278 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jeremy Hanna Assignee: Matthew F. Dennis Fix For: 0.7.5 Attachments: 1278-cassandra-0.7-v2.txt, 1278-cassandra-0.7.1.txt, 1278-cassandra-0.7.txt Original Estimate: 40h Time Spent: 40h 40m Remaining Estimate: 0h Currently bulk loading into Cassandra is a black art. People are either directed to just do it responsibly with thrift or a higher level client, or they have to explore the contrib/bmt example - http://wiki.apache.org/cassandra/BinaryMemtable That contrib module requires delving into the code to find out how it works and then applying it to the given problem. Using either method, the user also needs to keep in mind that overloading the cluster is possible - which will hopefully be addressed in CASSANDRA-685 This improvement would be to create a contrib module or set of documents dealing with bulk loading. Perhaps it could include code in the Core to make it more pluggable for external clients of different types. It is just that this is something that many that are new to Cassandra need to do - bulk load their data into Cassandra. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018557#comment-13018557 ] Pavel Yaskevich commented on CASSANDRA-2441: Can you please re-attach git apply and patch both say that patch is corrupted at line 90?.. Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8 Attachments: 2441.txt, jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2420) row cache / streaming aren't aware of each other
[ https://issues.apache.org/jira/browse/CASSANDRA-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018560#comment-13018560 ] Jonathan Ellis commented on CASSANDRA-2420: --- I would be more comfortable having LCR throw UnsupportedOperation if asked for full row, since You Shouldn't Do That. Would prefer the updateCache case to be AES: ... default: invalidate and break; it's more obvious looking at it what the point is, and unnecessary invalidate calls will be harmless. row cache / streaming aren't aware of each other Key: CASSANDRA-2420 URL: https://issues.apache.org/jira/browse/CASSANDRA-2420 Project: Cassandra Issue Type: Bug Affects Versions: 0.6 Reporter: Matthew F. Dennis Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 Attachments: 0001-Handle-the-row-cache-for-streamed-row.patch SSTableWriter.Builder.build() takes tables that resulted from streaming, repair, bootstrapping, et cetera and builds the indexes and bloom filters before adding it so the current node is aware of it. However, if there is data present in the cache for a row that is also present in the streamed table the row cache can over shadow the data in the newly built table. In other words, until the row in row cache is removed from the cache (e.g. because it's pushed out because of size, the node is restarted, the cache is manually cleared) the data in the newly built table will never be returned to clients. The solution that seems most reasonable at this point is to have SSTableWriter.Builder.build() (or something below it) update the row cache if the row key in the table being built is also present in the cache. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2441: -- Attachment: 2441.txt manually ripped the binary portion out of the patch. Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8 Attachments: 2441.txt, 2441.txt, jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2451) Make clean compactions cleanup the row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018562#comment-13018562 ] Jonathan Ellis commented on CASSANDRA-2451: --- +1 Make clean compactions cleanup the row cache Key: CASSANDRA-2451 URL: https://issues.apache.org/jira/browse/CASSANDRA-2451 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Minor Fix For: 0.7.5 Attachments: 0001-Cleanup-cache-during-cleanup-compaction.patch Original Estimate: 1h Remaining Estimate: 1h We uselessly keep in cache keys that are cleanup, which is not a big deal because they will get expunged eventually but there is no point in wasting the memory in the meantime. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018566#comment-13018566 ] Pavel Yaskevich commented on CASSANDRA-2441: +1 Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 Key: CASSANDRA-2441 URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Environment: Both servers have identical hardware configuration: Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) Java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Pavel Yaskevich Assignee: Jonathan Ellis Priority: Critical Fix For: 0.8 Attachments: 2441.txt, 2441.txt, jamm-0.2.1.jar Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from 0.7), branch: trunk. What I did is cloned git://git.apache.org/cassandra.git and did git reset each commit with `ant clean ant ./bin/cassandra -f` until I got cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2453) node.js cql driver
node.js cql driver -- Key: CASSANDRA-2453 URL: https://issues.apache.org/jira/browse/CASSANDRA-2453 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Gary Dusbabek Assignee: Gary Dusbabek Priority: Minor Fix For: 0.8 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2454) Possible deadlock for counter mutations
Possible deadlock for counter mutations
---
Key: CASSANDRA-2454
URL: https://issues.apache.org/jira/browse/CASSANDRA-2454
Project: Cassandra
Issue Type: Bug
Reporter: Stu Hood
Fix For: 0.8
{{StorageProxy.applyCounterMutation}} is executed on the mutation stage, but it
also submits tasks to the mutation stage, and then blocks for them. If there
are more than a few concurrent mutations, this can lead to deadlock.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2454) Possible deadlock for counter mutations
[
https://issues.apache.org/jira/browse/CASSANDRA-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2454:
Attachment: 0001-Don-t-re-submit-to-the-mutation-stage.txt
Patch. (credit to Kelvin Kakugawa)
Possible deadlock for counter mutations
---
Key: CASSANDRA-2454
URL: https://issues.apache.org/jira/browse/CASSANDRA-2454
Project: Cassandra
Issue Type: Bug
Reporter: Stu Hood
Fix For: 0.8
Attachments: 0001-Don-t-re-submit-to-the-mutation-stage.txt
{{StorageProxy.applyCounterMutation}} is executed on the mutation stage, but
it also submits tasks to the mutation stage, and then blocks for them. If
there are more than a few concurrent mutations, this can lead to deadlock.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-494) add remove_slice to the api
[ https://issues.apache.org/jira/browse/CASSANDRA-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-494. -- Resolution: Later add remove_slice to the api --- Key: CASSANDRA-494 URL: https://issues.apache.org/jira/browse/CASSANDRA-494 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Dan Di Spaltro Priority: Minor It would be nice to mimic how get_slice works for removing values. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-571) API for requesting sub-slices of a range of supercolumns
[
https://issues.apache.org/jira/browse/CASSANDRA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-571.
--
Resolution: Won't Fix
As we've figured out best practices around supercolumns, this isn't really
necessary. (Supercolumns are best used for denormalizing data that is accessed
at the same time.)
API for requesting sub-slices of a range of supercolumns
Key: CASSANDRA-571
URL: https://issues.apache.org/jira/browse/CASSANDRA-571
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Ellis
Priority: Minor
Suhail Doshi wrote in a comment to CASSANDRA-570 (a different issue):
Ability to slice a column and specify an exact super column key, for example:
column_1 {
sc1: {}
}
column_2 {
sc1: {}
sc2: {}
}
Be able to slice by column_1 to column_2 but instead of grabbing every
column, grab only super column sc1 from each? The reasoning is that it's
terrible to have to slice by column and get *every* super column and have it
held in memory for the client application.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-940) Allow multiple fat clients to connect from the same host
[ https://issues.apache.org/jira/browse/CASSANDRA-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-940. -- Resolution: Later Allow multiple fat clients to connect from the same host -- Key: CASSANDRA-940 URL: https://issues.apache.org/jira/browse/CASSANDRA-940 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Johan Oskarsson Priority: Minor Currently no more then one so called fat client can access Cassandra from the same machine due to assumptions being made about the port used for communication. It would be useful in many scenarios such as reading data using Hadoop to allow many clients to connect from the same node, even from the same machine that Cassandra is running on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-941) Replace InetAddress usage with EndPoint
[ https://issues.apache.org/jira/browse/CASSANDRA-941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-941. -- Resolution: Later Replace InetAddress usage with EndPoint --- Key: CASSANDRA-941 URL: https://issues.apache.org/jira/browse/CASSANDRA-941 Project: Cassandra Issue Type: Sub-task Components: Core Reporter: Johan Oskarsson Attachments: CASSANDRA-941.patch, CASSANDRA-941.patch, CASSANDRA-941.patch, CASSANDRA-941.patch An end point should be represented by both address and port instead of port. Create an EndPoint class that takes an InetAddress and port and use this where needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
