[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: CASSANDRA-1714-v6-0.7.patch zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1979) CassandraServiceDataCleaner.prepare() fails with IOException.
CassandraServiceDataCleaner.prepare() fails with IOException. - Key: CASSANDRA-1979 URL: https://issues.apache.org/jira/browse/CASSANDRA-1979 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.7.0 Environment: Windows XP Reporter: Chris Ng CassandraServiceDataCleaner.prepare() fails with an IOException if run in isolation. It seems that initializing the DataDescriptor creates a new CommitLog file, and then the cleaner tries to delete this file and fails. 16:06:07.204 [main] INFO o.a.c.config.DatabaseDescriptor - Loading settings from file:/C:/workspace/sandbox/target/classes/cassandra.yaml 16:06:07.282 [main] DEBUG o.a.c.config.DatabaseDescriptor - Syncing log with a period of 1 16:06:07.282 [main] INFO o.a.c.config.DatabaseDescriptor - DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard 16:06:07.797 [main] DEBUG o.a.c.config.DatabaseDescriptor - setting auto_bootstrap to false 16:06:07.797 [main] INFO o.a.c.db.commitlog.CommitLogSegment - Creating new commitlog segment target/var/lib/cassandra/commitlog\CommitLog-1294934767797.log 16:06:07.813 [main] DEBUG o.apache.cassandra.io.util.FileUtils - Deleting CommitLog-1294934767797.log Exception in thread main java.io.IOException: Failed to delete C:\workspace\sandbox\target\var\lib\cassandra\commitlog\CommitLog-1294934767797.log at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54) at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:201) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanDir(CassandraServiceDataCleaner.java:99) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.cleanupDataDirectories(CassandraServiceDataCleaner.java:53) at org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner.prepare(CassandraServiceDataCleaner.java:44) at cng.sandbox.App.main(App.java:15) This also seems to leave a bunch of threads running in the background, so the process has to be manually killed. This was tested with the javautils in the 0.7.0 branch. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1980) Add support for client-side encryption
Add support for client-side encryption -- Key: CASSANDRA-1980 URL: https://issues.apache.org/jira/browse/CASSANDRA-1980 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jeremy Hanna Fix For: 0.8 With thrift-106 in place coming in thrift version 0.6, it will be necessary to add a way for making the server aware of that. We could allow the option in the server configuration (cassandra.yaml) to use encrypted or non-encrypted client connections. However if they choose encrypted, the only clients able to connect would be java clients at this point. It might be better to take some more time and look at how to enable accepting both encrypted and unencrypted connections at the same time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1948) Disambiguate messages from different versions of cassandra.
[ https://issues.apache.org/jira/browse/CASSANDRA-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek resolved CASSANDRA-1948. -- Resolution: Duplicate This is basically the same as CASSANDRA-1970. Disambiguate messages from different versions of cassandra. --- Key: CASSANDRA-1948 URL: https://issues.apache.org/jira/browse/CASSANDRA-1948 Project: Cassandra Issue Type: Sub-task Reporter: Gary Dusbabek Assignee: Gary Dusbabek -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.
[ https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981381#action_12981381 ] Vijay commented on CASSANDRA-1969: -- Update: Reclaiming the space of the invalid cache objects gets complicated running OOM (Native) is not within the GC control (until a full GC runs on the JVM automatically, free space are not automatically/manually reclaimed), and I couldn't find a clean way to free the native memory from Java. Option 1 (Easier option): Set a hard limit on the Native memory which can be allocated by the GC once it is full, load the Objects in cache in the Heap Memory. Eventually the cache will be cleaned by GC (always staying within 2 limits hard- total objects cached and soft - total objects which can be cached in the heap). Option 2: Allocate chunks of memory in the native memory and reclame it when they are needed. Example: when you need 25.5 K block always allocate 26K block and once it is free allocate a new Object which is of the same size into the cache (This is only if GC didn't reclaim the object, if reclaimed by GC reassign memory native), this can be done using PhantomReference, 'if' enqued then allocate new 'else' use the same block of memory. Use BB for row cache - To Improve GC performance. - Key: CASSANDRA-1969 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux and Mac Reporter: Vijay Assignee: Vijay Priority: Minor Attachments: BB_Cache-1945.png, JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt Java BB.allocateDirect() will allocate native memory out of the JVM and will help reducing the GC pressure in the JVM with a large Cache. From some of the basic tests it shows around 50% improvement than doing a normal Object cache. In addition this patch provide the users an option to choose BB.allocateDirect or store everything in the heap. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: (was: CASSANDRA-1714-v6-0.7.patch) zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: CASSANDRA-1714-v6-0.7.patch zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714-v6-0.7.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1714: --- Attachment: (was: CASSANDRA-1714-v6-0.7.patch) zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[Cassandra Wiki] Trivial Update of Committers by Chris Goffinet
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The Committers page has been changed by Chris Goffinet. http://wiki.apache.org/cassandra/Committers?action=diffrev1=10rev2=11 -- ||Jonathan Ellis||Mar 2009||Riptano||Project chair|| ||Eric Evans||Jun 2009||Rackspace||PMC member, Release manager, Debian packager|| ||Jun Rao||Jun 2009||!LinkedIn||PMC member|| - ||Chris Goffinet||Sept 2009||Digg||PMC member|| + ||Chris Goffinet||Sept 2009||Twitter||PMC member|| ||Johan Oskarsson||Nov 2009||Twitter||Also a [[http://hadoop.apache.org/|Hadoop]] committer|| ||Gary Dusbabek||Dec 2009||Rackspace||PMC member|| ||Jaakko Laine||Dec 2009||?|| ||
[jira] Commented: (CASSANDRA-1714) zero-copy reads
[ https://issues.apache.org/jira/browse/CASSANDRA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981420#action_12981420 ] Jonathan Ellis commented on CASSANDRA-1714: --- won't the buffers for column name / value in row cache be coming from MFDI? (we don't care about truncate et al, but we don't want to keep the mappedbytebuffers alive post-compaction, so we do want to copy for that reason) zero-copy reads --- Key: CASSANDRA-1714 URL: https://issues.apache.org/jira/browse/CASSANDRA-1714 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: 1714-v5-07.txt, CASSANDRA-1714-0.7.patch, CASSANDRA-1714-v2.patch, CASSANDRA-1714-v3.patch, CASSANDRA-1714-v4-trunk.patch, CASSANDRA-1714.patch, zerocopy.txt Original Estimate: 32h Time Spent: 32h Remaining Estimate: 0h Since we are already using mmap'd ByteBuffers in MappedFileDataInput we should be able to do zero-copy reads (via buffer.slice()), which would give us better performance than CASSANDRA-1651 without having to worry about buffer management. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981426#action_12981426 ] Nick Bailey commented on CASSANDRA-1898: I think importSorted should check that the keys are actually in order to prevent writing any bad sstables. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981436#action_12981436 ] Pavel Yaskevich commented on CASSANDRA-1898: I'm not sure about that - user is warned about -s option. If you think that this is required I will add reading of the keys to check if everything is in order and read-write from json to sstable after. WDYT? json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1981) The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072
[ https://issues.apache.org/jira/browse/CASSANDRA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-1981: Attachment: 0001-Add-back-writting-of-statistics-in-SSTableWriter.patch The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072 -- Key: CASSANDRA-1981 URL: https://issues.apache.org/jira/browse/CASSANDRA-1981 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8 Attachments: 0001-Add-back-writting-of-statistics-in-SSTableWriter.patch Everything's in the summary -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1981) The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072
The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072 -- Key: CASSANDRA-1981 URL: https://issues.apache.org/jira/browse/CASSANDRA-1981 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8 Attachments: 0001-Add-back-writting-of-statistics-in-SSTableWriter.patch Everything's in the summary -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1982) When creating snapshots, flush doesn't have to return before the snapshots are created
When creating snapshots, flush doesn't have to return before the snapshots are created -- Key: CASSANDRA-1982 URL: https://issues.apache.org/jira/browse/CASSANDRA-1982 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0, 0.6.9 Reporter: Joaquin Casares Priority: Minor Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But he flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981443#action_12981443 ] Nick Bailey commented on CASSANDRA-1898: I don't think its too hard to add and actually stopping the user from doing anything dumb is better than warning them not to do dumb things. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1058716 - /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Author: jake
Date: Thu Jan 13 19:41:57 2011
New Revision: 1058716
URL: http://svn.apache.org/viewvc?rev=1058716view=rev
Log:
turn back on sstable statistics generation after it was removed by #1072
mistakenly.
Patch by Sylvain Lebresne; reviewed by Jake Luciani for CASSANDRA-1981
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1058716r1=1058715r2=1058716view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Thu Jan 13 19:41:57 2011
@@ -382,6 +382,7 @@ public class SSTableWriter extends SSTab
rows++;
}
+writeStatistics(desc, rowSizes, columnCounts);
return rows;
}
}
@@ -451,6 +452,7 @@ public class SSTableWriter extends SSTab
dfile.sync();
}
+writeStatistics(desc, rowSizes, columnCounts);
if (writeRowPosition != readRowPosition)
{
[jira] Updated: (CASSANDRA-1982) When creating snapshots, flush isn't required to finish before the snapshots are created
[ https://issues.apache.org/jira/browse/CASSANDRA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joaquin Casares updated CASSANDRA-1982: --- Description: Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. was: Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But he flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. Summary: When creating snapshots, flush isn't required to finish before the snapshots are created (was: When creating snapshots, flush doesn't have to return before the snapshots are created) When creating snapshots, flush isn't required to finish before the snapshots are created Key: CASSANDRA-1982 URL: https://issues.apache.org/jira/browse/CASSANDRA-1982 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6.9, 0.7.0 Reporter: Joaquin Casares Priority: Minor Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1968) Increase JVM young generation size
[ https://issues.apache.org/jira/browse/CASSANDRA-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981459#action_12981459 ] T Jake Luciani commented on CASSANDRA-1968: --- I tested this with Solandra and saw a 25% boost in write speed, read however was unchanged (though Solandra doesn't use thrift) Increase JVM young generation size -- Key: CASSANDRA-1968 URL: https://issues.apache.org/jira/browse/CASSANDRA-1968 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Brandon Williams Assignee: Brandon Williams Fix For: 0.7.1 Attachments: 1968.txt, 1968_v2.txt Original Estimate: 4h Remaining Estimate: 4h Increasing the young generation size has a profound effect on performance and cpu utilization. We should set it explicitly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1898: --- Attachment: CASSANDRA-1898-v3.patch json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981483#action_12981483 ] Brandon Williams edited comment on CASSANDRA-1898 at 1/13/11 4:05 PM: -- +1 to checking behavior is correct rather than trusting the user. was (Author: brandon.williams): +1 to checking behavior is correct rather trusting the user. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981483#action_12981483 ] Brandon Williams commented on CASSANDRA-1898: - +1 to checking behavior is correct rather trusting the user. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981487#action_12981487 ] Pavel Yaskevich commented on CASSANDRA-1898: v3 patch supports it. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1981) The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072
[ https://issues.apache.org/jira/browse/CASSANDRA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981489#action_12981489 ] Hudson commented on CASSANDRA-1981: --- Integrated in Cassandra #667 (See [https://hudson.apache.org/hudson/job/Cassandra/667/]) turn back on sstable statistics generation after it was removed by #1072 mistakenly. Patch by Sylvain Lebresne; reviewed by Jake Luciani for CASSANDRA-1981 The writing of statistics in SSTableWrite.Builder has been mistakenly removed by #1072 -- Key: CASSANDRA-1981 URL: https://issues.apache.org/jira/browse/CASSANDRA-1981 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8 Attachments: 0001-Add-back-writting-of-statistics-in-SSTableWriter.patch Everything's in the summary -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1983) Make sstable filenames contain a UUID instead of increasing integer
Make sstable filenames contain a UUID instead of increasing integer --- Key: CASSANDRA-1983 URL: https://issues.apache.org/jira/browse/CASSANDRA-1983 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: David King Priority: Minor sstable filenames look like CFName-1569-Index.db, containing an integer for uniqueness. This makes it possible (however unlikely) that the integer could overflow, which could be a problem. It also makes it difficult to collapse multiple nodes into a single one with rsync. I do this occasionally for testing: I'll copy our 20 node cluster into only 3 nodes by copying all of the data files and running cleanup; at present this requires a manual step of uniqifying the overlapping sstable names. If instead of an incrementing integer, it would be handy if these contained a UUID or somesuch that guarantees uniqueness across the cluster. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1982) When creating snapshots, flush isn't required to finish before the snapshots are created
[ https://issues.apache.org/jira/browse/CASSANDRA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981499#action_12981499 ] Yang Yang commented on CASSANDRA-1982: -- it's probably useful to enable the user to do 2 modes of snapshot: synchronous and async. in synchronous snapshot, the snapshot() call should wait for the memtable flushing Callable object to finish, for all ColumnFamilyStores. the async mode is just the current implementation. the semantic for synchronous snapshot means that the resulting snapshot dir will contain all the data in the system (including memtable and sstables) ***at the exact moment *** that the user invoked snapshot() the semantic for async snapshot is that the information contained in the snapshot dir is from ***some moment between when the user invoked snapshot() and when the last time an sstable was fully dumped When creating snapshots, flush isn't required to finish before the snapshots are created Key: CASSANDRA-1982 URL: https://issues.apache.org/jira/browse/CASSANDRA-1982 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6.9, 0.7.0 Reporter: Joaquin Casares Priority: Minor Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1983) Make sstable filenames contain a UUID instead of increasing integer
[ https://issues.apache.org/jira/browse/CASSANDRA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981507#action_12981507 ] Ryan King commented on CASSANDRA-1983: -- Alternatively, since we'll need a host-uuid mapping for counters we can put that uuid in the filename along with a serial integer (make it a long and we should be ok, right?) Make sstable filenames contain a UUID instead of increasing integer --- Key: CASSANDRA-1983 URL: https://issues.apache.org/jira/browse/CASSANDRA-1983 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: David King Priority: Minor sstable filenames look like CFName-1569-Index.db, containing an integer for uniqueness. This makes it possible (however unlikely) that the integer could overflow, which could be a problem. It also makes it difficult to collapse multiple nodes into a single one with rsync. I do this occasionally for testing: I'll copy our 20 node cluster into only 3 nodes by copying all of the data files and running cleanup; at present this requires a manual step of uniqifying the overlapping sstable names. If instead of an incrementing integer, it would be handy if these contained a UUID or somesuch that guarantees uniqueness across the cluster. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1983) Make sstable filenames contain a UUID instead of increasing integer
[ https://issues.apache.org/jira/browse/CASSANDRA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981509#action_12981509 ] David King commented on CASSANDRA-1983: --- As long as the host is still willing to read filenames without its own uuid, sure Make sstable filenames contain a UUID instead of increasing integer --- Key: CASSANDRA-1983 URL: https://issues.apache.org/jira/browse/CASSANDRA-1983 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: David King Priority: Minor sstable filenames look like CFName-1569-Index.db, containing an integer for uniqueness. This makes it possible (however unlikely) that the integer could overflow, which could be a problem. It also makes it difficult to collapse multiple nodes into a single one with rsync. I do this occasionally for testing: I'll copy our 20 node cluster into only 3 nodes by copying all of the data files and running cleanup; at present this requires a manual step of uniqifying the overlapping sstable names. If instead of an incrementing integer, it would be handy if these contained a UUID or somesuch that guarantees uniqueness across the cluster. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1058764 - in /cassandra/branches/cassandra-0.7: conf/cassandra.yaml src/java/org/apache/cassandra/cli/CliClient.java src/java/org/apache/cassandra/config/KSMetaData.java
Author: brandonwilliams
Date: Thu Jan 13 22:13:45 2011
New Revision: 1058764
URL: http://svn.apache.org/viewvc?rev=1058764view=rev
Log:
Make NetworkTopologyStrategy the default.
Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-1960
Modified:
cassandra/branches/cassandra-0.7/conf/cassandra.yaml
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java
Modified: cassandra/branches/cassandra-0.7/conf/cassandra.yaml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/conf/cassandra.yaml?rev=1058764r1=1058763r2=1058764view=diff
==
--- cassandra/branches/cassandra-0.7/conf/cassandra.yaml (original)
+++ cassandra/branches/cassandra-0.7/conf/cassandra.yaml Thu Jan 13 22:13:45
2011
@@ -309,8 +309,9 @@ index_interval: 128
# With NetworkTopologyStrategy,
# for each datacenter, you can specify how many replicas you want
# on a per-keyspace basis. Replicas are placed on different racks
-# within each DC, if possible. This strategy also requires rack aware
-# snitch, such as RackInferringSnitch or PropertyFileSnitch.
+# within each DC, if possible. With SimpleSnitch, this will be
+# equivalent to SimpleStrategy. Later, the snitch can be upgraded to
+# one such as RackInferringSnitch or PropertyFileSnitch.
# An example:
#- name: Keyspace1
# replica_placement_strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy
@@ -405,7 +406,7 @@ index_interval: 128
# /NOTE
keyspaces:
- name: Keyspace1
- replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy
+ replica_placement_strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy
replication_factor: 1
column_families:
- name: Standard1
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java?rev=1058764r1=1058763r2=1058764view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
Thu Jan 13 22:13:45 2011
@@ -18,6 +18,8 @@
package org.apache.cassandra.cli;
import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.util.*;
@@ -29,6 +31,7 @@ import org.apache.cassandra.config.Confi
import org.apache.cassandra.db.ColumnFamilyStoreMBean;
import org.apache.cassandra.db.CompactionManagerMBean;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.locator.SimpleSnitch;
import org.apache.cassandra.thrift.*;
import org.apache.cassandra.tools.NodeProbe;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -92,7 +95,8 @@ public class CliClient extends CliUserHe
PLACEMENT_STRATEGY,
STRATEGY_OPTIONS
}
-private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.SimpleStrategy;
+
+private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.NetworkTopologyStrategy;
private Cassandra.Client thriftClient = null;
private CliSessionState sessionState = null;
@@ -803,6 +807,30 @@ public class CliClient extends CliUserHe
}
}
+// using default snitch options if strategy is NetworkTopologyStrategy
and no options were set.
+if (ksDef.getStrategy_class().contains(.NetworkTopologyStrategy))
+{
+MapString, String currentStrategyOptions =
ksDef.getStrategy_options();
+
+// adding default data center from SimpleSnitch
+if (currentStrategyOptions == null ||
currentStrategyOptions.isEmpty())
+{
+SimpleSnitch snitch = new SimpleSnitch();
+MapString, String options = new HashMapString, String();
+
+try
+{
+
options.put(snitch.getDatacenter(InetAddress.getLocalHost()), 1);
+}
+catch (UnknownHostException e)
+{
+throw new RuntimeException(e.getMessage());
+}
+
+ksDef.setStrategy_options(options);
+}
+}
+
return ksDef;
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java?rev=1058764r1=1058763r2=1058764view=diff
==
---
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981524#action_12981524 ] Nick Bailey commented on CASSANDRA-1898: I think we need to delete the temporarily created sstables if we hit a sorting error. Or do those get cleaned up by something else at some point? json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1960: Fix Version/s: (was: 0.8) 0.7.1 Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.1 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981528#action_12981528 ] Brandon Williams commented on CASSANDRA-1898: - If they have the 'tmp' name, they'll get cleaned at startup. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981532#action_12981532 ] Nick Bailey commented on CASSANDRA-1898: Well I think we should at least log a message that they may want to do it manually. Besides that +1. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1975) Multi NIC/IP support
[ https://issues.apache.org/jira/browse/CASSANDRA-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Sanders updated CASSANDRA-1975: --- Fix Version/s: 0.7.1 Affects Version/s: (was: 0.7.0) Multi NIC/IP support Key: CASSANDRA-1975 URL: https://issues.apache.org/jira/browse/CASSANDRA-1975 Project: Cassandra Issue Type: Improvement Reporter: Max Sanders Fix For: 0.7.1 Every node should listen on more than one IP. When a node is contacted a random IP for that node is chosen. If the node can not be reached by this IP an other IP is used. This way you don't need multipathing switches or Distributed Split Multi-Link Trunking hardware that is very expensive. You could use normal hardware that is inexpensive. If a cable is plugged of or a switch failed it is no problem. Also you can use the second/third network for load balancing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1898: --- Attachment: CASSANDRA-1898-v4.patch Notice for user about deleting SSTables if json was unsorted. json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898-v4.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1984) Implement check and set
Implement check and set - Key: CASSANDRA-1984 URL: https://issues.apache.org/jira/browse/CASSANDRA-1984 Project: Cassandra Issue Type: New Feature Reporter: Max Sanders Fix For: 0.7.1 When fetching data from cassandra you should ask for a hash value for the stored value. Then when updating a value it should be possible to submit this hash value and only update the data when the hash key is still the same. This way you can be sure that the data was not changed on this node. It is the same as the memcached CAS Feature. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981539#action_12981539 ] Hudson commented on CASSANDRA-1960: --- Integrated in Cassandra-0.7 #155 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/155/]) Make NetworkTopologyStrategy the default. Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-1960 Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.1 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1982) When creating snapshots, flush isn't required to finish before the snapshots are created
[ https://issues.apache.org/jira/browse/CASSANDRA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1982. --- Resolution: Invalid if you look at the code, snapshot calls forceBlockingFlush, which does what its name says and blocks for the flush to complete. When creating snapshots, flush isn't required to finish before the snapshots are created Key: CASSANDRA-1982 URL: https://issues.apache.org/jira/browse/CASSANDRA-1982 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6.9, 0.7.0 Reporter: Joaquin Casares Priority: Minor Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1982) When creating snapshots, flush isn't required to finish before the snapshots are created
[ https://issues.apache.org/jira/browse/CASSANDRA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981543#action_12981543 ] Jonathan Ellis commented on CASSANDRA-1982: --- (ColumnFamilyStore.snapshot) When creating snapshots, flush isn't required to finish before the snapshots are created Key: CASSANDRA-1982 URL: https://issues.apache.org/jira/browse/CASSANDRA-1982 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.6.9, 0.7.0 Reporter: Joaquin Casares Priority: Minor Right now ColumnFamilyStore.snapshot() works by doing a flush() first, then do symlinks on the sstables. But flush() is actually submitted through Executor, so by the time the code reaches the symlink part, the flush() may have not happened at all. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981546#action_12981546 ] Jonathan Ellis commented on CASSANDRA-1960: --- If this was tagged 0.7.1 that was an error, we should only commit this to 0.8. Not because it is unsafe but because it changes behavior that shouldn't change in a stable series. Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.7.1 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1975) Multi NIC/IP support
[ https://issues.apache.org/jira/browse/CASSANDRA-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1975: -- Fix Version/s: (was: 0.7.1) Multi NIC/IP support Key: CASSANDRA-1975 URL: https://issues.apache.org/jira/browse/CASSANDRA-1975 Project: Cassandra Issue Type: Improvement Reporter: Max Sanders Every node should listen on more than one IP. When a node is contacted a random IP for that node is chosen. If the node can not be reached by this IP an other IP is used. This way you don't need multipathing switches or Distributed Split Multi-Link Trunking hardware that is very expensive. You could use normal hardware that is inexpensive. If a cable is plugged of or a switch failed it is no problem. Also you can use the second/third network for load balancing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-1984) Implement check and set
[ https://issues.apache.org/jira/browse/CASSANDRA-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-1984. --- Resolution: Won't Fix You can't implement CAS in an eventually consistent system. Implement check and set - Key: CASSANDRA-1984 URL: https://issues.apache.org/jira/browse/CASSANDRA-1984 Project: Cassandra Issue Type: New Feature Reporter: Max Sanders Fix For: 0.7.1 When fetching data from cassandra you should ask for a hash value for the stored value. Then when updating a value it should be possible to submit this hash value and only update the data when the hash key is still the same. This way you can be sure that the data was not changed on this node. It is the same as the memcached CAS Feature. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1058793 - in /cassandra/branches/cassandra-0.7: conf/cassandra.yaml src/java/org/apache/cassandra/cli/CliClient.java src/java/org/apache/cassandra/config/KSMetaData.java
Author: jbellis
Date: Thu Jan 13 23:23:14 2011
New Revision: 1058793
URL: http://svn.apache.org/viewvc?rev=1058793view=rev
Log:
revert r1058764
Modified:
cassandra/branches/cassandra-0.7/conf/cassandra.yaml
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java
Modified: cassandra/branches/cassandra-0.7/conf/cassandra.yaml
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/conf/cassandra.yaml?rev=1058793r1=1058792r2=1058793view=diff
==
--- cassandra/branches/cassandra-0.7/conf/cassandra.yaml (original)
+++ cassandra/branches/cassandra-0.7/conf/cassandra.yaml Thu Jan 13 23:23:14
2011
@@ -309,9 +309,8 @@ index_interval: 128
# With NetworkTopologyStrategy,
# for each datacenter, you can specify how many replicas you want
# on a per-keyspace basis. Replicas are placed on different racks
-# within each DC, if possible. With SimpleSnitch, this will be
-# equivalent to SimpleStrategy. Later, the snitch can be upgraded to
-# one such as RackInferringSnitch or PropertyFileSnitch.
+# within each DC, if possible. This strategy also requires rack aware
+# snitch, such as RackInferringSnitch or PropertyFileSnitch.
# An example:
#- name: Keyspace1
# replica_placement_strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy
@@ -406,7 +405,7 @@ index_interval: 128
# /NOTE
keyspaces:
- name: Keyspace1
- replica_placement_strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy
+ replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy
replication_factor: 1
column_families:
- name: Standard1
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java?rev=1058793r1=1058792r2=1058793view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
Thu Jan 13 23:23:14 2011
@@ -18,8 +18,6 @@
package org.apache.cassandra.cli;
import java.io.IOException;
-import java.net.InetAddress;
-import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.util.*;
@@ -31,7 +29,6 @@ import org.apache.cassandra.config.Confi
import org.apache.cassandra.db.ColumnFamilyStoreMBean;
import org.apache.cassandra.db.CompactionManagerMBean;
import org.apache.cassandra.db.marshal.*;
-import org.apache.cassandra.locator.SimpleSnitch;
import org.apache.cassandra.thrift.*;
import org.apache.cassandra.tools.NodeProbe;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -95,8 +92,7 @@ public class CliClient extends CliUserHe
PLACEMENT_STRATEGY,
STRATEGY_OPTIONS
}
-
-private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.NetworkTopologyStrategy;
+private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.SimpleStrategy;
private Cassandra.Client thriftClient = null;
private CliSessionState sessionState = null;
@@ -807,30 +803,6 @@ public class CliClient extends CliUserHe
}
}
-// using default snitch options if strategy is NetworkTopologyStrategy
and no options were set.
-if (ksDef.getStrategy_class().contains(.NetworkTopologyStrategy))
-{
-MapString, String currentStrategyOptions =
ksDef.getStrategy_options();
-
-// adding default data center from SimpleSnitch
-if (currentStrategyOptions == null ||
currentStrategyOptions.isEmpty())
-{
-SimpleSnitch snitch = new SimpleSnitch();
-MapString, String options = new HashMapString, String();
-
-try
-{
-
options.put(snitch.getDatacenter(InetAddress.getLocalHost()), 1);
-}
-catch (UnknownHostException e)
-{
-throw new RuntimeException(e.getMessage());
-}
-
-ksDef.setStrategy_options(options);
-}
-}
-
return ksDef;
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java?rev=1058793r1=1058792r2=1058793view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/config/KSMetaData.java
(original)
+++
[jira] Updated: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1960: Fix Version/s: (was: 0.7.1) 0.8 Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1971) Let Ivy manage all dependencies and create POM file
[ https://issues.apache.org/jira/browse/CASSANDRA-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Folke Behrens updated CASSANDRA-1971: - Attachment: (was: ivy-all-dep.patch.txt) Let Ivy manage all dependencies and create POM file --- Key: CASSANDRA-1971 URL: https://issues.apache.org/jira/browse/CASSANDRA-1971 Project: Cassandra Issue Type: Improvement Components: Packaging Affects Versions: 0.8 Reporter: Folke Behrens Fix For: 0.8 Attachments: v0.7-ivy-all-dep-v2.patch.txt Attached patch changes the ivy configuration to manage all dependencies. The patch is not complete and still very experimental. * ivy.xml *# Different configurations defined. *# All JARs from /lib/ as dependencies. *# libthrift gets fake org/module for special handling. * ivysettings.xml *# New resolver for dependencies inside the project. *# Module filter for libthrift to use this resolver. * build.xml *# New target: ivy-makepom create a POM file in /build/ next to the .jar and -sources.jar files. *# New target: ivy-retrieve-libs copies dependencies back into /lib/. (For IDE users without Ivy plugin.) Now all JARs except libthrift should be remove from /lib/. Thoughts? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1971) Let Ivy manage all dependencies and create POM file
[ https://issues.apache.org/jira/browse/CASSANDRA-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Folke Behrens updated CASSANDRA-1971: - Attachment: v0.7-ivy-all-dep-v2.patch.txt New patch for 0.7 branch. Ivy now requires a project-local repository in /repo/ with the JARs of libthrift and avro. /lib/ is empty and can be used for copies of all requires JARs. So far, all the most common targets work. Let Ivy manage all dependencies and create POM file --- Key: CASSANDRA-1971 URL: https://issues.apache.org/jira/browse/CASSANDRA-1971 Project: Cassandra Issue Type: Improvement Components: Packaging Affects Versions: 0.8 Reporter: Folke Behrens Fix For: 0.8 Attachments: v0.7-ivy-all-dep-v2.patch.txt Attached patch changes the ivy configuration to manage all dependencies. The patch is not complete and still very experimental. * ivy.xml *# Different configurations defined. *# All JARs from /lib/ as dependencies. *# libthrift gets fake org/module for special handling. * ivysettings.xml *# New resolver for dependencies inside the project. *# Module filter for libthrift to use this resolver. * build.xml *# New target: ivy-makepom create a POM file in /build/ next to the .jar and -sources.jar files. *# New target: ivy-retrieve-libs copies dependencies back into /lib/. (For IDE users without Ivy plugin.) Now all JARs except libthrift should be remove from /lib/. Thoughts? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
svn commit: r1058797 - in /cassandra/trunk: CHANGES.txt conf/cassandra.yaml src/java/org/apache/cassandra/cli/CliClient.java src/java/org/apache/cassandra/config/KSMetaData.java
Author: jbellis
Date: Thu Jan 13 23:38:55 2011
New Revision: 1058797
URL: http://svn.apache.org/viewvc?rev=1058797view=rev
Log:
make NetworkTopologyStrategy the default
patch by Pavel Yaskevich; reviewed by brandonwilliams for CASSANDRA-1960
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/conf/cassandra.yaml
cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/trunk/src/java/org/apache/cassandra/config/KSMetaData.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1058797r1=1058796r2=1058797view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Thu Jan 13 23:38:55 2011
@@ -1,6 +1,7 @@
0.8-dev
* avoid double RowMutation serialization on write path (CASSANDRA-1800)
* adds support for columns that act as incr/decr counters (CASSANDRA-1072)
+ * make NetworkTopologyStrategy the default (CASSANDRA-1960)
0.7.1-dev
Modified: cassandra/trunk/conf/cassandra.yaml
URL:
http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1058797r1=1058796r2=1058797view=diff
==
--- cassandra/trunk/conf/cassandra.yaml (original)
+++ cassandra/trunk/conf/cassandra.yaml Thu Jan 13 23:38:55 2011
@@ -314,8 +314,7 @@ index_interval: 128
# With NetworkTopologyStrategy,
# for each datacenter, you can specify how many replicas you want
# on a per-keyspace basis. Replicas are placed on different racks
-# within each DC, if possible. This strategy also requires rack aware
-# snitch, such as RackInferringSnitch or PropertyFileSnitch.
+# within each DC, if possible.
# An example:
#- name: Keyspace1
# replica_placement_strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy
Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1058797r1=1058796r2=1058797view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Thu Jan 13
23:38:55 2011
@@ -18,6 +18,8 @@
package org.apache.cassandra.cli;
import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.util.*;
@@ -29,6 +31,7 @@ import org.apache.cassandra.config.Confi
import org.apache.cassandra.db.ColumnFamilyStoreMBean;
import org.apache.cassandra.db.CompactionManagerMBean;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.locator.SimpleSnitch;
import org.apache.cassandra.thrift.*;
import org.apache.cassandra.tools.NodeProbe;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -92,7 +95,8 @@ public class CliClient extends CliUserHe
PLACEMENT_STRATEGY,
STRATEGY_OPTIONS
}
-private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.SimpleStrategy;
+
+private static final String DEFAULT_PLACEMENT_STRATEGY =
org.apache.cassandra.locator.NetworkTopologyStrategy;
private Cassandra.Client thriftClient = null;
private CliSessionState sessionState = null;
@@ -803,6 +807,30 @@ public class CliClient extends CliUserHe
}
}
+// using default snitch options if strategy is NetworkTopologyStrategy
and no options were set.
+if (ksDef.getStrategy_class().contains(.NetworkTopologyStrategy))
+{
+MapString, String currentStrategyOptions =
ksDef.getStrategy_options();
+
+// adding default data center from SimpleSnitch
+if (currentStrategyOptions == null ||
currentStrategyOptions.isEmpty())
+{
+SimpleSnitch snitch = new SimpleSnitch();
+MapString, String options = new HashMapString, String();
+
+try
+{
+
options.put(snitch.getDatacenter(InetAddress.getLocalHost()), 1);
+}
+catch (UnknownHostException e)
+{
+throw new RuntimeException(e.getMessage());
+}
+
+ksDef.setStrategy_options(options);
+}
+}
+
return ksDef;
}
Modified: cassandra/trunk/src/java/org/apache/cassandra/config/KSMetaData.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/KSMetaData.java?rev=1058797r1=1058796r2=1058797view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/config/KSMetaData.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/config/KSMetaData.java Thu
Jan
[jira] Commented: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981555#action_12981555 ] Jonathan Ellis commented on CASSANDRA-1960: --- reverted from 0.7 and committed to trunk Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1984) Implement check and set
[ https://issues.apache.org/jira/browse/CASSANDRA-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981568#action_12981568 ] Jonathan Ellis commented on CASSANDRA-1984: --- See: CASSANDRA-48 Implement check and set - Key: CASSANDRA-1984 URL: https://issues.apache.org/jira/browse/CASSANDRA-1984 Project: Cassandra Issue Type: New Feature Reporter: Max Sanders Fix For: 0.7.1 When fetching data from cassandra you should ask for a hash value for the stored value. Then when updating a value it should be possible to submit this hash value and only update the data when the hash key is still the same. This way you can be sure that the data was not changed on this node. It is the same as the memcached CAS Feature. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981570#action_12981570 ] Hudson commented on CASSANDRA-1898: --- Integrated in Cassandra-0.7 #156 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/156/]) Streaming support for json2sstable. Patch by Pavel Yaskevich, reviewed by brandonwilliams for CASSANDRA-1898 json2sstable should support streaming - Key: CASSANDRA-1898 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Nick Bailey Assignee: Pavel Yaskevich Fix For: 0.7.1 Attachments: CASSANDRA-1898-v2.patch, CASSANDRA-1898-v3.patch, CASSANDRA-1898-v4.patch, CASSANDRA-1898.patch Original Estimate: 8h Time Spent: 8h Remaining Estimate: 0h json2sstable loads the entire json file into memory. This is so it can sort the file before creating an sstable. If the file was created using sstable2json and the partitioner isn't changing, this isn't necessary. For very large files this means json2sstable requires a huge amount of memory. There should be an option to stream the file. A simple check for out of order keys will prevent writing bad sstables. This should be possible with the SAX style parser available in our current json library. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (CASSANDRA-1985) read repair on CL.ONE regression
read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kelvin Kakugawa updated CASSANDRA-1985: --- Attachment: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch ensure RR happens in the background. read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1960) Make NetworkTopologyStrategy the default
[ https://issues.apache.org/jira/browse/CASSANDRA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981590#action_12981590 ] Hudson commented on CASSANDRA-1960: --- Integrated in Cassandra #668 (See [https://hudson.apache.org/hudson/job/Cassandra/668/]) make NetworkTopologyStrategy the default patch by Pavel Yaskevich; reviewed by brandonwilliams for CASSANDRA-1960 Make NetworkTopologyStrategy the default Key: CASSANDRA-1960 URL: https://issues.apache.org/jira/browse/CASSANDRA-1960 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 0.8 Attachments: CASSANDRA-1960.patch Original Estimate: 4h Time Spent: 2h Remaining Estimate: 0h Now that NTS doesn't require a sophisticated snitch (e.g., SimpleSnitch assumes everything is in Datacenter 'default' and rack 'default') we should make it the replication strategy out of the box, so adding a new datacenter doesn't require the pain of switching strategies first. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1971) Let Ivy manage all dependencies and create POM file
[ https://issues.apache.org/jira/browse/CASSANDRA-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-1971: -- Reviewer: urandom Let Ivy manage all dependencies and create POM file --- Key: CASSANDRA-1971 URL: https://issues.apache.org/jira/browse/CASSANDRA-1971 Project: Cassandra Issue Type: Improvement Components: Packaging Affects Versions: 0.8 Reporter: Folke Behrens Fix For: 0.8 Attachments: v0.7-ivy-all-dep-v2.patch.txt Attached patch changes the ivy configuration to manage all dependencies. The patch is not complete and still very experimental. * ivy.xml *# Different configurations defined. *# All JARs from /lib/ as dependencies. *# libthrift gets fake org/module for special handling. * ivysettings.xml *# New resolver for dependencies inside the project. *# Module filter for libthrift to use this resolver. * build.xml *# New target: ivy-makepom create a POM file in /build/ next to the .jar and -sources.jar files. *# New target: ivy-retrieve-libs copies dependencies back into /lib/. (For IDE users without Ivy plugin.) Now all JARs except libthrift should be remove from /lib/. Thoughts? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1971) Let Ivy manage all dependencies and create POM file
[ https://issues.apache.org/jira/browse/CASSANDRA-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981594#action_12981594 ] Eric Evans commented on CASSANDRA-1971: --- The reason for the current split, (ivy for build deps, and lib/ for runtime), is that the runtime dependencies are shipped in our binary artifacts, and so need to have licensing and attribution documented according to the respective terms. Since ivy fetches these dynamically from differently administered repositories (none by us), including transitive dependencies, we were reluctant to trust that what we were shipping lined up with hand-crafted documentation. Also, were you aware of CASSANDRA-1851? If I understand correctly, Stephen uploaded a copy of what is found in lib/ to the maven repo under out groupId to get around the above problem. Let Ivy manage all dependencies and create POM file --- Key: CASSANDRA-1971 URL: https://issues.apache.org/jira/browse/CASSANDRA-1971 Project: Cassandra Issue Type: Improvement Components: Packaging Affects Versions: 0.8 Reporter: Folke Behrens Fix For: 0.8 Attachments: v0.7-ivy-all-dep-v2.patch.txt Attached patch changes the ivy configuration to manage all dependencies. The patch is not complete and still very experimental. * ivy.xml *# Different configurations defined. *# All JARs from /lib/ as dependencies. *# libthrift gets fake org/module for special handling. * ivysettings.xml *# New resolver for dependencies inside the project. *# Module filter for libthrift to use this resolver. * build.xml *# New target: ivy-makepom create a POM file in /build/ next to the .jar and -sources.jar files. *# New target: ivy-retrieve-libs copies dependencies back into /lib/. (For IDE users without Ivy plugin.) Now all JARs except libthrift should be remove from /lib/. Thoughts? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1718) cassandra should chdir / when daemonizing
[
https://issues.apache.org/jira/browse/CASSANDRA-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981615#action_12981615
]
paul cannon commented on CASSANDRA-1718:
-XX:ErrorFile ?
cassandra should chdir / when daemonizing
-
Key: CASSANDRA-1718
URL: https://issues.apache.org/jira/browse/CASSANDRA-1718
Project: Cassandra
Issue Type: Bug
Components: Packaging
Environment: Debian squeeze, Cassandra 0.7.0-beta3 and trunk
(r1032649)
Reporter: paul cannon
Assignee: Eric Evans
Priority: Minor
Fix For: 0.7.1
Attachments:
v1-0001-CASSANDRA-1718-switch-to-home-directory-on-startup.txt
Common practice when daemonizing is to cd / to avoid pinning a filesystem.
For example, if the oper happens to start Cassandra (by itself, or with a
manual jsvc invocation, or with the initscript) in /mnt/usb-storage, and
there is something mounted there, then the oper will not be able to unmount
the usb device that was mounted at that location, since the cassandra process
has it open as its cwd.
evidence that this isn't being done already:
{noformat}
~% sudo lsof -p 9775 | awk '$4==cwd'
jsvc9775 cassandra cwdDIR8,1 4096 147675
/home/paul/packages/cassandra/trunk
{noformat}
(That instance was invoked using the Debian initscript.)
Obviously chdir(/) isn't necessary when not daemonizing, although it
shouldn't hurt either.
If there are concerns about Cassandra having an ongoing ability to open
filenames relative to its original working directory, then it should be
sufficient just to do a cd / in the initscript before starting Cassandra.
That case, at least, is particularly important.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1718) cassandra should chdir / when daemonizing
[
https://issues.apache.org/jira/browse/CASSANDRA-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981617#action_12981617
]
Jonathan Ellis commented on CASSANDRA-1718:
---
Bingo.
cassandra should chdir / when daemonizing
-
Key: CASSANDRA-1718
URL: https://issues.apache.org/jira/browse/CASSANDRA-1718
Project: Cassandra
Issue Type: Bug
Components: Packaging
Environment: Debian squeeze, Cassandra 0.7.0-beta3 and trunk
(r1032649)
Reporter: paul cannon
Assignee: Eric Evans
Priority: Minor
Fix For: 0.7.1
Attachments:
v1-0001-CASSANDRA-1718-switch-to-home-directory-on-startup.txt
Common practice when daemonizing is to cd / to avoid pinning a filesystem.
For example, if the oper happens to start Cassandra (by itself, or with a
manual jsvc invocation, or with the initscript) in /mnt/usb-storage, and
there is something mounted there, then the oper will not be able to unmount
the usb device that was mounted at that location, since the cassandra process
has it open as its cwd.
evidence that this isn't being done already:
{noformat}
~% sudo lsof -p 9775 | awk '$4==cwd'
jsvc9775 cassandra cwdDIR8,1 4096 147675
/home/paul/packages/cassandra/trunk
{noformat}
(That instance was invoked using the Debian initscript.)
Obviously chdir(/) isn't necessary when not daemonizing, although it
shouldn't hurt either.
If there are concerns about Cassandra having an ongoing ability to open
filenames relative to its original working directory, then it should be
sufficient just to do a cd / in the initscript before starting Cassandra.
That case, at least, is particularly important.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1972) Default concurrency values are improperly proportioned
[
https://issues.apache.org/jira/browse/CASSANDRA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981625#action_12981625
]
Stu Hood commented on CASSANDRA-1972:
-
Based on IO/seek bound tests on two machines with 4 and 12 drives in RAID-0 and
RAID-10, we saw the best performance with around 20 threads per drive. Not
everyone will be completely IO bound though (hopefully _something_ is in
cache), so perhaps a better recommended value is 16 threads per drive. I'll
write a new blurb for the config file.
Default concurrency values are improperly proportioned
--
Key: CASSANDRA-1972
URL: https://issues.apache.org/jira/browse/CASSANDRA-1972
Project: Cassandra
Issue Type: Bug
Reporter: Stu Hood
Assignee: Stu Hood
Priority: Minor
Fix For: 0.7.1
The default/suggested {{concurrent_reads}} value is much too low. It
assumes that CPU will be the bottleneck, rather than IO, and for most
deployments, this will not be the case. Additionally it is better to be
queued for IO in the kernel or on your device than in user space, because the
former work to optimize queue order.
Additionally, reads are much cheaper than writes in terms of CPU time (since
writes can experience contention due to retries), so while
{{concurrent_writes}} should probably factor in the number of cores on the
machine, {{concurrent_reads}} should probably be calculated purely by number
of spindles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1985) read repair on CL.ONE regression
[ https://issues.apache.org/jira/browse/CASSANDRA-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12981626#action_12981626 ] Jonathan Ellis commented on CASSANDRA-1985: --- It's possible that we missed something, but we we did test read repair post-982. This is the part that does the resolve: if (repairs.contains(command)) repairExecutor.schedule(new RepairRunner(readCallback.resolver, command, endpoints), DatabaseDescriptor.getRpcTimeout(), TimeUnit.MILLISECONDS); read repair on CL.ONE regression Key: CASSANDRA-1985 URL: https://issues.apache.org/jira/browse/CASSANDRA-1985 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0, 0.7.1, 0.8 Reporter: Kelvin Kakugawa Assignee: Kelvin Kakugawa Fix For: 0.7.1, 0.8 Attachments: CASSANDRA-1985-0001-fix-CL.ONE-read-repair-regression.patch read repair w/ CL.ONE had a regression. The RepairCallback was dropped (in the background for CL.ONE), so ReadResponseResolver : resolve() was never called. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1972) Default concurrency values are improperly proportioned
[
https://issues.apache.org/jira/browse/CASSANDRA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-1972:
Attachment: 0001-Recommendation-changes.txt
Default concurrency values are improperly proportioned
--
Key: CASSANDRA-1972
URL: https://issues.apache.org/jira/browse/CASSANDRA-1972
Project: Cassandra
Issue Type: Bug
Reporter: Stu Hood
Assignee: Stu Hood
Priority: Minor
Fix For: 0.7.1
Attachments: 0001-Recommendation-changes.txt
The default/suggested {{concurrent_reads}} value is much too low. It
assumes that CPU will be the bottleneck, rather than IO, and for most
deployments, this will not be the case. Additionally it is better to be
queued for IO in the kernel or on your device than in user space, because the
former work to optimize queue order.
Additionally, reads are much cheaper than writes in terms of CPU time (since
writes can experience contention due to retries), so while
{{concurrent_writes}} should probably factor in the number of cores on the
machine, {{concurrent_reads}} should probably be calculated purely by number
of spindles.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1964) MutationTest of the distributed-test suite fails
[ https://issues.apache.org/jira/browse/CASSANDRA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-1964: Attachment: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt Attaching a fix for the first failing testcase: when doing a write at ONE and a read at ONE, we retry to ensure that the read eventually succeeds. The other two failures are more interesting, and are not bugs in the testcase. Kelvin is investigating, but it appears that reads and writes at ALL are not blocking appropriately on the server side: he'll open a separate ticket for this tomorrow. MutationTest of the distributed-test suite fails Key: CASSANDRA-1964 URL: https://issues.apache.org/jira/browse/CASSANDRA-1964 Project: Cassandra Issue Type: Bug Components: Core Environment: OS X Version 10.6.6 java version 1.6.0_22 Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261) Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode) Apache Ant version 1.8.1 compiled on September 21 2010 Reporter: Michael Allen Assignee: Stu Hood Priority: Minor Fix For: 0.8 Attachments: 0001-Add-a-builder-for-a-retrying-get-attempt-and-use-to-im.txt MutationTest of the distributed-test test suite causes errors on trunk. To reproduce, issue: ant distributed-test -Dwhirr.config=path_to_whirr_config_file from the project root. relevant whirr configuration settings used: whirr.service-name=cassandra whirr.cluster-name=cassandra_test whirr.instance-templates=4 cassandra whirr.version=0.3.0-incubating-SNAPSHOT whirr.location-id=us-west-1 whirr.image-id=us-west-1/ami-16f3a253 whirr.hardware-id=m1.large whirr.blobstore.provider=s3 whirr.blobstore.container=tawamuducassandratests whirr.provider=ec2 whirr.run-url-base=http://hoodidge.net/scripts/ Traceback: distributed-test: [echo] running distributed tests [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/Users/mallen/Desktop/cassandra-trunk/build/lib/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Testsuite: org.apache.cassandra.MovementTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 446.65 sec [junit] [junit] - Standard Error - [junit] SLF4J: Class path contains multiple SLF4J bindings. [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: Found binding in [jar:file:/Users/mallen/Desktop/cassandra-trunk/build/test/lib/jars/whirr-cli-0.3.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit] WARN 12:12:46,654 over limit 471283/262144: wrote temp file [junit] WARN 12:12:48,572 over limit 374979/262144: wrote temp file [junit] WARN 12:12:50,701 over limit 892174/262144: wrote temp file [junit] WARN 12:12:54,442 over limit 612358/262144: wrote temp file [junit] - --- [junit] Testsuite: org.apache.cassandra.MutationTest [junit] Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 110.971 sec [junit] [junit] Testcase: testInsert(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testInsert(MutationTest.java:66) [junit] [junit] [junit] Testcase: testWriteAllReadOne(org.apache.cassandra.MutationTest): Caused an ERROR [junit] null [junit] NotFoundException() [junit] at org.apache.cassandra.thrift.Cassandra$get_result.read(Cassandra.java:6900) [junit] at org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:568) [junit] at org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:541) [junit] at org.apache.cassandra.MutationTest.getColumn(MutationTest.java:210) [junit] at org.apache.cassandra.MutationTest.testWriteAllReadOne(MutationTest.java:87) [junit] [junit] [junit] Testcase:
