[jira] [Created] (CASSANDRA-3097) Missing encryption_options in results in NPE when repair:ing
Missing encryption_options in results in NPE when repair:ing Key: CASSANDRA-3097 URL: https://issues.apache.org/jira/browse/CASSANDRA-3097 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Marcus Eriksson Priority: Minor if encryption_options is not in cassandra.yaml, an NPE is thrown when trying to do nodetool repair. ERROR [AntiEntropyStage:1] 2011-08-29 09:11:05,602 AbstractCassandraDaemon.java (line 134) Fatal exception in thread Thread[AntiEntropyStage:1,5,main] java.lang.RuntimeException: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:796) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:820) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:792) ... 3 more Caused by: java.lang.NullPointerException at org.apache.cassandra.net.MessagingService.stream(MessagingService.java:420) at org.apache.cassandra.streaming.StreamOutSession.begin(StreamOutSession.java:176) at org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:166) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:814) ... 4 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3097) Missing encryption_options in results in NPE when repair:ing
[ https://issues.apache.org/jira/browse/CASSANDRA-3097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-3097: --- Attachment: 3097.txt Missing encryption_options in results in NPE when repair:ing Key: CASSANDRA-3097 URL: https://issues.apache.org/jira/browse/CASSANDRA-3097 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Marcus Eriksson Priority: Minor Attachments: 3097.txt if encryption_options is not in cassandra.yaml, an NPE is thrown when trying to do nodetool repair. ERROR [AntiEntropyStage:1] 2011-08-29 09:11:05,602 AbstractCassandraDaemon.java (line 134) Fatal exception in thread Thread[AntiEntropyStage:1,5,main] java.lang.RuntimeException: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:796) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:820) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:792) ... 3 more Caused by: java.lang.NullPointerException at org.apache.cassandra.net.MessagingService.stream(MessagingService.java:420) at org.apache.cassandra.streaming.StreamOutSession.begin(StreamOutSession.java:176) at org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:166) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:814) ... 4 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3097) Missing encryption_options in results in NPE when repair:ing
[ https://issues.apache.org/jira/browse/CASSANDRA-3097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-3097: --- Attachment: 3097v2.txt doh fixed braces Missing encryption_options in results in NPE when repair:ing Key: CASSANDRA-3097 URL: https://issues.apache.org/jira/browse/CASSANDRA-3097 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Marcus Eriksson Priority: Minor Attachments: 3097.txt, 3097v2.txt if encryption_options is not in cassandra.yaml, an NPE is thrown when trying to do nodetool repair. ERROR [AntiEntropyStage:1] 2011-08-29 09:11:05,602 AbstractCassandraDaemon.java (line 134) Fatal exception in thread Thread[AntiEntropyStage:1,5,main] java.lang.RuntimeException: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:796) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Streaming repair failed. at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:820) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.run(AntiEntropyService.java:792) ... 3 more Caused by: java.lang.NullPointerException at org.apache.cassandra.net.MessagingService.stream(MessagingService.java:420) at org.apache.cassandra.streaming.StreamOutSession.begin(StreamOutSession.java:176) at org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:166) at org.apache.cassandra.service.AntiEntropyService$RepairSession$Differencer.performStreamingRepair(AntiEntropyService.java:814) ... 4 more -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3098) Weird/hex Column Name: formatting with describe keyspaces
Weird/hex Column Name: formatting with describe keyspaces -- Key: CASSANDRA-3098 URL: https://issues.apache.org/jira/browse/CASSANDRA-3098 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Environment: Cassandra 0.8.4 (and today's cassandra-0.8 branch) java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.9) (fedora-54.1.9.9.fc14-x86_64) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Jonas Borgström Displaying a newly created column family with column_metadata displays some kind of hex representation of the column names instead of something more human readable: {code} [default@test] create column family Foo3 with column_metadata = [{ column_name: mycolumn, validation_class: UTF8Type }, { column_name: mycolumn2, validation_class: UTF8Type }]; 4f266c60-d236-11e0--242d50cf1fbf Waiting for schema agreement... ... schemas agree across the cluster [default@test] describe keyspace; Keyspace: test: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true Options: [datacenter1:1] Column Families: ColumnFamily: Foo3 Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.BytesType Row cache size / save period in seconds: 0.0/0 Key cache size / save period in seconds: 20.0/14400 Memtable thresholds: 0.2953125/1440/63 (millions of ops/minutes/MB) GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Built indexes: [] Column Metadata: Column Name: fffcf2 --- I expected this to say 'mycolumn' or 'mycolumn2' Validation Class: org.apache.cassandra.db.marshal.UTF8Type Column Name: --- I expected this to say 'mycolumn' or 'mycolumn2' Validation Class: org.apache.cassandra.db.marshal.UTF8Type {code} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3098) Weird/hex Column Name: formatting with describe keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3098. --- Resolution: Not A Problem column names are formatted based on the CF comparator. default is bytes. Weird/hex Column Name: formatting with describe keyspaces -- Key: CASSANDRA-3098 URL: https://issues.apache.org/jira/browse/CASSANDRA-3098 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.4 Environment: Cassandra 0.8.4 (and today's cassandra-0.8 branch) java version 1.6.0_20 OpenJDK Runtime Environment (IcedTea6 1.9.9) (fedora-54.1.9.9.fc14-x86_64) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Reporter: Jonas Borgström Displaying a newly created column family with column_metadata displays some kind of hex representation of the column names instead of something more human readable: {code} [default@test] create column family Foo3 with column_metadata = [{ column_name: mycolumn, validation_class: UTF8Type }, { column_name: mycolumn2, validation_class: UTF8Type }]; 4f266c60-d236-11e0--242d50cf1fbf Waiting for schema agreement... ... schemas agree across the cluster [default@test] describe keyspace; Keyspace: test: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true Options: [datacenter1:1] Column Families: ColumnFamily: Foo3 Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.BytesType Row cache size / save period in seconds: 0.0/0 Key cache size / save period in seconds: 20.0/14400 Memtable thresholds: 0.2953125/1440/63 (millions of ops/minutes/MB) GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Built indexes: [] Column Metadata: Column Name: fffcf2 --- I expected this to say 'mycolumn' or 'mycolumn2' Validation Class: org.apache.cassandra.db.marshal.UTF8Type Column Name: --- I expected this to say 'mycolumn' or 'mycolumn2' Validation Class: org.apache.cassandra.db.marshal.UTF8Type {code} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3085) Race condition in sstable reference counting
[ https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3085: -- Reviewer: slebresne (was: bcoverston) Race condition in sstable reference counting Key: CASSANDRA-3085 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Critical Fix For: 1.0 Attachments: 3085-v2.txt, 3085.txt DataTracker gives us an atomic View of memtable/sstables, but acquiring references is not atomic. So it is possible to acquire references to an SSTableReader object that is no longer valid, as in this example: View V contains sstables {A, B}. We attempt a read in thread T using this View. Meanwhile, A and B are compacted to {C}, yielding View W. No references exist to A or B so they are cleaned up. Back in thread T we acquire references to A and B. This does not cause an error, but it will when we attempt to read from them next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162775 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/cli/ src/resources/org/apache/cassandra/cli/ test/unit/org/apache/cassandra/cli/
Author: xedin Date: Mon Aug 29 12:53:02 2011 New Revision: 1162775 URL: http://svn.apache.org/viewvc?rev=1162775view=rev Log: Improvements of the CLI `describe` command patch by satishbabu; reviewed by xedin for CASSANDRA-2630 Modified: cassandra/trunk/CHANGES.txt cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java cassandra/trunk/src/java/org/apache/cassandra/cli/CliUtils.java cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162775r1=1162774r2=1162775view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Aug 29 12:53:02 2011 @@ -45,7 +45,7 @@ * Add timeouts to client request schedulers (CASSANDRA-3079) * Cli to use hashes rather than array of hashes for strategy options (CASSANDRA-3081) * LeveledCompactionStrategy (CASSANDRA-1608) - + * Improvements of the CLI `describe` command (CASSANDRA-2630) 0.8.5 * fix NPE when encryption_options is unspecified (CASSANDRA-3007) Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g?rev=1162775r1=1162774r2=1162775view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g (original) +++ cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g Mon Aug 29 12:53:02 2011 @@ -35,7 +35,7 @@ tokens { // various top-level CLI statements. // NODE_CONNECT; -NODE_DESCRIBE_TABLE; +NODE_DESCRIBE; NODE_DESCRIBE_CLUSTER; NODE_USE_TABLE; NODE_EXIT; @@ -180,8 +180,8 @@ helpStatement - ^(NODE_HELP NODE_CONNECT) | HELP USE - ^(NODE_HELP NODE_USE_TABLE) -| HELP DESCRIBE KEYSPACE -- ^(NODE_HELP NODE_DESCRIBE_TABLE) +| HELP DESCRIBE +- ^(NODE_HELP NODE_DESCRIBE) | HELP DESCRIBE 'CLUSTER' - ^(NODE_HELP NODE_DESCRIBE_CLUSTER) | HELP EXIT @@ -366,8 +366,8 @@ showSchema ; describeTable -: DESCRIBE KEYSPACE (keyspace)? -- ^(NODE_DESCRIBE_TABLE (keyspace)?) +: DESCRIBE (keyspace)? +- ^(NODE_DESCRIBE (keyspace)?) ; describeCluster Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1162775r1=1162774r2=1162775view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Mon Aug 29 12:53:02 2011 @@ -250,8 +250,8 @@ public class CliClient case CliParser.NODE_SHOW_SCHEMA: executeShowSchema(tree); break; -case CliParser.NODE_DESCRIBE_TABLE: -executeDescribeKeySpace(tree); +case CliParser.NODE_DESCRIBE: +executeDescribe(tree); break; case CliParser.NODE_DESCRIBE_CLUSTER: executeDescribeCluster(); @@ -1857,92 +1857,10 @@ public class CliClient sessionState.out.println( Column Families:); -boolean isSuper; - Collections.sort(ks_def.cf_defs, new CfDefNamesComparator()); -for (CfDef cf_def : ks_def.cf_defs) -{ -// fetching bean for current column family store -ColumnFamilyStoreMBean cfMBean = (probe == null) ? null : probe.getCfsProxy(ks_def.getName(), cf_def.getName()); -isSuper = cf_def.column_type.equals(Super); -sessionState.out.printf(ColumnFamily: %s%s%n, cf_def.name, isSuper ? (Super) : ); - -if (cf_def.comment != null !cf_def.comment.isEmpty()) -{ -sessionState.out.printf(\%s\%n, cf_def.comment); -} -if (cf_def.key_validation_class != null) -sessionState.out.printf( Key Validation Class: %s%n, cf_def.key_validation_class); -if (cf_def.default_validation_class != null) -sessionState.out.printf( Default column value validator: %s%n, cf_def.default_validation_class); -sessionState.out.printf( Columns sorted by: %s%s%n, cf_def.comparator_type, cf_def.column_type.equals(Super) ? / + cf_def.subcomparator_type : ); -sessionState.out.printf( Row cache size / save period in seconds / keys to save : %s/%s/%s%n, -
[jira] [Commented] (CASSANDRA-2630) CLI - 'describe column family' would be nice
[ https://issues.apache.org/jira/browse/CASSANDRA-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092821#comment-13092821 ] Hudson commented on CASSANDRA-2630: --- Integrated in Cassandra #1053 (See [https://builds.apache.org/job/Cassandra/1053/]) Improvements of the CLI `describe` command patch by satishbabu; reviewed by xedin for CASSANDRA-2630 xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162775 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g * /cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java * /cassandra/trunk/src/java/org/apache/cassandra/cli/CliUtils.java * /cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml * /cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java CLI - 'describe column family' would be nice Key: CASSANDRA-2630 URL: https://issues.apache.org/jira/browse/CASSANDRA-2630 Project: Cassandra Issue Type: Improvement Affects Versions: 0.8.4 Reporter: Jeremy Hanna Assignee: satish babu krishnamoorthy Priority: Minor Labels: cli, lhf Fix For: 1.0 Attachments: cassandra-0.8.2-2630-1.txt, cassandra-0.8.2-2630-2.txt, cassandra-0.8.2-2630-2.txt, cassandra-0.8.2-2630.txt I end up verifying column families a lot and using 'describe keyspace keyspace;' spits out a whole bunch of data since our keyspace has a lot of metadata. It would be really useful to have a 'describe column family;' for a given column family in the currently authenticated keyspace. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3015) Inter-Node Compression
[ https://issues.apache.org/jira/browse/CASSANDRA-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3015: --- Fix Version/s: 1.1 Inter-Node Compression -- Key: CASSANDRA-3015 URL: https://issues.apache.org/jira/browse/CASSANDRA-3015 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.1 Attachments: 0001-stream-based-compression.patch Although we have CASSANDRA-47 for on-disk the code used for streaming actually sends un-compressed blocks over the wire. I propose we compress this data. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2475) Prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2475: -- Fix Version/s: (was: 1.0) 1.1 Assignee: (was: Pavel Yaskevich) Prepared statements --- Key: CASSANDRA-2475 URL: https://issues.apache.org/jira/browse/CASSANDRA-2475 Project: Cassandra Issue Type: Sub-task Components: API, Core Reporter: Eric Evans Labels: cql Fix For: 1.1 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3050) Global row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3050: -- Fix Version/s: (was: 1.0) 1.1 Global row cache Key: CASSANDRA-3050 URL: https://issues.apache.org/jira/browse/CASSANDRA-3050 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.1 Row-cache-per-columnfamily is difficult to configure well as columnfamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2405: -- Fix Version/s: (was: 0.8.5) 1.1 should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.1 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3049) Global key cache
[ https://issues.apache.org/jira/browse/CASSANDRA-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3049: -- Fix Version/s: (was: 1.0) 1.1 Global key cache Key: CASSANDRA-3049 URL: https://issues.apache.org/jira/browse/CASSANDRA-3049 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.1 Key-cache-per-columnfamily is difficult to configure well as columnfamilies are added, similar to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3010) Java CQL command-line shell
[ https://issues.apache.org/jira/browse/CASSANDRA-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3010: -- Priority: Minor (was: Major) Java CQL command-line shell --- Key: CASSANDRA-3010 URL: https://issues.apache.org/jira/browse/CASSANDRA-3010 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.0 We need a real CQL shell that: - does not require installing additional environments - includes show keyspaces and other introspection tools - does not break existing cli scripts I.e., it needs to be java, but it should be a new tool instead of replacing the existing cli. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092834#comment-13092834 ] Gary Dusbabek commented on CASSANDRA-1391: -- Pavel, can you please rebase? Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2731) Impelement in-house file caching.
[ https://issues.apache.org/jira/browse/CASSANDRA-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2731: -- Priority: Minor (was: Major) Affects Version/s: (was: 1.0) Impelement in-house file caching. - Key: CASSANDRA-2731 URL: https://issues.apache.org/jira/browse/CASSANDRA-2731 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Priority: Minor Implement FileCache, CachedRandomAccessFile (to replace BufferedRandomAccessFile) and RadixTree (to play role of the backend cache storage) classes. FileCache class with be responsible for storing/retrieving data from Radix Tree and also flushing of the dirty pages to the disk, page management such as adding new pages, utilizing old/unused pages. CRAF Linux only features (via JNI): 1). O_DIRECT for both read/write operations. 2). AIO's lio_listio write operation batching. Provide possibility to migrate hot data directly from Memtable to CRAF cache to keep live-reads data always hot in memory. To minimise compaction effects CRAF should provide a way to by-pass a caching data if it does not already exists. Provide a way to make pointers in the cache which will be useful to minimize impact on performance when a single column is distributed among multiple SSTable files (except counter columns). Use jemalloc (http://www.canonware.com/jemalloc/) for cache memory management. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1740: -- Fix Version/s: (was: 0.8.5) 1.1 Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.1 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-1391: --- Attachment: CASSANDRA-1391.patch rebased with trunk (last commit 0a4b1667bee674f7c0a22057cbdab97e368a20d1) Allow Concurrent Schema Migrations -- Key: CASSANDRA-1391 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Stu Hood Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-1391.patch, CASSANDRA-1391.patch CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use. It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092845#comment-13092845 ] Jonathan Ellis commented on CASSANDRA-3056: --- I don't think I'm a big fan of adding -env.sh to the list of things that you need to modify before Cassandra will start up. Right now we can tell people paths are all in cassandra.yaml which keeps things simple. Maybe putting it in /tmp would be better from that perspective. Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3095) java.lang.NegativeArraySizeException during compacting large row
[ https://issues.apache.org/jira/browse/CASSANDRA-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3095: -- Attachment: 3095-debug.txt Can you try the attached patch against 0.8 head to get some debug logging about the BF being created? java.lang.NegativeArraySizeException during compacting large row Key: CASSANDRA-3095 URL: https://issues.apache.org/jira/browse/CASSANDRA-3095 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Environment: Linux 2.6.26-2-amd64 #1 SMP Thu Feb 11 00:59:32 UTC 2010 x86_64 GNU/Linux JDK 1.6.0_27 (Java 6 update 27), with JNA. Reporter: Pas Attachments: 3095-debug.txt Hello, It's a 4 node ring, 3 on 0.7.4, I've upgraded one to 0.8.4. This particular node was having issues with compaction that's why I've tried the upgrade (it looks likely that this solved the compaction issues). Here's the stack trace from system.log. INFO [CompactionExecutor:22] 2011-08-28 18:12:46,566 CompactionController.java (line 136) Compacting large row (36028797018963968 bytes) incrementally ERROR [CompactionExecutor:22] 2011-08-28 18:12:46,609 AbstractCassandraDaemon.java (line 134) Fatal exception in thread Thread[CompactionExecutor:22,1,main] java.lang.NegativeArraySizeException at org.apache.cassandra.utils.obs.OpenBitSet.init(OpenBitSet.java:85) at org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:56) at org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:73) at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62) at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50) at org.apache.cassandra.db.compaction.LazilyCompactedRow.init(LazilyCompactedRow.java:89) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:138) at org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:123) at org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:74) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:569) at org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:506) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:141) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:107) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) We've ~70 files still in f format. And 80 in g. We've ~100 GB of data on this node. Thanks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3093) delete multiple rows on secondary equals
[ https://issues.apache.org/jira/browse/CASSANDRA-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3093. --- Resolution: Won't Fix I think it's a good thing to make it explicit that fetch rows matching condition X and delete rows are inherently two separate operations. delete multiple rows on secondary equals Key: CASSANDRA-3093 URL: https://issues.apache.org/jira/browse/CASSANDRA-3093 Project: Cassandra Issue Type: Improvement Reporter: Tongguo Pang For now if we want to delete rows on secondary index equals, we have to read the keys back. This very inefficient, especially when the result set is big(for example, 1,000,000 rows). If the delete operation can accept a secondary index query as input, that will be very helpful for this situation -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092853#comment-13092853 ] Jonathan Ellis commented on CASSANDRA-3091: --- I feel like the cure might be worse than the disease here. Schema typically only changes underneath a connection during development -- if you're changing schema in production, you're going to redeploy your app to take advantage of that, creating new connections. So the benefit feels small vs the cost of re-requesting and parsing metadata for each Statement. Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092858#comment-13092858 ] Jonathan Ellis commented on CASSANDRA-3078: --- bq. require specifying the classname as a index_option I like that approach too. Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-pluggable-secondary-indexes.txt, v1-0002-CASSANDRA-3078-pluggable-secondary-index-thrift.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3099) Counters are not always hinted
[ https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3099: Attachment: 0001-hint-counters.patch Counters are not always hinted -- Key: CASSANDRA-3099 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.5 Attachments: 0001-hint-counters.patch CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints that were supposed to be stored on the local node (that is, instead of removing from the hintedEndpoints multimap only the local write (since it has been already applied), we were removing everything having the local node as destination). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3099) Counters are not always hinted
Counters are not always hinted -- Key: CASSANDRA-3099 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.5 Attachments: 0001-hint-counters.patch CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints that were supposed to be stored on the local node (that is, instead of removing from the hintedEndpoints multimap only the local write (since it has been already applied), we were removing everything having the local node as destination). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092862#comment-13092862 ] Rick Shaw commented on CASSANDRA-3091: -- It is cached in the statement as well so it will do the fetch one time when the statement is first created and then just check the cache to make sure the metadata is there and attempt a reload only if it does not find it. Once you acquire a statement you usually hold on to it for the duration of your task. So not significantly more than if you acquired a connection. Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3099) Counters are not always hinted
[ https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092868#comment-13092868 ] Jonathan Ellis commented on CASSANDRA-3099: --- +1 Counters are not always hinted -- Key: CASSANDRA-3099 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.5 Attachments: 0001-hint-counters.patch CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints that were supposed to be stored on the local node (that is, instead of removing from the hintedEndpoints multimap only the local write (since it has been already applied), we were removing everything having the local node as destination). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092869#comment-13092869 ] Rick Shaw commented on CASSANDRA-3091: -- In JDBC you can pool both! Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092873#comment-13092873 ] Jonathan Ellis commented on CASSANDRA-3091: --- I suppose, but if you're pooling statements, how is making metadata per-statement a win? Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092877#comment-13092877 ] Rick Shaw commented on CASSANDRA-3091: -- This approach puts the caller in better control of when the refreshed data is acquired. It is easy to close one statement and reacquire. If you do with a pooled resource it will not reacquire until the physical connection goes stale from no use. On a busy system and when the app acquires the connection itself infrequently, it may be a long time before you see a new refreshed set of data in a new connection. Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2942) If you drop a CF when one node is down the files are orphaned on the downed node
[ https://issues.apache.org/jira/browse/CASSANDRA-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092884#comment-13092884 ] Sylvain Lebresne commented on CASSANDRA-2942: - nit: we could log an info message when awaitTermination returns false. It also look like DeletionService could just go away with with this. But otherwise, +1. I agree it is good enough and not worth going for more complicated. If you drop a CF when one node is down the files are orphaned on the downed node Key: CASSANDRA-2942 URL: https://issues.apache.org/jira/browse/CASSANDRA-2942 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Cathy Daw Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0 Attachments: 2942.txt * Bring up 3 node cluster * From node1: Run Stress Tool {code} stress --num-keys=10 --columns=10 --consistency-level=ALL --average-size-values --replication-factor=3 --nodes=node1,node2 {code} * Shutdown node3 * From node1: drop the Standard1 CF in Keyspace1 * Shutdown node2 and node3 * Bring up node1 and node2. Check that the Standard1 files are gone. {code} ls -al /var/lib/cassandra/data/Keyspace1/ {code} * Bring up node3. The log file shows the drop column family occurs {code} INFO 00:51:25,742 Applying migration 9a76f880-b4c5-11e0--8901a7c5c9ce Drop column family: Keyspace1.Standard1 {code} * Restart node3 to clear out dropped tables from the filesystem {code} root@cathy3:~/cass-0.8/bin# ls -al /var/lib/cassandra/data/Keyspace1/ total 36 drwxr-xr-x 3 root root 4096 Jul 23 00:51 . drwxr-xr-x 6 root root 4096 Jul 23 00:48 .. -rw-r--r-- 1 root root0 Jul 23 00:51 Standard1-g-1-Compacted -rw-r--r-- 2 root root 5770 Jul 23 00:51 Standard1-g-1-Data.db -rw-r--r-- 2 root root 32 Jul 23 00:51 Standard1-g-1-Filter.db -rw-r--r-- 2 root root 120 Jul 23 00:51 Standard1-g-1-Index.db -rw-r--r-- 2 root root 4276 Jul 23 00:51 Standard1-g-1-Statistics.db drwxr-xr-x 3 root root 4096 Jul 23 00:51 snapshots {code} *Bug: The files for Standard1 are orphaned on node3* -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092883#comment-13092883 ] Jonathan Ellis commented on CASSANDRA-3025: --- Thanks! bq. Parsing into object would be problematic if the UUID is used as a column name Why is that? Does PDO require only primitive or string column names? PHP/PDO driver for Cassandra CQL Key: CASSANDRA-3025 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025 Project: Cassandra Issue Type: New Feature Components: API Reporter: Mikko Koppanen Labels: php Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, php_test_results_20110818_2317.txt Hello, attached is the initial version of the PDO driver for Cassandra CQL language. This is a native PHP extension written in what I would call a combination of C and C++, due to PHP being C. The thrift API used is the C++. The API looks roughly following: {code} ?php $db = new PDO('cassandra:host=127.0.0.1;port=9160'); $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor=1;); $db-exec (USE mytest); $db-exec (CREATE COLUMNFAMILY users ( my_key varchar PRIMARY KEY, full_name varchar );); $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, :full_name);); $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' )); {code} Currently prepared statements are emulated on the client side but I understand that there is a plan to add prepared statements to Cassandra CQL API as well. I will add this feature in to the extension as soon as they are implemented. Additional documentation can be found in github https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered MarkDown file. Tests are currently not included in the package file and they can be found in the github for now as well. I have created documentation in docbook format as well, but have not yet rendered it. Comments and feedback are welcome. Thanks, Mikko -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3061) Optionally skip log4j configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092886#comment-13092886 ] T Jake Luciani commented on CASSANDRA-3061: --- Aaron any update here? this is working fine for me. Optionally skip log4j configuration --- Key: CASSANDRA-3061 URL: https://issues.apache.org/jira/browse/CASSANDRA-3061 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.4 Reporter: Aaron Morton Assignee: Aaron Morton Priority: Minor Fix For: 0.8.5 Attachments: 0001-3061.patch, 3061_v2.txt from this thread http://groups.google.com/group/brisk-users/browse_thread/thread/3a18f4679673bea8 When brisk accesses cassandra classes inside of a Hadoop Task JVM the AbstractCassandraDaemon uses a log4j PropertyConfigurator to setup cassandra logging. This closes all the existing appenders, including the TaskLogAppender for the hadoop task. They are not opened again because they are not in the config. log4j has Logger Repositories to handle multiple configs in the same process, but there is a bit of suck involved in making a RepositorySelector. Two examples... http://www.mail-archive.com/log4j-dev@jakarta.apache.org/msg02972.html http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2/html/Getting_Started_Guide/logging.log4j.reposelect.html Basically all the selector has access to thread local storage, and it looks like normally people get the class loader from the current thread. A thread will inherit it's class loader from the thread that created it, unless otherwise specified. We have code in the same thread the uses hadoop and cassandra classes, so this could be a dead end. As a work around i've added cassandra.log4j.configure JVM param and made the AbstractCassandraServer skip the log4j config if it's false. My job completes and I can see the cassandra code logging an extra message I put in into the Hadoop task log file... 2011-08-19 15:56:06,442 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-maptask.properties, hadoop-metrics2.properties 2011-08-19 15:56:06,776 INFO org.apache.cassandra.service.AbstractCassandraDaemon: Logging initialized externally 2011-08-19 15:56:07,332 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 The param has to be passed to the task JVM, so need to modify Haddop mapred-site.xml as follows property namemapred.child.java.opts/name value-Xmx256m -Dcassandra.log4j.configure=false/value description Tune your mapred jvm arguments for best performance. Also see documentation from jvm vendor. /description /property It's not pretty but it works. In my extra log4j logging I can see the second reset() call is gone. Change the to Hadoop TaskLogAppender also stops the NPE but there may also be some lost log messages https://issues.apache.org/jira/browse/HADOOP-7556 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092887#comment-13092887 ] Jonathan Ellis commented on CASSANDRA-3091: --- What would that look like with statement pooling? Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092885#comment-13092885 ] Jonathan Ellis commented on CASSANDRA-3025: --- Still to-do: bq. should test an actual big int ( 64bit) anything else? PHP/PDO driver for Cassandra CQL Key: CASSANDRA-3025 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025 Project: Cassandra Issue Type: New Feature Components: API Reporter: Mikko Koppanen Labels: php Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, php_test_results_20110818_2317.txt Hello, attached is the initial version of the PDO driver for Cassandra CQL language. This is a native PHP extension written in what I would call a combination of C and C++, due to PHP being C. The thrift API used is the C++. The API looks roughly following: {code} ?php $db = new PDO('cassandra:host=127.0.0.1;port=9160'); $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor=1;); $db-exec (USE mytest); $db-exec (CREATE COLUMNFAMILY users ( my_key varchar PRIMARY KEY, full_name varchar );); $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, :full_name);); $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' )); {code} Currently prepared statements are emulated on the client side but I understand that there is a plan to add prepared statements to Cassandra CQL API as well. I will add this feature in to the extension as soon as they are implemented. Additional documentation can be found in github https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered MarkDown file. Tests are currently not included in the package file and they can be found in the github for now as well. I have created documentation in docbook format as well, but have not yet rendered it. Comments and feedback are welcome. Thanks, Mikko -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2857) initialize log4j correctly in EmbeddedCassandraService
[ https://issues.apache.org/jira/browse/CASSANDRA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092890#comment-13092890 ] T Jake Luciani commented on CASSANDRA-2857: --- needs rebase but +1 initialize log4j correctly in EmbeddedCassandraService -- Key: CASSANDRA-2857 URL: https://issues.apache.org/jira/browse/CASSANDRA-2857 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Trivial Fix For: 0.8.5 Attachments: 2857-drivers.txt, 2857.txt Currently, ECS.cleanUpOldStuff calls CleanupHelper.cleanupAndLeaveDirs(), which initialized DatabaseDescriptor which does some logging. When we go to initialize log4j later in AbstractCassandraService, it's too late. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162844 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/service/StorageProxy.java
Author: slebresne Date: Mon Aug 29 15:04:57 2011 New Revision: 1162844 URL: http://svn.apache.org/viewvc?rev=1162844view=rev Log: Always hint counters patch by slebresne; reviewed by jbellis for CASSANDRA-3099 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1162844r1=1162843r2=1162844view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Aug 29 15:04:57 2011 @@ -37,6 +37,7 @@ * fix UnavailableException with writes at CL.EACH_QUORM (CASSANDRA-3084) * fix parsing of the Keyspace and ColumnFamily names in numeric and string representations in CLI (CASSANDRA-3075) + * always hint counters (CASSANDRA-3099) 0.8.4 * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972) Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1162844r1=1162843r2=1162844view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java Mon Aug 29 15:04:57 2011 @@ -446,7 +446,8 @@ public class StorageProxy implements Sto responseHandler.response(null); // then send to replicas, if any -hintedEndpoints.removeAll(FBUtilities.getLocalAddress()); +InetAddress local = FBUtilities.getLocalAddress(); +hintedEndpoints.remove(local, local); if (cm.shouldReplicateOnWrite() !hintedEndpoints.isEmpty()) { // We do the replication on another stage because it involves a read (see CM.makeReplicationMutation)
[jira] [Updated] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3051: --- Attachment: CASSANDRA-3051.patch Removed SSLFileStreamTask and added EncryptionOptions to the constructor of the FileStreamTask. Rebased with latest trunk (last commit 0a4b1667bee674f7c0a22057cbdab97e368a20d1) On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162846 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/
Author: slebresne Date: Mon Aug 29 15:07:04 2011 New Revision: 1162846 URL: http://svn.apache.org/viewvc?rev=1162846view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:07:04 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1160444,1160825,1161607 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162846r1=1162845r2=1162846view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:07:04 2011 @@ -78,6 +78,7 @@ * avoid including cwd in classpath for deb and rpm packages (CASSANDRA-2881) * remove gossip state when a new IP takes over a token (CASSANDRA-3071) * allow sstable2json to work on index sstable files (CASSANDRA-3059) + * always hint counters (CASSANDRA-3099) 0.8.4 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:07:04 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1160444,1160825,1161607 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:07:04 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1160444,1160825,1161607 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1161708 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1161708,1162844 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:07:04 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1160444,1160825,1161607
[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901 ] Jeremy Hanna commented on CASSANDRA-3056: - Do people generally configure their systems to have enough space in /tmp for the whole Cassandra heap? Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162849 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/db/commitlog/ src/java/org/apache/cassandra/io/ src/java/org/apache/cassandra/io/util/ src/java/org/apache/cassandra/servic
Author: jbellis Date: Mon Aug 29 15:15:32 2011 New Revision: 1162849 URL: http://svn.apache.org/viewvc?rev=1162849view=rev Log: reduce window where dropped CF sstables may not be deleted patch by jbellis; reviewed by slebresne for CASSANDRA-2942 Modified: cassandra/trunk/CHANGES.txt cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java cassandra/trunk/src/java/org/apache/cassandra/io/DeletionService.java cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162849r1=1162848r2=1162849view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:15:32 2011 @@ -46,6 +46,8 @@ * Cli to use hashes rather than array of hashes for strategy options (CASSANDRA-3081) * LeveledCompactionStrategy (CASSANDRA-1608) * Improvements of the CLI `describe` command (CASSANDRA-2630) + * reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942) + 0.8.5 * fix NPE when encryption_options is unspecified (CASSANDRA-3007) Modified: cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java?rev=1162849r1=1162848r2=1162849view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java Mon Aug 29 15:15:32 2011 @@ -42,7 +42,6 @@ import org.apache.cassandra.concurrent.S import org.apache.cassandra.concurrent.StageManager; import org.apache.cassandra.config.Config; import org.apache.cassandra.config.DatabaseDescriptor; -import org.apache.cassandra.io.DeletionService; import org.apache.cassandra.io.util.FastByteArrayInputStream; import org.apache.cassandra.utils.ByteBufferUtil; import org.apache.cassandra.utils.FBUtilities; @@ -486,7 +485,7 @@ public class CommitLog implements Commit { logger.info(Discarding obsolete commit log: + segment); segment.close(); -DeletionService.executeDelete(segment.getPath()); +FileUtils.deleteAsync(segment.getPath()); // usually this will be the first (remaining) segment, but not always, if segment A contains // writes to a CF that is unflushed but is followed by segment B whose CFs are all flushed. iter.remove(); Modified: cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java?rev=1162849r1=1162848r2=1162849view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java Mon Aug 29 15:15:32 2011 @@ -20,13 +20,15 @@ package org.apache.cassandra.io.util; import java.io.*; import java.text.DecimalFormat; -import java.util.Collection; import java.util.Comparator; import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.utils.WrappedRunnable; + public class FileUtils { @@ -167,6 +169,18 @@ public class FileUtils } } +public static void deleteAsync(final String file) +{ +Runnable runnable = new WrappedRunnable() +{ +protected void runMayThrow() throws IOException +{ +deleteWithConfirm(new File(file)); +} +}; +StorageService.tasks.execute(runnable); +} + public static String stringifyFileSize(double value) { double d; Modified: cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1162849r1=1162848r2=1162849view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java Mon Aug 29 15:15:32 2011 @@ -46,7 +46,6 @@ import org.apache.cassandra.db.*; import org.apache.cassandra.db.commitlog.CommitLog; import org.apache.cassandra.dht.*; import org.apache.cassandra.gms.*; -import org.apache.cassandra.io.DeletionService; import org.apache.cassandra.io.sstable.SSTableDeletingTask; import org.apache.cassandra.io.sstable.SSTableLoader; import
[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement
[ https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092904#comment-13092904 ] Rick Shaw commented on CASSANDRA-3091: -- I think (in retrospect) I would clear the cache upon close for either approach so it would force a reload on acquisition, so that may be a red herring. I think the only issue is: is doing it in the statement any more expensive in time vs the advantage of a more current set of metadata? If the general usage pattern is to open a lot of statements vs opening connections, then it is undeniably more time intensive to do in statement but the benefit is you would SELDOM have mismatch with the state of the server. If the average use is to open a connection and then open a statement and do all accesses on the statement then its not really a penalty in time and the additional ability to do a reload on demand when needed in the statement (at the price of the boolean check for CF metadata) is a win. If you think this is ill advised having said all that :) we can close the ticket, and I'll put some of the additional cleanup of the {{ColumnDecoder}} in another related patch at a later date. I could also look to see if I could get the check for missing metadata against the data hanging off connection, but that's along way from {{ColumnDecoder}} where the info is needed. Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement - Key: CASSANDRA-3091 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091 Project: Cassandra Issue Type: Improvement Components: Drivers Affects Versions: 0.8.4 Reporter: Rick Shaw Assignee: Rick Shaw Priority: Minor Labels: JDBC Fix For: 0.8.5 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, move-metadata-for-decoder-to-statement-level-v2.txt Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is loaded and held in the {{CassandraConnection}} class. The implication of this is that any activity on the connected server from the time the connection is established is not reflected in the KSs and CF that can be accessed by the {{ResultSet, Statement}} and {{PreparedStatement}}. By moving the cached metadata to the {{Statement}} level, the currency of the metadata can be checked within the {{Statement}} and reloaded if it is seen to be absent. And by instantiating a new {{Statement}} (on any existing connection) you are assured of getting the most current copy of the metadata known to the server at the new time of instantiation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162851 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java src/java/org/apache/cassandra/thrift/CassandraDaemon.java
Author: jbellis Date: Mon Aug 29 15:19:16 2011 New Revision: 1162851 URL: http://svn.apache.org/viewvc?rev=1162851view=rev Log: fix log4j initialization in EmbeddedCassandraService patch by jbellis; reviewed by tjake for CASSANDRA-2857 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1162851r1=1162850r2=1162851view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Aug 29 15:19:16 2011 @@ -38,6 +38,8 @@ * fix parsing of the Keyspace and ColumnFamily names in numeric and string representations in CLI (CASSANDRA-3075) * always hint counters (CASSANDRA-3099) + * fix log4j initialization in EmbeddedCassandraService (CASSANDRA-2857) + 0.8.4 * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972) Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java?rev=1162851r1=1162850r2=1162851view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java Mon Aug 29 15:19:16 2011 @@ -62,17 +62,19 @@ import org.apache.cassandra.utils.Mx4jTo */ public abstract class AbstractCassandraDaemon implements CassandraDaemon { -//Initialize logging in such a way that it checks for config changes every 10 seconds. -static +/** + * Initialize logging in such a way that it checks for config changes every 10 seconds. + */ +public static void initLog4j() { String config = System.getProperty(log4j.configuration, log4j-server.properties); URL configLocation = null; -try +try { // try loading from a physical location first. configLocation = new URL(config); } -catch (MalformedURLException ex) +catch (MalformedURLException ex) { // then try loading from the classpath. configLocation = AbstractCassandraDaemon.class.getClassLoader().getResource(config); Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java?rev=1162851r1=1162850r2=1162851view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java Mon Aug 29 15:19:16 2011 @@ -26,6 +26,7 @@ import java.util.concurrent.ExecutorServ import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; +import org.apache.cassandra.service.AbstractCassandraDaemon; import org.apache.thrift.server.TNonblockingServer; import org.apache.thrift.server.TThreadPoolServer; import org.slf4j.Logger; @@ -53,6 +54,11 @@ import org.apache.thrift.transport.TTran public class CassandraDaemon extends org.apache.cassandra.service.AbstractCassandraDaemon { +static +{ +AbstractCassandraDaemon.initLog4j(); +} + private static Logger logger = LoggerFactory.getLogger(CassandraDaemon.class); private final static String SYNC = sync; private final static String ASYNC = async;
svn commit: r1162860 - in /cassandra/trunk: ./ contrib/ drivers/java/test/org/apache/cassandra/cql/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/ src/ja
Author: jbellis Date: Mon Aug 29 15:24:08 2011 New Revision: 1162860 URL: http://svn.apache.org/viewvc?rev=1162860view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraDaemon.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:24:08 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1160444,1160825,1161607 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844,1162851 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162860r1=1162859r2=1162860view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:24:08 2011 @@ -81,6 +81,7 @@ * remove gossip state when a new IP takes over a token (CASSANDRA-3071) * allow sstable2json to work on index sstable files (CASSANDRA-3059) * always hint counters (CASSANDRA-3099) + * fix log4j initialization in EmbeddedCassandraService (CASSANDRA-2857) 0.8.4 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:24:08 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1160444,1160825,1161607 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844,1162851 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Modified: cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java URL: http://svn.apache.org/viewvc/cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java?rev=1162860r1=1162859r2=1162860view=diff == --- cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java (original) +++ cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java Mon Aug 29 15:24:08 2011 @@ -28,6 +28,7 @@ import java.net.UnknownHostException; import org.apache.cassandra.CleanupHelper; import org.apache.cassandra.SchemaLoader; import org.apache.cassandra.config.*; +import org.apache.cassandra.service.AbstractCassandraDaemon; import org.apache.cassandra.service.EmbeddedCassandraService; import org.junit.BeforeClass; @@ -39,7 +40,12 @@ public abstract class EmbeddedServiceBas /** The embedded server cassandra. */ private static EmbeddedCassandraService cassandra; -@BeforeClass +static +{ +AbstractCassandraDaemon.initLog4j(); +} + +@BeforeClass public static void cleanUpOldStuff() throws IOException { CleanupHelper.cleanupAndLeaveDirs(); Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 29 15:24:08 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092915#comment-13092915 ] Pavel Yaskevich commented on CASSANDRA-3051: I don't see anh reason why it won't :) but can't write a test because SSL is treacky with it's stores... On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2034) Make Read Repair unnecessary when Hinted Handoff is enabled
[ https://issues.apache.org/jira/browse/CASSANDRA-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2034: -- Attachment: 2034-v19-rebased.txt rebased v19. no other changes made. Make Read Repair unnecessary when Hinted Handoff is enabled --- Key: CASSANDRA-2034 URL: https://issues.apache.org/jira/browse/CASSANDRA-2034 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Patricio Echague Fix For: 1.0 Attachments: 2034-formatting.txt, 2034-v16.txt, 2034-v17.txt, 2034-v18.txt, 2034-v19-rebased.txt, 2034-v19.txt, CASSANDRA-2034-trunk-v10.patch, CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v12.patch, CASSANDRA-2034-trunk-v13.patch, CASSANDRA-2034-trunk-v14.patch, CASSANDRA-2034-trunk-v15.patch, CASSANDRA-2034-trunk-v2.patch, CASSANDRA-2034-trunk-v3.patch, CASSANDRA-2034-trunk-v4.patch, CASSANDRA-2034-trunk-v5.patch, CASSANDRA-2034-trunk-v6.patch, CASSANDRA-2034-trunk-v7.patch, CASSANDRA-2034-trunk-v8.patch, CASSANDRA-2034-trunk-v9.patch, CASSANDRA-2034-trunk.patch Original Estimate: 8h Remaining Estimate: 8h Currently, HH is purely an optimization -- if a machine goes down, enabling HH means RR/AES will have less work to do, but you can't disable RR entirely in most situations since HH doesn't kick in until the FailureDetector does. Let's add a scheduled task to the mutate path, such that we return to the client normally after ConsistencyLevel is achieved, but after RpcTimeout we check the responseHandler write acks and write local hints for any missing targets. This would making disabling RR when HH is enabled a much more reasonable option, which has a huge impact on read throughput. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3100) Secondary index still does minor compacting after deleting index
Secondary index still does minor compacting after deleting index Key: CASSANDRA-3100 URL: https://issues.apache.org/jira/browse/CASSANDRA-3100 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.8 Reporter: Jeremy Hanna We deleted all of our secondary indexes. A couple of days later I was watching compactionstats on one of the nodes and it was in the process of minor compacting one of the deleted secondary indexes. I double checked the keyspace definitions on the CLI and there were no secondary indexes defined. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092921#comment-13092921 ] Gary Dusbabek commented on CASSANDRA-3051: -- wait, you tested it locally first, right? It's not difficult to set up a streaming environment locally. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092924#comment-13092924 ] Jonathan Ellis commented on CASSANDRA-3051: --- do we have a ssl howto somewhere? I was hoping it would be in cassandra.yaml by encryption_options but no. Or at least, not sufficiently for dummies for me. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092929#comment-13092929 ] Pavel Yaskevich commented on CASSANDRA-3051: No, unfortunately I haven't found any info about how to do that so you are welcome to test if you can... On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting
[ https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092928#comment-13092928 ] Sylvain Lebresne commented on CASSANDRA-3085: - * The try of the try...finally block should start just after the markReferenced to be safe in CollationControler. * In CFS.getRangeSlice, I am not confortable using the same try...finally block for both the view and the iterator. RowIteratorFactory.getIterator() does make seeks and could throw an error that would leave a bunch of sstable referenced. * Nit: Maybe we could use a plain old DataTracker.View instead of introducing ViewFragment, since the View constructor is accessible to CFS anyway ? Race condition in sstable reference counting Key: CASSANDRA-3085 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Critical Fix For: 1.0 Attachments: 3085-v2.txt, 3085.txt DataTracker gives us an atomic View of memtable/sstables, but acquiring references is not atomic. So it is possible to acquire references to an SSTableReader object that is no longer valid, as in this example: View V contains sstables {A, B}. We attempt a read in thread T using this View. Meanwhile, A and B are compacted to {C}, yielding View W. No references exist to A or B so they are cleaned up. Back in thread T we acquire references to A and B. This does not cause an error, but it will when we attempt to read from them next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092930#comment-13092930 ] Gary Dusbabek commented on CASSANDRA-3051: -- Not that I know of. If someone wants to write one it would flesh out these basic steps: # follow the steps for generating a keystore and a trust store here: http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore # plug those files into encryption_options in cassandra.yaml # make sure encryption_options.internode_encryption = all in the yaml. I was mostly raising a voice of caution against committing code backed up by statements like I don't see anh reason why it won't... This is usually a prelude to something like we need to quickly release a new version because XYZ broke streaming. Just sayin'. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish babu krishnamoorthy updated CASSANDRA-3056: -- Attachment: CASSANDRA-3056-2.txt Updated the patch to set HeapDump in cassandra-env.sh, this value is parsed in the following order. 1) look for jvm_heap_dump_directory in cassandra.yaml 2) if jvm_heap_dump_directory is not set in cassandra.yaml, use /tmp Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting
[ https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092949#comment-13092949 ] Sylvain Lebresne commented on CASSANDRA-3085: - I would argue that even now a View is not meant to be updated either (we pick new Views but a View itself is fixed), nor does it really ensure that it contains the full set of sstables in a way since that set is a moving target. But I suppose this is just a matter of perspective on what View is, and I'm fine with ViewFragment. Race condition in sstable reference counting Key: CASSANDRA-3085 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Critical Fix For: 1.0 Attachments: 3085-v2.txt, 3085.txt DataTracker gives us an atomic View of memtable/sstables, but acquiring references is not atomic. So it is possible to acquire references to an SSTableReader object that is no longer valid, as in this example: View V contains sstables {A, B}. We attempt a read in thread T using this View. Meanwhile, A and B are compacted to {C}, yielding View W. No references exist to A or B so they are cleaned up. Back in thread T we acquire references to A and B. This does not cause an error, but it will when we attempt to read from them next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092951#comment-13092951 ] Pavel Yaskevich commented on CASSANDRA-3051: Following your logic - person who was working on ssl should've done that at first place, there is no guarantee that it works in the current state. I'm not pushing things forward just wondering why testing wasn't done before. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901 ] Jeremy Hanna edited comment on CASSANDRA-3056 at 8/29/11 4:20 PM: -- Do people generally configure their systems to have enough space in /tmp for the whole Cassandra heap? I guess /tmp is generally the best place though. was (Author: jeromatron): Do people generally configure their systems to have enough space in /tmp for the whole Cassandra heap? Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901 ] Jeremy Hanna edited comment on CASSANDRA-3056 at 8/29/11 4:21 PM: -- Do people generally configure their systems to have enough space in /tmp for the whole Cassandra heap? What about the cassandra log directory as a default? /tmp seems fine but this bit us because the heap dump is so big. was (Author: jeromatron): Do people generally configure their systems to have enough space in /tmp for the whole Cassandra heap? I guess /tmp is generally the best place though. Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3099) Counters are not always hinted
[ https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092953#comment-13092953 ] Hudson commented on CASSANDRA-3099: --- Integrated in Cassandra-0.8 #298 (See [https://builds.apache.org/job/Cassandra-0.8/298/]) Always hint counters patch by slebresne; reviewed by jbellis for CASSANDRA-3099 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162844 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java Counters are not always hinted -- Key: CASSANDRA-3099 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 0.8.5 Attachments: 0001-hint-counters.patch CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints that were supposed to be stored on the local node (that is, instead of removing from the hintedEndpoints multimap only the local write (since it has been already applied), we were removing everything having the local node as destination). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2942) If you drop a CF when one node is down the files are orphaned on the downed node
[ https://issues.apache.org/jira/browse/CASSANDRA-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092955#comment-13092955 ] Hudson commented on CASSANDRA-2942: --- Integrated in Cassandra #1054 (See [https://builds.apache.org/job/Cassandra/1054/]) reduce window where dropped CF sstables may not be deleted patch by jbellis; reviewed by slebresne for CASSANDRA-2942 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162849 Files : * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java * /cassandra/trunk/src/java/org/apache/cassandra/io/DeletionService.java * /cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java * /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java If you drop a CF when one node is down the files are orphaned on the downed node Key: CASSANDRA-2942 URL: https://issues.apache.org/jira/browse/CASSANDRA-2942 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Cathy Daw Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0 Attachments: 2942.txt * Bring up 3 node cluster * From node1: Run Stress Tool {code} stress --num-keys=10 --columns=10 --consistency-level=ALL --average-size-values --replication-factor=3 --nodes=node1,node2 {code} * Shutdown node3 * From node1: drop the Standard1 CF in Keyspace1 * Shutdown node2 and node3 * Bring up node1 and node2. Check that the Standard1 files are gone. {code} ls -al /var/lib/cassandra/data/Keyspace1/ {code} * Bring up node3. The log file shows the drop column family occurs {code} INFO 00:51:25,742 Applying migration 9a76f880-b4c5-11e0--8901a7c5c9ce Drop column family: Keyspace1.Standard1 {code} * Restart node3 to clear out dropped tables from the filesystem {code} root@cathy3:~/cass-0.8/bin# ls -al /var/lib/cassandra/data/Keyspace1/ total 36 drwxr-xr-x 3 root root 4096 Jul 23 00:51 . drwxr-xr-x 6 root root 4096 Jul 23 00:48 .. -rw-r--r-- 1 root root0 Jul 23 00:51 Standard1-g-1-Compacted -rw-r--r-- 2 root root 5770 Jul 23 00:51 Standard1-g-1-Data.db -rw-r--r-- 2 root root 32 Jul 23 00:51 Standard1-g-1-Filter.db -rw-r--r-- 2 root root 120 Jul 23 00:51 Standard1-g-1-Index.db -rw-r--r-- 2 root root 4276 Jul 23 00:51 Standard1-g-1-Statistics.db drwxr-xr-x 3 root root 4096 Jul 23 00:51 snapshots {code} *Bug: The files for Standard1 are orphaned on node3* -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2857) initialize log4j correctly in EmbeddedCassandraService
[ https://issues.apache.org/jira/browse/CASSANDRA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092954#comment-13092954 ] Hudson commented on CASSANDRA-2857: --- Integrated in Cassandra-0.8 #298 (See [https://builds.apache.org/job/Cassandra-0.8/298/]) fix log4j initialization in EmbeddedCassandraService patch by jbellis; reviewed by tjake for CASSANDRA-2857 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162851 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java initialize log4j correctly in EmbeddedCassandraService -- Key: CASSANDRA-2857 URL: https://issues.apache.org/jira/browse/CASSANDRA-2857 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Trivial Fix For: 0.8.5 Attachments: 2857-drivers.txt, 2857.txt Currently, ECS.cleanUpOldStuff calls CleanupHelper.cleanupAndLeaveDirs(), which initialized DatabaseDescriptor which does some logging. When we go to initialize log4j later in AbstractCassandraService, it's too late. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092957#comment-13092957 ] Gary Dusbabek edited comment on CASSANDRA-3051 at 8/29/11 4:25 PM: --- Pavel, I tested SSL prior to committing the feature. I was under the impression that this ticket exists because compression, when enabled, breaks SSL. The implication is that it was working prior, else the ticket would be something more like SSL is broken. was (Author: gdusbabek): Pavel, I tested SSL prior to committing the feature. I was under the impression that ticket exists because compression, when enabled, breaks SSL. The implication is that it was working prior, else the ticket would be something more like SSL is broken. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092957#comment-13092957 ] Gary Dusbabek commented on CASSANDRA-3051: -- Pavel, I tested SSL prior to committing the feature. I was under the impression that ticket exists because compression, when enabled, breaks SSL. The implication is that it was working prior, else the ticket would be something more like SSL is broken. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092967#comment-13092967 ] Pavel Yaskevich commented on CASSANDRA-3051: You misunderstood that, it is not breaking SSL it was just special cased in FileStreamTask so it wasn't using ssl socket. This patch removes special casing for SSL streaming by creating ssl socket directly in FileStreamTask if encryption options were set. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-3084: --- Attachment: 3084.txt 3084-unit-test.txt 3084.txt covers all cases of possible range differences. 3084-unit-test.txt provides unit tests that fail before the patch. The logic is definitely a bit hairy here, so any thoughts on how to simplify it are welcome. o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs reassigned CASSANDRA-3084: -- Assignee: Tyler Hobbs o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: v1-0002-CASSANDRA-3078-thrift-generated-changes.txt v1-0001-CASSANDRA-3078-add-custom-index-type.txt Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: (was: v1-0002-CASSANDRA-3078-pluggable-secondary-index-thrift.txt) Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: (was: v1-0001-CASSANDRA-3078-pluggable-secondary-indexes.txt) Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093005#comment-13093005 ] T Jake Luciani commented on CASSANDRA-3078: --- added CUSTOM type with a class_name parameter Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093010#comment-13093010 ] Jonathan Ellis commented on CASSANDRA-3051: --- Pavel, can you try to set up local SSL w/ a ccm cluster based on Gary's instructions to verify? On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-3084: Reviewer: stuhood o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093042#comment-13093042 ] Stu Hood edited comment on CASSANDRA-3084 at 8/29/11 6:16 PM: -- Couldn't a bunch of difference cases be eliminated by taking advantage of the intersection implementation? The difference between ranges x and y would be {{z = x.intersect\(y\); y.subtract(z)}}, and I think a subtract method with a y must contain z precondition would be much easier to implement. was (Author: stuhood): Couldn't a bunch of difference cases be eliminated by taking advantage of the intersection implementation? The difference between ranges x and y would be {{z = x.intersect(y); y.subtract(z)}}, and I think a subtract method with a y must contain z precondition would be much easier to implement. o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093042#comment-13093042 ] Stu Hood commented on CASSANDRA-3084: - Couldn't a bunch of difference cases be eliminated by taking advantage of the intersection implementation? The difference between ranges x and y would be {{z = x.intersect(y); y.subtract(z)}}, and I think a subtract method with a y must contain z precondition would be much easier to implement. o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093092#comment-13093092 ] Jonathan Ellis commented on CASSANDRA-3078: --- It would be a lot more clear to me that there isn't a subtle bug in the type - class conversion (and the thrift - native - avro surroundings) if you just added CUSTOM to the old switch statement instead of the indexTypes map business. Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3101) Should check for errors when calling /bin/ln
[ https://issues.apache.org/jira/browse/CASSANDRA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3101: -- Affects Version/s: (was: 0.8.4) (was: 0.7.8) 0.4 Labels: lhf (was: ) Should check for errors when calling /bin/ln Key: CASSANDRA-3101 URL: https://issues.apache.org/jira/browse/CASSANDRA-3101 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.4 Reporter: paul cannon Priority: Minor Labels: lhf It looks like cassandra.utils.CLibrary.createHardLinkWithExec() does not check for any errors in the execution of the hard-link-making utility. This could be bad if, for example, the user has put the snapshot directory on a different filesystem from the data directory. The hard linking would fail and the sstable snapshots would not exist, but no error would be reported. It does look like errors with the more direct JNA link() call are handled correctly- an exception is thrown. The WithExec version should probably do the same thing. Definitely it would be enough to check the process exit value from /bin/ln for nonzero in the *nix case, but I don't know whether 'fsutil hardlink create' or 'cmd /c mklink /H' return nonzero on failure. For bonus points, use any output from the Process's error stream in the text of the exception, to aid in debugging problems. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env
[ https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093110#comment-13093110 ] Eric Evans commented on CASSANDRA-3056: --- cassandra-env.sh is used to setup the environment, (the jvm and everything that is passed to it); cassandra.yaml is really the wrong place for this. But, I agree with Jonathan that it doesn't seem right to add cassandra-env.sh to the list of first-start criteria, particularly for something that only a small fraction of users will be concerned with. Why not make this env-only? If CASSANDRA_HEAP_DUMP_PATH (or whatever) is set, then use it to construct the argument (from cassandra-env.sh). Able to set path location of HeapDump in cassandra-env -- Key: CASSANDRA-3056 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056 Project: Cassandra Issue Type: Improvement Affects Versions: 0.7.8, 0.8.4 Reporter: David Talbott Priority: Minor Labels: lhf Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt We should be able to designate the path location to put any perf dumps that are performed. By Default with this not set the perf dump can occur on the root disk and fill the drive. Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option available and set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
[ https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-3084: --- Attachment: 3084-v2.txt Good idea! 3084-v2.txt simplifies the logic by adding a subtractContained() method. o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly -- Key: CASSANDRA-3084 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Tyler Hobbs Assignee: Tyler Hobbs Attachments: 3084-unit-test.txt, 3084-v2.txt, 3084.txt It's possible that differenceToFetch is making implicit assumptions about the relationship between the two ranges, but the following cases are not handled correctly (the old range is (A, B], the new is (C, D]: {noformat} --C--A-B--D-- {noformat} Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D]. {noformat} --C--A-D--B-- {noformat} The result will be (C, D] instead of just (C, A]. {noformat} --A--C-D--B-- {noformat} The result will be (B, D] when nothing needs to be transfered. If there is some kind of implicit assumption that these cases won't arise, it either needs to be explicit (assertions, exceptions) or the cases need to be handled. It should be easy to cover this with unit tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: (was: v1-0001-CASSANDRA-3078-add-custom-index-type.txt) Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, v1-0004-CASSANDRA-3078-add-cli-show-support.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: (was: v1-0002-CASSANDRA-3078-thrift-generated-changes.txt) Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, v1-0004-CASSANDRA-3078-add-cli-show-support.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-3078: -- Attachment: v1-0004-CASSANDRA-3078-add-cli-show-support.txt v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt v1-0002-CASSANDRA-3078-thrift-generated-changes.txt v1-0001-CASSANDRA-3078-add-custom-index-type.txt Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, v1-0004-CASSANDRA-3078-add-cli-show-support.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093129#comment-13093129 ] T Jake Luciani commented on CASSANDRA-3078: --- ok, added back switch statement. Also added cli support for displaying column metadata. Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, v1-0004-CASSANDRA-3078-add-cli-show-support.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1162988 - in /cassandra/trunk: CHANGES.txt src/java/org/apache/cassandra/db/CollationController.java src/java/org/apache/cassandra/db/ColumnFamilyStore.java src/java/org/apache/cassandra/
Author: jbellis Date: Mon Aug 29 20:28:46 2011 New Revision: 1162988 URL: http://svn.apache.org/viewvc?rev=1162988view=rev Log: fix race condition in sstable reference counting patch by jbellis; reviewed by slebresne for CASSANDRA-3085 Modified: cassandra/trunk/CHANGES.txt cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162988r1=1162987r2=1162988view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Mon Aug 29 20:28:46 2011 @@ -44,7 +44,7 @@ Thrift-Avro conversion methods (CASSANDRA-3032) * Add timeouts to client request schedulers (CASSANDRA-3079) * Cli to use hashes rather than array of hashes for strategy options (CASSANDRA-3081) - * LeveledCompactionStrategy (CASSANDRA-1608) + * LeveledCompactionStrategy (CASSANDRA-1608, 3085) * Improvements of the CLI `describe` command (CASSANDRA-2630) * reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942) Modified: cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java?rev=1162988r1=1162987r2=1162988view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java Mon Aug 29 20:28:46 2011 @@ -72,15 +72,12 @@ public class CollationController { logger.debug(collectTimeOrderedData); -ListIColumnIterator iterators; -ColumnFamily container; -while (true) -{ -DataTracker.View dataview = cfs.getDataTracker().getView(); -iterators = new ArrayListIColumnIterator(); -container = ColumnFamily.create(cfs.metadata, factory, filter.filter.isReversed()); -ListSSTableReader sstables; -for (Memtable memtable : Iterables.concat(dataview.memtablesPendingFlush, Collections.singleton(dataview.memtable))) +ColumnFamily container = ColumnFamily.create(cfs.metadata, factory, filter.filter.isReversed()); +ListIColumnIterator iterators = new ArrayListIColumnIterator(); +ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key); +try +{ +for (Memtable memtable : view.memtables) { IColumnIterator iter = filter.getMemtableColumnIterator(memtable, cfs.metadata.comparator); if (iter != null) @@ -99,42 +96,33 @@ public class CollationController QueryFilter reducedFilter = new QueryFilter(filter.key, filter.path, new NamesQueryFilter(filterColumns)); /* add the SSTables on disk */ -sstables = dataview.intervalTree.search(new Interval(filter.key, filter.key)); -Collections.sort(sstables, SSTable.maxTimestampComparator); -if (!SSTableReader.acquireReferences(sstables)) -continue; // retry w/ new view +Collections.sort(view.sstables, SSTable.maxTimestampComparator); -try +// read sorted sstables +for (SSTableReader sstable : view.sstables) { -// read sorted sstables -for (SSTableReader sstable : sstables) +long currentMaxTs = sstable.getMaxTimestamp(); +reduceNameFilter(reducedFilter, container, currentMaxTs); +if (((NamesQueryFilter) reducedFilter.filter).columns.isEmpty()) +break; + +IColumnIterator iter = reducedFilter.getSSTableColumnIterator(sstable); +iterators.add(iter); +if (iter.getColumnFamily() != null) { -long currentMaxTs = sstable.getMaxTimestamp(); -reduceNameFilter(reducedFilter, container, currentMaxTs); -if (((NamesQueryFilter) reducedFilter.filter).columns.isEmpty()) -break; - -IColumnIterator iter = reducedFilter.getSSTableColumnIterator(sstable); -iterators.add(iter); -if (iter.getColumnFamily() != null) -{ -container.delete(iter.getColumnFamily()); -sstablesIterated++; -while (iter.hasNext()) -container.addColumn(iter.next()); -} +container.delete(iter.getColumnFamily()); +
[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting
[ https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093166#comment-13093166 ] Jonathan Ellis commented on CASSANDRA-3085: --- committed with requested fixes, fix for interval computation involving stopAt.isEmpty(), and javadoc for CFS.markReferenced Race condition in sstable reference counting Key: CASSANDRA-3085 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Critical Fix For: 1.0 Attachments: 3085-v2.txt, 3085.txt DataTracker gives us an atomic View of memtable/sstables, but acquiring references is not atomic. So it is possible to acquire references to an SSTableReader object that is no longer valid, as in this example: View V contains sstables {A, B}. We attempt a read in thread T using this View. Meanwhile, A and B are compacted to {C}, yielding View W. No references exist to A or B so they are cleaned up. Back in thread T we acquire references to A and B. This does not cause an error, but it will when we attempt to read from them next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable
[ https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093177#comment-13093177 ] Jonathan Ellis commented on CASSANDRA-3078: --- - ThriftValidation should probably try to instantiate the given index type, w/ the options map, so it can check for and raise exception on invalid options, the way we do for replication strategy - spaces around operators please Make Secondary Indexes Pluggable - Key: CASSANDRA-3078 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0 Reporter: T Jake Luciani Assignee: T Jake Luciani Labels: secondary_index Fix For: 1.0 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, v1-0004-CASSANDRA-3078-add-cli-show-support.txt CASSANDRA-2982 got us most of the way there... This ticket removes the IndexType enum (while keeping support for KEYS internally from old cf metadata). You now specify a index_class rather than index_type. index_class is the full classname of the SecondaryIndex impl. This also adds a index_options map to pass extra info to the secondary index impl if needed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1827) Batching across stages
[ https://issues.apache.org/jira/browse/CASSANDRA-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-1827: -- Fix Version/s: (was: 1.0) Batching across stages -- Key: CASSANDRA-1827 URL: https://issues.apache.org/jira/browse/CASSANDRA-1827 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet We might be able to get some improvement if we start batching tasks for every stage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093222#comment-13093222 ] Mikko Koppanen commented on CASSANDRA-3025: --- bq. Why is that? Does PDO require only primitive or string column names? This is not actually a PDO requirement but rather PHP array type requirement. A key in array can be integer or string. I looked at phpcassa code briefly and it doesn't seem that they automatically marshal the UUIDs but rather provide a method for doing that. bq. should test an actual big int ( 64bit) I added a test for very big integers. There is really no native datatype in PHP that could handle this, so I took the following approach: 1. If the integer is larger than 8 bytes, return LONG_MIN 2. If user sees LONG_MIN, it means that binary presentation might be different 3. Turn on CASSANDRA_ATTR_PRESERVE_VALUES and convert manually, using bcmath or gmp libraries. Currently the conversion back to userland will overflow on 32bit platforms when values larger than LONG_MAX or smaller than INT_MIN. Not sure if there is a clean and portable way to handle this as PHP uses platform long for integer types rather than fixed width integers. PHP/PDO driver for Cassandra CQL Key: CASSANDRA-3025 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025 Project: Cassandra Issue Type: New Feature Components: API Reporter: Mikko Koppanen Labels: php Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, php_test_results_20110818_2317.txt Hello, attached is the initial version of the PDO driver for Cassandra CQL language. This is a native PHP extension written in what I would call a combination of C and C++, due to PHP being C. The thrift API used is the C++. The API looks roughly following: {code} ?php $db = new PDO('cassandra:host=127.0.0.1;port=9160'); $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor=1;); $db-exec (USE mytest); $db-exec (CREATE COLUMNFAMILY users ( my_key varchar PRIMARY KEY, full_name varchar );); $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, :full_name);); $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' )); {code} Currently prepared statements are emulated on the client side but I understand that there is a plan to add prepared statements to Cassandra CQL API as well. I will add this feature in to the extension as soon as they are implemented. Additional documentation can be found in github https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered MarkDown file. Tests are currently not included in the package file and they can be found in the github for now as well. I have created documentation in docbook format as well, but have not yet rendered it. Comments and feedback are welcome. Thanks, Mikko -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093224#comment-13093224 ] Mikko Koppanen commented on CASSANDRA-3025: --- Forgot to link to test: https://github.com/mkoppanen/php-pdo_cassandra/blob/master/tests/026-bigint.phpt In case of PHP the users are better of using string types rather than actual long integers. PHP/PDO driver for Cassandra CQL Key: CASSANDRA-3025 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025 Project: Cassandra Issue Type: New Feature Components: API Reporter: Mikko Koppanen Labels: php Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, php_test_results_20110818_2317.txt Hello, attached is the initial version of the PDO driver for Cassandra CQL language. This is a native PHP extension written in what I would call a combination of C and C++, due to PHP being C. The thrift API used is the C++. The API looks roughly following: {code} ?php $db = new PDO('cassandra:host=127.0.0.1;port=9160'); $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor=1;); $db-exec (USE mytest); $db-exec (CREATE COLUMNFAMILY users ( my_key varchar PRIMARY KEY, full_name varchar );); $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, :full_name);); $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' )); {code} Currently prepared statements are emulated on the client side but I understand that there is a plan to add prepared statements to Cassandra CQL API as well. I will add this feature in to the extension as soon as they are implemented. Additional documentation can be found in github https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered MarkDown file. Tests are currently not included in the package file and they can be found in the github for now as well. I have created documentation in docbook format as well, but have not yet rendered it. Comments and feedback are welcome. Thanks, Mikko -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093230#comment-13093230 ] Pavel Yaskevich commented on CASSANDRA-3051: I figured out the problem - SSLSocket always returns null on getChannel even on current code, I will refactor FileStreamingTask to support DataOutputStream instead of SocketChannel to unify normal and SSL transfers. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3100) Secondary index still does minor compacting after deleting index
[ https://issues.apache.org/jira/browse/CASSANDRA-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093255#comment-13093255 ] Jeremy Hanna commented on CASSANDRA-3100: - Also relevant, when bootstrapping a new node into the ring, it streamed data from a node and in the compactionstats I also see it's trying to build a secondary index that shouldn't be there. Secondary index still does minor compacting after deleting index Key: CASSANDRA-3100 URL: https://issues.apache.org/jira/browse/CASSANDRA-3100 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.8 Reporter: Jeremy Hanna We deleted all of our secondary indexes. A couple of days later I was watching compactionstats on one of the nodes and it was in the process of minor compacting one of the deleted secondary indexes. I double checked the keyspace definitions on the CLI and there were no secondary indexes defined. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093263#comment-13093263 ] Todd Nine commented on CASSANDRA-2915: -- Could we also use this feature as a standard way for building our lucene documents? This would accomplish what we want, as well as giving a hook for more user functionality. CASSANDRA-1311 Lucene based Secondary Indexes -- Key: CASSANDRA-2915 URL: https://issues.apache.org/jira/browse/CASSANDRA-2915 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Jason Rutherglen Labels: secondary_index Secondary indexes (of type KEYS) suffer from a number of limitations in their current form: - Multiple IndexClauses only work when there is a subset of rows under the highest clause - One new column family is created per index this means 10 new CFs for 10 secondary indexes This ticket will use the Lucene library to implement secondary indexes as one index per CF, and utilize the Lucene query engine to handle multiple index clauses. Also, by using the Lucene we get a highly optimized file format. There are a few parallels we can draw between Cassandra and Lucene. Lucene indexes segments in memory then flushes them to disk so we can sync our memtable flushes to lucene flushes. Lucene also has optimize() which correlates to our compaction process, so these can be sync'd as well. We will also need to correlate column validators to Lucene tokenizers, so the data can be stored properly, the big win in once this is done we can perform complex queries within a column like wildcard searches. The downside of this approach is we will need to read before write since documents in Lucene are written as complete documents. For random workloads with lot's of indexed columns this means we need to read the document from the index, update it and write it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3051: --- Attachment: CASSANDRA-3051.patch CompressedRandomAccessReader.transfer method was removed with special casing for compressed files, SocketChannel based transfer changed to DataOuputStream based to unify SSL and normal modes. SSLFileStreamTask removed as unused. On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093283#comment-13093283 ] Mikko Koppanen commented on CASSANDRA-3025: --- Changed the behaviour a bit: https://github.com/mkoppanen/php-pdo_cassandra/blob/master/tests/026-bigint.phpt There is an error/exception if the conversion fails and user is forced to handle this scenario. This allows user to react if they are handling very large integers. PHP/PDO driver for Cassandra CQL Key: CASSANDRA-3025 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025 Project: Cassandra Issue Type: New Feature Components: API Reporter: Mikko Koppanen Labels: php Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, php_test_results_20110818_2317.txt Hello, attached is the initial version of the PDO driver for Cassandra CQL language. This is a native PHP extension written in what I would call a combination of C and C++, due to PHP being C. The thrift API used is the C++. The API looks roughly following: {code} ?php $db = new PDO('cassandra:host=127.0.0.1;port=9160'); $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor=1;); $db-exec (USE mytest); $db-exec (CREATE COLUMNFAMILY users ( my_key varchar PRIMARY KEY, full_name varchar );); $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, :full_name);); $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' )); {code} Currently prepared statements are emulated on the client side but I understand that there is a plan to add prepared statements to Cassandra CQL API as well. I will add this feature in to the extension as soon as they are implemented. Additional documentation can be found in github https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered MarkDown file. Tests are currently not included in the package file and they can be found in the github for now as well. I have created documentation in docbook format as well, but have not yet rendered it. Comments and feedback are welcome. Thanks, Mikko -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3095) java.lang.NegativeArraySizeException during compacting large row
[ https://issues.apache.org/jira/browse/CASSANDRA-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093294#comment-13093294 ] Pas commented on CASSANDRA-3095: Hi, as I've still 3 nodes on 0.7.4, I'll try to reproduce this error on one of them. (ETA ~2 days.) java.lang.NegativeArraySizeException during compacting large row Key: CASSANDRA-3095 URL: https://issues.apache.org/jira/browse/CASSANDRA-3095 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Environment: Linux 2.6.26-2-amd64 #1 SMP Thu Feb 11 00:59:32 UTC 2010 x86_64 GNU/Linux JDK 1.6.0_27 (Java 6 update 27), with JNA. Reporter: Pas Attachments: 3095-debug.txt Hello, It's a 4 node ring, 3 on 0.7.4, I've upgraded one to 0.8.4. This particular node was having issues with compaction that's why I've tried the upgrade (it looks likely that this solved the compaction issues). Here's the stack trace from system.log. INFO [CompactionExecutor:22] 2011-08-28 18:12:46,566 CompactionController.java (line 136) Compacting large row (36028797018963968 bytes) incrementally ERROR [CompactionExecutor:22] 2011-08-28 18:12:46,609 AbstractCassandraDaemon.java (line 134) Fatal exception in thread Thread[CompactionExecutor:22,1,main] java.lang.NegativeArraySizeException at org.apache.cassandra.utils.obs.OpenBitSet.init(OpenBitSet.java:85) at org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:56) at org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:73) at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62) at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50) at org.apache.cassandra.db.compaction.LazilyCompactedRow.init(LazilyCompactedRow.java:89) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:138) at org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:123) at org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:74) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:569) at org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:506) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:141) at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:107) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) We've ~70 files still in f format. And 80 in g. We've ~100 GB of data on this node. Thanks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093298#comment-13093298 ] Jason Rutherglen commented on CASSANDRA-2915: - Todd, Another option is to add a [user optional] class that converts raw Cassandra columns into a Lucene document. Implicitly the Cassandra columns do not need to map to Lucene document fields. This is more of a slight change in the user's expectations for CQL rather than a core functional change. Eg, the CQL submitted to a Lucene secondary index may refer to Lucene fields that do not exist as columns. Lucene based Secondary Indexes -- Key: CASSANDRA-2915 URL: https://issues.apache.org/jira/browse/CASSANDRA-2915 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Jason Rutherglen Labels: secondary_index Secondary indexes (of type KEYS) suffer from a number of limitations in their current form: - Multiple IndexClauses only work when there is a subset of rows under the highest clause - One new column family is created per index this means 10 new CFs for 10 secondary indexes This ticket will use the Lucene library to implement secondary indexes as one index per CF, and utilize the Lucene query engine to handle multiple index clauses. Also, by using the Lucene we get a highly optimized file format. There are a few parallels we can draw between Cassandra and Lucene. Lucene indexes segments in memory then flushes them to disk so we can sync our memtable flushes to lucene flushes. Lucene also has optimize() which correlates to our compaction process, so these can be sync'd as well. We will also need to correlate column validators to Lucene tokenizers, so the data can be stored properly, the big win in once this is done we can perform complex queries within a column like wildcard searches. The downside of this approach is we will need to read before write since documents in Lucene are written as complete documents. For random workloads with lot's of indexed columns this means we need to read the document from the index, update it and write it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093341#comment-13093341 ] Ed Anuff commented on CASSANDRA-2915: - +1 on having the ability to provide a conversion class for handling transformations from columns to Lucene documents. It's not uncommon for people to store objects serialized to JSON or other some other serialization format into columns. CQL will have to catch up with this practice at some point. Lucene based Secondary Indexes -- Key: CASSANDRA-2915 URL: https://issues.apache.org/jira/browse/CASSANDRA-2915 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Jason Rutherglen Labels: secondary_index Secondary indexes (of type KEYS) suffer from a number of limitations in their current form: - Multiple IndexClauses only work when there is a subset of rows under the highest clause - One new column family is created per index this means 10 new CFs for 10 secondary indexes This ticket will use the Lucene library to implement secondary indexes as one index per CF, and utilize the Lucene query engine to handle multiple index clauses. Also, by using the Lucene we get a highly optimized file format. There are a few parallels we can draw between Cassandra and Lucene. Lucene indexes segments in memory then flushes them to disk so we can sync our memtable flushes to lucene flushes. Lucene also has optimize() which correlates to our compaction process, so these can be sync'd as well. We will also need to correlate column validators to Lucene tokenizers, so the data can be stored properly, the big win in once this is done we can perform complex queries within a column like wildcard searches. The downside of this approach is we will need to read before write since documents in Lucene are written as complete documents. For random workloads with lot's of indexed columns this means we need to read the document from the index, update it and write it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093344#comment-13093344 ] Todd Nine commented on CASSANDRA-2915: -- I think forcing users to install classes for common use cases would cause issues with adoption. What about creating new CQL commands to handle this? When creating an index in a db, you would define the fields and the manner in which they are indexed. Could we do something like the following? create index [colname] in [colfamily] using [index type 1] as [indexFieldName], [index type 2] as [indexFieldName], [index type n] as [indexFieldName]? drop index [indexFieldName] in [colfamily] on [colname] This way clients such as JPA can update and create indexes, without the need to install custom classes on Cassandra itself. They also have the ability to directly reference the field name when using CQL queries. Assuming that the index class types exist in the Lucene classpath, you get the 1 to many mappings for column to indexing strategy. This would allow more advanced clients such as the JPA plugin to automatically add indexes to the document based on indexes defined on persistent fields, without generating any code the user has to install in the Cassandra runtime. If users want to install custom analyzers, they still have the option to do so, and would gain access to it via CQL. Lucene based Secondary Indexes -- Key: CASSANDRA-2915 URL: https://issues.apache.org/jira/browse/CASSANDRA-2915 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Jason Rutherglen Labels: secondary_index Secondary indexes (of type KEYS) suffer from a number of limitations in their current form: - Multiple IndexClauses only work when there is a subset of rows under the highest clause - One new column family is created per index this means 10 new CFs for 10 secondary indexes This ticket will use the Lucene library to implement secondary indexes as one index per CF, and utilize the Lucene query engine to handle multiple index clauses. Also, by using the Lucene we get a highly optimized file format. There are a few parallels we can draw between Cassandra and Lucene. Lucene indexes segments in memory then flushes them to disk so we can sync our memtable flushes to lucene flushes. Lucene also has optimize() which correlates to our compaction process, so these can be sync'd as well. We will also need to correlate column validators to Lucene tokenizers, so the data can be stored properly, the big win in once this is done we can perform complex queries within a column like wildcard searches. The downside of this approach is we will need to read before write since documents in Lucene are written as complete documents. For random workloads with lot's of indexed columns this means we need to read the document from the index, update it and write it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2884) CQL processing engine should provide server side data binding to pre-compiled CQL scripts
[ https://issues.apache.org/jira/browse/CASSANDRA-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093372#comment-13093372 ] Rick Shaw commented on CASSANDRA-2884: -- Yes. Seems like it to me as well. CQL processing engine should provide server side data binding to pre-compiled CQL scripts - Key: CASSANDRA-2884 URL: https://issues.apache.org/jira/browse/CASSANDRA-2884 Project: Cassandra Issue Type: New Feature Components: API Affects Versions: 0.8.1 Reporter: Rick Shaw Labels: API, CQL Fix For: 1.0 The notion of Prepared statements is derived from JDBC lore, but is equally useful for the other supported clients of CQL. In order to support server-side binding and pre-compiled code referenceable from the client side, the API (currently Thrift based) will need to be enhanced to pass both script (CQL) and the binding arguments to the server. In addition the parser will need to recognize the place holders for the bound variables (?). And the product of the parse will need to be savable and a reference or handle will need to be returned to the client side caller. The execution of C* API calls will need to be moved from the parsers action routines where it is currently executed as the script is parsed, and moved to another execution engine that calls API methods and uses the bound variables and the stored pre-compiled code to drive the process. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093373#comment-13093373 ] Jonathan Ellis commented on CASSANDRA-3051: --- - feels like we lose more than we gain by making writeHeader/write separate methods. they aren't really self-contained so you have to keep the context they were called in, around mentally. and if they were in-line, it would be obvious that you don't need to re-seek for each call to write(). - comments in write() don't really add much to what the code says, imo - is flushing with each chunk necessary? seems like that would harm performance On Disk Compression breaks SSL Encryption - Key: CASSANDRA-3051 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051 Project: Cassandra Issue Type: Bug Affects Versions: 1.0 Environment: Trunk Reporter: Benjamin Coverston Assignee: Pavel Yaskevich Fix For: 1.0 Attachments: CASSANDRA-3051.patch Encryption depends on FileStreamTask.write [1] protected member to be called because the SSLFileStreamTask.write overrides this to write back to the server. When enabled, compression circumvents the call and the client does not communicate using an SSL socket back to the server. [1] protected long write(FileChannel fc, PairLong, Long section, long length, long bytesTransferred) throws IOException -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2915) Lucene based Secondary Indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093344#comment-13093344 ] Todd Nine edited comment on CASSANDRA-2915 at 8/30/11 2:13 AM: --- I think forcing users to install classes for common use cases would cause issues with adoption. What about creating new CQL commands to handle this? When creating an index in a db, you would define the fields and the manner in which they are indexed. Could we do something like the following? create index on [colname] in [colfamily] using [index type 1] as [indexFieldName], [index type 2] as [indexFieldName], [index type n] as [indexFieldName]? drop index [indexFieldName] in [colfamily] on [colname] This way clients such as JPA can update and create indexes, without the need to install custom classes on Cassandra itself. They also have the ability to directly reference the field name when using CQL queries. Assuming that the index class types exist in the Lucene classpath, you get the 1 to many mappings for column to indexing strategy. This would allow more advanced clients such as the JPA plugin to automatically add indexes to the document based on indexes defined on persistent fields, without generating any code the user has to install in the Cassandra runtime. If users want to install custom analyzers, they still have the option to do so, and would gain access to it via CQL. was (Author: tnine): I think forcing users to install classes for common use cases would cause issues with adoption. What about creating new CQL commands to handle this? When creating an index in a db, you would define the fields and the manner in which they are indexed. Could we do something like the following? create index [colname] in [colfamily] using [index type 1] as [indexFieldName], [index type 2] as [indexFieldName], [index type n] as [indexFieldName]? drop index [indexFieldName] in [colfamily] on [colname] This way clients such as JPA can update and create indexes, without the need to install custom classes on Cassandra itself. They also have the ability to directly reference the field name when using CQL queries. Assuming that the index class types exist in the Lucene classpath, you get the 1 to many mappings for column to indexing strategy. This would allow more advanced clients such as the JPA plugin to automatically add indexes to the document based on indexes defined on persistent fields, without generating any code the user has to install in the Cassandra runtime. If users want to install custom analyzers, they still have the option to do so, and would gain access to it via CQL. Lucene based Secondary Indexes -- Key: CASSANDRA-2915 URL: https://issues.apache.org/jira/browse/CASSANDRA-2915 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Jason Rutherglen Labels: secondary_index Secondary indexes (of type KEYS) suffer from a number of limitations in their current form: - Multiple IndexClauses only work when there is a subset of rows under the highest clause - One new column family is created per index this means 10 new CFs for 10 secondary indexes This ticket will use the Lucene library to implement secondary indexes as one index per CF, and utilize the Lucene query engine to handle multiple index clauses. Also, by using the Lucene we get a highly optimized file format. There are a few parallels we can draw between Cassandra and Lucene. Lucene indexes segments in memory then flushes them to disk so we can sync our memtable flushes to lucene flushes. Lucene also has optimize() which correlates to our compaction process, so these can be sync'd as well. We will also need to correlate column validators to Lucene tokenizers, so the data can be stored properly, the big win in once this is done we can perform complex queries within a column like wildcard searches. The downside of this approach is we will need to read before write since documents in Lucene are written as complete documents. For random workloads with lot's of indexed columns this means we need to read the document from the index, update it and write it back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2884) CQL processing engine should provide server side data binding to pre-compiled CQL scripts
[ https://issues.apache.org/jira/browse/CASSANDRA-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2884. --- Resolution: Duplicate Fix Version/s: (was: 1.0) CQL processing engine should provide server side data binding to pre-compiled CQL scripts - Key: CASSANDRA-2884 URL: https://issues.apache.org/jira/browse/CASSANDRA-2884 Project: Cassandra Issue Type: New Feature Components: API Affects Versions: 0.8.1 Reporter: Rick Shaw Labels: API, CQL The notion of Prepared statements is derived from JDBC lore, but is equally useful for the other supported clients of CQL. In order to support server-side binding and pre-compiled code referenceable from the client side, the API (currently Thrift based) will need to be enhanced to pass both script (CQL) and the binding arguments to the server. In addition the parser will need to recognize the place holders for the bound variables (?). And the product of the parse will need to be savable and a reference or handle will need to be returned to the client side caller. The execution of C* API calls will need to be moved from the parsers action routines where it is currently executed as the script is parsed, and moved to another execution engine that calls API methods and uses the bound variables and the stored pre-compiled code to drive the process. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira