date:20110829


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3098.
---

Resolution: Not A Problem

column names are formatted based on the CF comparator.  default is bytes.

 Weird/hex Column Name:  formatting with describe keyspaces
 --

 Key: CASSANDRA-3098
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3098
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.8.4
 Environment: Cassandra 0.8.4 (and today's cassandra-0.8 branch)
 java version 1.6.0_20
 OpenJDK Runtime Environment (IcedTea6 1.9.9) (fedora-54.1.9.9.fc14-x86_64)
 OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
Reporter: Jonas Borgström

 Displaying a newly created column family with column_metadata displays some 
 kind of hex representation of the column names instead of something more 
 human readable:
 {code}
 [default@test] create column family Foo3 with column_metadata = [{ 
 column_name: mycolumn, validation_class: UTF8Type }, { column_name: 
 mycolumn2, validation_class: UTF8Type }];
 4f266c60-d236-11e0--242d50cf1fbf
 Waiting for schema agreement...
 ... schemas agree across the cluster
 [default@test] describe keyspace; 
   
 
 Keyspace: test:
   Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
   Durable Writes: true
 Options: [datacenter1:1]
   Column Families:
 ColumnFamily: Foo3
   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
   Default column value validator: 
 org.apache.cassandra.db.marshal.BytesType
   Columns sorted by: org.apache.cassandra.db.marshal.BytesType
   Row cache size / save period in seconds: 0.0/0
   Key cache size / save period in seconds: 20.0/14400
   Memtable thresholds: 0.2953125/1440/63 (millions of ops/minutes/MB)
   GC grace seconds: 864000
   Compaction min/max thresholds: 4/32
   Read repair chance: 1.0
   Replicate on write: true
   Built indexes: []
   Column Metadata:
 Column Name: fffcf2   --- I expected this to say 'mycolumn' or 
 'mycolumn2'
   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
 Column Name:  --- I expected this to say 'mycolumn' or 
 'mycolumn2'
   Validation Class: org.apache.cassandra.db.marshal.UTF8Type
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3085) Race condition in sstable reference counting


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3085:
--

Reviewer: slebresne  (was: bcoverston)

 Race condition in sstable reference counting
 

 Key: CASSANDRA-3085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Critical
 Fix For: 1.0

 Attachments: 3085-v2.txt, 3085.txt


 DataTracker gives us an atomic View of memtable/sstables, but acquiring 
 references is not atomic.  So it is possible to acquire references to an 
 SSTableReader object that is no longer valid, as in this example:
 View V contains sstables {A, B}.  We attempt a read in thread T using this 
 View.
 Meanwhile, A and B are compacted to {C}, yielding View W.  No references 
 exist to A or B so they are cleaned up.
 Back in thread T we acquire references to A and B.  This does not cause an 
 error, but it will when we attempt to read from them next.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162775 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/cli/ src/resources/org/apache/cassandra/cli/ test/unit/org/apache/cassandra/cli/

2011-08-29 Thread xedin

Author: xedin
Date: Mon Aug 29 12:53:02 2011
New Revision: 1162775

URL: http://svn.apache.org/viewvc?rev=1162775view=rev
Log:
Improvements of the CLI `describe` command
patch by satishbabu; reviewed by xedin for CASSANDRA-2630

Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g
cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/trunk/src/java/org/apache/cassandra/cli/CliUtils.java
cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml
cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162775r1=1162774r2=1162775view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Aug 29 12:53:02 2011
@@ -45,7 +45,7 @@
  * Add timeouts to client request schedulers (CASSANDRA-3079)
  * Cli to use hashes rather than array of hashes for strategy options 
(CASSANDRA-3081)
  * LeveledCompactionStrategy (CASSANDRA-1608)
-
+ * Improvements of the CLI `describe` command (CASSANDRA-2630)
 
 0.8.5
  * fix NPE when encryption_options is unspecified (CASSANDRA-3007)

Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g?rev=1162775r1=1162774r2=1162775view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g Mon Aug 29 12:53:02 
2011
@@ -35,7 +35,7 @@ tokens {
 // various top-level CLI statements.
 //
 NODE_CONNECT;
-NODE_DESCRIBE_TABLE;
+NODE_DESCRIBE;
 NODE_DESCRIBE_CLUSTER;
 NODE_USE_TABLE;
 NODE_EXIT;
@@ -180,8 +180,8 @@ helpStatement
 - ^(NODE_HELP NODE_CONNECT)
 | HELP USE 
 - ^(NODE_HELP NODE_USE_TABLE)
-| HELP DESCRIBE KEYSPACE 
-- ^(NODE_HELP NODE_DESCRIBE_TABLE)
+| HELP DESCRIBE
+- ^(NODE_HELP NODE_DESCRIBE)
 | HELP DESCRIBE 'CLUSTER'
 - ^(NODE_HELP NODE_DESCRIBE_CLUSTER)
 | HELP EXIT 
@@ -366,8 +366,8 @@ showSchema
 ;
 
 describeTable
-: DESCRIBE KEYSPACE (keyspace)?
-- ^(NODE_DESCRIBE_TABLE (keyspace)?)
+: DESCRIBE (keyspace)?
+- ^(NODE_DESCRIBE (keyspace)?)
 ;
 
 describeCluster

Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1162775r1=1162774r2=1162775view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Mon Aug 29 
12:53:02 2011
@@ -250,8 +250,8 @@ public class CliClient
 case CliParser.NODE_SHOW_SCHEMA:
 executeShowSchema(tree);
 break;
-case CliParser.NODE_DESCRIBE_TABLE:
-executeDescribeKeySpace(tree);
+case CliParser.NODE_DESCRIBE:
+executeDescribe(tree);
 break;
 case CliParser.NODE_DESCRIBE_CLUSTER:
 executeDescribeCluster();
@@ -1857,92 +1857,10 @@ public class CliClient
 
 sessionState.out.println(  Column Families:);
 
-boolean isSuper;
-
 Collections.sort(ks_def.cf_defs, new CfDefNamesComparator());
-for (CfDef cf_def : ks_def.cf_defs)
-{
-// fetching bean for current column family store
-ColumnFamilyStoreMBean cfMBean = (probe == null) ? null : 
probe.getCfsProxy(ks_def.getName(), cf_def.getName());
 
-isSuper = cf_def.column_type.equals(Super);
-sessionState.out.printf(ColumnFamily: %s%s%n, 
cf_def.name, isSuper ?  (Super) : );
-
-if (cf_def.comment != null  !cf_def.comment.isEmpty())
-{
-sessionState.out.printf(\%s\%n, cf_def.comment);
-}
-if (cf_def.key_validation_class != null)
-sessionState.out.printf(  Key Validation Class: 
%s%n, cf_def.key_validation_class);
-if (cf_def.default_validation_class != null)
-sessionState.out.printf(  Default column value 
validator: %s%n, cf_def.default_validation_class);
-sessionState.out.printf(  Columns sorted by: %s%s%n, 
cf_def.comparator_type, cf_def.column_type.equals(Super) ? / + 
cf_def.subcomparator_type : );
-sessionState.out.printf(  Row cache size / save period in 
seconds / keys to save : %s/%s/%s%n,
-

[jira] [Commented] (CASSANDRA-2630) CLI - 'describe column family' would be nice


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092821#comment-13092821
 ] 

Hudson commented on CASSANDRA-2630:
---

Integrated in Cassandra #1053 (See 
[https://builds.apache.org/job/Cassandra/1053/])
Improvements of the CLI `describe` command
patch by satishbabu; reviewed by xedin for CASSANDRA-2630

xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162775
Files : 
* /cassandra/trunk/CHANGES.txt
* /cassandra/trunk/src/java/org/apache/cassandra/cli/Cli.g
* /cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
* /cassandra/trunk/src/java/org/apache/cassandra/cli/CliUtils.java
* /cassandra/trunk/src/resources/org/apache/cassandra/cli/CliHelp.yaml
* /cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java


 CLI - 'describe column family' would be nice
 

 Key: CASSANDRA-2630
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2630
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.8.4
Reporter: Jeremy Hanna
Assignee: satish babu krishnamoorthy
Priority: Minor
  Labels: cli, lhf
 Fix For: 1.0

 Attachments: cassandra-0.8.2-2630-1.txt, cassandra-0.8.2-2630-2.txt, 
 cassandra-0.8.2-2630-2.txt, cassandra-0.8.2-2630.txt


 I end up verifying column families a lot and using 'describe keyspace 
 keyspace;' spits out a whole bunch of data since our keyspace has a lot of 
 metadata.  It would be really useful to have a 'describe column family;' 
 for a given column family in the currently authenticated keyspace.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3015) Inter-Node Compression


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-3015:
---

Fix Version/s: 1.1

 Inter-Node Compression
 --

 Key: CASSANDRA-3015
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3015
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.1

 Attachments: 0001-stream-based-compression.patch


 Although we have CASSANDRA-47 for on-disk the code used for streaming 
 actually sends un-compressed blocks over the wire. I propose we compress this 
 data.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2475) Prepared statements


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2475:
--

Fix Version/s: (was: 1.0)
   1.1
 Assignee: (was: Pavel Yaskevich)

 Prepared statements
 ---

 Key: CASSANDRA-2475
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2475
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
  Labels: cql
 Fix For: 1.1




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3050) Global row cache


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3050:
--

Fix Version/s: (was: 1.0)
   1.1

 Global row cache
 

 Key: CASSANDRA-3050
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3050
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.1


 Row-cache-per-columnfamily is difficult to configure well as columnfamilies 
 are added, similar to how memtables were difficult pre-CASSANDRA-2006.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2405:
--

Fix Version/s: (was: 0.8.5)
   1.1

 should expose 'time since last successful repair' for easier aes monitoring
 ---

 Key: CASSANDRA-2405
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Peter Schuller
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.1

 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, 
 CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch


 The practical implementation issues of actually ensuring repair runs is 
 somewhat of an undocumented/untreated issue.
 One hopefully low hanging fruit would be to at least expose the time since 
 last successful repair for a particular column family, to make it easier to 
 write a correct script to monitor for lack of repair in a non-buggy fashion.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3049) Global key cache


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3049:
--

Fix Version/s: (was: 1.0)
   1.1

 Global key cache
 

 Key: CASSANDRA-3049
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3049
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.1


 Key-cache-per-columnfamily is difficult to configure well as columnfamilies 
 are added, similar to how memtables were difficult pre-CASSANDRA-2006.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3010) Java CQL command-line shell


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3010:
--

Priority: Minor  (was: Major)

 Java CQL command-line shell
 ---

 Key: CASSANDRA-3010
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3010
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.0


 We need a real CQL shell that:
 - does not require installing additional environments
 - includes show keyspaces and other introspection tools
 - does not break existing cli scripts
 I.e., it needs to be java, but it should be a new tool instead of replacing 
 the existing cli.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema Migrations


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092834#comment-13092834
 ] 

Gary Dusbabek commented on CASSANDRA-1391:
--

Pavel, can you please rebase?

 Allow Concurrent Schema Migrations
 --

 Key: CASSANDRA-1391
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-1391.patch


 CASSANDRA-1292 fixed multiple migrations started from the same node to 
 properly queue themselves, but it is still possible for migrations initiated 
 on different nodes to conflict and leave the cluster in a bad state. Since 
 the system_add/drop/rename methods are accessible directly from the client 
 API, they should be completely safe for concurrent use.
 It should be possible to allow for most types of concurrent migrations by 
 converting the UUID schema ID into a VersionVectorClock (as provided by 
 CASSANDRA-580).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2731) Impelement in-house file caching.

[
https://issues.apache.org/jira/browse/CASSANDRA-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-2731:
--

Priority: Minor (was: Major)
Affects Version/s: (was: 1.0)

Impelement in-house file caching.
-

Key: CASSANDRA-2731
URL: https://issues.apache.org/jira/browse/CASSANDRA-2731
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Priority: Minor

Implement FileCache, CachedRandomAccessFile (to replace
BufferedRandomAccessFile) and RadixTree (to play role of the backend cache
storage) classes.
FileCache class with be responsible for storing/retrieving data from Radix
Tree and also flushing of the dirty pages to the disk, page management such
as adding new pages, utilizing old/unused pages.
CRAF Linux only features (via JNI):
1). O_DIRECT for both read/write operations.
2). AIO's lio_listio write operation batching.
Provide possibility to migrate hot data directly from Memtable to CRAF cache
to keep live-reads data always hot in memory. To minimise compaction effects
CRAF should provide a way to by-pass a caching data if it does not already
exists.
Provide a way to make pointers in the cache which will be useful to minimize
impact on performance when a single column is distributed among multiple
SSTable files (except counter columns).
Use jemalloc (http://www.canonware.com/jemalloc/) for cache memory management.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-1740:
--

Fix Version/s: (was: 0.8.5)
   1.1

 Nodetool commands to query and stop compaction, repair, cleanup and scrub
 -

 Key: CASSANDRA-1740
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Chip Salzenberg
Assignee: Pavel Yaskevich
Priority: Minor
 Fix For: 1.1

 Attachments: CASSANDRA-1740.patch

   Original Estimate: 24h
  Remaining Estimate: 24h

 The only way to stop compaction, repair, cleanup, or scrub in progress is to 
 stop and restart the entire Cassandra server.  Please provide nodetool 
 commands to query whether such things are running, and stop them if they are.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1391) Allow Concurrent Schema Migrations


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-1391:
---

Attachment: CASSANDRA-1391.patch

rebased with trunk (last commit 0a4b1667bee674f7c0a22057cbdab97e368a20d1)

 Allow Concurrent Schema Migrations
 --

 Key: CASSANDRA-1391
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Stu Hood
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-1391.patch, CASSANDRA-1391.patch


 CASSANDRA-1292 fixed multiple migrations started from the same node to 
 properly queue themselves, but it is still possible for migrations initiated 
 on different nodes to conflict and leave the cluster in a bad state. Since 
 the system_add/drop/rename methods are accessible directly from the client 
 API, they should be completely safe for concurrent use.
 It should be possible to allow for most types of concurrent migrations by 
 converting the UUID schema ID into a VersionVectorClock (as provided by 
 CASSANDRA-580).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092845#comment-13092845
 ] 

Jonathan Ellis commented on CASSANDRA-3056:
---

I don't think I'm a big fan of adding -env.sh to the list of things that you 
need to modify before Cassandra will start up.  Right now we can tell people 
paths are all in cassandra.yaml which keeps things simple.

Maybe putting it in /tmp would be better from that perspective.

 Able to set path location of HeapDump in cassandra-env
 --

 Key: CASSANDRA-3056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
  Labels: lhf
 Attachments: CASSANDRA-3056-1.txt


 We should be able to designate the path location to put any perf dumps that 
 are performed. By Default with this not set the perf dump can occur on the 
 root disk and fill the drive. 
 Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS 
 -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option 
 available and set. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3095) java.lang.NegativeArraySizeException during compacting large row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3095:
--

Attachment: 3095-debug.txt

Can you try the attached patch against 0.8 head to get some debug logging about 
the BF being created?

 java.lang.NegativeArraySizeException during compacting large row
 

 Key: CASSANDRA-3095
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3095
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
 Environment: Linux 2.6.26-2-amd64 #1 SMP Thu Feb 11 00:59:32 UTC 2010 
 x86_64 GNU/Linux
 JDK 1.6.0_27 (Java 6 update 27), with JNA.
Reporter: Pas
 Attachments: 3095-debug.txt


 Hello,
 It's a 4 node ring, 3 on 0.7.4, I've upgraded one to 0.8.4. This particular 
 node was having issues with compaction that's why I've tried the upgrade (it 
 looks likely that this solved the compaction issues).
 Here's the stack trace from system.log.
  INFO [CompactionExecutor:22] 2011-08-28 18:12:46,566 
 CompactionController.java (line 136) Compacting large row  (36028797018963968 
 bytes) incrementally
 ERROR [CompactionExecutor:22] 2011-08-28 18:12:46,609 
 AbstractCassandraDaemon.java (line 134) Fatal exception in thread 
 Thread[CompactionExecutor:22,1,main]
 java.lang.NegativeArraySizeException
 at 
 org.apache.cassandra.utils.obs.OpenBitSet.init(OpenBitSet.java:85)
 at 
 org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:56)
 at 
 org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:73)
 at 
 org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62)
 at 
 org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
 at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.init(LazilyCompactedRow.java:89)
 at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:138)
 at 
 org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:123)
 at 
 org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:43)
 at 
 org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:74)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 at 
 org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
 at 
 org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:569)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:506)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:141)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:107)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 We've ~70 files still in f format. And 80 in g. We've ~100 GB of data on 
 this node.
 Thanks.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-3093) delete multiple rows on secondary equals


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3093.
---

Resolution: Won't Fix

I think it's a good thing to make it explicit that fetch rows matching 
condition X and delete rows are inherently two separate operations.

 delete multiple rows on secondary equals
 

 Key: CASSANDRA-3093
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3093
 Project: Cassandra
  Issue Type: Improvement
Reporter: Tongguo Pang

 For now if we want to delete rows on secondary index equals, we have to read 
 the keys back. This very inefficient, especially when the result set is 
 big(for example,  1,000,000 rows). If the delete operation can accept a 
 secondary index query as input, that will be very helpful for this situation

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092853#comment-13092853
 ] 

Jonathan Ellis commented on CASSANDRA-3091:
---

I feel like the cure might be worse than the disease here.  Schema typically 
only changes underneath a connection during development -- if you're changing 
schema in production, you're going to redeploy your app to take advantage of 
that, creating new connections.  So the benefit feels small vs the cost of 
re-requesting and parsing metadata for each Statement.

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092858#comment-13092858
 ] 

Jonathan Ellis commented on CASSANDRA-3078:
---

bq. require specifying the classname as a index_option

I like that approach too.

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-pluggable-secondary-indexes.txt, 
 v1-0002-CASSANDRA-3078-pluggable-secondary-index-thrift.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3099) Counters are not always hinted


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-3099:


Attachment: 0001-hint-counters.patch

 Counters are not always hinted
 --

 Key: CASSANDRA-3099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 0.8.5

 Attachments: 0001-hint-counters.patch


 CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints 
 that were supposed to be stored on the local node (that is, instead of 
 removing from the hintedEndpoints multimap only the local write (since it has 
 been already applied), we were removing everything having the local node as 
 destination).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3099) Counters are not always hinted

Counters are not always hinted
--

 Key: CASSANDRA-3099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 0.8.5
 Attachments: 0001-hint-counters.patch

CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints 
that were supposed to be stored on the local node (that is, instead of removing 
from the hintedEndpoints multimap only the local write (since it has been 
already applied), we were removing everything having the local node as 
destination).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092862#comment-13092862
 ] 

Rick Shaw commented on CASSANDRA-3091:
--

It is cached in the statement as well so it will do the fetch one time when the 
statement is first created and then just check the cache to make sure the 
metadata is there and attempt a reload only if it does not find it. Once you 
acquire a statement you usually hold on to it for the duration of your task. So 
not significantly more than if you acquired a connection.

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3099) Counters are not always hinted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092868#comment-13092868
 ] 

Jonathan Ellis commented on CASSANDRA-3099:
---

+1

 Counters are not always hinted
 --

 Key: CASSANDRA-3099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 0.8.5

 Attachments: 0001-hint-counters.patch


 CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints 
 that were supposed to be stored on the local node (that is, instead of 
 removing from the hintedEndpoints multimap only the local write (since it has 
 been already applied), we were removing everything having the local node as 
 destination).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092869#comment-13092869
 ] 

Rick Shaw commented on CASSANDRA-3091:
--

In JDBC you can pool both!

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092873#comment-13092873
 ] 

Jonathan Ellis commented on CASSANDRA-3091:
---

I suppose, but if you're pooling statements, how is making metadata 
per-statement a win?

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092877#comment-13092877
 ] 

Rick Shaw commented on CASSANDRA-3091:
--

This approach puts the caller in better control of when the refreshed data is 
acquired. It is easy to close one statement and reacquire. If you do with a 
pooled resource it will not reacquire until the physical connection goes stale 
from no use. On a busy system and when the app acquires the connection itself 
infrequently, it may be a long time before you see a new refreshed set of data 
in a new connection.

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2942) If you drop a CF when one node is down the files are orphaned on the downed node


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092884#comment-13092884
 ] 

Sylvain Lebresne commented on CASSANDRA-2942:
-

nit: we could log an info message when awaitTermination returns false. It also 
look like DeletionService could just go away with with this.

But otherwise, +1. I agree it is good enough and not worth going for more 
complicated.

 If you drop a CF when one node is down the files are orphaned on the downed 
 node
 

 Key: CASSANDRA-2942
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2942
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Cathy Daw
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.0

 Attachments: 2942.txt


 * Bring up 3 node cluster
 * From node1: Run Stress Tool
 {code} stress --num-keys=10 --columns=10 --consistency-level=ALL 
 --average-size-values --replication-factor=3 --nodes=node1,node2 {code}
 * Shutdown node3
 * From node1: drop the Standard1 CF in Keyspace1
 * Shutdown node2 and node3
 * Bring up node1 and node2. Check that the Standard1 files are gone.
 {code}
 ls -al /var/lib/cassandra/data/Keyspace1/
 {code}
 * Bring up node3. The log file shows the drop column family occurs
 {code}
  INFO 00:51:25,742 Applying migration 9a76f880-b4c5-11e0--8901a7c5c9ce 
 Drop column family: Keyspace1.Standard1
 {code}
 * Restart node3 to clear out dropped tables from the filesystem
 {code}
 root@cathy3:~/cass-0.8/bin# ls -al /var/lib/cassandra/data/Keyspace1/
 total 36
 drwxr-xr-x 3 root root 4096 Jul 23 00:51 .
 drwxr-xr-x 6 root root 4096 Jul 23 00:48 ..
 -rw-r--r-- 1 root root0 Jul 23 00:51 Standard1-g-1-Compacted
 -rw-r--r-- 2 root root 5770 Jul 23 00:51 Standard1-g-1-Data.db
 -rw-r--r-- 2 root root   32 Jul 23 00:51 Standard1-g-1-Filter.db
 -rw-r--r-- 2 root root  120 Jul 23 00:51 Standard1-g-1-Index.db
 -rw-r--r-- 2 root root 4276 Jul 23 00:51 Standard1-g-1-Statistics.db
 drwxr-xr-x 3 root root 4096 Jul 23 00:51 snapshots
 {code}
 *Bug:  The files for Standard1 are orphaned on node3*

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092883#comment-13092883
 ] 

Jonathan Ellis commented on CASSANDRA-3025:
---

Thanks!

bq. Parsing into object would be problematic if the UUID is used as a column 
name

Why is that?  Does PDO require only primitive or string column names?

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, 
 php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3061) Optionally skip log4j configuration

[
https://issues.apache.org/jira/browse/CASSANDRA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092886#comment-13092886
]

T Jake Luciani commented on CASSANDRA-3061:
---

Aaron any update here? this is working fine for me.

Optionally skip log4j configuration
---

Key: CASSANDRA-3061
URL: https://issues.apache.org/jira/browse/CASSANDRA-3061
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 0.8.4
Reporter: Aaron Morton
Assignee: Aaron Morton
Priority: Minor
Fix For: 0.8.5

Attachments: 0001-3061.patch, 3061_v2.txt

from this thread
http://groups.google.com/group/brisk-users/browse_thread/thread/3a18f4679673bea8
When brisk accesses cassandra classes inside of a Hadoop Task JVM the
AbstractCassandraDaemon uses a log4j PropertyConfigurator to setup cassandra
logging. This closes all the existing appenders, including the
TaskLogAppender for the hadoop task. They are not opened again because they
are not in the config.
log4j has Logger Repositories to handle multiple configs in the same process,
but there is a bit of suck involved in making a RepositorySelector.
Two examples...
http://www.mail-archive.com/log4j-dev@jakarta.apache.org/msg02972.html
http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2/html/Getting_Started_Guide/logging.log4j.reposelect.html
Basically all the selector has access to thread local storage, and it looks
like normally people get the class loader from the current thread. A thread
will inherit it's class loader from the thread that created it, unless
otherwise specified.
We have code in the same thread the uses hadoop and cassandra classes, so
this could be a dead end.
As a work around i've added cassandra.log4j.configure JVM param and made the
AbstractCassandraServer skip the log4j config if it's false. My job completes
and I can see the cassandra code logging an extra message I put in into the
Hadoop task log file...
2011-08-19 15:56:06,442 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics system not
started: Cannot locate configuration: tried
hadoop-metrics2-maptask.properties, hadoop-metrics2.properties
2011-08-19 15:56:06,776 INFO
org.apache.cassandra.service.AbstractCassandraDaemon: Logging initialized
externally
2011-08-19 15:56:07,332 INFO org.apache.hadoop.mapred.MapTask:
numReduceTasks: 0

The param has to be passed to the task JVM, so need to modify Haddop
mapred-site.xml as follows
property
namemapred.child.java.opts/name
value-Xmx256m -Dcassandra.log4j.configure=false/value
description
Tune your mapred jvm arguments for best performance.
Also see documentation from jvm vendor.
/description
/property
It's not pretty but it works. In my extra log4j logging I can see the second
reset() call is gone.
Change the to Hadoop TaskLogAppender also stops the NPE but there may also be
some lost log messages
https://issues.apache.org/jira/browse/HADOOP-7556

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092887#comment-13092887
 ] 

Jonathan Ellis commented on CASSANDRA-3091:
---

What would that look like with statement pooling?

 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092885#comment-13092885
 ] 

Jonathan Ellis commented on CASSANDRA-3025:
---

Still to-do: 

bq. should test an actual big int ( 64bit)

anything else?

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, 
 php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2857) initialize log4j correctly in EmbeddedCassandraService


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092890#comment-13092890
 ] 

T Jake Luciani commented on CASSANDRA-2857:
---

needs rebase but +1

 initialize log4j correctly in EmbeddedCassandraService
 --

 Key: CASSANDRA-2857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2857
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Trivial
 Fix For: 0.8.5

 Attachments: 2857-drivers.txt, 2857.txt


 Currently, ECS.cleanUpOldStuff calls CleanupHelper.cleanupAndLeaveDirs(), 
 which initialized DatabaseDescriptor which does some logging.  When we go to 
 initialize log4j later in AbstractCassandraService, it's too late.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162844 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/service/StorageProxy.java

2011-08-29 Thread slebresne

Author: slebresne
Date: Mon Aug 29 15:04:57 2011
New Revision: 1162844

URL: http://svn.apache.org/viewvc?rev=1162844view=rev
Log:
Always hint counters
patch by slebresne; reviewed by jbellis for CASSANDRA-3099

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1162844r1=1162843r2=1162844view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Aug 29 15:04:57 2011
@@ -37,6 +37,7 @@
  * fix UnavailableException with writes at CL.EACH_QUORM (CASSANDRA-3084)
  * fix parsing of the Keyspace and ColumnFamily names in numeric
and string representations in CLI (CASSANDRA-3075)
+ * always hint counters (CASSANDRA-3099)
 
 0.8.4
  * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972)

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1162844r1=1162843r2=1162844view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java
 Mon Aug 29 15:04:57 2011
@@ -446,7 +446,8 @@ public class StorageProxy implements Sto
 responseHandler.response(null);
 
 // then send to replicas, if any
-hintedEndpoints.removeAll(FBUtilities.getLocalAddress());
+InetAddress local = FBUtilities.getLocalAddress();
+hintedEndpoints.remove(local, local);
 if (cm.shouldReplicateOnWrite()  !hintedEndpoints.isEmpty())
 {
 // We do the replication on another stage because it 
involves a read (see CM.makeReplicationMutation)

[jira] [Updated] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-3051:
---

Attachment: CASSANDRA-3051.patch

Removed SSLFileStreamTask and added EncryptionOptions to the constructor of the 
FileStreamTask.

Rebased with latest trunk (last commit 0a4b1667bee674f7c0a22057cbdab97e368a20d1)

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162846 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/

2011-08-29 Thread slebresne

Author: slebresne
Date: Mon Aug 29 15:07:04 2011
New Revision: 1162846

URL: http://svn.apache.org/viewvc?rev=1162846view=rev
Log:
merge from 0.8

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:07:04 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
 /cassandra/branches/cassandra-0.7:1026516-1160444,1160825,1161607
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162846r1=1162845r2=1162846view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:07:04 2011
@@ -78,6 +78,7 @@
  * avoid including cwd in classpath for deb and rpm packages (CASSANDRA-2881)
  * remove gossip state when a new IP takes over a token (CASSANDRA-3071)
  * allow sstable2json to work on index sstable files (CASSANDRA-3059)
+ * always hint counters (CASSANDRA-3099)
 
 
 0.8.4

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:07:04 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
 /cassandra/branches/cassandra-0.7/contrib:1026516-1160444,1160825,1161607
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:07:04 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1160444,1160825,1161607
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
-/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1161708
+/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1161708,1162844
 
/cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369
 
/cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:07:04 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291
 
/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1160444,1160825,1161607

[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901
 ] 

Jeremy Hanna commented on CASSANDRA-3056:
-

Do people generally configure their systems to have enough space in /tmp for 
the whole Cassandra heap?

 Able to set path location of HeapDump in cassandra-env
 --

 Key: CASSANDRA-3056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
  Labels: lhf
 Attachments: CASSANDRA-3056-1.txt


 We should be able to designate the path location to put any perf dumps that 
 are performed. By Default with this not set the perf dump can occur on the 
 root disk and fill the drive. 
 Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS 
 -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option 
 available and set. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162849 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/db/commitlog/ src/java/org/apache/cassandra/io/ src/java/org/apache/cassandra/io/util/ src/java/org/apache/cassandra/servic

Author: jbellis
Date: Mon Aug 29 15:15:32 2011
New Revision: 1162849

URL: http://svn.apache.org/viewvc?rev=1162849view=rev
Log:
reduce window where dropped CF sstables may not be deleted
patch by jbellis; reviewed by slebresne for CASSANDRA-2942

Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
cassandra/trunk/src/java/org/apache/cassandra/io/DeletionService.java
cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162849r1=1162848r2=1162849view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:15:32 2011
@@ -46,6 +46,8 @@
  * Cli to use hashes rather than array of hashes for strategy options 
(CASSANDRA-3081)
  * LeveledCompactionStrategy (CASSANDRA-1608)
  * Improvements of the CLI `describe` command (CASSANDRA-2630)
+ * reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942)
+
 
 0.8.5
  * fix NPE when encryption_options is unspecified (CASSANDRA-3007)

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java?rev=1162849r1=1162848r2=1162849view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
Mon Aug 29 15:15:32 2011
@@ -42,7 +42,6 @@ import org.apache.cassandra.concurrent.S
 import org.apache.cassandra.concurrent.StageManager;
 import org.apache.cassandra.config.Config;
 import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.io.DeletionService;
 import org.apache.cassandra.io.util.FastByteArrayInputStream;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
@@ -486,7 +485,7 @@ public class CommitLog implements Commit
 {
 logger.info(Discarding obsolete commit log: + segment);
 segment.close();
-DeletionService.executeDelete(segment.getPath());
+FileUtils.deleteAsync(segment.getPath());
 // usually this will be the first (remaining) segment, but not 
always, if segment A contains
 // writes to a CF that is unflushed but is followed by segment B 
whose CFs are all flushed.
 iter.remove();

Modified: cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java?rev=1162849r1=1162848r2=1162849view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java Mon 
Aug 29 15:15:32 2011
@@ -20,13 +20,15 @@ package org.apache.cassandra.io.util;
 
 import java.io.*;
 import java.text.DecimalFormat;
-import java.util.Collection;
 import java.util.Comparator;
 import java.util.List;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.WrappedRunnable;
+
 
 public class FileUtils
 {
@@ -167,6 +169,18 @@ public class FileUtils
 }
 }
 
+public static void deleteAsync(final String file)
+{
+Runnable runnable = new WrappedRunnable()
+{
+protected void runMayThrow() throws IOException
+{
+deleteWithConfirm(new File(file));
+}
+};
+StorageService.tasks.execute(runnable);
+}
+
 public static String stringifyFileSize(double value)
 {
 double d;

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1162849r1=1162848r2=1162849view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java 
Mon Aug 29 15:15:32 2011
@@ -46,7 +46,6 @@ import org.apache.cassandra.db.*;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.dht.*;
 import org.apache.cassandra.gms.*;
-import org.apache.cassandra.io.DeletionService;
 import org.apache.cassandra.io.sstable.SSTableDeletingTask;
 import org.apache.cassandra.io.sstable.SSTableLoader;
 import

[jira] [Commented] (CASSANDRA-3091) Move the caching of KS and CF metadata in the JDBC suite from Connection to Statement


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092904#comment-13092904
 ] 

Rick Shaw commented on CASSANDRA-3091:
--

I think (in retrospect) I would clear the cache upon close for either approach 
so it would force a reload on acquisition, so that may be a red herring. I 
think the only issue is: is doing it in the statement any more expensive in 
time vs the advantage of a more current set of metadata? If the general usage 
pattern is to open a lot of statements vs opening connections, then it is 
undeniably more time intensive to do in statement but the benefit is you would 
SELDOM have mismatch with the state of the server. If the average use is to 
open a connection and then open a statement and do all accesses on the 
statement then its not really a penalty in time and the additional ability to 
do a reload on demand when needed in the statement (at the price of the boolean 
check for CF metadata) is a win.

If you think this is ill advised having said all that :) we can close the 
ticket, and I'll put some of the additional cleanup of the {{ColumnDecoder}} in 
another related patch at a later date. I could also look to see if I could get 
the check for missing metadata against the data hanging off connection, but 
that's along way from {{ColumnDecoder}} where the info is needed.


 Move the caching of KS and CF metadata in the JDBC suite from Connection to 
 Statement
 -

 Key: CASSANDRA-3091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3091
 Project: Cassandra
  Issue Type: Improvement
  Components: Drivers
Affects Versions: 0.8.4
Reporter: Rick Shaw
Assignee: Rick Shaw
Priority: Minor
  Labels: JDBC
 Fix For: 0.8.5

 Attachments: move-metadata-for decoder-to-statement-level-v1.txt, 
 move-metadata-for-decoder-to-statement-level-v2.txt


 Currently, all caching of metadata used in JDBC's {{ColumnDecoder}} class is 
 loaded and held in the {{CassandraConnection}} class. The implication of this 
 is that any activity on the connected server from the time the connection is 
 established is not reflected in the KSs and CF that can be accessed by the 
 {{ResultSet, Statement}} and {{PreparedStatement}}.
 By moving the cached metadata to the {{Statement}} level, the currency of the 
 metadata can be checked within the {{Statement}} and reloaded if it is seen 
 to be absent. And by instantiating a new {{Statement}} (on any existing 
 connection) you are assured of getting the most current copy of the metadata 
 known to the server at the new time of instantiation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162851 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java src/java/org/apache/cassandra/thrift/CassandraDaemon.java

Author: jbellis
Date: Mon Aug 29 15:19:16 2011
New Revision: 1162851

URL: http://svn.apache.org/viewvc?rev=1162851view=rev
Log:
fix log4j initialization in EmbeddedCassandraService
patch by jbellis; reviewed by tjake for CASSANDRA-2857

Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java

cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java

Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1162851r1=1162850r2=1162851view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Aug 29 15:19:16 2011
@@ -38,6 +38,8 @@
  * fix parsing of the Keyspace and ColumnFamily names in numeric
and string representations in CLI (CASSANDRA-3075)
  * always hint counters (CASSANDRA-3099)
+ * fix log4j initialization in EmbeddedCassandraService (CASSANDRA-2857)
+
 
 0.8.4
  * include files-to-be-streamed in StreamInSession.getSources (CASSANDRA-2972)

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java?rev=1162851r1=1162850r2=1162851view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
 Mon Aug 29 15:19:16 2011
@@ -62,17 +62,19 @@ import org.apache.cassandra.utils.Mx4jTo
  */
 public abstract class AbstractCassandraDaemon implements CassandraDaemon
 {
-//Initialize logging in such a way that it checks for config changes every 
10 seconds.
-static
+/**
+ * Initialize logging in such a way that it checks for config changes 
every 10 seconds.
+ */
+public static void initLog4j()
 {
 String config = System.getProperty(log4j.configuration, 
log4j-server.properties);
 URL configLocation = null;
-try 
+try
 {
 // try loading from a physical location first.
 configLocation = new URL(config);
 }
-catch (MalformedURLException ex) 
+catch (MalformedURLException ex)
 {
 // then try loading from the classpath.
 configLocation = 
AbstractCassandraDaemon.class.getClassLoader().getResource(config);

Modified: 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java?rev=1162851r1=1162850r2=1162851view=diff
==
--- 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java
 (original)
+++ 
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java
 Mon Aug 29 15:19:16 2011
@@ -26,6 +26,7 @@ import java.util.concurrent.ExecutorServ
 import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.TimeUnit;
 
+import org.apache.cassandra.service.AbstractCassandraDaemon;
 import org.apache.thrift.server.TNonblockingServer;
 import org.apache.thrift.server.TThreadPoolServer;
 import org.slf4j.Logger;
@@ -53,6 +54,11 @@ import org.apache.thrift.transport.TTran
 
 public class CassandraDaemon extends 
org.apache.cassandra.service.AbstractCassandraDaemon
 {
+static
+{
+AbstractCassandraDaemon.initLog4j();
+}
+
 private static Logger logger = 
LoggerFactory.getLogger(CassandraDaemon.class);
 private final static String SYNC = sync;
 private final static String ASYNC = async;

svn commit: r1162860 - in /cassandra/trunk: ./ contrib/ drivers/java/test/org/apache/cassandra/cql/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/service/ src/ja

Author: jbellis
Date: Mon Aug 29 15:24:08 2011
New Revision: 1162860

URL: http://svn.apache.org/viewvc?rev=1162860view=rev
Log:
merge from 0.8

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/CHANGES.txt
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)

cassandra/trunk/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraDaemon.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:24:08 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291
 /cassandra/branches/cassandra-0.7:1026516-1160444,1160825,1161607
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
-/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844
+/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1161708,1162844,1162851
 /cassandra/branches/cassandra-0.8.0:1125021-1130369
 /cassandra/branches/cassandra-0.8.1:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162860r1=1162859r2=1162860view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Aug 29 15:24:08 2011
@@ -81,6 +81,7 @@
  * remove gossip state when a new IP takes over a token (CASSANDRA-3071)
  * allow sstable2json to work on index sstable files (CASSANDRA-3059)
  * always hint counters (CASSANDRA-3099)
+ * fix log4j initialization in EmbeddedCassandraService (CASSANDRA-2857)
 
 
 0.8.4

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:24:08 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
 /cassandra/branches/cassandra-0.7/contrib:1026516-1160444,1160825,1161607
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
-/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844
+/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1161708,1162844,1162851
 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369
 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689

Modified: 
cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java?rev=1162860r1=1162859r2=1162860view=diff
==
--- 
cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java
 (original)
+++ 
cassandra/trunk/drivers/java/test/org/apache/cassandra/cql/EmbeddedServiceBase.java
 Mon Aug 29 15:24:08 2011
@@ -28,6 +28,7 @@ import java.net.UnknownHostException;
 import org.apache.cassandra.CleanupHelper;
 import org.apache.cassandra.SchemaLoader;
 import org.apache.cassandra.config.*;
+import org.apache.cassandra.service.AbstractCassandraDaemon;
 import org.apache.cassandra.service.EmbeddedCassandraService;
 import org.junit.BeforeClass;
 
@@ -39,7 +40,12 @@ public abstract class EmbeddedServiceBas
 /** The embedded server cassandra. */
 private static EmbeddedCassandraService cassandra;
 
-@BeforeClass 
+static
+{
+AbstractCassandraDaemon.initLog4j();
+}
+
+@BeforeClass
 public static void cleanUpOldStuff() throws IOException
 {
 CleanupHelper.cleanupAndLeaveDirs();

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Aug 29 15:24:08 2011
@@ -1,7 +1,7 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092915#comment-13092915
 ] 

Pavel Yaskevich commented on CASSANDRA-3051:


I don't see anh reason why it won't :) but can't write a test because SSL is 
treacky with it's stores...

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2034) Make Read Repair unnecessary when Hinted Handoff is enabled


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2034:
--

Attachment: 2034-v19-rebased.txt

rebased v19.  no other changes made.

 Make Read Repair unnecessary when Hinted Handoff is enabled
 ---

 Key: CASSANDRA-2034
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2034
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Patricio Echague
 Fix For: 1.0

 Attachments: 2034-formatting.txt, 2034-v16.txt, 2034-v17.txt, 
 2034-v18.txt, 2034-v19-rebased.txt, 2034-v19.txt, 
 CASSANDRA-2034-trunk-v10.patch, CASSANDRA-2034-trunk-v11.patch, 
 CASSANDRA-2034-trunk-v11.patch, CASSANDRA-2034-trunk-v12.patch, 
 CASSANDRA-2034-trunk-v13.patch, CASSANDRA-2034-trunk-v14.patch, 
 CASSANDRA-2034-trunk-v15.patch, CASSANDRA-2034-trunk-v2.patch, 
 CASSANDRA-2034-trunk-v3.patch, CASSANDRA-2034-trunk-v4.patch, 
 CASSANDRA-2034-trunk-v5.patch, CASSANDRA-2034-trunk-v6.patch, 
 CASSANDRA-2034-trunk-v7.patch, CASSANDRA-2034-trunk-v8.patch, 
 CASSANDRA-2034-trunk-v9.patch, CASSANDRA-2034-trunk.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 Currently, HH is purely an optimization -- if a machine goes down, enabling 
 HH means RR/AES will have less work to do, but you can't disable RR entirely 
 in most situations since HH doesn't kick in until the FailureDetector does.
 Let's add a scheduled task to the mutate path, such that we return to the 
 client normally after ConsistencyLevel is achieved, but after RpcTimeout we 
 check the responseHandler write acks and write local hints for any missing 
 targets.
 This would making disabling RR when HH is enabled a much more reasonable 
 option, which has a huge impact on read throughput.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3100) Secondary index still does minor compacting after deleting index

Secondary index still does minor compacting after deleting index


 Key: CASSANDRA-3100
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3100
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.8
Reporter: Jeremy Hanna


We deleted all of our secondary indexes.  A couple of days later I was watching 
compactionstats on one of the nodes and it was in the process of minor 
compacting one of the deleted secondary indexes.  I double checked the keyspace 
definitions on the CLI and there were no secondary indexes defined.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092921#comment-13092921
 ] 

Gary Dusbabek commented on CASSANDRA-3051:
--

wait, you tested it locally first, right?  It's not difficult to set up a 
streaming environment locally.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092924#comment-13092924
 ] 

Jonathan Ellis commented on CASSANDRA-3051:
---

do we have a ssl howto somewhere?  I was hoping it would be in cassandra.yaml 
by encryption_options but no.  Or at least, not sufficiently for dummies for 
me.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092929#comment-13092929
 ] 

Pavel Yaskevich commented on CASSANDRA-3051:


No, unfortunately I haven't found any info about how to do that so you are 
welcome to test if you can...

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting

[
https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092928#comment-13092928
]

Sylvain Lebresne commented on CASSANDRA-3085:
-

* The try of the try...finally block should start just after the markReferenced
to be safe in CollationControler.
* In CFS.getRangeSlice, I am not confortable using the same try...finally block
for both the view and the iterator. RowIteratorFactory.getIterator() does make
seeks and could throw an error that would leave a bunch of sstable referenced.
* Nit: Maybe we could use a plain old DataTracker.View instead of introducing
ViewFragment, since the View constructor is accessible to CFS anyway ?

Race condition in sstable reference counting

Key: CASSANDRA-3085
URL: https://issues.apache.org/jira/browse/CASSANDRA-3085
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Critical
Fix For: 1.0

Attachments: 3085-v2.txt, 3085.txt

DataTracker gives us an atomic View of memtable/sstables, but acquiring
references is not atomic. So it is possible to acquire references to an
SSTableReader object that is no longer valid, as in this example:
View V contains sstables {A, B}. We attempt a read in thread T using this
View.
Meanwhile, A and B are compacted to {C}, yielding View W. No references
exist to A or B so they are cleaned up.
Back in thread T we acquire references to A and B. This does not cause an
error, but it will when we attempt to read from them next.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092930#comment-13092930
 ] 

Gary Dusbabek commented on CASSANDRA-3051:
--

Not that I know of.  If someone wants to write one it would flesh out these 
basic steps:
# follow the steps for generating a keystore and a trust store here: 
http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore
# plug those files into encryption_options in cassandra.yaml
# make sure encryption_options.internode_encryption = all in the yaml.

I was mostly raising a voice of caution against committing code backed up by 
statements like I don't see anh reason why it won't...  This is usually a 
prelude to something like we need to quickly release a new version because XYZ 
broke streaming.  Just sayin'.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env

2011-08-29 Thread satish babu krishnamoorthy (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

satish babu krishnamoorthy updated CASSANDRA-3056:
--

Attachment: CASSANDRA-3056-2.txt

Updated the patch to set HeapDump in cassandra-env.sh, 
this value is parsed in the following order.
1) look for jvm_heap_dump_directory in cassandra.yaml
2) if jvm_heap_dump_directory is not set in cassandra.yaml, use /tmp


 Able to set path location of HeapDump in cassandra-env
 --

 Key: CASSANDRA-3056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
  Labels: lhf
 Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt


 We should be able to designate the path location to put any perf dumps that 
 are performed. By Default with this not set the perf dump can occur on the 
 root disk and fill the drive. 
 Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS 
 -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option 
 available and set. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting

[
https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092949#comment-13092949
]

Sylvain Lebresne commented on CASSANDRA-3085:
-

I would argue that even now a View is not meant to be updated either (we pick
new Views but a View itself is fixed), nor does it really ensure that it
contains the full set of sstables in a way since that set is a moving target.
But I suppose this is just a matter of perspective on what View is, and I'm
fine with ViewFragment.

Race condition in sstable reference counting

Attachments: 3085-v2.txt, 3085.txt

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092951#comment-13092951
 ] 

Pavel Yaskevich commented on CASSANDRA-3051:


Following your logic - person who was working on ssl should've done that at 
first place, there is no guarantee that it works in the current state. I'm not 
pushing things forward just wondering why testing wasn't done before.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901
 ] 

Jeremy Hanna edited comment on CASSANDRA-3056 at 8/29/11 4:20 PM:
--

Do people generally configure their systems to have enough space in /tmp for 
the whole Cassandra heap?  I guess /tmp is generally the best place though.

  was (Author: jeromatron):
Do people generally configure their systems to have enough space in /tmp 
for the whole Cassandra heap?
  
 Able to set path location of HeapDump in cassandra-env
 --

 Key: CASSANDRA-3056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
  Labels: lhf
 Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt


 We should be able to designate the path location to put any perf dumps that 
 are performed. By Default with this not set the perf dump can occur on the 
 root disk and fill the drive. 
 Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS 
 -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option 
 available and set. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092901#comment-13092901
 ] 

Jeremy Hanna edited comment on CASSANDRA-3056 at 8/29/11 4:21 PM:
--

Do people generally configure their systems to have enough space in /tmp for 
the whole Cassandra heap?  What about the cassandra log directory as a default? 
 /tmp seems fine but this bit us because the heap dump is so big.

  was (Author: jeromatron):
Do people generally configure their systems to have enough space in /tmp 
for the whole Cassandra heap?  I guess /tmp is generally the best place though.
  
 Able to set path location of HeapDump in cassandra-env
 --

 Key: CASSANDRA-3056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
  Labels: lhf
 Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt


 We should be able to designate the path location to put any perf dumps that 
 are performed. By Default with this not set the perf dump can occur on the 
 root disk and fill the drive. 
 Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS 
 -XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option 
 available and set. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3099) Counters are not always hinted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092953#comment-13092953
 ] 

Hudson commented on CASSANDRA-3099:
---

Integrated in Cassandra-0.8 #298 (See 
[https://builds.apache.org/job/Cassandra-0.8/298/])
Always hint counters
patch by slebresne; reviewed by jbellis for CASSANDRA-3099

slebresne : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162844
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageProxy.java


 Counters are not always hinted
 --

 Key: CASSANDRA-3099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 0.8.5

 Attachments: 0001-hint-counters.patch


 CASSANDRA-2892 mistakenly removed some hints for counters, namely the hints 
 that were supposed to be stored on the local node (that is, instead of 
 removing from the hintedEndpoints multimap only the local write (since it has 
 been already applied), we were removing everything having the local node as 
 destination).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2942) If you drop a CF when one node is down the files are orphaned on the downed node


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092955#comment-13092955
 ] 

Hudson commented on CASSANDRA-2942:
---

Integrated in Cassandra #1054 (See 
[https://builds.apache.org/job/Cassandra/1054/])
reduce window where dropped CF sstables may not be deleted
patch by jbellis; reviewed by slebresne for CASSANDRA-2942

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162849
Files : 
* /cassandra/trunk/CHANGES.txt
* /cassandra/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
* /cassandra/trunk/src/java/org/apache/cassandra/io/DeletionService.java
* /cassandra/trunk/src/java/org/apache/cassandra/io/util/FileUtils.java
* /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java


 If you drop a CF when one node is down the files are orphaned on the downed 
 node
 

 Key: CASSANDRA-2942
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2942
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Cathy Daw
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.0

 Attachments: 2942.txt


 * Bring up 3 node cluster
 * From node1: Run Stress Tool
 {code} stress --num-keys=10 --columns=10 --consistency-level=ALL 
 --average-size-values --replication-factor=3 --nodes=node1,node2 {code}
 * Shutdown node3
 * From node1: drop the Standard1 CF in Keyspace1
 * Shutdown node2 and node3
 * Bring up node1 and node2. Check that the Standard1 files are gone.
 {code}
 ls -al /var/lib/cassandra/data/Keyspace1/
 {code}
 * Bring up node3. The log file shows the drop column family occurs
 {code}
  INFO 00:51:25,742 Applying migration 9a76f880-b4c5-11e0--8901a7c5c9ce 
 Drop column family: Keyspace1.Standard1
 {code}
 * Restart node3 to clear out dropped tables from the filesystem
 {code}
 root@cathy3:~/cass-0.8/bin# ls -al /var/lib/cassandra/data/Keyspace1/
 total 36
 drwxr-xr-x 3 root root 4096 Jul 23 00:51 .
 drwxr-xr-x 6 root root 4096 Jul 23 00:48 ..
 -rw-r--r-- 1 root root0 Jul 23 00:51 Standard1-g-1-Compacted
 -rw-r--r-- 2 root root 5770 Jul 23 00:51 Standard1-g-1-Data.db
 -rw-r--r-- 2 root root   32 Jul 23 00:51 Standard1-g-1-Filter.db
 -rw-r--r-- 2 root root  120 Jul 23 00:51 Standard1-g-1-Index.db
 -rw-r--r-- 2 root root 4276 Jul 23 00:51 Standard1-g-1-Statistics.db
 drwxr-xr-x 3 root root 4096 Jul 23 00:51 snapshots
 {code}
 *Bug:  The files for Standard1 are orphaned on node3*

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2857) initialize log4j correctly in EmbeddedCassandraService


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092954#comment-13092954
 ] 

Hudson commented on CASSANDRA-2857:
---

Integrated in Cassandra-0.8 #298 (See 
[https://builds.apache.org/job/Cassandra-0.8/298/])
fix log4j initialization in EmbeddedCassandraService
patch by jbellis; reviewed by tjake for CASSANDRA-2857

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1162851
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/AbstractCassandraDaemon.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java


 initialize log4j correctly in EmbeddedCassandraService
 --

 Key: CASSANDRA-2857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2857
 Project: Cassandra
  Issue Type: Bug
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Trivial
 Fix For: 0.8.5

 Attachments: 2857-drivers.txt, 2857.txt


 Currently, ECS.cleanUpOldStuff calls CleanupHelper.cleanupAndLeaveDirs(), 
 which initialized DatabaseDescriptor which does some logging.  When we go to 
 initialize log4j later in AbstractCassandraService, it's too late.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption

[
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092957#comment-13092957
]

Gary Dusbabek edited comment on CASSANDRA-3051 at 8/29/11 4:25 PM:
---

Pavel, I tested SSL prior to committing the feature.

I was under the impression that this ticket exists because compression, when
enabled, breaks SSL. The implication is that it was working prior, else the
ticket would be something more like SSL is broken.

was (Author: gdusbabek):
Pavel, I tested SSL prior to committing the feature.

I was under the impression that ticket exists because compression, when
enabled, breaks SSL. The implication is that it was working prior, else the
ticket would be something more like SSL is broken.

On Disk Compression breaks SSL Encryption
-

Key: CASSANDRA-3051
URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.0
Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
Fix For: 1.0

Attachments: CASSANDRA-3051.patch

Encryption depends on FileStreamTask.write [1] protected member to be called
because the SSLFileStreamTask.write overrides this to write back to the
server.
When enabled, compression circumvents the call and the client does not
communicate using an SSL socket back to the server.
[1]
protected long write(FileChannel fc, PairLong, Long section, long length,
long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092957#comment-13092957
 ] 

Gary Dusbabek commented on CASSANDRA-3051:
--

Pavel, I tested SSL prior to committing the feature.

I was under the impression that ticket exists because compression, when 
enabled, breaks SSL.  The implication is that it was working prior, else the 
ticket would be something more like SSL is broken.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092967#comment-13092967
 ] 

Pavel Yaskevich commented on CASSANDRA-3051:


You misunderstood that, it is not breaking SSL it was just special cased in 
FileStreamTask so it wasn't using ssl socket. This patch removes special casing 
for SSL streaming by creating ssl socket directly in FileStreamTask if 
encryption options were set.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Tyler Hobbs (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-3084:
---

Attachment: 3084.txt
3084-unit-test.txt

3084.txt covers all cases of possible range differences.

3084-unit-test.txt provides unit tests that fail before the patch.

The logic is definitely a bit hairy here, so any thoughts on how to simplify it 
are welcome.

 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Tyler Hobbs (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-3084:
--

Assignee: Tyler Hobbs

 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: v1-0002-CASSANDRA-3078-thrift-generated-changes.txt
v1-0001-CASSANDRA-3078-add-custom-index-type.txt

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: (was: 
v1-0002-CASSANDRA-3078-pluggable-secondary-index-thrift.txt)

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: (was: 
v1-0001-CASSANDRA-3078-pluggable-secondary-indexes.txt)

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093005#comment-13093005
 ] 

T Jake Luciani commented on CASSANDRA-3078:
---

added CUSTOM type with a class_name parameter

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093010#comment-13093010
 ] 

Jonathan Ellis commented on CASSANDRA-3051:
---

Pavel, can you try to set up local SSL w/ a ccm cluster based on Gary's 
instructions to verify?

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Stu Hood (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stu Hood updated CASSANDRA-3084:


Reviewer: stuhood

 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Stu Hood (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093042#comment-13093042
 ] 

Stu Hood edited comment on CASSANDRA-3084 at 8/29/11 6:16 PM:
--

Couldn't a bunch of difference cases be eliminated by taking advantage of the 
intersection implementation? The difference between ranges x and y would be {{z 
= x.intersect\(y\); y.subtract(z)}}, and I think a subtract method with a y 
must contain z precondition would be much easier to implement.

  was (Author: stuhood):
Couldn't a bunch of difference cases be eliminated by taking advantage of 
the intersection implementation? The difference between ranges x and y would be 
{{z = x.intersect(y); y.subtract(z)}}, and I think a subtract method with a y 
must contain z precondition would be much easier to implement.
  
 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Stu Hood (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093042#comment-13093042
 ] 

Stu Hood commented on CASSANDRA-3084:
-

Couldn't a bunch of difference cases be eliminated by taking advantage of the 
intersection implementation? The difference between ranges x and y would be {{z 
= x.intersect(y); y.subtract(z)}}, and I think a subtract method with a y must 
contain z precondition would be much easier to implement.

 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093092#comment-13093092
 ] 

Jonathan Ellis commented on CASSANDRA-3078:
---

It would be a lot more clear to me that there isn't a subtle bug in the type - 
class conversion (and the thrift - native - avro surroundings) if you just 
added CUSTOM to the old switch statement instead of the indexTypes map business.

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3101) Should check for errors when calling /bin/ln

[
https://issues.apache.org/jira/browse/CASSANDRA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-3101:
--

Affects Version/s: (was: 0.8.4)
(was: 0.7.8)
0.4
Labels: lhf (was: )

Should check for errors when calling /bin/ln

Key: CASSANDRA-3101
URL: https://issues.apache.org/jira/browse/CASSANDRA-3101
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.4
Reporter: paul cannon
Priority: Minor
Labels: lhf

It looks like cassandra.utils.CLibrary.createHardLinkWithExec() does not
check for any errors in the execution of the hard-link-making utility. This
could be bad if, for example, the user has put the snapshot directory on a
different filesystem from the data directory. The hard linking would fail and
the sstable snapshots would not exist, but no error would be reported.
It does look like errors with the more direct JNA link() call are handled
correctly- an exception is thrown. The WithExec version should probably do
the same thing.
Definitely it would be enough to check the process exit value from /bin/ln
for nonzero in the *nix case, but I don't know whether 'fsutil hardlink
create' or 'cmd /c mklink /H' return nonzero on failure.
For bonus points, use any output from the Process's error stream in the text
of the exception, to aid in debugging problems.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3056) Able to set path location of HeapDump in cassandra-env

2011-08-29 Thread Eric Evans (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093110#comment-13093110
]

Eric Evans commented on CASSANDRA-3056:
---

cassandra-env.sh is used to setup the environment, (the jvm and everything
that is passed to it); cassandra.yaml is really the wrong place for this. But,
I agree with Jonathan that it doesn't seem right to add cassandra-env.sh to the
list of first-start criteria, particularly for something that only a small
fraction of users will be concerned with.

Why not make this env-only? If CASSANDRA_HEAP_DUMP_PATH (or whatever) is set,
then use it to construct the argument (from cassandra-env.sh).

Able to set path location of HeapDump in cassandra-env
--

Key: CASSANDRA-3056
URL: https://issues.apache.org/jira/browse/CASSANDRA-3056
Project: Cassandra
Issue Type: Improvement
Affects Versions: 0.7.8, 0.8.4
Reporter: David Talbott
Priority: Minor
Labels: lhf
Attachments: CASSANDRA-3056-1.txt, CASSANDRA-3056-2.txt

We should be able to designate the path location to put any perf dumps that
are performed. By Default with this not set the perf dump can occur on the
root disk and fill the drive.
Should be able to solve this by simply inserting JVM_OPTS=$JVM_OPTS
-XX:HeapDumpPath=path to dir into cassandra-env.sh as a default option
available and set.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3084) o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly

2011-08-29 Thread Tyler Hobbs (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-3084:
---

Attachment: 3084-v2.txt

Good idea!

3084-v2.txt simplifies the logic by adding a subtractContained() method.

 o.a.c.dht.Range.differenceToFetch() doesn't handle all cases correctly
 --

 Key: CASSANDRA-3084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3084
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Attachments: 3084-unit-test.txt, 3084-v2.txt, 3084.txt


 It's possible that differenceToFetch is making implicit assumptions about the 
 relationship between the two ranges, but the following cases are not handled 
 correctly (the old range is (A, B], the new is (C, D]:
 {noformat}
 --C--A-B--D--
 {noformat}
 Here, the result will be (C, A] and (D, B], instead of (C, A] and (B, D].
 {noformat}
 --C--A-D--B--
 {noformat}
 The result will be (C, D] instead of just (C, A].
 {noformat}
 --A--C-D--B--
 {noformat}
 The result will be (B, D] when nothing needs to be transfered.
 If there is some kind of implicit assumption that these cases won't arise, it 
 either needs to be explicit (assertions, exceptions) or the cases need to be 
 handled.  It should be easy to cover this with unit tests.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: (was: v1-0001-CASSANDRA-3078-add-custom-index-type.txt)

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, 
 v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, 
 v1-0004-CASSANDRA-3078-add-cli-show-support.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: (was: v1-0002-CASSANDRA-3078-thrift-generated-changes.txt)

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, 
 v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, 
 v1-0004-CASSANDRA-3078-add-cli-show-support.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3078) Make Secondary Indexes Pluggable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-3078:
--

Attachment: v1-0004-CASSANDRA-3078-add-cli-show-support.txt
v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt
v1-0002-CASSANDRA-3078-thrift-generated-changes.txt
v1-0001-CASSANDRA-3078-add-custom-index-type.txt

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, 
 v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, 
 v1-0004-CASSANDRA-3078-add-cli-show-support.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093129#comment-13093129
 ] 

T Jake Luciani commented on CASSANDRA-3078:
---

ok, added back switch statement.

Also added cli support for displaying column metadata.

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, 
 v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, 
 v1-0004-CASSANDRA-3078-add-cli-show-support.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1162988 - in /cassandra/trunk: CHANGES.txt src/java/org/apache/cassandra/db/CollationController.java src/java/org/apache/cassandra/db/ColumnFamilyStore.java src/java/org/apache/cassandra/

Author: jbellis
Date: Mon Aug 29 20:28:46 2011
New Revision: 1162988

URL: http://svn.apache.org/viewvc?rev=1162988view=rev
Log:
fix race condition in sstable reference counting
patch by jbellis; reviewed by slebresne for CASSANDRA-3085

Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/trunk/src/java/org/apache/cassandra/db/RowIteratorFactory.java

Modified: cassandra/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1162988r1=1162987r2=1162988view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Aug 29 20:28:46 2011
@@ -44,7 +44,7 @@
Thrift-Avro conversion methods (CASSANDRA-3032)
  * Add timeouts to client request schedulers (CASSANDRA-3079)
  * Cli to use hashes rather than array of hashes for strategy options 
(CASSANDRA-3081)
- * LeveledCompactionStrategy (CASSANDRA-1608)
+ * LeveledCompactionStrategy (CASSANDRA-1608, 3085)
  * Improvements of the CLI `describe` command (CASSANDRA-2630)
  * reduce window where dropped CF sstables may not be deleted (CASSANDRA-2942)
 

Modified: 
cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java?rev=1162988r1=1162987r2=1162988view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/CollationController.java 
Mon Aug 29 20:28:46 2011
@@ -72,15 +72,12 @@ public class CollationController
 {
 logger.debug(collectTimeOrderedData);
 
-ListIColumnIterator iterators;
-ColumnFamily container;
-while (true)
-{
-DataTracker.View dataview = cfs.getDataTracker().getView();
-iterators = new ArrayListIColumnIterator();
-container = ColumnFamily.create(cfs.metadata, factory, 
filter.filter.isReversed());
-ListSSTableReader sstables;
-for (Memtable memtable : 
Iterables.concat(dataview.memtablesPendingFlush, 
Collections.singleton(dataview.memtable)))
+ColumnFamily container = ColumnFamily.create(cfs.metadata, factory, 
filter.filter.isReversed());
+ListIColumnIterator iterators = new ArrayListIColumnIterator();
+ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
+try
+{
+for (Memtable memtable : view.memtables)
 {
 IColumnIterator iter = 
filter.getMemtableColumnIterator(memtable, cfs.metadata.comparator);
 if (iter != null)
@@ -99,42 +96,33 @@ public class CollationController
 QueryFilter reducedFilter = new QueryFilter(filter.key, 
filter.path, new NamesQueryFilter(filterColumns));
 
 /* add the SSTables on disk */
-sstables = dataview.intervalTree.search(new Interval(filter.key, 
filter.key));
-Collections.sort(sstables, SSTable.maxTimestampComparator);
-if (!SSTableReader.acquireReferences(sstables))
-continue; // retry w/ new view
+Collections.sort(view.sstables, SSTable.maxTimestampComparator);
 
-try
+// read sorted sstables
+for (SSTableReader sstable : view.sstables)
 {
-// read sorted sstables
-for (SSTableReader sstable : sstables)
+long currentMaxTs = sstable.getMaxTimestamp();
+reduceNameFilter(reducedFilter, container, currentMaxTs);
+if (((NamesQueryFilter) 
reducedFilter.filter).columns.isEmpty())
+break;
+
+IColumnIterator iter = 
reducedFilter.getSSTableColumnIterator(sstable);
+iterators.add(iter);
+if (iter.getColumnFamily() != null)
 {
-long currentMaxTs = sstable.getMaxTimestamp();
-reduceNameFilter(reducedFilter, container, currentMaxTs);
-if (((NamesQueryFilter) 
reducedFilter.filter).columns.isEmpty())
-break;
-
-IColumnIterator iter = 
reducedFilter.getSSTableColumnIterator(sstable);
-iterators.add(iter);
-if (iter.getColumnFamily() != null)
-{
-container.delete(iter.getColumnFamily());
-sstablesIterated++;
-while (iter.hasNext())
-container.addColumn(iter.next());
-}
+container.delete(iter.getColumnFamily());
+

[jira] [Commented] (CASSANDRA-3085) Race condition in sstable reference counting


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093166#comment-13093166
 ] 

Jonathan Ellis commented on CASSANDRA-3085:
---

committed with requested fixes, fix for interval computation involving 
stopAt.isEmpty(), and javadoc for CFS.markReferenced

 Race condition in sstable reference counting
 

 Key: CASSANDRA-3085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3085
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Critical
 Fix For: 1.0

 Attachments: 3085-v2.txt, 3085.txt


 DataTracker gives us an atomic View of memtable/sstables, but acquiring 
 references is not atomic.  So it is possible to acquire references to an 
 SSTableReader object that is no longer valid, as in this example:
 View V contains sstables {A, B}.  We attempt a read in thread T using this 
 View.
 Meanwhile, A and B are compacted to {C}, yielding View W.  No references 
 exist to A or B so they are cleaned up.
 Back in thread T we acquire references to A and B.  This does not cause an 
 error, but it will when we attempt to read from them next.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3078) Make Secondary Indexes Pluggable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093177#comment-13093177
 ] 

Jonathan Ellis commented on CASSANDRA-3078:
---

- ThriftValidation should probably try to instantiate the given index type, w/ 
the options map, so it can check for and raise exception on invalid options, 
the way we do for replication strategy
- spaces around operators please

 Make Secondary Indexes Pluggable 
 -

 Key: CASSANDRA-3078
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3078
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.0
Reporter: T Jake Luciani
Assignee: T Jake Luciani
  Labels: secondary_index
 Fix For: 1.0

 Attachments: v1-0001-CASSANDRA-3078-add-custom-index-type.txt, 
 v1-0002-CASSANDRA-3078-thrift-generated-changes.txt, 
 v1-0003-CASSANDRA-3078-remove-class-name-lookup.txt, 
 v1-0004-CASSANDRA-3078-add-cli-show-support.txt


 CASSANDRA-2982 got us most of the way there...
 This ticket removes the IndexType enum (while keeping support for KEYS 
 internally from old cf metadata).
 You now specify a index_class rather than index_type.  index_class is the 
 full classname of the SecondaryIndex impl.  This also adds a index_options 
 map to pass extra info to the secondary index impl if needed.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1827) Batching across stages


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-1827:
--

Fix Version/s: (was: 1.0)

 Batching across stages
 --

 Key: CASSANDRA-1827
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1827
 Project: Cassandra
  Issue Type: Improvement
Reporter: Chris Goffinet

 We might be able to get some improvement if we start batching tasks for every 
 stage.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL

2011-08-29 Thread Mikko Koppanen (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093222#comment-13093222
 ] 

Mikko Koppanen commented on CASSANDRA-3025:
---

bq. Why is that? Does PDO require only primitive or string column names?

This is not actually a PDO requirement but rather PHP array type requirement. A 
key in array can be integer or string. I looked at phpcassa code briefly and it 
doesn't seem that they automatically marshal the UUIDs but rather provide a 
method for doing that.

bq. should test an actual big int ( 64bit)

I added a test for very big integers. There is really no native datatype in PHP 
that could handle this, so I took the following approach:

1. If the integer is larger than 8 bytes, return LONG_MIN
2. If user sees LONG_MIN, it means that binary presentation might be different
3. Turn on CASSANDRA_ATTR_PRESERVE_VALUES and convert manually, using bcmath or 
gmp libraries.


Currently the conversion back to userland will overflow on 32bit platforms when 
values larger than LONG_MAX or smaller than INT_MIN. Not sure if there is a 
clean and portable way to handle this as PHP uses platform long for integer 
types rather than fixed width integers. 

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, 
 php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL

2011-08-29 Thread Mikko Koppanen (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093224#comment-13093224
 ] 

Mikko Koppanen commented on CASSANDRA-3025:
---

Forgot to link to test: 
https://github.com/mkoppanen/php-pdo_cassandra/blob/master/tests/026-bigint.phpt

In case of PHP the users are better of using string types rather than actual 
long integers.

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, 
 php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093230#comment-13093230
 ] 

Pavel Yaskevich commented on CASSANDRA-3051:


I figured out the problem - SSLSocket always returns null on getChannel even on 
current code, I will refactor FileStreamingTask to support DataOutputStream 
instead of SocketChannel to unify normal and SSL transfers.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3100) Secondary index still does minor compacting after deleting index


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093255#comment-13093255
 ] 

Jeremy Hanna commented on CASSANDRA-3100:
-

Also relevant, when bootstrapping a new node into the ring, it streamed data 
from a node and in the compactionstats I also see it's trying to build a 
secondary index that shouldn't be there.

 Secondary index still does minor compacting after deleting index
 

 Key: CASSANDRA-3100
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3100
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.8
Reporter: Jeremy Hanna

 We deleted all of our secondary indexes.  A couple of days later I was 
 watching compactionstats on one of the nodes and it was in the process of 
 minor compacting one of the deleted secondary indexes.  I double checked the 
 keyspace definitions on the CLI and there were no secondary indexes defined.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes

2011-08-29 Thread Todd Nine (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093263#comment-13093263
]

Todd Nine commented on CASSANDRA-2915:
--

Could we also use this feature as a standard way for building our lucene
documents? This would accomplish what we want, as well as giving a hook for
more user functionality.

CASSANDRA-1311

Lucene based Secondary Indexes
--

Key: CASSANDRA-2915
URL: https://issues.apache.org/jira/browse/CASSANDRA-2915
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: T Jake Luciani
Assignee: Jason Rutherglen
Labels: secondary_index

Secondary indexes (of type KEYS) suffer from a number of limitations in their
current form:
- Multiple IndexClauses only work when there is a subset of rows under the
highest clause
- One new column family is created per index this means 10 new CFs for 10
secondary indexes
This ticket will use the Lucene library to implement secondary indexes as one
index per CF, and utilize the Lucene query engine to handle multiple index
clauses. Also, by using the Lucene we get a highly optimized file format.
There are a few parallels we can draw between Cassandra and Lucene.
Lucene indexes segments in memory then flushes them to disk so we can sync
our memtable flushes to lucene flushes. Lucene also has optimize() which
correlates to our compaction process, so these can be sync'd as well.
We will also need to correlate column validators to Lucene tokenizers, so the
data can be stored properly, the big win in once this is done we can perform
complex queries within a column like wildcard searches.
The downside of this approach is we will need to read before write since
documents in Lucene are written as complete documents. For random workloads
with lot's of indexed columns this means we need to read the document from
the index, update it and write it back.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-3051:
---

Attachment: CASSANDRA-3051.patch

CompressedRandomAccessReader.transfer method was removed with special casing 
for compressed files, SocketChannel based transfer changed to DataOuputStream 
based to unify SSL and normal modes. SSLFileStreamTask removed as unused.

 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3025) PHP/PDO driver for Cassandra CQL

2011-08-29 Thread Mikko Koppanen (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093283#comment-13093283
 ] 

Mikko Koppanen commented on CASSANDRA-3025:
---

Changed the behaviour a bit: 
https://github.com/mkoppanen/php-pdo_cassandra/blob/master/tests/026-bigint.phpt

There is an error/exception if the conversion fails and user is forced to 
handle this scenario. This allows user to react if they are handling very large 
integers.

 PHP/PDO driver for Cassandra CQL
 

 Key: CASSANDRA-3025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3025
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Mikko Koppanen
  Labels: php
 Attachments: pdo_cassandra-0.1.0.tgz, pdo_cassandra-0.1.1.tgz, 
 pdo_cassandra-0.1.2.tgz, pdo_cassandra-0.1.3.tgz, 
 php_test_results_20110818_2317.txt


 Hello,
 attached is the initial version of the PDO driver for Cassandra CQL language. 
 This is a native PHP extension written in what I would call a combination of 
 C and C++, due to PHP being C. The thrift API used is the C++.
 The API looks roughly following:
 {code}
 ?php
 $db = new PDO('cassandra:host=127.0.0.1;port=9160');
 $db-exec (CREATE KEYSPACE mytest with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor=1;);
 $db-exec (USE mytest);
 $db-exec (CREATE COLUMNFAMILY users (
   my_key varchar PRIMARY KEY,
   full_name varchar ););
   
 $stmt = $db-prepare (INSERT INTO users (my_key, full_name) VALUES (:key, 
 :full_name););
 $stmt-execute (array (':key' = 'mikko', ':full_name' = 'Mikko K' ));
 {code}
 Currently prepared statements are emulated on the client side but I 
 understand that there is a plan to add prepared statements to Cassandra CQL 
 API as well. I will add this feature in to the extension as soon as they are 
 implemented.
 Additional documentation can be found in github 
 https://github.com/mkoppanen/php-pdo_cassandra, in the form of rendered 
 MarkDown file. Tests are currently not included in the package file and they 
 can be found in the github for now as well.
 I have created documentation in docbook format as well, but have not yet 
 rendered it.
 Comments and feedback are welcome.
 Thanks,
 Mikko

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3095) java.lang.NegativeArraySizeException during compacting large row

2011-08-29 Thread Pas (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093294#comment-13093294
 ] 

Pas commented on CASSANDRA-3095:


Hi, as I've still 3 nodes on 0.7.4, I'll try to reproduce this error on one of 
them. (ETA ~2 days.)

 java.lang.NegativeArraySizeException during compacting large row
 

 Key: CASSANDRA-3095
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3095
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.4
 Environment: Linux 2.6.26-2-amd64 #1 SMP Thu Feb 11 00:59:32 UTC 2010 
 x86_64 GNU/Linux
 JDK 1.6.0_27 (Java 6 update 27), with JNA.
Reporter: Pas
 Attachments: 3095-debug.txt


 Hello,
 It's a 4 node ring, 3 on 0.7.4, I've upgraded one to 0.8.4. This particular 
 node was having issues with compaction that's why I've tried the upgrade (it 
 looks likely that this solved the compaction issues).
 Here's the stack trace from system.log.
  INFO [CompactionExecutor:22] 2011-08-28 18:12:46,566 
 CompactionController.java (line 136) Compacting large row  (36028797018963968 
 bytes) incrementally
 ERROR [CompactionExecutor:22] 2011-08-28 18:12:46,609 
 AbstractCassandraDaemon.java (line 134) Fatal exception in thread 
 Thread[CompactionExecutor:22,1,main]
 java.lang.NegativeArraySizeException
 at 
 org.apache.cassandra.utils.obs.OpenBitSet.init(OpenBitSet.java:85)
 at 
 org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:56)
 at 
 org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:73)
 at 
 org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62)
 at 
 org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
 at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.init(LazilyCompactedRow.java:89)
 at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:138)
 at 
 org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:123)
 at 
 org.apache.cassandra.db.compaction.CompactionIterator.getReduced(CompactionIterator.java:43)
 at 
 org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:74)
 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 at 
 org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
 at 
 org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.doCompactionWithoutSizeEstimation(CompactionManager.java:569)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.doCompaction(CompactionManager.java:506)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:141)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:107)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 We've ~70 files still in f format. And 80 in g. We've ~100 GB of data on 
 this node.
 Thanks.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes

2011-08-29 Thread Jason Rutherglen (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093298#comment-13093298
]

Jason Rutherglen commented on CASSANDRA-2915:
-

Todd,

Another option is to add a [user optional] class that converts raw Cassandra
columns into a Lucene document. Implicitly the Cassandra columns do not need
to map to Lucene document fields. This is more of a slight change in the
user's expectations for CQL rather than a core functional change. Eg, the CQL
submitted to a Lucene secondary index may refer to Lucene fields that do not
exist as columns.

Lucene based Secondary Indexes
--

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes

2011-08-29 Thread Ed Anuff (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093341#comment-13093341
]

Ed Anuff commented on CASSANDRA-2915:
-

+1 on having the ability to provide a conversion class for handling
transformations from columns to Lucene documents. It's not uncommon for people
to store objects serialized to JSON or other some other serialization format
into columns. CQL will have to catch up with this practice at some point.

Lucene based Secondary Indexes
--

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2915) Lucene based Secondary Indexes

2011-08-29 Thread Todd Nine (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093344#comment-13093344
]

Todd Nine commented on CASSANDRA-2915:
--

I think forcing users to install classes for common use cases would cause
issues with adoption. What about creating new CQL commands to handle this?
When creating an index in a db, you would define the fields and the manner in
which they are indexed. Could we do something like the following?

create index [colname] in [colfamily] using [index type 1] as [indexFieldName],
[index type 2] as [indexFieldName], [index type n] as [indexFieldName]?

drop index [indexFieldName] in [colfamily] on [colname]

This way clients such as JPA can update and create indexes, without the need to
install custom classes on Cassandra itself. They also have the ability to
directly reference the field name when using CQL queries.

Assuming that the index class types exist in the Lucene classpath, you get the
1 to many mappings for column to indexing strategy. This would allow more
advanced clients such as the JPA plugin to automatically add indexes to the
document based on indexes defined on persistent fields, without generating any
code the user has to install in the Cassandra runtime. If users want to
install custom analyzers, they still have the option to do so, and would gain
access to it via CQL.

Lucene based Secondary Indexes
--

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2884) CQL processing engine should provide server side data binding to pre-compiled CQL scripts

[
https://issues.apache.org/jira/browse/CASSANDRA-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093372#comment-13093372
]

Rick Shaw commented on CASSANDRA-2884:
--

Yes. Seems like it to me as well.

CQL processing engine should provide server side data binding to pre-compiled
CQL scripts
-

Key: CASSANDRA-2884
URL: https://issues.apache.org/jira/browse/CASSANDRA-2884
Project: Cassandra
Issue Type: New Feature
Components: API
Affects Versions: 0.8.1
Reporter: Rick Shaw
Labels: API, CQL
Fix For: 1.0

The notion of Prepared statements is derived from JDBC lore, but is equally
useful for the other supported clients of CQL. In order to support
server-side binding and pre-compiled code referenceable from the client side,
the API (currently Thrift based) will need to be enhanced to pass both script
(CQL) and the binding arguments to the server. In addition the parser will
need to recognize the place holders for the bound variables (?). And the
product of the parse will need to be savable and a reference or handle will
need to be returned to the client side caller. The execution of C* API calls
will need to be moved from the parsers action routines where it is currently
executed as the script is parsed, and moved to another execution engine
that calls API methods and uses the bound variables and the stored
pre-compiled code to drive the process.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3051) On Disk Compression breaks SSL Encryption


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093373#comment-13093373
 ] 

Jonathan Ellis commented on CASSANDRA-3051:
---

- feels like we lose more than we gain by making writeHeader/write separate 
methods.  they aren't really self-contained so you have to keep the context 
they were called in, around mentally.  and if they were in-line, it would be 
obvious that you don't need to re-seek for each call to write().
- comments in write() don't really add much to what the code says, imo
- is flushing with each chunk necessary?  seems like that would harm performance


 On Disk Compression breaks SSL Encryption
 -

 Key: CASSANDRA-3051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3051
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0
 Environment: Trunk
Reporter: Benjamin Coverston
Assignee: Pavel Yaskevich
 Fix For: 1.0

 Attachments: CASSANDRA-3051.patch


 Encryption depends on FileStreamTask.write [1] protected member to be called 
 because the SSLFileStreamTask.write overrides this to write back to the 
 server.
 When enabled, compression circumvents the call and the client does not 
 communicate using an SSL socket back to the server.
 [1]
 protected long write(FileChannel fc, PairLong, Long section, long length, 
 long bytesTransferred) throws IOException

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-2915) Lucene based Secondary Indexes

2011-08-29 Thread Todd Nine (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093344#comment-13093344
]

Todd Nine edited comment on CASSANDRA-2915 at 8/30/11 2:13 AM:
---

create index on [colname] in [colfamily] using [index type 1] as
[indexFieldName], [index type 2] as [indexFieldName], [index type n] as
[indexFieldName]?

drop index [indexFieldName] in [colfamily] on [colname]

was (Author: tnine):
I think forcing users to install classes for common use cases would cause
issues with adoption. What about creating new CQL commands to handle this?
When creating an index in a db, you would define the fields and the manner in
which they are indexed. Could we do something like the following?

create index [colname] in [colfamily] using [index type 1] as [indexFieldName],
[index type 2] as [indexFieldName], [index type n] as [indexFieldName]?

drop index [indexFieldName] in [colfamily] on [colname]

Lucene based Secondary Indexes
--

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2884) CQL processing engine should provide server side data binding to pre-compiled CQL scripts